WO2022068557A1

WO2022068557A1 - Biological information verification method and device

Info

Publication number: WO2022068557A1
Application number: PCT/CN2021/117858
Authority: WO
Inventors: 韩亚
Original assignee: 华为技术有限公司
Priority date: 2020-09-30
Filing date: 2021-09-11
Publication date: 2022-04-07
Also published as: CN114331448A

Abstract

A biological information verification method and device. In the method, a device can collect voiceprint information of a user and invoke a nearby face collection device to collect facial information of the user. When the device receives a user operation of requesting payment for a pay audio resource, the device can verify the identity of the user by combining voiceprint verification and face verification. When the verification is passed, the device can request a payment server to deduct money from a payment account, so as to pay for the pay audio resource. By means of the method for verifying the identity of a user by combining a plurality of pieces of biological information, the security of identity verification can be improved. In addition, a device which is not configured with a face collection apparatus can also complete identity verification during a payment process by combining voiceprint verification and face verification, and in this way, a payment operation by a user can be simplified.

Description

Biological information verification method and equipment

This application claims the priority of the Chinese patent application with the application number 202011060748.2 and the application name "Biological Information Verification Method and Equipment" filed with the China Patent Office on September 30, 2020, the entire contents of which are incorporated into this application by reference.

technical field

The present application relates to the field of terminal technology, and in particular, to a method and device for verifying biological information.

Background technique

With the development of smart home devices such as speakers, users can easily listen to various audio resources, such as music, audio books, etc. through speakers. Paying for audio resources has also become a trend.

Completing the identity verification in the payment process by means of biometric information verification can improve the convenience of the payment process. However, a general speaker only includes a voice collection device for collecting voiceprint information, and is not equipped with a device for collecting other types of biological information. For example, a face collection device, a fingerprint collection device, etc. Only through a single biometric information verification (such as voiceprint verification) to complete the identity verification in the payment process is not conducive to ensuring the security of identity verification.

SUMMARY OF THE INVENTION

The present application provides a biological information verification method and device. This method can be used for authentication during the payment process. The device (such as a speaker) can collect the first biological information of the user, and call a nearby device with the second biological information collection capability to collect the second biological information of the user. The first biological information and the second biological information are different types of biological information. The device can combine the first biometric information verification and the second biometric information verification to verify the user's identity during the payment process. In this way, the user can complete the identity verification in the payment process through his own first biometric information and second biometric information. The above method can simplify the user's payment operation. In addition, a device that does not have the ability to collect the second biological information can also verify the identity of the user by combining the first biological information verification and the second biological information verification, thereby improving the security of the identity verification.

In a first aspect, the present application provides a biological information verification method. The method includes: the first device can collect the first biological information. The first device may discover a second device with a second biological information collection capability through a short-range wireless communication connection. The first device may receive second biological information collected from the second device, where the second biological information is different from the first biological information. Further, the first device may determine that the first biological information matches the third biological information, and the second biological information matches the fourth biological information. The above-mentioned third biological information may be stored in the first device or a cloud storage space accessible by the first device. The third biological information is the biological information of the first user pre-collected by the first device. The fourth biological information may be stored in the first device or a cloud storage space accessible by the first device. The fourth biological information is the biological information of the first user pre-collected by the second device.

Through the biometric information verification method provided in this application, the first device can complete the identity verification in the payment process. The first device that does not have the ability to collect the second biological information can also verify the identity of the user by combining the first biological information verification and the second biological information verification to improve the security of the identity verification. In addition, the user can quickly complete the identity verification in the payment process through his first biometric information and the second biometric information, thereby improving the convenience of the payment process.

The above-mentioned third biological information and the above-mentioned fourth biological information can be used as the reference biological information of the first user, and are respectively used to determine whether the first biological information obtained by the first device is the biological information of the first user and the second biological information obtained by the first device. Whether the biological information is the biological information of the first user.

The above short-range wireless communication connection may be one or more of the following: a near field communication connection, a Bluetooth communication connection, a WLAN direct connection communication connection, and a ZigBee communication connection.

In a possible implementation manner, when the above-mentioned authentication is successful, the first device may send the first message to the first server. The first message may be used to instruct the first server to debit the first payment account. The first payment account is the payment account of the first user. The above-mentioned first server may be a payment server.

With reference to the first aspect, in some embodiments, the above-mentioned first device is a device configured with a voice collection device, for example, a speaker. The above-mentioned second device is a device configured with a face collection device, for example, a TV. Both the first biometric information and the third biometric information may be voiceprint information. Both the second biological information and the fourth biological information may be face information.

In this way, the speaker can combine voiceprint verification and face verification to verify the user's identity. That is, users can complete identity verification through their own voice and face, so as to pay conveniently.

With reference to the first aspect, in other embodiments, the above-mentioned first device is a device configured with a face acquisition device, for example, a television. The above-mentioned second device may be a device equipped with a voice collection device, for example, a speaker. Both the first biological information and the third biological information may be face information. Both the second biometric information and the fourth biometric information may be voiceprint information.

With reference to the first aspect, in some embodiments, before the first device collects the first biological information, the method further includes: the first device can log in to the first account on the second server. The first account is the account of the first user on the second server. The second server may be configured to send the content purchased by the first user to the first device after the first server successfully debits the first payment account. The first device may send an acquisition request for the first content to the second server. The first device may receive price information of the first content sent by the second server.

Wherein, the above-mentioned first server may be a content server. Specifically, the content server may be, for example, a music server, a video server, and the like. The above-mentioned first content may be, for example, audio data, video data, or the like.

It should be noted that the above-mentioned first device may be bound with the above-mentioned first account. Exemplarily, the first device establishes a communication connection with the third device. An application for controlling the first device is installed in the third device. The above-mentioned third device may be, for example, a mobile phone, a tablet, or the like. The third device may log in to the above-mentioned first account in the application. Further, the third device may send the relevant information of the first account to the first device. In this way, the first device can establish a binding relationship with the first account.

Before the first device collects the third biometric information and the fourth biometric information of the first user, the third device may perform identity verification on the first payment account. The third device may receive the payment password of the first payment account input by the user. The third device may then compare the received payment password with the stored payment password for the first payment account. If they are the same, the authentication of the first payment account by the third device is successful. The first payment account may be associated with the first account number. Further, the third device may send an instruction for collecting the third biological information and the fourth biological information to the first device. In this way, the first device can collect the third biological information and the fourth biological information, and associate the remaining first accounts. During the payment process, if the first biometric information matches the third biometric information, and the second biometric information matches the fourth biometric information, the first device can request the payment server for the payment account associated with the first account, that is, the first payment The account is debited to purchase the above-mentioned first content.

That is to say, the first user can first complete the identity verification of the first payment account on the third device, and then instruct the first device to enter the above-mentioned third biometric information and the above-mentioned fourth biometric information through the third device.

In addition, identifiers of multiple accounts may be stored in the content server. The above-mentioned multiple accounts include a first account. The content server may determine the paid resources purchased by each account according to the identifiers of the above-mentioned multiple accounts. When the first device logs into the first account on the content server and obtains the first content from the content server, the content server may determine that the first account has not purchased the first content. Further, the content server may send a message to the first device to prompt the first device to pay for acquiring the first content. The content server may also send price information of the first content to the first device.

The following specifically takes the first device as a speaker equipped with a voice acquisition device, and the second device as a TV equipped with a face acquisition device as an example to illustrate that the speaker completes identity verification through biometric information, and requests the payment server to deduct the first payment account method of implementation.

Exemplarily, the first user wakes up the speaker and instructs the speaker to play music A. The music A is a paid resource. The speaker can receive the information sent by the music server to indicate that music A can only be obtained after payment. The speaker can voice broadcast "you can listen to the complete music A after payment, whether to pay or not" to remind the first user that the music A that has not been purchased is currently unavailable. Further, the speaker can receive a voice command "please help me complete the payment". That is, the first user requests to purchase music A.

The speaker can start voiceprint verification and face verification. Specifically, the speaker can voice broadcast "Okay, start the voiceprint verification. Please follow me and say the following verification word: Xiaoyi Xiaoyi" to prompt the first user to enter the voiceprint information. The first user can say the verification word "Xiaoyi Xiaoyi" according to the prompt of the speaker. The speaker can extract voiceprint information from the received voice input and compare it with the reference voiceprint information of the first user. If they match, the voiceprint verification is successful.

Then, the speaker can find and call a nearby TV equipped with a camera to collect face information. The speaker can voice broadcast "Voiceprint verification is successful, start face verification. Please aim your face at the camera of Li Ming's TV and wink" to prompt the first user to enter through the TV named "Li Ming" face information. Among them, the speaker can prompt the user to adjust the face by means of voice broadcast, so that the face is aimed at the camera. When receiving the face information collected from the TV, the speaker can compare the face information with the reference face information of the first user. If it matches, the face verification is successful. In this way, the speaker can complete the authentication of the user.

Further, the speaker may request the payment server to debit the first payment account. Among them, the payment server can perform trusted authentication on the speaker to determine that the speaker is a trusted device. The speaker may send the identification information of the first payment account and the order information of the paid content to be purchased by the first user (ie, the above-mentioned music A) to the payment server. In this way, the payment server can debit the first payment account according to the above order information. When the deduction is successful, the payment server may send a message indicating that the payment is successful to the music server. Thus, the music server can send the resource of music A to the speaker. The speaker can play the successfully purchased music A for the user.

In combination with the first aspect, in some embodiments, the third biological information and the fourth biological information stored in the cloud storage space can be accessed by multiple devices. The plurality of devices includes a first device. The multiple devices can share the same account on the content server, or the multiple accounts of the multiple devices on the content server belong to the same account group; belonging to the same account group means that multiple accounts in the same account group share each of the accounts Content purchased in the second server.

Among them, since the third biological information and the fourth biological information stored in the cloud storage space can be accessed by multiple devices sharing the same account, these multiple devices can obtain the third biological information and the fourth biological information from the cloud storage space to authenticate. In this way, the user only needs to input the reference biometric information once, and then the paid resources requested by the first device can be paid for by the first biometric information verification and the second biometric information verification on multiple speakers. The above implementation manner can simplify the user's operation of entering the reference biometric information for payment, and improve the user's use experience.

In conjunction with the first aspect, in some embodiments, the first user may authorize the second user to establish reference biometric information of the second user on the first device. Wherein, the method may include: the first device may collect fifth biological information of the second user. The fifth biological information is the same type of biological information as the first biological information. The fifth biological information may be used to determine whether the biological information obtained by the first device is the biological information of the second user. The fifth biological information may be stored in the first device or a cloud storage space accessible by the first device. The fifth biological information is bound with the fourth biological information.

If the first biological information is voiceprint information, the fifth biological information is also voiceprint information.

The above-mentioned first user may be a primary user associated with the first account. The above-mentioned second user may be an authorized user associated with the first account. The primary user associated with the first account may be: when the first device or the cloud storage space accessible to the first device does not store the reference biological information associated with the first account, the reference biological information established by the first device the corresponding user. The authorized user associated with the first account may be: when the reference biological information associated with the first account is stored in the first device or the cloud storage space accessible to the first device, the reference biological information established by the first device corresponds to the reference biological information. user. The first account can be associated with one primary user and one or more authorized users.

When the second user (that is, the authorized user) uses his own biometric information for verification, the main user's authorization is required to complete the identity verification. Specifically, the first device may collect sixth biological information. The sixth biological information is the same type of biological information as the first biological information. The first device may receive the second biological information collected from the second device. The first device may determine that the sixth biological information matches the fifth biological information, and the third biological information matches the fourth biological information. The first device may send a second message to the first server, where the second message is used to instruct the first server to debit the first payment account.

It can be seen that the above-mentioned main user authorization process may be a process of verifying the second biological information of the main user. That is to say, when the authorized user's voiceprint information is verified, combined with the verification of the main user's face information, the first device can request the payment server to debit the main user's payment account (ie, the first payment account). to purchase paid resources. In this way, it is not limited to the primary user to complete the payment through the first biometric information verification and the second biometric information verification. Authorized users, such as the primary user's family members and friends, can also pass the first biometric information verification under the authorization of the primary user. and the second biometric information verification method to complete the payment. Moreover, since the first payment account associated with the first account bound to the first device can be considered as the main user's payment account, the method of verifying in combination with the authorized user's voiceprint information and the main user's face information can be performed in the first payment account. A payment account must be confirmed by the main user before making payment, so as to avoid the authorized user from abusing the authority and over-consuming the main user's payment account.

With reference to the first aspect, in other embodiments, the first user may authorize the second user to establish reference biometric information of the second user on the first device. The method includes: the first device can collect fifth biological information of the second user. The fifth biological information is the same type of biological information as the first biological information. The fifth biological information may be used to determine whether the biological information obtained by the first device is the biological information of the second user. The fifth biological information may be stored in the first device or a cloud storage space accessible by the first device. The first device may receive seventh biological information of the second user collected by the second device. The seventh biological information is the same type of biological information as the second biological information. The seventh biological information may be used to determine whether the biological information obtained by the first device is the biological information of the second user. The seventh biological information may be stored in the first device or a cloud storage space accessible by the first device. Wherein, the fifth biological information is bound with the seventh biological information.

When the second user (ie, the authorized user) uses his own biometric information for authentication, the authorization of the main user may not be required. That is, the second user can independently complete the identity verification by using his own biometric information. The first device may request the payment server to debit the first payment account after the second user's identity verification is successful. Specifically, the first device may collect sixth biological information. The sixth biological information is the same type of biological information as the first biological information. The first device may receive the eighth biological information collected from the second device. The eighth biological information is the same type of biological information as the second biological information. The first device may determine that the sixth biological information matches the fifth biological information, and the eighth biological information matches the seventh biological information. Further, the first device may send the second message to the first server. The second message is used to instruct the first server to debit the first payment account.

In the above method, the second user (ie, the authorized user) can independently complete the authentication. In this way, in a scenario where the primary user is no longer near the first device, the second user can also complete identity verification on the first device, and use the primary user's payment account to purchase paid resources.

Optionally, the purchase of paid resources by the second user using the payment account of the primary user still requires the consent of the primary user.

Specifically, after the sixth biometric information verification and the eighth biometric information verification are both successful, the method further includes: the first device may send a third message to the third device, and receive a fourth message from the third device. The third device is a device installed with an application program for controlling the first device, the indication content of the third message can be displayed on the third device, and the indication content of the third message includes asking the third device whether to allow the first device to send the second device message, and the indication content of the fourth message includes that the third device agrees to the first device to send the second message.

That is to say, after the second user completes the authentication on the first device, the first device can send the third message to the third device. The third message may prompt the first user that the second user requests to use the payment account of the first user to purchase paid resources. If the first user agrees that the second user uses his own payment account to purchase paid resources, the third device may send a fourth message to the first device. When receiving the fourth message, the first device may request the payment server to debit the first payment account to purchase the paid resource.

It can be seen that the first user (ie the main user) can remotely authorize the first device to request payment resources from the payment server through the third device. The above-mentioned primary user remote authorization may be: when the first detection of the biometric information of the authorized user is successful, the first device may send a message for confirming whether to pay to the third device. The third device may display the above message for confirming whether to pay on the user interface. Furthermore, the primary user may authorize the first device through the third device to request the payment server to debit the first payment account.

In this way, in a scenario where the primary user is not near the first device, the second user (ie, the authorized user) can also complete the identity verification, thereby purchasing the paid resource. In combination with the above-mentioned method of remote authorization of the main user, it can be ensured that the payment account of the main user is debited after the consent of the main user. This not only simplifies the payment operation when acquiring the paid resources on the first device, but also improves the security of the payment account, and effectively prevents the authorized user from over-consuming the payment account of the main user.

In some embodiments, in conjunction with the first aspect, the first device does not have the capability to collect the second biological information. The first device may collect the second biological information by finding and calling a nearby device having the capability of collecting the second biological information.

In this way, a device that does not have the capability of collecting the second biological information can also verify the identity of the user by combining the first biological information verification and the second biological information verification, thereby improving the security of the identity verification performed by the first device.

In a second aspect, the present application provides a device, which is a first device. The first device may include: a first collection device, a communication device, a memory and a processor, wherein: the first collection device can be used to collect the first biological information. The communication device can be used by the first device to discover the second device with the second biological information collection capability through the wireless communication connection. The second device includes a second collection device, and the second collection device can be used to collect the above-mentioned second biological information. The communication device can also be used to receive the second biological information collected from the second collecting device. The memory can be used to store the first biological information and the second biological information, and can also be used to store a computer program. The memory can be used to invoke the above computer program, so that the first device executes any possible implementation method of the above first aspect.

In a third aspect, the present application provides a computer-readable storage medium, including instructions, when the above-mentioned instructions are executed on the device provided in the above-mentioned second aspect, the device enables the device to execute any one of the possible implementation methods of the above-mentioned first aspect.

In a fourth aspect, the present application provides a computer program product that, when the computer program product is executed on the device provided in the second aspect, enables the device to execute any of the possible implementation methods of the first aspect.

In a fifth aspect, the present application provides a chip, the chip is applied to the device provided in the second aspect, the chip includes one or more processors, and the one or more processors are used for invoking computer instructions to make the device provided by the second aspect The device executes any one of the possible implementation methods of the first aspect above.

Understandably, the device provided in the second aspect, the computer-readable storage medium provided in the third aspect, the computer program product provided in the fourth aspect, and the chip provided in the fifth aspect are all used to execute the method provided by the embodiments of the present application. . Therefore, for the beneficial effects that can be achieved, reference may be made to the beneficial effects in the corresponding method, which will not be repeated here.

Description of drawings

1 is a schematic structural diagram of a sound box provided by an embodiment of the present application;

2A to 2E are a series of schematic diagrams of user interfaces for performing identity authentication on a payment account provided by an embodiment of the present application;

2F to 2J are schematic diagrams of scenarios in which some speakers according to the embodiments of the present application record reference voiceprint information and reference face information;

3 is a flowchart of a method for inputting reference voiceprint information and reference face information in a sound box provided by an embodiment of the present application;

4A to 4E are schematic diagrams of scenarios in which speakers pay for paid audio resources according to an embodiment of the present application;

5 is a flowchart of a method for paying for paid audio resources for a speaker provided by an embodiment of the present application;

6A to 6F are schematic diagrams of scenarios in which reference voiceprint information and reference face information are entered into other speakers provided by the embodiments of the present application;

7 is a flowchart of another method for inputting reference voiceprint information and reference face information in a speaker provided by an embodiment of the present application;

8A to 8E are schematic diagrams of another scenario in which speakers pay for paid audio resources according to an embodiment of the present application;

9 is a flowchart of a method for paying for paid audio resources for another speaker provided by an embodiment of the present application;

10A to 10E are schematic diagrams of scenarios in which reference voiceprint information and reference face information are entered into other speakers provided by the embodiments of the present application;

11A to 11D are schematic diagrams of another scenario in which speakers pay for paid audio resources according to an embodiment of the present application;

12A to 12D are schematic diagrams of another scenario in which a speaker pays for paid audio resources provided by an embodiment of the present application.

Detailed ways

The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to be used as limitations of the present application. As used in the specification of this application and the appended claims, the singular expressions "a," "an," "the," "above," "the," and "the" are intended to also Plural expressions are included unless the context clearly dictates otherwise. It will also be understood that, as used in this application, the term "and/or" refers to and includes any and all possible combinations of one or more of the listed items.

Hereinafter, the terms "first" and "second" are only used for descriptive purposes, and should not be construed as implying or implying relative importance or implying the number of indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, unless otherwise specified, the "multiple" The meaning is two or more.

Currently, when listening to paid audio resources through a speaker, a user can complete the payment through an electronic device installed with a speaker application (application, APP) associated with the speaker, so as to listen to paid audio resources on the speaker. Specifically, a speaker APP is installed on an electronic device, such as a mobile phone. In response to a user operation for pairing the speaker with the above-mentioned mobile phone, the speaker may complete the pairing with the above-mentioned mobile phone. In this way, the speaker can establish a communication connection with the above-mentioned mobile phone. Further, in response to a user operation of logging into the first account of the speaker APP on the above-mentioned mobile phone, the speaker APP can log in to the above-mentioned first account. When the login is successful, the first account of the above speaker APP can be bound to the speaker. The first account of the speaker APP may be associated with the payment account of the user (such as a Huawei wallet payment account, an Alipay payment account, and a WeChat payment account). In this way, in the case where the account bound to the speaker is the above-mentioned first account, the payment account associated with the above-mentioned first account can perform payment in response to a user operation for paying for the paid audio resources.

When the playing audio resources need to be paid, the speaker can prompt the user to make payment on the above-mentioned speaker APP by means of voice broadcast. That is to say, the user needs to open the speaker APP, determine the audio resources that need to be paid, and enter the payment password of the payment account and a series of operations. Then, the electronic device installed with the speaker APP can complete the payment and send a payment success message to the speaker. In this way, the speakers can play paid audio sources.

It can be seen that the above operation of completing the payment process when using the speaker to listen to the paid audio resources is cumbersome. Especially when the user is inconvenient to use the electronic device with the speaker APP installed, for example, when both hands are occupied, the user cannot complete the above payment process. As a result, the user cannot listen to the paid audio resources in time.

In addition, the speaker can also complete the identity verification during the payment process by means of biometric information verification. The speaker can call the payment account to pay for the paid resources after the biometric information verification is successful. This improves the convenience of the payment process. However, the speakers are often only equipped with a voice collection device for collecting voiceprint information. That is, the speaker is generally not equipped with a device for collecting other types of biological information, such as a face collecting device, a fingerprint collecting device, a bone voiceprint collecting device, and so on. Then, the speaker can only complete the identity verification during the payment process through voiceprint verification. The above single biometric information verification method is not conducive to ensuring the security of identity verification.

The present application provides a biological information verification method. The biometric information verification method can be used for a speaker without a face acquisition device (such as a camera) to complete the identity verification in the payment process. Specifically, the speaker can realize payment by combining voiceprint verification and face verification. The speaker can extract voiceprint information from the user's voice input and store it as reference voiceprint information. Further, the speaker can call a nearby device configured with a face collection device, such as a mobile phone, tablet, laptop, TV, surveillance camera, etc., to collect the user's face image. When the face image is obtained, the above-mentioned device configured with the face acquisition device can send the face image to the speaker. The speaker can extract face information from the above face image and store it as reference face information, and bind the reference face information with the reference voiceprint information. When the playing audio resources need to be paid, the speaker can prompt the user to enter the voiceprint information and face information by means of voice broadcast. Furthermore, the speaker can compare the voiceprint information and face information obtained by the above input with the stored reference voiceprint information and reference face information, respectively. If the above-mentioned voiceprint information and face information match, the speaker can call the payment account for payment. Wherein, when the face information is obtained in the above payment process, the speaker can still call a device configured with a face collecting device nearby to collect the face image.

In this way, users can complete the payment only by speaking the preset verification words and using their own face information, and then listen to paid audio resources. This greatly simplifies the payment process when listening to premium audio sources with speakers. Speakers without a face capture device can also pay for paid audio resources by combining voiceprint verification and face verification. Moreover, the combination of voiceprint verification and face verification can improve the security in the payment process.

FIG. 1 exemplarily shows a schematic structural diagram of a sound box 100 provided by an embodiment of the present application.

The embodiments of the present application will be described in detail below by taking the sound box 100 as an example. It should be understood that the loudspeaker 100 shown in FIG. 1 is only an example, and the loudspeaker 100 may have more or fewer components than those shown in FIG. 1 , may combine two or more components, or may have different component configuration. The various components shown in FIG. 1 may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and/or application specific integrated circuits.

The speaker 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 2, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a sensor module 180, a key 190, a motor 191, an indicator 192, and the like.

It can be understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the sound box 100 . In other embodiments of the present application, the sound box 100 may include more or less components than shown, or combine some components, or separate some components, or arrange different components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), controller, memory, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural-network processing unit (NPU) Wait. Wherein, different processing units may be independent devices, or may be integrated in one or more processors.

In some embodiments, the processor 110 may include a voice wake-up module and a voice command recognition module. Among them, the voice wake-up module and the voice command recognition module can be integrated in different processor chips and executed by different chips. For example, the voice wake-up module can be integrated in a coprocessor or DSP chip with lower power consumption, and the voice command recognition module can be integrated in an AP or NPU or other chips. In this way, after the voice wakeup module recognizes the preset voice wakeup word, the chip where the voice command recognition module is located can trigger the voice command recognition function, thereby saving the power consumption of the electronic device. Alternatively, the voice wake-up module and the voice command recognition module can be integrated in the same processor chip, and the same chip performs related functions. For example, both the voice wake-up module and the voice command recognition module can be integrated in the AP chip or the NPU or other chips.

The processor 110 may further include a voice instruction execution module, that is, after recognizing the voice instruction, execute the operation corresponding to the voice instruction.

In some embodiments, the processor 110 may also include a security module. The security module may be a module integrated in the AP chip. Alternatively, the security module can also be integrated on a separate security chip. The security module can be used to store reference voiceprint information and reference face information, and complete local verification in the payment process. Exemplarily, during the payment process, the security module may compare the voiceprint information and face information to be verified with the reference voiceprint information and the reference face information, respectively. If the voiceprint information to be verified matches the reference voiceprint information, and the face information to be verified matches the reference face information, the local verification is passed. The speaker 200 can call the payment account for payment.

A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in processor 110 is cache memory. This memory may hold instructions or data that have just been used or recycled by the processor 110 . If the processor 110 needs to use the instruction or data again, it can be called directly from the memory. Repeated accesses are avoided and the latency of the processor 110 is reduced, thereby increasing the efficiency of the system.

The charging management module 140 is used to receive charging input from the charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from the wired charger through the USB interface 130 . In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through the wireless charging coil of the speaker 100 . While the charging management module 140 charges the battery 142 , it can also supply power to the speaker through the power management module 141 .

The power management module 141 is used for connecting the battery 142 , the charging management module 140 and the processor 110 . The power management module 141 receives input from the battery 142 and/or the charging management module 140 and supplies power to the processor 110 , the internal memory 121 , the external memory, the display screen 194 , the camera 193 , and the wireless communication module 160 . The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle times, battery health status (leakage, impedance). In some other embodiments, the power management module 141 may also be provided in the processor 110 . In other embodiments, the power management module 141 and the charging management module 140 may also be provided in the same device.

The wireless communication function of the speaker 100 can be realized by the antenna 2, the wireless communication module 160, the modem processor, the baseband processor, and the like.

The antenna 2 is used to transmit and receive electromagnetic wave signals. Each antenna in loudspeaker 100 may be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization.

The wireless communication module 160 can provide wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), and global navigation satellite systems applied on the speaker 100 . (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field communication technology (near field communication, NFC), infrared technology (infrared, IR) and other wireless communication solutions. The wireless communication module 160 may be one or more devices integrating at least one communication processing module.

The NPU is a neural-network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transfer mode between neurons in the human brain, it can quickly process the input information, and can continuously learn by itself. Applications such as intelligent cognition of the speaker 100 can be implemented through the NPU, such as face verification, voiceprint verification, text understanding, speech synthesis, and the like.

The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the speaker 100 . The external memory card communicates with the processor 110 through the external memory interface 120 to realize the data storage function. For example to save files like music, video etc in external memory card.

Internal memory 121 may be used to store computer executable program code, which includes instructions. The processor 110 executes various functional applications and data processing of the speaker 100 by executing the instructions stored in the internal memory 121 . The internal memory 121 may include a storage program area and a storage data area. The storage program area may store an operating system, an application program required for at least one function (such as a sound playback function), and the like. The storage data area can store data (such as audio data) created during the use of the speaker 100 and the like. In addition, the internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (UFS), and the like.

The speaker 100 may implement audio functions through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an application processor, and the like. Such as music playback, recording, etc.

The audio module 170 is used for converting digital audio information into analog audio signal output, and also for converting analog audio input into digital audio signal. Audio module 170 may also be used to encode and decode audio signals. In some embodiments, the audio module 170 may be provided in the processor 110 , or some functional modules of the audio module 170 may be provided in the processor 110 .

Speaker 170A, also referred to as a "speaker", is used to convert audio electrical signals into sound signals. The speaker 100 can listen to music or answer calls through the speaker 170A.

The receiver 170B, also referred to as "earpiece", is used to convert audio electrical signals into sound signals.

The microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. Exemplarily, the speaker 100 may collect the user's voice input through the microphone 170C, and extract voiceprint information from the voice input. When making a call or sending a voice message, the user can make a sound by approaching the microphone 170C through a human mouth, and input the sound signal into the microphone 170C. The speaker 100 may be provided with at least one microphone 170C. In other embodiments, the sound box 100 may be provided with two microphones 170C, which in addition to collecting sound signals, may also implement a noise reduction function. In other embodiments, three, four or more microphones 170C may also be provided in the speaker 100 to collect sound signals, reduce noise, identify sound sources, and implement directional recording functions.

The keys 190 include a power-on key, a volume key, and the like. Keys 190 may be mechanical keys. It can also be a touch key. The speaker 100 can receive key inputs and generate key signal inputs related to user settings and function control of the speaker 100 .

Motor 191 can generate vibrating cues. The motor 191 can be used for vibrating alerts for incoming calls, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as audio playback) may correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

The indicator 192 can be an indicator light, which can be used to indicate the charging state, the change of the power, and can also be used to indicate a message, a missed call, a notification, and the like.

In this application, the speaker 100 needs to obtain the user's reference voiceprint information and reference face information before making payment through voiceprint verification and face verification. In a possible implementation manner, the speaker 100 may receive a user's voice input and a face image after receiving an instruction for instructing to create biometric information. Further, the speaker 100 can extract voiceprint information from the above voice input, and store the voiceprint information in the security module as reference voiceprint information. The speaker can extract face information from the above face image, and store the face information in the security module as reference face information.

The above-mentioned biological information includes voiceprint information and face information. The above-mentioned instruction for instructing to create new biological information may come from the electronic device 200 . The electronic device 200 may be a device in which a speaker APP associated with the speaker 100 is installed. For example, mobile phones, tablets, etc. For the schematic structural diagram of the electronic device 200 , reference may be made to the structural schematic diagram of the sound box 100 shown in FIG. 1 . Wherein, it is not limited to the components shown in FIG. 1 , and the electronic device 200 may also include more or less components. For example, the electronic device 200 may also include components such as a display screen, a camera, and the like.

The above speaker APP may be an application named "speaker". The embodiments of the present application do not specifically limit the above speaker APP.

This embodiment of the present application does not specifically limit the type of the above electronic device 200 .

In this application, the speaker 100 may be a device without a face acquisition device (eg, a camera). When face verification needs to be performed, the speaker 100 can call the face collection device 300 that is configured with a face collection device nearby. When the face image is collected, the face collection device 300 can send the face image to the speaker 300 . Further, the speaker 100 can extract face information from the face image.

In a possible implementation manner, after the face collection device 300 collects a face image, face information can be extracted from the face image, and the face information can be sent to the speaker 100 . In this way, the speaker 100 can directly store the received face information in the security module as reference face information. This embodiment of the present application does not limit the device for extracting face information from a face image.

For the schematic structural diagram of the face collecting device 300, reference may be made to the structural schematic diagram of the sound box 100 shown in FIG. 1 . Wherein, the face collection device 300 may include a face collection device, such as a camera. Not limited to the components shown in FIG. 1 , the face acquisition device 300 may also include more or less components.

The face collection device 300 may be, for example, a mobile phone, a tablet, a notebook computer, a TV, a surveillance camera, and the like. In some embodiments, the face collection device 300 and the electronic device 200 may be the same device.

This embodiment of the present application does not limit the type of the face collection device 300 .

The following describes a schematic diagram of a scene in which the speaker 100 according to the embodiment of the present application obtains the reference voiceprint information and the reference face information.

In a possible implementation manner, the speaker 100 may obtain the reference voiceprint information and the reference face information for payment after receiving the instruction from the electronic device 200 to obtain the reference voiceprint information and the reference face information operation. The electronic device 200 may send the above-mentioned instruction for obtaining the reference voiceprint information and the reference face information after the payment password of the payment account is successfully verified. In this way, the electronic device 200 can perform identity authentication on the user who has entered the reference voiceprint information and the reference face information to ensure that the user is the user corresponding to the above payment account.

FIGS. 2A to 2E exemplarily show a user interface for the electronic device 200 to perform identity authentication of a payment account.

As shown in FIG. 2A , the electronic device 200 may display a user interface 210 of the speaker APP. The user interface 210 may include a login account 211 , speaker status 212 , and setting options 213 . in:

The login account 211 can be used to display the account name currently logged into the speaker APP. Exemplarily, the account name currently logged into the speaker APP may be "Li Ming". In response to the user's operation on the login account 211, the electronic device 200 may display contents such as switching account options, logging out options, and the like. The above switch account option can be used to switch the account currently logged into the speaker APP. The above logout options can be used to logout of the account currently logged into the speaker APP.

The speaker status 212 may be used to display status information of the speakers paired with the electronic device 200 . Specifically, the electronic device 200 is successfully paired with the speaker 100 . Speaker status 212 may include Wi-Fi connection status 212A, Bluetooth status 212B, power level 212C, and speaker settings 212D. The Wi-Fi connection status 212A can indicate whether the speaker is connected to the network, and the name of the Wi-Fi when connected to the Wi-Fi. For example, the speaker 100 is connected to a Wi-Fi named "liming". The Bluetooth status 212B can indicate whether the Bluetooth of the speaker is turned on. For example, when the bluetooth of the speaker 100 is turned on, the bluetooth status 212B may display the prompt "turned on". The power 212C can indicate the current remaining power of the speaker. For example, the current remaining power of the speaker 100 is 80% of the total power. Speaker Setup 212D can be used to set up information about the speakers. For example, setting the speaker name, setting the speaker can use biometric information for payment, etc. In response to a user operation acting on the speaker setup 212D, the electronic device 200 may display a user interface 220 as shown in FIG. 2B.

It should be noted that the electronic device 200 can be paired with the speaker 100 through a Bluetooth connection. Specifically, when connecting for the first time, the speaker 100 turns on Bluetooth. The electronic device 200 starts the speaker APP, turns on the Bluetooth, and scans for nearby Bluetooth. When the Bluetooth of the speaker 100 is scanned, the electronic device 200 can be paired and connected with the speaker 100 . When the Bluetooth connection is subsequently disconnected and paired again, if both the electronic device 200 and the speaker 100 have Bluetooth enabled, the electronic device 200 can be automatically paired when approaching the speaker 100 . For the specific process of establishing the Bluetooth connection above, reference may be made to the implementation manner of the prior art, which will not be repeated here.

The embodiments of the present application do not specifically limit the way of pairing the electronic device 200 with the speaker 100. In addition to pairing through Bluetooth, other wired (such as data cables, optical fibers, etc.) or wireless (such as NFC, ZigBee, Wi-Fi, etc.) etc.) are paired.

In addition, when the electronic device 200 is successfully paired with the speaker 100 , the electronic device 200 can configure the network for the speaker 100 . Exemplarily, after the electronic device 200 is connected to a network (eg WiFi), the name and password of the WiFi may be sent to the speaker 100 . After the speaker 100 receives the name and password of the WiFi, the speaker 100 can connect to the WiFi, and then access the network. The speaker 100 connected to the network can search for audio resources on the network, obtain the audio resources from a server that provides the audio resources, such as a music server, and play them.

When the speaker 100 is connected to the network, the speaker 100 can be bound with the account logged in the speaker APP in the electronic device 200 . For example, an account named "Li Ming" is logged in the speaker APP, and the speaker 100 can be bound with the account "Li Ming".

The above-mentioned binding between the speaker and the account may mean that the speaker can obtain the data associated with the account, such as the audio resources collected by the account, the purchased audio resources, the historical audio resources listened to, and so on. For example, the speaker 100 is bound to an account whose account name is "Li Ming". For the paid audio resources already purchased by the account "Li Ming", the speaker 100 can play them directly without the need for the account "Li Ming" to pay again. Moreover, when the paid audio resource needs to be paid, the speaker 100 can call the payment account associated with the account "Li Ming" to pay.

The embodiments of the present application do not limit the way in which the electronic device 200 is paired with the speaker 100, the way in which the electronic device 200 configures the network for the speaker 100, and the way in which the speaker 100 establishes a binding relationship with the account logged in the speaker APP. The above specific implementation methods can be Refer to the implementation in the prior art.

The setting options 213 may include setting options such as music collection, purchased programs, to-be-purchased programs, music preferences, audio subscription, APP message push, questions and suggestions, and the like. Exemplarily, the above-mentioned music collection may be used to display the music collected by the currently logged-in account. The above-mentioned purchased programs and programs to be purchased can be respectively used to display the audio resources that have been purchased by the currently logged in account and the audio resources that are added to the shopping cart and are waiting to be purchased. The above music preferences can be used to set the preferred music genres of the currently logged-in account, such as rock music, classical music, light music, etc. The above audio subscription can be used to display the audio programs subscribed by the currently logged in account. The above APP message push can be used to enable or disable the function of the speaker APP to push messages to the electronic device.

In response to a user operation acting on the speaker setup 212D described above, the electronic device 200 may display a user interface 220 as shown in FIG. 2B . The user interface 220 may include the following setting options: speaker name 221 , Bluetooth 222 , wakeup response 223 , alarm tone 224 , home device authorization management 225 , speaker information 226 , unbinding 227 , and biometric payment 228 . in,

The speaker name 221 can be used to set the name of the speaker 100 . For example, the speaker 100 is currently named "Li Ming's Speaker". In response to a user operation acting on the speaker name 221 and inputting a new name, the electronic device 200 may modify the name of the speaker 100 .

The Bluetooth 222 can be used to turn on or turn off the Bluetooth of the speaker 100 .

The wake-up response 223 can be used to turn on or turn off the function of the voice wake-up speaker 100 . For example, when the function of waking up the speaker 100 by voice is enabled, the speaker 100 can wake up the processor after receiving a preset wake-up word (such as Xiaoyi Xiaoyi), and then execute the voice command spoken by the user after the wake-up word.

The alarm tone 224 may be used to set the alarm of the speaker 100 .

The home device authorization management 225 can be used to set the home devices that the speaker 100 can control.

The speaker information 226 may be used to display configuration information such as the model of the speaker 100, the version number, the speaker identification, and the size of the storage space.

Unbinding 227 can be used to unbind the account currently logged in the speaker APP and the speaker 100 .

The biometric payment 228 may be used to instruct the speaker 100 to create reference biometric information for paying for the paid resource. The reference biological information may include reference voiceprint information and reference face information. The reference biometric information can be used for the speaker 100 to verify the user's identity in the process of paying for the paid resources. That is to say, in the process of paying for the paid resources, the speaker 100 collects the user's voiceprint information and face information, and compares the voiceprint information and the face information with the above-mentioned reference voiceprint information and reference face information respectively. . If both match, the user authentication is successful.

Illustratively, in response to a user operation acting on the biometric payment 228, the electronic device 200 may display a user interface 230 as shown in FIG. 2C. The user interface 230 may include a biometrics function 231 and a new biometrics option 232 . in:

The biological information function 231 may be used for a function of prompting the user's biological information. For example, the biometric information function 231 may include the prompt "biological information is used to pay for paid resources". A switch 231A may also be included in the biological information function 231 . The switch 231A can be used to turn on or off the function of paying for resources using biological information.

Not limited to the above setting options, the user interface 220 may also contain more or less setting options.

As shown in FIG. 2C , no biological information for payment is currently stored in the speaker 100 . When no biometric information for payment is stored in the speaker 100, the switch 231A is off by default.

The New Biometrics option 232 may be used to begin creating reference biometrics for payment.

In a possible implementation manner, the process of establishing reference biometric information for payment may include: performing identity authentication on the payment account, and entering reference voiceprint information and reference face information after the identity authentication is passed.

Wherein, in response to a user operation acting on the newly created biological information option 232, the electronic device 200 may display a user interface 240 as shown in FIG. 2D. User interface 240 may include a password input box 241 and a keyboard 243 .

The password input box 241 can be used to input the payment password of the payment account associated with the account currently logged in the speaker APP.

The keyboard 243 can be used to enter the payment password in the password input box 241 .

When the password in the password input box 241 matches the payment password of the payment account stored in the electronic device 200, the identity authentication is successful. As shown in FIG. 2E , when the identity authentication is successful, the electronic device 200 may display a prompt box 242 on the user interface 240 . The prompt box 242 may be used to prompt the user to perform successful identity authentication and to perform operations for entering voiceprint information and face information. The prompt box 242 may contain the prompt "authentication is successful! Please continue to complete the voice entry and face entry". The specific content of the prompt word in the prompt box 242 is not limited in this embodiment of the present application.

That is to say, the user can instruct the speaker to create biological information through the speaker APP on the electronic device 200 . Wherein, after the user completes the identity authentication by entering the payment password of the payment account on the speaker APP, the user can further enter the reference voiceprint information and reference face information for payment on the speaker 100 .

The user interface shown in FIG. 2A to FIG. 2E may further include more or less content, which is not limited in this embodiment of the present application.

When the above-mentioned identity authentication of the payment account is completed, the electronic device 200 may send an instruction to the speaker 100 for instructing to obtain the reference voiceprint information and the reference face information. Further, the speaker 100 may prompt the user to input voiceprint information and face information.

FIGS. 2F to 2I exemplarily show schematic diagrams of scenarios in which the first user enters reference voiceprint information and reference face information.

As shown in FIG. 2F , the speaker 100 may prompt the first user to input voiceprint information by means of voice broadcast. Exemplarily, the speaker 100 may voice broadcast "Please follow me and say the following verification word: Xiaoyi Xiaoyi". The first user can say "Xiaoyi Xiaoyi" after hearing the prompt from the speaker 100 . Furthermore, the speaker 100 may collect the voice input of the first user saying the above verification word, and extract voiceprint information from the voice input.

When the voiceprint information of the first user is obtained, the speaker 100 can compare the voiceprint information with the reference voiceprint information stored in the security module. If the voiceprint information matches one of the stored reference voiceprint information, the speaker 100 may prompt the first user to have entered the voiceprint information by means of voice broadcast.

If the voiceprint information does not match the stored reference voiceprint information, the speaker 100 may store the voiceprint information in the security module as the reference voiceprint information of the first user.

Further, the speaker 100 may prompt the first user to input face information by means of voice broadcast.

Wherein, since the speaker 100 is not equipped with a face collection device, and cannot collect a face image, the speaker 100 can search for and call a nearby available face collection device to collect a face image. Specifically, the speaker 100 can send a broadcast signal to inquire whether a nearby device is equipped with a face acquisition device. For example, the devices near the speaker 100 include electronic devices 200, face acquisition devices (eg, televisions, mobile phones, tablets, and laptops) 300, and the like. When receiving the broadcast signal from the speaker 100 , a device equipped with a face acquisition device near the speaker 100 can send a response message to the speaker 100 . The response message may include the configuration of the face acquisition device (eg, a 2D camera, a 3D camera, an infrared camera).

If receiving response messages sent by multiple devices configured with face collection devices, the speaker 100 can select one of the multiple devices to collect people according to the sequence of the received response messages, the configuration of the face collection device, and other factors. face image. For example, 3D face authentication is more reliable than 2D face authentication. The speaker 100 can choose a better equipped device, such as a device equipped with a 3D camera.

The order in which the response messages are received may reflect the communication delay between the speaker 100 and the device configured with the face acquisition device and the response speed of the device. The speaker 100 can select the device corresponding to the response message received first. This embodiment of the present application does not limit the manner in which the speaker 100 selects a device configured with a face collecting device nearby.

The device near the sound box 100 may be a device in the same local area network as the sound box, or may be a device within a preset distance range of the sound box 100 . The above-mentioned preset distance may be determined by the farthest communication distance that can be achieved by the communication method in which the speaker 100 sends the above-mentioned broadcast signal. The embodiments of the present application do not limit the electronic devices near the sound box 100 described above.

The speaker 100 can send the above-mentioned broadcast signal through a short-distance communication channel such as a near-field communication channel, a Bluetooth communication channel, and a WLAN direct communication channel. Reference may be made to the prior art for an implementation manner of the speaker 100 sending the above broadcast signal, which is not limited in this embodiment of the present application.

Exemplarily, the speaker 100 may determine to call the face collection device 300, such as a TV, to collect a face image according to the aforementioned method of finding and selecting a nearby device configured with a face collection device. The face collection device 300 is configured with a face collection device 301 . In subsequent embodiments of the present application, the face collection device 300 is used as a television for specific description. The face acquisition device 301 configured on the TV can be a 3D camera. Not limited to a TV, the face collection device 300 may also be other electronic devices configured with a face collection state, such as a mobile phone, a tablet, a notebook computer, and the like.

The speaker 100 may send an instruction to the face collection device 300 for instructing to turn on the face collection device 301, such as a camera, to collect face images. And, as shown in FIG. 2G , the speaker 100 can voice broadcast "Voiceprint recording is successful. Please complete face recording on Li Ming's TV". "Li Ming's TV" in the voice broadcast may be the name of the face collecting device 300 .

When receiving an instruction from the speaker 100 for instructing to turn on the face collection device 301 to collect face images, the face collection device 300 can turn on the camera and send a message to the speaker 100 indicating that the camera is turned on.

As shown in FIG. 2H , when receiving the above-mentioned message indicating that the camera is turned on, the speaker 100 can voice broadcast "Please aim your face at the camera of Li Ming's TV and blink." In this way, the speaker 100 can prompt the first user to perform face entry on the face collection device 300 (ie, Li Ming's TV).

In a possible implementation manner, the face acquisition device 300 includes a display screen. When collecting a face image, the face collecting device 300 can also light up the display screen to display the image collected by the camera and the text prompt "Aim at the camera and blink." In this way, the first user can adjust his position according to the image collected by the camera displayed on the display screen, so that his face is aligned with the camera.

In another possible implementation manner, the face acquisition device 300 does not include a display screen. Or the face collecting device 300 includes a display screen, but when collecting a face image, the display screen is not lit. Then, during the process of collecting the face image by the face collecting device 300, the speaker 100 may prompt the first user to point the face at the face collecting device by means of voice broadcast. For example, a voice announces "The face is not aligned with the camera, please move a little to the left". In this way, when the face collection device 300 does not include a display screen or includes a display screen but the display screen is not lit, the first user can align his face with the face collection device according to the voice prompt of the speaker.

When the face image is obtained, the face acquisition device 300 can encrypt the face image and send it to the speaker 100 . The face acquisition device 300 can encrypt the face image according to an encryption method such as a symmetric encryption algorithm and an asymmetric encryption algorithm. For the specific implementation process of the above encryption, reference may be made to the prior art, which will not be repeated here.

After receiving the encrypted face image, the speaker 100 can decrypt it according to the encryption method negotiated with the face acquisition device 300 to obtain the face image of the first user. Further, the speaker 100 can extract face information from the face image and store it in the security module as reference face information. The speaker 100 may bind the above-mentioned reference voiceprint information and the above-mentioned reference face information.

That is to say, in the process of using biological information for payment, the speaker 100 needs to verify whether the voiceprint information and the face information match with the bound reference voiceprint information and the reference face information respectively. When both the voiceprint information and the face information are verified, the speaker 100 can call the payment account associated with the account bound to the speaker 100 for payment. Compared with separate voiceprint verification and separate face verification, the combination of voiceprint verification and face verification requires that the voiceprint information and face information entered by the user are correct, which can improve the security of using biometric information for payment .

As shown in FIG. 2I , when the reference voiceprint information and the reference face information are successfully bound, the speaker 100 can voice broadcast "The face entry is successful, you have enabled biometric payment". In this way, the speaker 100 can prompt the first user that the biometric payment has been successfully activated. When the paid audio resources need to be paid later, the first user can complete the payment by entering his own voiceprint information and face information.

In addition, the speaker 100 may also send a message to the electronic device 200 for indicating that the biometric information is successfully established. As shown in FIG. 2J , when receiving the above-mentioned message indicating that the biometric information is established successfully, the electronic device 200 may display a biometric information list 233 on the user interface 230 . The biological information list 233 may be used to display reference biological information that has been established on the speaker 100 .

Exemplarily, after the operations shown in FIG. 2A to FIG. 2I above, the biological information of the first user has been established on the speaker 100 . The biometric information list 233 of the electronic device 200 may include "User 1" 233A indicating biometric information of the first user. In response to an operation, such as a touch operation, acting on the "user 1" 233A, the electronic device 200 may display an option to change the name of the biometric information, an option to delete the biometric information, and the like. That is, the user can modify the name of the biological information, for example, modify the name "User 1" to "Li Ming". And, the user can delete the biological information. In response to a user operation to delete the biometric information, the electronic device 200 may transmit an instruction to delete the biometric information named "User 1" to the speaker 100 . In response to the above instruction, the speaker 100 can delete the biometric information (including the reference voiceprint information and the reference face information) named "User 1" stored in the security module.

This embodiment of the present application does not limit the specific content of the voice broadcast by the speaker 100 .

For the above-mentioned implementation method of extracting voiceprint information in voice input and extracting face information in face image, reference may be made to the prior art, which is not specifically limited in the embodiment of the present application.

This embodiment of the present application does not limit the above process of entering user voiceprint information. The speaker 100 may prompt the user to speak the preset verification word for many times. The above verification word is not limited to "Xiaoyi Xiaoyi", but can also be other words or sentences. Alternatively, the speaker 100 may not contain a preset verification word. That is, the user can speak any word or sentence. The speaker 100 can extract the user's voiceprint information from the user's voice input.

In some embodiments, the speaker 100 is configured with a face capture device, such as a camera. When the above-mentioned voiceprint information is successfully entered, and the face information is further entered, the speaker 100 can turn on its own face collection device to collect the face image and extract the face information in the face image. Exemplarily, the sound box 100 is configured with a camera. When entering face information, the speaker 100 can voice broadcast "Please aim your face at my camera and wink". In addition, the speaker 100 can also prompt the user to adjust the position through voice broadcast, so that the user's face is aligned with the camera of the speaker 100 . For example, when the user's face is inclined to the right of the area where the camera captures the image, the speaker 100 can voice broadcast "The face is not aligned with the camera, please move a little to the left".

When the speaker 100 collects a face image through its own camera, the speaker 100 can extract the face information from the face image, and store the face information in the security module as reference face information. In this way, the speaker equipped with the voice collecting device and the face collecting device can independently complete the input of the reference biometric information, and use the reference biometric information to verify the user's identity in the process of paying for the paid resources.

This embodiment of the present application does not limit the storage space of the above-mentioned reference biological information. In addition to the above-mentioned security module of the speaker 100, the reference biological information may also be stored in the cloud storage space. The above-mentioned cloud storage space may be a storage space that can be accessed by the speaker 100 and the face collecting device 300 .

In some embodiments, when the speaker 100 obtains the voiceprint information of the first user, the speaker 100 may store the voiceprint information in the above-mentioned cloud storage space. Further, the speaker 100 can find and call the face collecting device 300 to collect the face image. The face collection device 300 may perform feature extraction on the collected face image of the first user to obtain face information of the first user.

Further, the face collection device 300 may store the face information of the first user in a cloud storage space. Wherein, in the cloud storage space, the voiceprint information and face information of the first user may be associated. Alternatively, the face collection device 300 may send the collected face image of the first user to the speaker 100 . The speaker 100 can perform feature extraction on the face image to obtain the face information of the first user. Further, the speaker 100 may store the face information of the first user and the voiceprint information of the first user in a cloud storage space in association with each other.

Optionally, the above-mentioned reference biological information includes reference voiceprint information and reference face information. One of the reference voiceprint information and the reference face information is stored in the above cloud storage space. Exemplarily, the speaker 100 may collect the voice input of the first user and obtain voiceprint information. The speaker 100 can store the voiceprint information in a local security module. Further, the speaker 100 can call a nearby face collection device to collect face images and obtain face information. The face collection device can extract face information from the face image, and store the face information in the cloud storage space.

It can be seen from the schematic diagrams of the scenarios shown in FIGS. 2A to 2I that a speaker without a face collection device can record face information by calling a nearby face collection device. In this way, the speaker can record the user's reference voiceprint information and reference face information, and verify the user's identity in the process of paying for the paid resources, thereby simplifying the user's payment operation when listening to the paid resources on the speaker.

The following describes a method for inputting reference voiceprint information and reference face information in a speaker provided by an embodiment of the present application with reference to the schematic diagram of the above scenario.

FIG. 3 exemplarily shows a flowchart of a method for recording reference voiceprint information and reference face information in a speaker. The method may include steps S101-S109. in:

S101. The electronic device 200 receives a user operation for opening the speaker APP, pairing the electronic device 200 with the speaker 100, and establishing a binding relationship between the first account of the speaker APP and the speaker 100.

In some embodiments, the sound box 100 and the electronic device 200 may be paired first when connected for the first time. Exemplarily, the Bluetooth of the electronic device 200 and the speaker 100 are both turned on. The electronic device 200 starts the speaker APP, and after receiving a user operation for searching for nearby devices, scans for nearby Bluetooth. When the Bluetooth of the speaker 100 is scanned, the electronic device 200 can be paired and connected with the speaker 100 . The embodiment of the present application does not limit the pairing manner of the electronic device 200 and the sound box 100 .

When the electronic device 200 is successfully paired with the speaker 100 , the electronic device 200 can configure the network for the speaker 100 . The speaker 100 can be connected to the network. The manner in which the electronic device 200 configures the network for the speaker 100 may refer to the foregoing embodiments, which will not be repeated here.

Further, in response to a user operation of logging in the first account on the speaker APP, the electronic device 200 may log in the first account on the speaker APP. The first account may be the account named "Li Ming" as shown in FIG. 2A. For the user interface where the account logged in the speaker APP is the first account, reference may be made to the user interface 210 shown in FIG. 2A above.

When a first account is logged into the speaker APP, the speaker 100 can establish a binding relationship with the first account. For the implementation process of establishing a binding relationship between the speaker 100 and the first account logged in the speaker APP, reference may be made to the prior art, which is not limited in this embodiment of the present application.

S102 , the electronic device 200 is successfully paired with the speaker 100 , and the first account of the speaker APP establishes a binding relationship with the speaker 100 .

S103 , the electronic device 200 receives a user operation for requesting establishment of the first biometric information for payment, and sends a request for establishing the first biometric information for payment to the speaker 100 .

The above-mentioned user operation for requesting the establishment of the first biometric information for payment may be the user operation acting on the above-mentioned new biometric information option 232 shown in FIG. 2C . The electronic device 200 may send a request for establishing the first biometric information for payment to the speaker 100 after performing identity authentication on the payment account associated with the first account. For the implementation manner of performing identity authentication on the payment account above, reference may be made to the embodiments shown in the foregoing FIG. 2C to FIG. 2E , which will not be repeated here.

S104, the speaker 100 records the first voiceprint information of the first user.

The above-mentioned first voiceprint information may be the aforementioned reference voiceprint information. When the first voiceprint information is obtained, the speaker 100 may store the first voiceprint information in the security module. The first voiceprint information can be used for voiceprint verification in the subsequent payment process.

S105 , the speaker 100 finds and calls the available face collection device 300 .

S106. The face collection device 300 records the first face information of the first user.

The first face information may be the aforementioned reference face information. The face collection device 300 may use a face collection device to collect a face image, and extract the first face information from the face image.

Alternatively, the face collection device 300 may send the collected face image to the speaker 100, and the speaker 100 extracts the face image to obtain the first face information.

S107. The face collection device 300 sends the encrypted first face information to the speaker.

S108, the speaker 100 decrypts the encrypted first face information, binds and stores the first voiceprint information and the first face information.

When the first face information is obtained, the speaker 100 may store the first face information in the security module. The first face information can be used for face verification in the subsequent payment process.

In addition, the speaker 100 may bind the first voiceprint information and the first face information as the first biological information of the first user.

S109 , the speaker 100 sends a message to the electronic device 200 for indicating that the first biological information is successfully established.

For the specific implementation process of the foregoing steps S104 to S109, reference may be made to the foregoing description of the embodiments shown in FIG. 2F to FIG. 2J , which will not be repeated here.

The following specifically introduces a schematic diagram of a scenario in which the speaker 100 according to the embodiment of the present application pays for paid audio resources by combining voiceprint verification and face verification.

When the reference voiceprint information and reference face information of the first user are stored in the security module of the speaker 100, the speaker 100 can combine voiceprint verification and face verification to verify the identity of the user requesting payment.

As shown in FIG. 4A , the first user wakes up the speaker 100 through a preset wake-up word and issues a voice instruction to play music A. Specifically, the first user says "Xiaoyi Xiaoyi, I want to listen to music A" near the speaker 100 .

When a preset wake-up word is detected, the speaker 100 can wake up the application processor to recognize and execute the received voice command. In response to the voice instruction of "listen to music A" by the first user, the speaker 100 can acquire and play the resources of music A from the music server. In addition, the speaker 100 can also voice broadcast "OK, play music A for you" in response to the voice instruction of the first user.

If the music A is a paid audio resource, and the first account (the account bound to the speaker 100 ) has not purchased the music A, the music server may send a message indicating that the music A is paid music to the speaker 100 . As shown in FIG. 4B , when receiving a message indicating that music A is paid music, the speaker 100 can voice broadcast “you can listen to the complete music A after payment, whether to pay or not” to prompt the first user to pay to listen to music A .

When hearing the prompt of the speaker 100, the first user can say "please help me complete the payment" near the speaker 100. When the voice command "please help me complete the payment" is detected, the speaker 100 can start to perform voiceprint verification and face verification. Specifically, as shown in FIG. 4C , the speaker 100 can voice broadcast "OK, start voiceprint verification. Please follow me and say the following verification word: Xiaoyi Xiaoyi" to prompt the first user to enter the voiceprint information. The first user can say the verification word "Xiaoyi Xiaoyi" according to the prompt of the speaker 100 .

The speaker 100 can receive the voice input of the first user speaking the above verification word, and extract voiceprint information from the voice input. Further, the speaker 100 may compare the voiceprint information with the stored reference voiceprint information of the first user. If the voiceprint information matches the reference voiceprint information of the first user, the voiceprint verification is successful.

As shown in Fig. 4D, the speaker 100 can voice broadcast "Voiceprint verification is successful, face verification starts. Please aim your face at the camera of Li Ming's TV and blink." Among them, since the speaker 100 is not equipped with a face collection device, it cannot collect a face image. After the voiceprint verification is successful, the speaker 100 can find and call a nearby available face collection device to collect a face image.

The manner in which the speaker 100 finds and invokes the face acquisition device 300 (ie, Li Ming's TV) to acquire the face information may refer to the foregoing embodiments, which will not be repeated here. When the above-mentioned face information is obtained, the speaker 100 may compare the above-mentioned face information with the stored reference face information of the first user. If the above face information matches the reference face information of the first user, the face verification is successful.

When both the voiceprint verification and the face verification are successful, the speaker 100 can complete the local verification of the payment account. Further, the speaker 100 can be authenticated with the payment server of the payment account. When the verification between the speaker 100 and the aforementioned payment server is successful, the payment server may perform a deduction operation on the payment account associated with the account bound to the speaker 100 .

The above verification between the speaker 100 and the payment server can be used by the payment server to confirm that the speaker 100 is a trusted device. In this way, the payment server can complete the deduction of the payment account according to the instruction of the speaker 100 to debit the payment account. Specifically, the speaker 100 and the payment server can be verified according to fast identity online alliance (FIDO).

Wherein, the security module of the speaker 100 can generate a pair of asymmetric key pairs. The asymmetric key pair may include a private key and a public key. The security module of the speaker 100 can store the above-mentioned private key and send the above-mentioned public key to the above-mentioned payment server. The payment server may associate the payment account associated with the account bound to the speaker 100 with the above public key.

When both the voiceprint verification and the face verification are successful, the speaker 100 can use the private key to sign the verification request sent by the payment server, and send the verification request including the signature to the payment server. The payment server can then use the public key to verify the verification request containing the signature. If the verification is successful, the payment server can confirm that the speaker 100 is a trusted device.

In addition to the above-mentioned FIDO for verification, other trusted identity verification methods may also be used for verification between the speaker 100 and the payment server. For example, internet finance authentication (IFAA), Tencent user security infrastructure (TUSI) and other trusted authentication methods. For the verification between the speaker 100 and the payment server, reference may be made to the implementation manner of the prior art, which will not be repeated here.

In a possible implementation manner, when the payment server completes the deduction of the payment account, the payment server may send a message indicating that the payment is successful to the speaker 100 . The speaker 100 can send the message indicating that the payment is successful to the music server. When receiving the message indicating successful payment, the music server may send the resources of music A to the speaker 100 . In this way, the speaker 100 can play the complete music A.

In another possible implementation manner, the payment server establishes a communication connection with the music server. When the payment server completes the deduction of the payment account, the payment server may send a message to the music server to indicate that the payment account associated with the account bound to the speaker 100 is successfully debited. Further, the music server may send the resource of music A to the speaker 100 . In this way, the speaker 100 can play the complete music A. This embodiment of the present application does not limit the method for the music server to determine that the account bound to the speaker 100 has purchased the above-mentioned music A.

As shown in FIG. 4E , when receiving the message indicating successful payment sent by the payment server, the speaker 100 can voice broadcast “The face verification is successful, you have successfully purchased music A, and the complete music A will be played for you” to prompt The first user has paid successfully. Then, the speaker 100 can play the resource of the music A received from the music server.

It should be noted that the above music server may store music resources and account related information (such as paid audio resources purchased by the account, etc.). It is not limited to a music server that provides music, but can also be a server that provides other types of audio resources for the speaker 100 . The embodiments of the present application do not limit the types of the above-mentioned music servers and the included contents.

In the present application, the implementation process of waking up the application processor after the speaker 100 detects the wake-up word, and recognizing and executing the voice command may refer to the prior art, which will not be repeated here.

It can be seen from the scenarios shown in FIGS. 4A to 4E that when the first user enters the reference voiceprint information and the reference face information in the speaker, the first user can pass the voiceprint verification and face verification when paying for the paid audio resources. way to complete the payment. In this way, the first user can perform a series of cumbersome operations such as selecting a paid audio resource and inputting a payment password without opening the electronic device on which the speaker APP is installed. The above-mentioned payment method of voiceprint verification combined with face verification can simplify user operations and make the payment process more convenient.

In some embodiments, the speaker 100 may receive an instruction to instruct the speaker 100 to authenticate the user. The instruction may be, for example, "please help me complete the payment" by the first user's voice input as shown in FIG. 4B . Further, the speaker 100 may first record the user's voiceprint information, and call the face collection device 300 to obtain the user's face information. Then, the speaker 100 may compare the above-mentioned voiceprint information and the above-mentioned face information with the reference voiceprint information and reference face information of the first user, respectively. If they match, both voiceprint verification and face verification are successful.

In this method, the speaker 100 may first obtain voiceprint information and face information that need to be verified. Then, the speaker 100 can verify the above-mentioned voiceprint information and face information that need to be verified. By performing voiceprint verification and face verification at the same time, the speaker 100 can improve the efficiency of verifying the user's identity, thereby improving the above-mentioned payment efficiency by combining voiceprint verification and face verification.

In some embodiments, the reference biological information of the first user is stored in a cloud storage space. During the process of paying for the paid resources, the speaker 100 may obtain the reference biometric information of the first user from the cloud storage space to verify the user's identity.

Specifically, as shown in FIG. 4B , the first user requests the speaker 100 to pay for the paid resources. The speaker 100 may acquire the reference biological information of the first user from the above-mentioned cloud storage space. The speaker 100 can collect the voice input of the first user to obtain voiceprint information, and compare the voiceprint information with the reference voiceprint information in the reference biological information of the first user. If the voiceprint information matches the above-mentioned reference voiceprint information, the speaker 100 can call the face collecting device 300 to collect the face image of the first user and obtain the face information. The speaker 100 may compare the face information with the reference face information in the above-mentioned reference biological information of the first user. When the speaker 100 completes the above voiceprint verification and face verification, the speaker 100 can perform trusted identity verification with the payment server. For the implementation manner of the above-mentioned trusted identity verification, reference may be made to the foregoing embodiments, which will not be repeated here. If the above-mentioned trusted identity verification is successful, the payment server may debit the payment account of the first user. The payment account of the first user may be a payment account associated with the account bound to the speaker 100 .

In other embodiments, the reference biological information of the first user is stored in a cloud storage space. This cloud storage space can be accessed by the payment server. During the process of paying for the paid resources, the speaker 100 may send the obtained voiceprint information and face information of the first user to the payment server. The payment server may obtain the reference biometric information of the first user from the cloud storage space, and use the biometric information to verify the voiceprint information and face information from the speaker 100 .

Specifically, the speaker 100 can obtain the voiceprint information of the first user according to the method in the foregoing embodiment. The speaker 100 can send the voiceprint information to the payment server, and request the payment server to perform voiceprint verification. The payment server may acquire the reference biometric information of the first user from the cloud storage space, and compare the reference voiceprint information of the first user with the voiceprint information from the speaker 100 . If there is a match, the payment server may send a message to the speaker 100 to indicate that the voiceprint information is successfully verified.

Further, the speaker 100 can obtain the face information of the first user according to the method in the foregoing embodiment. The speaker 100 can send the face information to the payment server, and request the payment server to perform face verification. The payment server may compare the reference face information of the first user with the face information from the speaker 100 . If there is a match, the payment server may debit the payment account of the first user.

The embodiments of the present application do not limit the manner in which the speaker 100 invokes the payment account for payment. For details, reference may also be made to the implementation manner in which the electronic device invokes the payment account for payment in the prior art.

In a scenario where the first user has multiple speakers, the above-mentioned reference biological information stored in the cloud storage space can be shared by the multiple speakers. Exemplarily, the first user owns multiple speakers. The plurality of speakers may include the speakers 100 described above. These multiple speakers can be located in the bedroom, living room, etc. respectively. The multiple speakers can be bound to the account logged in the speaker APP in the electronic device 200 .

When the reference biological information of the first user is recorded in one of the plurality of speakers, the one speaker can store the reference biological information of the first user in the cloud storage space. These multiple speakers can access the above cloud storage space. In this way, users only need to enter the reference biometric information once, and then they can pay for the paid resources played by the speakers through voiceprint verification and face verification on multiple speakers. The above implementation manner can simplify the user's operation of entering the reference biometric information for payment, and improve the user's use experience.

It is not limited to the above-mentioned scenario of multiple speakers, but can also be a distributed scenario including electronic devices such as speakers, TVs, mobile phones, and story machines. In the above distributed scenario, multiple electronic devices bound to the same account can share the reference biological information stored in the cloud storage space associated with the account. That is, the user only needs to enter the reference biological information once, and these multiple electronic devices can obtain the reference biological information from the cloud storage space for verification, and call the payment account to pay for the paid resources, such as paid audio resources, paid video resources, etc.

In some embodiments, accounts bound to multiple electronic devices in the above distributed scenario may be different accounts. These different accounts may be accounts belonging to the same account group. Wherein, multiple accounts belonging to the same account group may share the content purchased by each account in the content server. The above-mentioned content server may be, for example, a music server, a video server, or the like.

Exemplarily, a distributed scenario of a home includes sound box A, sound box B, and sound box C. The accounts bound to Speaker A, Speaker B, and Speaker C are Account A, Account B, and Account C, respectively. Account A, Account B, and Account C can be respectively associated with reference biometric information of different family members in this family. The account A, the account B, and the account C are accounts belonging to one family group. When speaker A is bound to account A, it can purchase music A according to the payment method in the foregoing embodiment. Since speaker B is bound to account B, and speaker B is bound to account C, speaker B and speaker C can play music A without having to purchase it again.

The above accounts belonging to the same family group may be associated with the same payment account. That is to say, when the above-mentioned account A, account B, and account C are used as identifications to purchase paid resources, speaker A, speaker B, and speaker C can all instruct the payment server to assign the same one associated with account A, account B, and account B to the payment server. The payment account is debited. Using the above account A as the identity identifier to purchase paid resources, the speaker A can perform user identity verification according to the reference biometric information associated with the account A. To purchase paid resources with account B as an identity identifier, speaker B can perform user identity verification based on the reference biometric information associated with account B. Using account C as the identity identifier to purchase paid resources, speaker C can perform user identity verification based on the reference biometric information associated with account C.

This embodiment of the present application does not limit the manner in which different accounts are set to belong to the same family group. For details, reference may be made to the implementation manner in the prior art.

The following describes a payment method for verifying a user's identity through biometric information verification provided by an embodiment of the present application in conjunction with the scenarios shown in FIGS. 4A to 4E .

FIG. 5 exemplarily shows a flow chart of a payment method. The method may include steps S201-S209. in:

The security module of the speaker 100 may store the first biological information of the first user. The first biological information may include first voiceprint information and first face information. The first voiceprint information may be the reference voiceprint information in the foregoing embodiments. The first face information may be the reference face information in the foregoing embodiment.

S201, the speaker 100 establishes a binding relationship with the first account.

The first account may be an account that the first user logs in on the speaker APP of the electronic device 200, for example, an account whose account name is "Li Ming". The first account number may be associated with a payment account. When the account logged in the speaker APP is the first account, when paying for the paid audio resources, the payment server can debit the payment account associated with the first account.

For the method for establishing a binding relationship between the speaker 100 and the first account, reference may be made to steps S101 and S102 in the method shown in FIG. 3 , which will not be repeated here.

S202, the speaker 100 receives a user operation of waking up the speaker 100, playing the paid resource, and requesting payment.

The above-mentioned user operation for waking up the speaker 100 and playing the paid resource may be, for example, the first user as shown in FIG. Among them, music A is a paid resource.

The above-mentioned user operation for requesting payment may be, for example, the first user as shown in FIG. 4B uttering the voice instruction "please help me complete the payment" near the speaker 100 .

S203, the speaker 100 records the voiceprint information of the first user.

S204. The speaker 100 verifies the voiceprint information entered in step S203, and determines that the voiceprint information matches the first voiceprint information.

S205 , the speaker 100 finds and calls the available face collection device 300 .

S206, the face collection device 300 records the face information of the first user.

S207 , the face collection device 300 encrypts the face information entered in step S206 and sends it to the speaker 100 .

S208, the speaker 100 decrypts the encrypted face information, and performs verification to confirm that the face information matches the first face information.

S209, the speaker 100 calls the payment account associated with the first account to pay for the payment resource.

The specific process of the speaker 100 verifying the voiceprint information and the face information, and calling the payment account associated with the first account for payment may refer to the introduction of the foregoing embodiments, and will not be repeated here.

It can be seen from the method shown in FIG. 5 that when paying for the paid audio resources, the speaker can collect the user's voice input and extract voiceprint information for verification. In addition, a speaker without a face collecting device can call a nearby device configured with a face collecting device to collect a face image. Further, the speaker can verify the face information in the above face image. After the above verification of the voiceprint information and face information is passed, the speaker can call the payment account to pay for the paid audio resources. In this way, a speaker that is not equipped with a face acquisition device can also complete payment by combining voiceprint verification and face verification, which simplifies the user's operation of paying when using the speaker to listen to paid audio resources.

In some embodiments, when the speaker 100 is bound to the account of the first user, such as the first account (the account named "Li Ming" shown in FIG. 2A ), the first user can authorize his family Or when friends and the like use the speaker 100 to listen to paid audio resources, the payment is completed by the above-mentioned method combining voiceprint verification and face verification. Wherein, the payment account may be a payment account associated with the above-mentioned first account number. The above-mentioned payment account associated with the first account number may be the payment account of the first user. That is, the user authorized by the first user can pay for the paid audio resource from the payment account of the first user after successful verification using the biometric information.

The reference biometric information of the primary user is stored in the security module of the speaker 100 . The speaker 100 can also establish the reference biometric information of the authorized user after the authorization of the main user. The reference biological information may include reference voiceprint information and reference face information. The reference biometric information can be used to verify the identity of the user who instructed the speaker 100 to pay by comparing whether the biometric information obtained by the speaker 100 matches the reference biometric information during the process of paying for the paid resources by the speaker 100 .

The speaker 100 may establish the reference biological information of the primary user and the reference biological information of the authorized user for the first account. Wherein, the reference biological information of the primary user associated with the first account may be: when the speaker 100 does not store the reference biological information associated with the first account, the reference biological information established by the speaker 100. That is to say, the primary user associated with the first account may be a user corresponding to the reference biological information established by the speaker 100 when the speaker 100 does not store the reference biological information associated with the first account.

The authorized user associated with the first account may be a user corresponding to the reference biological information established by the speaker 100 when the speaker 100 stores the reference biological information associated with the first account. An account bound to the speaker 100 may have one main user and one or more authorized users. Exemplarily, the speaker 100 is bound to the first account. When the speaker 100 does not store the reference biological information associated with the first account, the speaker 100 records the biological information of the first user as the reference biological information. Then the first user is the primary user associated with the first account. When the sound box 100 stores the reference biological information of the first user, the sound box 100 records the biological information of the second user as the reference biological information. Then the second user is an authorized user associated with the first account. Wherein, in the process of paying for the paid resources, if the above-mentioned first user or the second user passes the verification by the speaker 100 using the reference biometric information, the speaker 100 can call the payment account associated with the first account to pay for the paid resources. That is, the speaker 100 can call the payment account of the main user (ie, the first user) for payment.

In a possible implementation manner, the reference biometric information of the primary user may include reference voiceprint information and reference face information. The reference biometric information of the authorized user may only include reference voiceprint information. The speaker 100 can bind the reference voiceprint information of the authorized user with the reference face information of the main user.

Wherein, when the main user uses biometric information for verification, the verification can be completed through the main user's voiceprint information and face information. When an authorized user uses biometric information for verification, in addition to the authorized user's voiceprint information for verification, it also needs to complete the verification through the main user's face information. That is to say, if the speaker 100 detects that the entered voiceprint information matches the reference voiceprint information of the authorized user, the speaker 100 may further prompt that the face information of the main user needs to be entered to complete the verification. In this way, in the case of authorization by the main user, the authorized user can use his own voiceprint information for verification, so as to listen to paid audio resources on the speaker 100 . Moreover, the main user authorization can also effectively prevent the authorized user from abusing the authority and over-consuming the payment account associated with the account bound to the speaker 100 .

The following describes a schematic diagram of a scenario in which a first user authorizes a second user to create reference biological information on the speaker 100 provided by the embodiment of the present application.

FIGS. 6A to 6F exemplarily show schematic diagrams of scenarios in which the first user authorizes the second user to establish reference biological information on the sound box 100 .

Wherein, the speaker APP of the electronic device 200 has a first account registered in the speaker APP, that is, an account whose account name is "Li Ming". The speaker 100 is bound to the first account. The first user may be the primary user associated with the first account in the foregoing embodiment. The second user may be an authorized user associated with the first account in the foregoing embodiment.

For the implementation process of the first user entering the reference biological information, reference may be made to the embodiments shown in the foregoing FIG. 2A to FIG. 2I . When the sound box 100 successfully records the reference biological information of the first user, the electronic device 200 may display the user interface 230 as shown in FIG. 6A . User interface 230 may include a list of biological information 233 . The biological information list 233 may be used to indicate reference biological information entered in the sound box 100 . For example, the reference biological information 233A named "User 1". The reference biological information named "user 1" is the reference biological information of the first user. Since this reference biological information is the reference biological information associated with the first user established by the speaker 100 under the condition that the speaker 100 does not store the reference biological information associated with the first user, the speaker 100 and the electronic device 200 can default to this reference biological information. The reference biometric information is the reference biometric information of the primary user associated with the first account. The electronic device 200 may use the prompt word "primary user" in the biological information list 233 to prompt the user which reference biological information in the list is the reference biological information of the primary user.

As shown in FIG. 6B , in response to a user operation acting on the newly created biological information option 232 shown in FIG. 6A , the electronic device 200 may display a prompt box 234 on the user interface 230 . The prompt box 234 may be used to prompt the user that, when the reference biometric information of the primary user already exists, the biometric information newly created requires the authorization of the primary user when being used for payment. Exemplarily, the prompt box 234 may include the prompt "This newly created biometric information will be bound with the primary user's biometric information. When using the newly created biometric information to make payment, the facial information of the primary user needs to be verified. Do you want to create a new one?" . The prompt box 234 may also include an OK button 234A and a cancel button 234B. The cancel button 234B can be used to cancel the establishment of the authorized user's reference biometric information. The OK button 234A may be used to establish the reference biometric information of the authorized user.

In response to the user operation on the determined button 234A, the electronic device 200 may perform identity authentication of the payment account. For the implementation process of performing identity authentication on the payment account, reference may be made to the embodiments shown in the foregoing FIG. 2D and FIG. 2E . When the authentication of the payment account is completed, the electronic device 200 may display a user interface 240 as shown in FIG. 6C . User interface 240 may include prompt box 244 . The prompt box 244 can be used to prompt that the user identity authentication is successful and further operations that need to be performed. For example, the prompt box 244 may contain the prompt "authentication is successful! Please continue to complete the voice input".

Further, when the identity authentication of the payment account is successful, the electronic device 200 may send an instruction to the speaker 100 for instructing to enter the reference voiceprint information of the authorized user.

As shown in FIG. 6D , when receiving an instruction from the electronic device 200 for instructing to enter the reference voiceprint information of the authorized user, the speaker 100 can prompt the user to enter the voiceprint information by way of voice broadcast. For example, the speaker 100 can voice broadcast "Please follow me and say the following verification word: Xiaoyi Xiaoyi". The second user (ie the authorized user) can say the verification word "Xiaoyi Xiaoyi" according to the voice prompt of the speaker 100 .

The speaker 100 can receive the voice input when the second user speaks the verification word, and extract voiceprint information from the voice input. When the voiceprint information is obtained, the speaker 100 can compare the voiceprint information with the reference voiceprint information already stored in the security module. If the voiceprint information matches one of the stored reference voiceprint information, the speaker 100 may prompt the user's voiceprint information to exist by means of voice broadcast.

If the voiceprint information does not match the stored reference voiceprint information, the speaker 100 may store the voiceprint information in the security module as the reference voiceprint information of an authorized user. Moreover, the speaker 100 can bind the voiceprint information with the reference face information of the main user. Further, the speaker 100 can prompt the main user and the authorized user by voice that the reference biological information has been successfully created. For example, the speaker 100 can voice broadcast "Voiceprint input is successful, you have created biometric information".

As shown in FIG. 6F , when the reference biological information is successfully created, the speaker 100 may send a message to the electronic device 200 indicating that the reference biological information is successfully created. Further, the electronic device 200 may display the name of the newly created reference biological information in the biological information list of the user interface 230, for example, the reference biological information 233B named "User 2".

That is to say, the first user can perform the operations shown in FIGS. 6A to 6C on the electronic device 200 . Further, the speaker 100 may prompt to start recording the voiceprint information of the authorized user through a voice broadcast. When it is determined that the entered voiceprint information does not exist in the stored reference voiceprint information, the speaker 100 can use the voiceprint information as the reference voiceprint information of the new authorized user. In this way, in the process of using the speaker 100 to listen to paid resources and make payment, the second user can use his own voiceprint information for verification, and complete the payment on the speaker 100 when the main user's face information is authorized.

In a possible implementation manner, when the above-mentioned reference voiceprint information of an authorized user is newly created, the speaker 100 needs to verify the face information of the main user. When the verification of the main user's face information is passed, the speaker 100 can determine that the entered voiceprint information is authorized by the main user. In this way, the reliability of local verification during the payment process can be improved.

Specifically, the speaker 100 may first prompt the authorized user to input the voiceprint information by means of voice broadcast. The specific process is shown in Figure 6D. When the voiceprint information is successfully entered, the speaker 100 can prompt the main user to enter the face information by way of voice broadcast. For example, a speaker 100 without a face collecting device can find and call the face collecting device 300 (ie, Li Ming's TV) to collect a face image. Then, the speaker 100 can voice broadcast "The new biometric information needs the consent of the main user, the main user is asked to point his face at the camera of Li Ming's TV and wink". The face collection device 300 can encrypt the collected face image and send it to the speaker 100 .

Further, the speaker 100 can extract the face information from the obtained face image and compare it with the stored reference face information of the main user. If it matches, the verification of the main user's face information is passed. The speaker 100 can bind the reference voiceprint information of the authorized user with the reference face information of the main user.

This embodiment of the present application does not limit the storage location of the reference biometric information of the primary user and the reference biometric information of the authorized user. In addition to the security module in the speaker 100, it can also be a location such as a cloud storage space that the speaker 100 can access.

FIG. 7 exemplarily shows a flowchart of a method for a first user to authorize a second user to create reference biological information on the speaker box 100 according to an embodiment of the present application.

The method may include steps S301-S306. in:

The speaker 100 is bound to the first account. The first user may be the primary user associated with the first account in the foregoing embodiment. The second user may be an authorized user associated with the first account in the foregoing embodiment. The security module of the speaker 100 stores the first biological information of the first user (including the first voiceprint information and the first face information). The first biological information is the reference biological information in the foregoing embodiments.

S301. The electronic device 200 receives a user operation for opening the speaker APP, pairing the electronic device 200 with the speaker 100, and establishing a binding relationship between the first account of the speaker APP and the speaker.

S302 , the electronic device 200 is successfully paired with the speaker 100 , and the first account of the speaker APP establishes a binding relationship with the speaker 100 .

For the above-mentioned steps S301 and S302, reference may be made to the steps S101 and S102 in the foregoing embodiments, respectively, which will not be repeated here.

S303 , the electronic device 200 receives a user operation for requesting establishment of the second biometric information for payment, and sends a request for establishing the second biometric information for payment to the speaker 100 .

The above-mentioned user operation for requesting establishment of the second biometric information for payment may be the user operation acting on the newly created biometric information option 232 in FIG. 6A . The second biological information may be the reference biological information of the authorized user. The second biological information may only include reference voiceprint information.

S304, the speaker 100 records the second voiceprint information of the second user.

S305, the speaker 100 stores the second voiceprint information, and binds the second voiceprint information with the first face information.

S306 , the speaker 100 sends a message to the electronic device 200 for indicating that the second biological information is successfully established.

For the specific process of the foregoing steps S304 to S306, reference may be made to the foregoing embodiments in FIG. 6D and FIG. 6E , which will not be repeated here.

The following specifically introduces a schematic diagram of another scenario in which the speaker 100 according to the embodiment of the present application pays for paid audio resources by combining voiceprint verification and face verification.

The security module of the speaker 100 stores the reference voiceprint information and the reference face information of the first user, and stores the reference voiceprint information of the second user. The reference face information of the first user is bound not only with the reference voiceprint information of the first user, but also with the reference voiceprint information of the second user.

The first user can pay for paid audio resources on the speaker 100 through voiceprint verification and face verification. The second user can pay for the paid audio resources on the speaker 100 through voiceprint verification and the first user authorizes using face verification.

Specifically, as shown in FIG. 8A , the second user wakes up the speaker 100 through a preset wake-up word and issues a voice instruction for playing music B. Among them, the second user can say "Xiaoyi Xiaoyi, I want to listen to music B" near the speaker 100 .

When a preset wake-up word is detected, the speaker 100 can wake up the application processor to recognize and execute the received voice command. In response to the second user's voice instruction of "listen to music B", the speaker 100 can acquire and play the resources of music B from the music server. In addition, the speaker 100 can also voice broadcast "OK, play music B for you" in response to the second user's voice instruction.

If the music B is a paid resource, and the first account (the account bound to the speaker 100 ) has not purchased the music B, the music server may send a message to the speaker 100 indicating that the music B is paid music. As shown in FIG. 8B , when receiving a message indicating that music B is paid music, the speaker 100 can voice broadcast “you can listen to the complete music B after payment, whether to pay or not” to prompt the first user to pay to listen to music B .

Further, the second user can say "please help me complete the payment" near the speaker 100 . When the voice command "please help me complete the payment" is detected, the speaker 100 can start to perform voiceprint verification and face verification. Specifically, as shown in FIG. 8C , the speaker 100 can voice broadcast "Okay, start the voiceprint verification. Please follow me and say the following verification word: Xiaoyi Xiaoyi" to prompt the second user to enter the voiceprint information. The second user can say the verification word "Xiaoyi Xiaoyi" according to the prompt of the speaker 100 .

The speaker 100 can receive the voice input of the second user speaking the above verification word, and extract voiceprint information from the voice input. Further, the speaker 100 may compare the voiceprint information with the stored reference voiceprint information, and determine that the voiceprint information matches the reference voiceprint information of the second user. That is, the speaker 100 can determine that the voiceprint verification is successful, and start face verification.

As shown in FIG. 8D , since the reference voiceprint information of the second user is bound to the reference face information of the main user, the speaker 100 can voice broadcast "Voiceprint verification is successful, and the face of the main user needs to be further verified. Face to the camera of Li Ming's TV and wink." Wherein, since the speaker 100 is not equipped with a face collection device, the speaker 100 can search for and call a nearby available face collection device to collect a face image after the voiceprint verification is successful.

The manner in which the speaker 100 finds and invokes the face acquisition device 300 (ie, Li Ming's TV) to acquire the face information may refer to the foregoing embodiments, which will not be repeated here. When the above-mentioned face information is obtained, the speaker 100 can compare the above-mentioned face information with the reference face information of the main user (ie, the first user). If the above face information matches the reference face information of the main user, the face verification is successful.

When both the voiceprint verification and the face verification are successful, the speaker 100 can complete the local verification of the payment account. Further, the speaker 100 can be authenticated with the payment server of the payment account.

The verification process between the speaker 100 and the payment server, the payment server debiting the payment account, and the implementation process of the music server sending the resources of music B to the speaker 100 after successful payment can all refer to the foregoing embodiments, which will not be repeated here. .

As shown in FIG. 8E , the speaker 100 may receive a message sent by the payment server to indicate that the payment is successful. Then, the speaker 100 can voice broadcast "face verification is successful, you have successfully purchased music B, and play the complete music B for you" to remind the user that the payment has been successful. Further, the speaker 100 can play the resource of music B received from the music server.

In some embodiments, when the account logged into the speaker APP is the first account named "Li Ming", the speaker 100 can be bound to the first account. The audio resources purchased by the first user (ie the primary user) after the biometric information verification is successful, and the audio resources purchased by the second user and other authorized users after the biometric information verification is successful can all belong to the audio resources purchased by the above-mentioned first account. The server providing the above audio resources may record the audio resources purchased by the first account. That is to say, the audio resources purchased under the condition that any reference biometric information created in the first account has passed the verification can be played when the account bound to the speaker 100 is the first account.

Exemplarily, the account currently bound to the speaker 100 is the first account. The reference biological information of the first user and the reference biological information of the second user have been established in the first account. The first user can complete the verification through his own biological information, and make the speaker 100 call the payment account to purchase the music A. The second user can complete the verification under the authorization of the first user through his biometric information, and make the speaker 100 call the payment account to purchase music B. Then, both music A and music B can belong to the music purchased by the above-mentioned first account. When the first user instructs the speaker 100 to play music B through a voice command, there is no need to pay again. Similarly, when the second user instructs the speaker 100 to play music A through a voice command, there is no need to pay again.

As can be seen from the scenarios shown in FIGS. 8A to 8E , the speaker 100 can store the reference biometric information of the main user (including the reference voiceprint information and the reference face information) and the reference biometric information of the authorized user (including only the reference voiceprint information). ). After the authorized user's voiceprint information is verified, combined with the verification of the main user's face information, the speaker 100 can call the payment account to pay for the paid audio. In this way, it is not limited to the main user to complete the payment through voiceprint verification and face verification. Authorized users, such as the main user's family and friends, can also pass voiceprint verification and face verification under the authorization of the main user. Complete the payment. In addition, since the payment account associated with the first account bound to the speaker 100 can be considered as the payment account of the main user, the method of verifying the voiceprint information of the authorized user and the face information of the main user can make payment in the payment account. The main user must confirm before, so that the authorized user can avoid abuse of authority and excessive consumption of the main user's payment account.

The following describes a payment method for verifying a user's identity through biometric information verification provided by an embodiment of the present application in conjunction with the scenarios shown in FIGS. 8A to 8E .

FIG. 9 exemplarily shows a flow chart of a payment method. The method may include steps S401-S409. in:

The security module of the speaker 100 stores the first biological information of the first user and the second biological information of the second user. The first biological information may include first voiceprint information and first face information. The first biological information may be the reference biological information of the first user in the foregoing embodiment. The second biological information may only include the second voiceprint information. The second biological information may be the reference biological information of the second user in the foregoing embodiment. That is, the second voiceprint information is the reference voiceprint information of the second user in the foregoing embodiment. The first user may be the main user in the foregoing embodiment. The second user may be the authorized user in the foregoing embodiment.

S401, the speaker 100 establishes a binding relationship with the first account.

S402, the speaker 100 receives a user operation of waking up the speaker 100, playing the paid resource, and requesting payment.

S403, the speaker 100 records the voiceprint information of the second user.

S404. The speaker 100 verifies the voiceprint information entered in step S403, and determines that the voiceprint information matches the second voiceprint information.

S405 , the speaker 100 finds and calls the available face collection device 300 .

S406, the face collection device 300 records the face information of the first user.

S407 , the face collection device 300 encrypts the face information entered in step S406 and sends it to the speaker 100 .

S408, the speaker 100 decrypts the encrypted face information, and performs verification to confirm that the face information matches the first face information.

S409, the speaker 100 calls the payment account associated with the first account to pay for the payment resource.

For the specific implementation process of the foregoing steps S401 to S409, reference may be made to the foregoing embodiments, which will not be repeated here.

In a possible implementation manner, the above-mentioned second biological information may further include second face information. The second face information may be the face information of the second user. The second face information can be used as reference face information to verify whether the face information acquired by the speaker 100 is the face information of the second user. The second voiceprint information in the second biological information may be bound with the second face information. That is to say, when the speaker 100 combines voiceprint verification and face verification to pay for the paid audio, if both the voiceprint information and the face information of the second user (that is, the authorized user) are verified, the speaker 100 can call the main The payment account of the user makes the payment. In this way, in a scenario where the main user is not near the speaker 100, the authorized user can also complete the verification by speaking the verification word and using his own face information, and then listen to the paid audio resources.

The following specifically introduces another schematic diagram of a scenario in which a first user authorizes a second user to create reference biological information on the speaker 100 provided in the embodiment of the present application.

The first user may be the primary user in the foregoing embodiment. The second user may be the authorized user in the foregoing embodiment.

As shown in FIG. 10A , the biological information list contains reference biological information named “User 1”. The reference biological information named "User 1" may represent the reference biological information of the main user. That is to say, the reference biological information of the main user is stored in the security module of the speaker 100 .

In response to a user operation acting on the new biometric information option 232 , the electronic device 200 may transmit an instruction to the sound box 100 for instructing to create a new authorized user's reference biometric information. Wherein, before the electronic device 200 sends the above-mentioned instruction for instructing the establishment of the reference biometric information of a new authorized user, the electronic device 200 may first perform identity authentication of the payment account. The first user (ie the main user) is the user corresponding to the payment account. That is, the first user knows the password of the payment account. The first user may input the password of the payment account in the user interface 240 shown in FIG. 10B to complete the identity authentication of the payment account. When the identity authentication of the payment account is successful, the electronic device 200 may consider that the first user agrees to establish the reference voiceprint information of the authorized user. Further, the speaker 100 can start to record the voiceprint information and the face information as the reference voiceprint information and the reference face information of the new authorized user, respectively.

For the specific implementation process of the identity authentication of the payment account performed by the electronic device 200, reference may be made to the embodiments shown in the foregoing FIG. 2C to FIG. 2E , which will not be repeated here.

As shown in FIG. 10C , upon receiving an instruction instructing to create the reference biometric information of the new authorized user, the speaker 100 may start recording the reference voiceprint information. Specifically, the speaker 100 can voice broadcast "Please follow me and say the following verification word: Xiaoyi Xiaoyi" to prompt the user to perform voiceprint input.

When hearing the voice prompt of the speaker 100, the second user can say the verification word "Xiaoyi Xiaoyi". The speaker 100 can receive the voice input when the second user speaks the verification word, and extract voiceprint information from the voice input. When the voiceprint information is obtained, the speaker 100 can compare the voiceprint information with the reference voiceprint information already stored in the security module. If the voiceprint information matches one of the stored reference voiceprint information, the speaker 100 may prompt the user's voiceprint information to exist by means of voice broadcast.

If the voiceprint information does not match the stored reference voiceprint information, the speaker 100 may store the voiceprint information in the security module as the reference voiceprint information of an authorized user.

When the reference voiceprint information is successfully entered, the speaker 100 can search for and call a nearby device configured with a face acquisition device to acquire face information. For a specific method for the speaker 100 to search for and call a device configured with a face acquisition device, reference may be made to the foregoing embodiments. Exemplarily, the speaker 100 may find and call the face acquisition device 300 to acquire a face image. The face collection device 300 may be a TV named "Li Ming's TV". The face collection device 300 may include a face collection apparatus 301 . The face collecting device 301 may be, for example, a camera.

As shown in Fig. 10D, when the face collection device 300 is found and called, the speaker 100 can voice broadcast "Voiceprint recording is successful. Please complete the face entry on Li Ming's TV, please point your face to Li Ming's TV's camera and wink". The second user can complete the face entry on the face collection device 300 according to the prompt of the speaker 100 . When the face image of the second user is collected, the face collection device 300 may encrypt the face image and send it to the speaker 100 .

Further, the speaker 100 can decrypt the encrypted face image, and extract face information therefrom. Then, the speaker 100 can store the face information in the security module as the reference face information, and bind it with the reference voiceprint information obtained in the above scenario shown in FIG. 10C . In this way, the speaker 100 can complete the input of the reference biometric information of the new authorized user. Among them, as shown in FIG. 10E , the speaker 100 can voice broadcast "face entry is successful, you have created biometric information" to remind the user that the reference biometric information of the new authorized user has been successfully created.

In addition, the speaker 100 may also send a message that the biometric information is successfully created to the electronic device 200 . The electronic device 200 may display a user interface as shown in FIG. 6F. That is, the name of the newly created biological information, such as "User 2", is added to the biological information list 233.

As can be seen from the scenarios shown in FIGS. 10A to 10E , the speaker 100 can establish reference biological information of multiple users. The reference biometric information of each user may include its own reference voiceprint information and reference face information. That is, when the speaker 100 prompts that the currently playing audio is paid audio, each of the multiple users can use the voiceprint verification and face verification to make the speaker 100 call the payment account for payment, thereby listening to the paid audio. The payment account used for payment may be the payment account associated with the account bound to the speaker 100 . That is, the payment account that has been authenticated in the above-mentioned FIG. 10B .

In some embodiments, after the authorized user enters the reference voiceprint information and the reference face information in the speaker 100, the speaker 100 can call a payment account to pay for the paid audio resources by combining voiceprint verification and face verification. In this way, in a scenario where the main user is not near the speaker, the authorized user can also use the payment account of the main user to purchase paid audio resources on the speaker 100 .

The following specifically introduces another payment method for verifying user identity through biometric information verification provided by the embodiment of the present application.

As shown in FIG. 11A , the sound box 100 may store the reference biological information of the first user (ie the main user) and the reference biological information of the second user (ie the authorized user). The above-mentioned reference biological information includes reference voiceprint information and reference face information.

The second user can wake up the speaker 100 through a preset wake-up word and issue a voice instruction for playing music B. Among them, the second user can say "Xiaoyi Xiaoyi, I want to listen to music B" near the speaker 100 . The above-mentioned music B is a paid resource, and the first account (the account bound to the speaker 100 ) has not purchased the music B. The music server providing the resource of music B may send a message to the speaker 100 indicating that music B is paid music.

As shown in FIG. 11B , when receiving the above-mentioned message indicating that the music B is paid music, the speaker 100 can voice broadcast “you can listen to the complete music B after paying, whether to pay” to remind the first user that the first user needs to pay to listen to the music B. Further, the second user can say "please help me complete the payment" near the speaker 100 . When the voice command "please help me complete the payment" is detected, the speaker 100 can start to perform voiceprint verification and face verification. Specifically, as shown in FIG. 11C and FIG. 11D , the speaker 100 can acquire voiceprint information and face information of the second user. For the method for acquiring the voiceprint information and face information of the second user by the speaker 100, reference may be made to the foregoing embodiments, which will not be repeated here.

When the voiceprint information and the face information are obtained, the speaker 100 can compare the voiceprint information and the face information with the stored reference voiceprint information and the reference face information respectively. The speaker 100 can determine that the voiceprint information and the face information are the voiceprint information and the face information of the second user. That is, both voiceprint verification and face verification are successful. Further, the speaker 100 can call the payment account associated with the above-mentioned first account to purchase the music B. In this way, the second user can listen to music B on the speaker 100 .

In other embodiments, after the authorized user enters the reference voiceprint information and the reference face information in the speaker 100, the authorized user can call the payment account on the speaker 100 in combination with voiceprint verification, face verification and remote authorization by the main user. Paid audio resources.

The above-mentioned main user remote authorization may be: when the speaker 100 detects that both the voiceprint verification and the face verification of the authorized user are successful, the speaker 100 may send a message to the electronic device 200 for confirming whether to pay. The electronic device 200 may display the above-mentioned message for confirming whether to pay on the user interface. Furthermore, the primary user may authorize the speaker 100 to call the payment account to pay for the paid audio resources through the electronic device 200 . In this way, in a scenario where the main user is not near the speaker, the authorized user can also complete voiceprint verification and face verification to request the speaker 100 to call the payment account to pay for the paid audio resources. In combination with the above-mentioned method of remote authorization of the main user, it can be ensured that the payment account of the main user is debited after the consent of the main user. This not only simplifies the operation of paying when listening to paid audio resources on the speaker 100, but also improves the security of the payment account, and effectively prevents the authorized user from over-consuming the payment account of the main user.

The speaker 100 can complete the voiceprint verification and face verification of the second user (ie, the authorized user) according to the processes shown in the aforementioned FIGS. 11A to 11D . When both the voiceprint verification and the face verification of the authorized user are successful, the speaker 100 may send a message for confirming the payment to the electronic device 200 .

As shown in FIG. 12A , the speaker 100 can also voice broadcast “Face verification is successful, waiting for confirmation from the mobile terminal” to prompt the authorized user to call the payment account for payment, which needs to be confirmed by the main user. When receiving a message for confirming payment from the speaker 100, the electronic device 200 may display a user interface 250 as shown in FIG. 12B. The user interface 250 may include a message notification box 251 . The message notification box 251 may be used to confirm to the user whether the speaker agrees to call the payment account to pay for the paid audio resources, such as the purchase of music B.

In response to a user operation acting on the message notification box 251, the electronic device 200 may open the speaker APP, and display the user interface 260 as shown in FIG. 12C. The user interface 260 may include a prompt 261 , an OK button 262 and a cancel button 263 . in:

The prompt 261 may be a text prompt "User 2 applies to purchase music B, whether to purchase it", to prompt the user whose main user name is "User 2" (ie, the aforementioned second user) to apply for purchasing music B.

The OK button 262 may be used to approve the purchase of Music B by the user named "User 2" using the primary user's payment account.

Cancel button 263 may be used to deny the user named "User 2" the purchase of Music B using the primary user's payment account.

In response to a user operation, such as a touch operation, acting on the determination button 262 , the electronic device 200 may transmit a message indicating approval to purchase the music B to the speaker 100 .

Further, when receiving the above-mentioned message indicating agreement to purchase music B, the speaker 100 can call the main user's payment account to purchase music B. When the payment is successful, the speaker 100 can receive the resources of music B. As shown in FIG. 12D , the speaker 100 can voice broadcast "The mobile terminal has been confirmed, you have successfully purchased music B, and the complete music B will be played for you". Then, the speaker 100 can play the music B. For the above-mentioned speaker 100 to call the main user's payment account to purchase music B, reference may be made to the foregoing embodiments, which will not be repeated here.

It should be noted that, it is not limited to be applied to the sound box 100 shown in FIG. 1 , and the biometric information verification method provided by the embodiments of the present application can also be applied to other electronic devices. For example, TV, mobile phone, tablet, laptop, story machine, etc. For the structures of these electronic devices, reference may be made to the schematic structural diagram of the sound box 100 shown in FIG. 1 . Not limited to the modules included in the schematic structural diagram shown in FIG. 1 , these electronic devices may also include more or less modules.

Exemplarily, in a scenario where a user plays a paid video resource on a TV, the TV equipped with a voice capture device but not configured with a face capture device can, according to the method in the foregoing embodiment, enter the user's reference biometric information and pay for it after payment. In the process of video resources, the reference biometric information is used to verify the user's identity. In this way, an electronic device without a face acquisition device can perform identity verification in the payment process by combining voiceprint verification and face verification.

Alternatively, the TV is equipped with a voice collecting device and a face collecting device. The TV can find out if there is a nearby face capture device equipped with a better configured face capture device. For example, a TV is configured with a 2D camera. When entering face information, if the TV finds that there is a face collection device equipped with a 3D camera nearby, the TV can call the face collection device equipped with a 3D camera to obtain the face information. Utilizing a better-configured face capture device can improve the reliability of identity verification during the payment process.

In some embodiments, the electronic device is configured with a face capture device but not a voice capture device. That is, the electronic device cannot independently collect the voice input to obtain the voiceprint information. Similar to the aforementioned method of finding and calling a nearby face collection device, the electronic device can search for and call a nearby device equipped with a voice collection device to collect the user's voice input and obtain voiceprint information. Or, the electronic device is not equipped with a face collecting device and a voice collecting device. The electronic device can search for and call a nearby face collection device to collect face images and obtain face information. In addition, the electronic device can also search for and call a nearby voice collecting device to collect voice input to obtain voiceprint information.

It can be known from the above method that when the electronic device performs identity verification in the payment process in combination with voiceprint verification and face verification, the electronic device may not be limited to an electronic device configured with a voice capture device and a face capture device at the same time. An electronic device that is not equipped with a voice capture device and/or a face capture device can also implement user identity verification by combining voiceprint verification and face verification.

In some embodiments, the device equipped with a face collecting device near the speaker 100 is a mobile phone. Then, in the process of inputting reference face information and in the process of face verification when paying for paid resources, the speaker 100 can call a face collection device on the mobile phone, such as a camera, to obtain the face information. Because mobile phones are used in many occasions, and mobile phones are generally equipped with face acquisition devices such as cameras (such as 2D cameras, 3D cameras, and infrared cameras), electronic devices that need to play paid resources can call mobile phones on various occasions to obtain face information. , and then perform face verification. For example, in the scenario of using a portable speaker outside, if the portable speaker plays paid resources, the portable speaker can easily find and call the user's mobile phone to obtain face information.

The embodiments of the present application do not specifically limit the content contained in the above-mentioned reference biological information. Not limited to including voiceprint information and face information, the reference biometric information may also include other data. For example, fingerprint information, bone voiceprint information, iris information and so on. That is to say, the speaker can also instruct the user to enter more information when entering the reference biological information.

In this application, the first device collects the first biological information. The above-mentioned first device may be the sound box 100 in the foregoing embodiment. The above-mentioned first device may also be other electronic devices that request to acquire paid resources, such as a TV, a story machine, and so on. Exemplarily, the first device is the speaker 100 . The first device is configured with a voice collection device. The above-mentioned first biological information may be the voiceprint information in the foregoing embodiments.

In this application, the first device may discover the second device with the second biological information collection capability through the short-range wireless communication connection. The first device is taken as an example of the sound box 100 in the foregoing embodiment for description. The second device may be the face collection device 300 in the foregoing embodiment. The face collection device 300 is configured with a face collection device. The face collection device may be, for example, a 2D camera, a 3D camera, an infrared camera, a millimeter wave radar, and other devices. The above-mentioned second biological information may be the face information in the foregoing embodiments. The above-mentioned short-range wireless communication connection may be, for example, a near field communication connection, a Bluetooth communication connection, a WiFi communication connection, a ZigBee communication connection, a WLAN direct communication connection, and the like. In other words, the first device can find and call devices near itself to obtain face information.

In this application, the first device may pre-collect the third biological information of the first user, and call the second device to pre-collect the fourth biological information of the first user. The above-mentioned third creature may be the reference voiceprint information of the first user in the foregoing embodiment. The above-mentioned fourth biological information may be the reference face information of the first user in the foregoing embodiment. The third biological information may be bound with the fourth biological information, and stored in the security module of the speaker 100 in the foregoing embodiment, or stored in a cloud storage space that the speaker 100 can access. For the above-mentioned specific methods of pre-collecting the third biological information and the fourth biological information, reference may be made to the embodiments shown in the foregoing FIG. 2A to FIG. 2I .

If the first device determines that the first biometric information matches the third biometric information, and the second biometric information matches the fourth biometric information, the identity verification is successful. That is, the first device may determine that the user corresponding to the first biological information and the second biological information is the above-mentioned first user.

When the authentication is successful, the first device can send the first message to the first server. The above-mentioned first server may be the payment server in the foregoing embodiment. The first message may include identification information of the first payment account, order information of the paid resources to be paid, and the like. This embodiment of the present application does not limit the content included in the foregoing first message. The payment server may debit the first payment account according to the first message. The above-mentioned first payment account may be a payment account associated with an account lock bound to the first device. For example, the user associated with the account named "Li Ming" in the foregoing embodiment.

In addition, before the payment server debits the first payment account, it may first perform trusted identity verification on the first device to determine that the first device is a trusted device. For the above-mentioned trusted identity verification method, reference may be made to the foregoing embodiments, which will not be repeated here.

When the payment server successfully debits the first payment account, the payment server may send a payment success message to the second server. Further, the second server may send the paid resources purchased when the account bound to the first device is used as the identity identifier to the first device. The above-mentioned second server may be a content server. The content server may store the paid resources requested by the first device. The content server may be, for example, a music server, a video server, or the like. This embodiment of the present application does not limit the types of the foregoing content servers.

In this application, the first device logs into the first account on the second server. The above-mentioned first account is the account bound to the first device in the foregoing embodiment. For example, the account name shown in FIG. 2A is an account named "Li Ming". For the method for binding the first device and the first account, reference may be made to the foregoing embodiments. The first device may log in to the first account on the content server. IDs of multiple accounts may be stored in the content server. The above-mentioned multiple accounts include a first account. The content server may determine the paid resources purchased by each account according to the identifiers of the above-mentioned multiple accounts. When the first device logs into the first account on the second server and obtains the first content from the second server, the second server may determine that the first account has not purchased the first content. Further, the second server may send a message to the first device to prompt the first device to pay for acquiring the first content. The second server may also send price information of the first content to the first device.

In this application, the first device may collect fifth biological information of the second user. Wherein, if the first device is the sound box 100 in the foregoing embodiment, the above-mentioned fifth biological information may be the reference voiceprint information of the second user in the foregoing embodiment. The second user may enter the fifth biometric information under the authorization of the first user. For a specific implementation process, reference may be made to the embodiments shown in the foregoing FIG. 6A to FIG. 6C . The above-mentioned first user may be the main user in the foregoing embodiment. The above-mentioned second user may be an authorized user in the foregoing embodiment.

In this application, the first device may collect sixth biological information. The sixth biological information may be the voiceprint information in the foregoing embodiments. The first device may also call the second to collect the second biological information. If the first device determines that the sixth biological information matches the fifth biological information, and the second biological information matches the fourth biological information, the identity verification is successful. That is to say, in the case that the authentication of the voiceprint information of the authorized user is successful, if the primary user is authorized, the first device may request the payment server to debit the first payment account. The above-mentioned main user authorization may indicate that the first device successfully authenticates the face information of the main user.

In this application, the first device may collect the fifth biological information of the second user, and call the second device to collect the seventh biological information of the second user. For the above fifth biological information, reference may be made to the introduction of the foregoing embodiments. The above-mentioned seventh biological information may be the reference face information of the second user. That is, the first device can obtain the reference voiceprint information and the reference face information of the authorized user.

In this application, the second user (ie, the authorized user) can use his own voiceprint information and face information for identity verification. The first device may collect sixth biological information, and call the second device to collect eighth biological information. For the above-mentioned sixth biological information, reference may be made to the introduction of the foregoing embodiments. The above-mentioned eighth biological information may be face information. If the speaker 100 determines that the fifth biometric information matches the sixth biometric information, and the seventh biometric information matches the eighth biometric information, the authentication of the authorized user is successful.

It can be known from the above embodiments that the biometric information verification method provided by the embodiments of the present application can be used for identity verification in the payment process. Not limited to identity verification in the payment process, the above biometric information verification method can also be used for identity verification in other scenarios. For example, log in to an account, obtain permissions, etc.

Among them, when listening to paid audio resources on the speaker needs to be paid, the speaker can perform user identity verification by combining voiceprint verification and face verification. After both voiceprint verification and face verification are successful, the speaker can request the payment server to debit the payment account to pay for the paid audio resources. This simplifies the operation for users to pay for listening to paid audio resources on the speakers. The above method of combining voiceprint verification and face verification can also improve the reliability of the payment process. In addition, a speaker without a face collection device can complete face verification by calling a nearby device equipped with a face collection device. That is, a speaker configured with or without a face collection device can implement the payment method provided by the embodiment of the present application, so as to simplify the user's operation of paying for paid audio resources.

In addition, after the main user has entered his own reference voiceprint information and reference face information in the speaker, he can also authorize other users, such as his family members and friends, to enter the reference voiceprint information and reference face information. The above-mentioned authorized users can also perform user identity verification by combining voiceprint verification and face verification. After the authentication of the authorized user is successful, the speaker can request the payment server to debit the payment account of the main user, thereby paying for the paid audio resource. That is to say, not limited to the main user, users authorized by the main user can also conveniently complete the operation of paying for the paid audio resources on the speaker.

As used in the above embodiments, the term "when" may be interpreted to mean "if" or "after" or "in response to determining..." or "in response to detecting..." depending on the context. Similarly, depending on the context, the phrases "in determining..." or "if detecting (the stated condition or event)" can be interpreted to mean "if determining..." or "in response to determining..." or "on detecting (the stated condition or event)" or "in response to the detection of (the stated condition or event)".

The above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: The recorded technical solutions are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

A method for verifying biological information, comprising:

The first device collects the first biological information;

The first device discovers a second device with a second biological information collection capability through a short-range wireless communication connection;

the first device receives the second biological information collected from the second device, the second biological information is different from the first biological information;

The first device determines that the first biological information matches the third biological information, and the second biological information matches the fourth biological information; the third biological information is stored in the first device or the The cloud storage space accessible by the first device, the third biometric information is the biometric information of the first user pre-collected by the first device; the fourth biometric information is stored in the first device or the first user A cloud storage space that can be accessed by a device, and the fourth biological information is the biological information of the first user pre-collected by the second device.
The method according to claim 1, wherein the third biological information is used to determine whether the first biological information obtained by the first device is the biological information of the first user, and the fourth biological information is used to determine whether the first biological information obtained by the first device is the biological information of the first user. The biological information is used to determine whether the second biological information obtained by the first device is the biological information of the first user.
The method according to claim 1 or 2, wherein the method further comprises:

The first device sends a first message to the first server; the first message is used to instruct the first server to debit a first payment account, where the first payment account is the payment of the first user account.
The method according to claim 3, wherein before the first device collects the first biological information, the method further comprises:

The first device logs in to the first account on the second server; the first account is the account of the first user on the second server, and the second server is used to log in from the first server from After successful deduction from the first payment account, send the content purchased by the first user to the first device;

sending, by the first device, an acquisition request of the first content to the second server;

The first device receives the price information of the first content sent by the second server.
The method according to claim 4, wherein the third biological information and the fourth biological information stored in the cloud storage space can be accessed by multiple devices, and the multiple devices include the first device , the multiple devices share the same account on the second server, or the multiple accounts of the multiple devices on the second server belong to the same account group; the belonging to the same account group indicates that the Multiple accounts in the same account group share the content purchased by each account in the second server.
The method according to any one of claims 1-5, wherein the first device is a device configured with a voice collection device, and the first biometric information and the third biometric information are both voiceprints information.
The method according to claim 6, wherein the second device is a device configured with a face collection device, and the second biological information and the fourth biological information are both human face information.
The method according to any one of claims 1-5, wherein the first device is a device configured with a face collection device, and the first biological information and the third biological information are both human face information .
The method according to claim 8, wherein the second device is a device configured with a voice collection device, and the second biometric information and the fourth biometric information are both voiceprint information.
The method according to any one of claims 1-9, wherein the method further comprises:

The first device collects the fifth biological information of the second user; the fifth biological information and the first biological information are the same type of biological information, and the fifth biological information is used to determine the first device Whether the obtained biometric information is the biometric information of the second user, the fifth biometric information is stored in the first device or a cloud storage space accessible to the first device, and the fifth biometric information is the same as that of the first device. Four biological information binding.
The method of claim 10, wherein the method further comprises:

The first device collects sixth biological information, and the sixth biological information is the same type of biological information as the first biological information;

the first device receives the second biological information collected from the second device;

the first device determines that the sixth biological information matches the fifth biological information, and the third biological information matches the fourth biological information;

The first device sends a second message to the first server, where the second message is used to instruct the first server to debit the first payment account.
The method according to any one of claims 1-9, wherein the method further comprises:

The first device collects the fifth biological information of the second user; the fifth biological information and the first biological information are the same type of biological information, and the fifth biological information is used to determine the first device Whether the obtained biological information is the biological information of the second user, the fifth biological information is stored in the first device or a cloud storage space accessible by the first device;

The first device receives seventh biological information of the second user collected from the second device; the seventh biological information and the second biological information are the same type of biological information, and the seventh biological information The information is used to determine whether the biological information obtained by the first device is the biological information of the second user, and the seventh biological information is stored in the first device or a cloud storage space accessible by the first device;

Wherein, the fifth biological information is bound with the seventh biological information.
The method of claim 12, wherein the method further comprises:

The first device collects sixth biological information, and the sixth biological information is the same type of biological information as the first biological information;

The first device receives the eighth biological information collected from the second device; the eighth biological information and the second biological information are the same type of biological information;

The first device determines that the sixth biological information matches the fifth biological information, and the eighth biological information matches the seventh biological information;

The first device sends a second message to the first server, where the second message is used to instruct the first server to debit the first payment account.
The method according to claim 13, wherein before the first device sends the second message to the first server, the method further comprises:

The first device sends a third message to a third device, and receives a fourth message from the third device; the third device is a device installed with an application program for controlling the first device, so The indication content of the third message is displayed on the third device, and the indication content of the third message includes asking the third device whether to allow the first device to send the second message, the fourth message The content of the indication includes that the third device agrees to the first device to send the second message.
The method according to any one of claims 1-14, wherein the first device does not have the capability of collecting the second biological information.
The method according to any one of claims 1-15, wherein the short-range wireless communication connection is one or more of the following: near field communication connection, Bluetooth communication connection, WLAN direct communication connection, ZigBee communication connection communication connection.
A device, the device is a first device, wherein the first device comprises: a first collection device, a communication device, a memory and a processor, wherein:

the first collection device, configured to collect first biological information;

The communication device is used for the first device to discover a second device with a second biological information collection capability through a wireless communication connection; the second device includes a second collection device, and the second collection device is used to collect the above-mentioned second biological information;

The communication device is further configured to receive the second biological information collected from the second collection device;

the memory, for storing the first biological information and the second biological information, and for storing a computer program;

The processor is configured to invoke the computer program to cause the first device to execute the method of any one of claims 1-16.
A computer-readable storage medium, comprising: computer instructions, which, when the computer instructions are executed on a device, cause the device to perform the method according to any one of claims 1-16.
A computer program product, characterized in that, when the computer program product is run on a device, the device is caused to perform the method according to any one of claims 1-16.