CN111352501A

CN111352501A - Service interaction method and device

Info

Publication number: CN111352501A
Application number: CN201911251433.3A
Authority: CN
Inventors: 白莹婷
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2019-12-09
Filing date: 2019-12-09
Publication date: 2020-06-30

Abstract

The application provides a service interaction method and a device, which relate to the technical field of artificial intelligence. When the user agrees to collect the personal data, the electronic equipment collects the personal data of the user and provides services for the user. In the process, the user embeds the privacy notification and the consent management into the business process in the interaction process with the electronic equipment, so that the flexible and complete privacy notification, the consent management and the like are realized, and the user can use the business provided by the electronic equipment without accessing a management end. Moreover, a special flow is used for users needing monitoring, such as children, namely personal data of the users are collected after the consent of the guardian is solicited, and the use requirements of the children and the requirements of privacy protection are met.

Description

Service interaction method and device

Technical Field

The embodiment of the application relates to the technical field of Artificial Intelligence (AI), in particular to a service interaction method and device.

Background

With the rapid development of the technology, more and more terminal devices integrate an autonomous system, and realize services through the capabilities of the autonomous system such as self-perception, self-decision, self-management and the like. Common terminal devices integrated with an autonomous system include an autonomous vehicle, a robot, an unmanned aerial vehicle, a smart speaker, a voice assistant, and the like.

In general, when a terminal device with an autonomous system interacts with a user to perform a service, it is inevitably necessary to acquire biometric information of the user, such as voice information and face information. However, if the terminal device does not remind the user to collect the biometric information of the user, the privacy protection requirement is violated; or, even if the terminal device reminds the user that the biometric information needs to be collected, the biometric information is collected on the premise that the user does not agree, and the privacy protection requirement is violated. In order to avoid violating the privacy protection requirement, when the biometric information needs to be collected, the autonomous system of the terminal device triggers an Application (APP) of the management end to pop up a popup window to prompt an administrator that the biometric information needs to be collected, and the prompting process is called a privacy notification process. After that, the autonomous system judges whether the administrator agrees to collect the biometric information according to the input of the administrator, which is called an agreement management process. The autonomous system implements the relevant service only when it is determined that the administrator agrees to collect the biometric information.

In the service interaction process, the privacy notification and the consent management only aim at the administrator with the access authority of the management terminal. However, in many cases, the administrator and the user are not the same user, and for a user without access authority, the autonomous system cannot perform privacy notification and consent management, and thus cannot provide related services for the user without management authority.

Disclosure of Invention

The embodiment of the application provides a service interaction method and a service interaction device, wherein electronic equipment sends privacy notification to users in a sensing range of the electronic equipment through voice and the like and performs consent management on the users, so that any user in the sensing range of the electronic equipment can use services provided by the electronic equipment without accessing a management terminal.

In a first aspect, an embodiment of the present application provides a service interaction method, which is described from the perspective of an electronic device, and the method includes: after receiving a first request sent by a first user, the electronic equipment sends a first prompt to the first user according to the first request so as to consult whether the first user is allowed to acquire first personal data of the first user. Then, the electronic device receives a first indication for indicating whether the electronic device is allowed to collect the first personal data of the first user, and when the first indication is used for indicating that the electronic device is allowed to collect the first personal data of the first user, the first personal data of the first user is collected, and the electronic device responds to the first user according to the first personal data. In the process, the first user embeds privacy notification and consent management into a business process in the process of interacting with the electronic equipment, so that flexible and complete privacy notification, consent management and the like are realized, and the user can use the business provided by the electronic equipment without accessing a management end. Moreover, when the first user is a user needing to be monitored, such as a child, a special flow is used for the child, namely the personal data of the first user is acquired after the consent of the guardian is solicited, and the use requirement of the child and the requirement of privacy protection are met.

In one possible design, after entering a visual perception range of the electronic device, a first user sends a first request to the electronic device through a gesture, an action, an expression or the like; or after recognizing that the first user enters the perception range, the electronic device actively initiates a first request to the first user in a voice mode; for another example, after recognizing that the first user enters the sensing range, the electronic device actively initiates a first request to the first user through the display screen. By adopting the scheme, the purpose that the electronic equipment actively or passively sends privacy notifications to the user is achieved.

In one possible design, the first user is a user with an age less than an age threshold, the second user is a user with an age greater than or equal to the age threshold, the first prompt is further used to prompt the first user for consent from the second user, and the first indication includes a voice, a limb action, a gesture, or an expression of the second user. By adopting the scheme, when the first user is a user needing to be monitored, such as a child, the special flow is used for the child, namely the personal data of the first user is acquired after the consent of the guardian is solicited, and the use requirement and the privacy protection requirement of the child are met.

In one possible design, when the electronic device interacts with the first service of the first user, a third user joining situation, i.e., a person joining in the middle of the process, is likely to occur. At this time, the electronic device issues a second prompt to the third user, the second prompt being used to consult whether the first personal data of the third user is allowed to be collected. Then, the electronic equipment receives a second instruction sent by a third user, wherein the second instruction is used for indicating whether the electronic equipment is allowed to acquire the first personal data of the third user; and responding to the third user according to the second indication. By adopting the scheme, the electronic equipment achieves the purposes of privacy notification and consent management to a third user, namely a user joining in the midway.

In one possible design, when the second indication allows the electronic device to collect the first personal data of the third user, such as an OK gesture, a nodding action, or an agreement, etc., the electronic device continues to interact with the first user and the third user to implement the first service; when the second indication does not allow the electronic device to acquire the first personal data of the third user, for example, the third user makes a hand-shaking motion, a head-shaking motion, or disagreement, the electronic device interacts with the first user to implement the first service and masks the third user, that is, even if the third user appears in a sensing range of the electronic device, the electronic device automatically desensitizes or masks the third user, ignores the presence of the third user, and avoids acquiring the first personal data of the third user.

In one possible design, to implement service switching, the electronic device receives a second request sent by the first user, where the second request is used to interact with the electronic device to implement a second service, and sends a second prompt to the first user according to the second request, where the second prompt is used to consult whether to allow the first user to acquire second personal data, and then the electronic device receives a third indication sent by the first user, where the third indication is used to indicate whether to allow the electronic device to acquire the first user's second personal data; and responding to the first user according to the third indication. By adopting the scheme, the electronic equipment only judges the personal data related to the stage before the business execution of each stage, if the first user does not agree to collect the personal data, the business of the stage can only be influenced, other businesses of the electronic equipment used by the first user cannot be influenced, and the user experience is good.

In one possible design, the first personal data of the first user is at least one, the second personal data of the first user is at least one, and the first personal data and the second personal data have intersection. By adopting the scheme, the electronic equipment can perform privacy notification and consent management on the user by taking the service as granularity.

In a possible design, after the electronic device receives the first indication, when the first indication does not allow the electronic device to collect the first personal data of the first user, the electronic device refuses to provide the first service for the first user. By adopting the scheme, the first user embeds the privacy notification and the consent management into the business process in the interaction process with the electronic equipment, thereby realizing flexible and complete privacy notification, consent management and the like, and ensuring that the user can use the business provided by the electronic equipment without accessing a management end.

In one possible design, the first response includes a voice, a limb action, a gesture, or an expression of the first user. By adopting the scheme, multi-mode input and output of the electronic equipment are realized.

In one possible design, the first personal data of the first user includes biometric information and/or non-biometric information of the first user, the non-biometric information including semantics of speech uttered by the first user.

In one possible design, when a first user is in a sensing range of an electronic device, before a first prompt is sent to the first user, sensing information including first biometric information of the first user is further acquired, the sensing information is analyzed to obtain the first biometric information of the first user, historical data is queried according to the first biometric information of the first user to obtain a first record, the first record is used for indicating an agreement state, a disagreement state or a null state, the agreement state indicates that the electronic device is allowed to collect the first personal data of the first user, the disagreement state indicates that the electronic device is not allowed to collect the first personal data of the first user, the null state indicates that the electronic device does not know whether the first personal data of the first user can be collected, and the first personal data is first personal data required by the electronic device for interacting with the first user for a first service, it is determined that the first record indicates an empty status. By adopting the scheme, the electronic equipment sends privacy notice to the user in the sensing range of the electronic equipment through voice and the like and performs consent management on the user, so that any user in the sensing range of the electronic equipment can use the service provided by the electronic equipment without accessing a management terminal. Meanwhile, when the electronic equipment needs to collect the first personal data of the first user, the historical consent state of the first user is firstly inquired, so that repeated privacy notification and consent management of the consented users can be avoided.

In a possible design, after receiving the first instruction, the electronic device further generates a second record according to the first instruction, the first service, the user identifier of the first user, the first biometric information, and the first personal data; the first record updated history data is replaced with the second record. By adopting the scheme, the electronic equipment supports the conversion of various forms of privacy notice and consent (text and voice) into a structured form for storage, and has the advantages of simple data structure and small occupied storage space.

In one possible design, querying the historical data according to the first biometric information of the first user to obtain the first record includes: determining a user Identity (ID) of a first user according to first biological characteristic information of the first user; the historical data is queried according to the user ID to obtain a first record. By adopting the scheme, the electronic equipment associates various historical records of the same user through the randomly distributed user identity ID, so that the management and the query are facilitated.

In one possible design, after the historical data is queried according to the first biological characteristic information of the first user to obtain the first record, and when the first record indicates an agreement state, the step of sending the first prompt to the first user is skipped, and the first user is directly interacted to realize the first service. By adopting the scheme, the electronic equipment is prevented from repeatedly prompting privacy notification to the user, the operation flow is simplified, the operation time is saved, and the user experience is improved.

In a second aspect, an embodiment of the present application provides a service interaction apparatus, including:

the system comprises an output unit, a first service processing unit and a second service processing unit, wherein the output unit is used for sending a first prompt to a first user when the first user is in a perception range of electronic equipment, the first prompt is used for consulting whether first personal data of the first user are allowed to be acquired or not, and the first personal data are personal data required for realizing a first service;

the electronic equipment comprises an input unit, a first display unit and a second display unit, wherein the input unit is used for receiving a first instruction which is used for indicating whether the electronic equipment is allowed to acquire first personal data of a first user;

the processing unit is used for collecting the first personal data of the first user when the first indication is used for indicating that the electronic equipment is allowed to collect the first personal data of the first user, and responding to the first user according to the first personal data.

In one possible design, the output unit, before issuing the first prompt to the first user, is further configured to:

receiving a first request sent by the first user, wherein the first request is used for requesting the first user to interact with the electronic equipment to realize a first service;

alternatively, the first and second electrodes may be,

issuing the first request to the first user.

In one possible design, the first user is a user with an age less than an age threshold, the second user is a user with an age greater than or equal to the age threshold, the first prompt is further used to prompt the first user to solicit consent from the second user, and the first indication includes a voice, a limb action, a gesture, or an expression of the second user.

In a possible design, when the processing unit interacts with the first user to implement the first service, the input unit is further configured to receive a first request sent by a third user, where the first request is used to request the first user to interact with the electronic device to implement the first service;

the output unit is further used for sending a second prompt to the third user, and the second prompt is used for consulting whether the third user is allowed to acquire the first personal data;

the input unit is further used for receiving a second instruction sent by the third user, and the second instruction is used for indicating whether the electronic equipment is allowed to acquire the first personal data of the third user;

the processing unit is further configured to respond to the third user according to the second indication.

In a possible design, when responding to the third user according to the second indication, the processing unit interacts with the first user and the third user to implement the first service when the second indication allows the electronic device to collect the first personal data of the third user, and interacts with the first user to implement the first service when the second indication does not allow the electronic device to collect the first personal data of the third user, and masks the third user.

In a possible design, when the processing unit interacts with the first user to implement the first service, the input unit is further configured to receive a second request sent by the first user, where the second request is used to interact with the electronic device to implement a second service;

the output unit is further configured to send a second prompt to the first user according to the second request, where the second prompt is used to consult whether the acquisition of second personal data of the first user is allowed;

the input unit is further used for receiving a third instruction, wherein the third instruction is used for indicating whether the electronic equipment is allowed to acquire second personal data of the first user;

the processing unit is further configured to respond to the first user according to the third indication.

In a possible design, the first personal data of the first user is at least one, the second personal data of the first user is at least one, and the first personal data and the second personal data have intersection.

In a possible design, the processing unit, after receiving the first indication by the input unit, is further configured to refuse to provide the first service to the first user when the first indication does not allow the electronic device to collect the first personal data of the first user.

In one possible design, the first response includes a voice, a limb action, a gesture, or an expression of the first user.

In one possible design, the first personal data of the first user includes biometric information and/or non-biometric information of the first user, and the non-biometric information includes semantics of a voice uttered by the first user.

In a possible design, before the output unit sends a first prompt to the first user when the first user is in a sensing range of the electronic device, the processing unit is further configured to obtain sensing information including first biometric information of the first user, parse the sensing information to obtain first biometric information of the first user, query historical data according to the first biometric information of the first user to obtain a first record, where the first record is used to indicate an agreement state, a disagreement state, or an empty state, where the agreement state indicates that the electronic device is allowed to collect the first personal data of the first user, where the disagreement state indicates that the electronic device is not allowed to collect the first personal data of the first user, and where the empty state indicates that the electronic device is not aware of whether the first personal data of the first user can be collected, the first personal data is first personal data required for the electronic device to interact with the first user for a first service, and it is determined that the first record indicates the empty state.

In a possible design, after the input unit receives the first instruction, the processing unit is further configured to generate a second record according to the first instruction, the first service, the user identifier of the first user, the first biometric information, and the first personal data, and replace the first record with the second record to update the historical data.

In a possible design, the processing unit is configured to determine a user ID of the first user according to the first biometric information of the first user, and query the historical data according to the user ID to obtain the first record.

In a possible design, after querying the historical data according to the first biometric information of the first user to obtain a first record, the processing unit is further configured to skip the step of sending the first prompt to the first user when the first record indicates an agreement state, and directly interact with the first user to implement the first service.

In a third aspect, an embodiment of the present application provides an electronic device, including: a processor, a memory, and a computer program stored on the memory and executable on the processor, the processor executing the program when performing the method as described above in the first aspect or in the various possible implementations of the first aspect.

In a fourth aspect, embodiments of the present application provide a computer program product containing instructions, which when run on an electronic device, cause the electronic device computer to perform the method of the first aspect or the various possible implementations of the first aspect.

In a fifth aspect, embodiments of the present application provide a computer-readable storage medium, which stores instructions that, when executed on an electronic device, cause the electronic device to perform the above-mentioned first aspect or the method in each possible implementation manner of the first aspect.

In a sixth aspect, an embodiment of the present application provides a chip system, where the chip system includes a processor and may further include a memory, and is configured to implement the functions of the electronic device in the foregoing method. The chip system may be formed by a chip, and may also include a chip and other discrete devices.

According to the service interaction method and device provided by the embodiment of the application, after the electronic equipment receives a first request sent by a first user, a first prompt is sent to the first user according to the first request so as to inquire whether the first user is allowed to acquire first personal data of the first user. Then, the electronic device receives a first indication for indicating whether the electronic device is allowed to collect the first personal data of the first user, and when the first indication is used for indicating that the electronic device is allowed to collect the first personal data of the first user, the first personal data of the first user is collected, and the electronic device responds to the first user according to the first personal data. In the process, the first user embeds privacy notification and consent management into a business process in the process of interacting with the electronic equipment, so that flexible and complete privacy notification, consent management and the like are realized, and the user can use the business provided by the electronic equipment without accessing a management end. Moreover, when the first user is a user needing to be monitored, such as a child, a special flow is used for the child, namely the personal data of the first user is acquired after the consent of the guardian is solicited, and the use requirement of the child and the requirement of privacy protection are met.

Drawings

FIG. 1 is a schematic diagram of a common privacy notification interface;

fig. 2A is a system architecture diagram used in the service interaction method provided in the embodiment of the present application;

fig. 2B is another system architecture diagram used in the service interaction method provided in the embodiment of the present application;

fig. 3 is a schematic frame diagram of a service interaction apparatus for executing a service interaction method provided by an embodiment of the present application;

FIG. 4 is a flowchart of a service interaction method provided in an embodiment of the present application;

FIG. 5 is a collaboration flow diagram of various modules included in the business layer of FIG. 3;

fig. 6A is a flowchart of a voice interaction between a first user and a robot according to a service interaction method provided in an embodiment of the present application;

FIG. 6B is a flow chart of a business interaction method corresponding to FIG. 6A;

fig. 7A is a flowchart of a voice interaction between a first user and a robot according to a service interaction method provided in an embodiment of the present application;

FIG. 7B is a flow chart of a business interaction method corresponding to FIG. 7A;

fig. 8A is a flowchart of a voice interaction between a child and a robot according to a service interaction method provided in an embodiment of the present application;

FIG. 8B is a flow chart of a business interaction method corresponding to FIG. 8A;

fig. 9A is a flowchart of a voice interaction between a newly added user and a robot in a service interaction method provided in an embodiment of the present application;

FIG. 9B is a flow chart of a business interaction method corresponding to FIG. 9A;

fig. 10 is a schematic structural diagram of a message display device according to an embodiment of the present application;

fig. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present application;

fig. 12 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present application.

Detailed Description

The autonomous system can cope with non-programmed or non-preset situations, has certain self-management and self-guiding capabilities, and comprises an automatic driving system, a robot, an unmanned aerial vehicle and the like. At present, the autonomous system is rapidly developed, and the input and output of multiple modes such as voice, vision and the like become trends. Modality refers to the source or form of information, including touch, hearing, vision, smell, etc.; alternatively, the source of the information includes various sensors such as radar, infrared, accelerometer, etc., and the form of the information includes voice, video, text, etc. Multimodal input and output provides great convenience to users. Taking a robot as an example, in the interaction process between the robot and the user, the robot may interact with the user by using voice, and the robot may also acquire the facial expression of the user and interact with the user according to the facial expression of the user. Meanwhile, the business process of the autonomous system also develops from a fixed process designed manually to an automatic autonomous decision-making direction. The automatic autonomous decision-making means that the autonomous system can automatically make a decision on the next action according to the feedback of the user. The interaction between the autonomous system and the user necessarily involves collecting the biometric information of the user. Taking the robot as an example, if the user interacts with the robot through voice, the robot must be allowed to open a microphone to collect voice information of the user; for another example, if a user interacts with the robot using facial expressions, the robot must be allowed to turn on a camera to collect face information of the user, where the face information, voice information, and the like are biometric information of the user.

Generally speaking, if the electronic device does not remind the user to collect the biometric information of the user, the privacy protection requirement is violated; alternatively, even if the electronic device reminds the user that biometric information needs to be collected, the biometric information is collected without the user's consent, which may also violate the privacy protection requirements. In order to avoid violating the privacy protection requirement, some personal terminal devices, such as mobile phones, tablet computers and the like, can obtain the consent of the user at one time before using the terminal device to provide services. For example, a cell phone loads a "mirror" Application (APP), which needs to acquire an image of the user. After the user installs mirror APP, when opening mirror APP for the first time, the cell-phone pops out a dialog box, and the suggestion user this APP need open the camera and acquire the image to the demonstration agrees the button and cancels the button. When the user agrees, then click and agree the button, afterwards, mirror APP can call the camera and acquire user's image.

Some electronic devices that possess autonomous system, such as robot etc. usually bind with the APP on the management end, and the administrator sets up electronic device through the APP on the management end, and the management end can be cell-phone, flat board etc.. And when the autonomous system needs to collect the biological characteristic information, triggering the management terminal to carry out privacy notification and consent management. For example, a privacy notice is popped up at a management side of the autonomous system, and consent management is performed according to an input of a user. For example, referring to fig. 1, fig. 1 is a schematic diagram of a common privacy notification interface. Please refer to fig. 1, the electronic device is a smart audio amplifier, the smart audio amplifier is bound with the APP on the administrator's mobile phone, and when the smart audio amplifier collects the voice and other biological feature information of the user, the APP interface of the management terminal is triggered to display the privacy notification, and the consent button and the cancel button are displayed. And then, judging whether the user agrees to acquire the biological characteristic information according to the selection of the user.

According to the above, it can be seen that: for an electronic device, such as a robot, the above-mentioned privacy notification and consent management can only be performed for a user having an access right of a management side, and for a user without an access right of a management side, the electronic device cannot perform the privacy notification and consent management. For example, the electronic device is a home service robot, the family member has a management-side access right, and the guest does not have the management-side access right, and when the guest is present in the service range of the home service robot, the home service robot cannot perform privacy notification and consent management. As another example, the electronic device is a home service robot, and only the parent has the administration side access right, and it is a child who actually interacts with the home service robot, and at this time, the setting subject and the use subject of the privacy notification and the consent management are separated.

In addition, to complete the above related operations of privacy notification and consent management, the APP of the management end, such as the management APP on the mobile phone, must be accessed, that is, the operation can only be executed when the user opens the management end APP for configuration. Usually, the electronic device notifies the administrator of the possible scenes through the APP on the management side at one time, however, it may be difficult for the administrator to associate each of the privacy notifications with the actual application scene. For example, the privacy notification of the APP at the management side includes "allow face information to be collected in the game". However, in an actual scene, the user is involved in collecting the face information when playing games such as a hobbyist, and the face information does not need to be collected when the user plays games such as finger recognition. That is, the privacy notification corresponds to the service scene which is not easy to be cleaned, the privacy notification is difficult to describe clearly every detailed scene in one-time notification, and the user is difficult to have a clear concept for the actual scene of each privacy notification.

In addition to the above privacy notification and consent management through APP, there is also a privacy notification and consent management scheme that fixedly embeds the privacy notification at the beginning of the business process. For example, when a call is connected every time, the telephone customer service system notifies the user through voice that the next call will be recorded; for another example, a robot may support game play that requires the acquisition of images of a user, and upon starting the game, the robot asks the user whether the user agrees to acquire images of the user. However, this solution can only perform privacy notification in a fixed link, and is only applicable to an autonomous system with a fixed flow, and is difficult to be applied to an autonomous system without a fixed flow. Moreover, the scheme adopts the same flow for all users, the users cannot be distinguished as children or adults, different flows cannot be adopted for the children and the adults, privacy protection requires that privacy notification and consent management of the children are implemented by a guardian, namely whether the biological characteristic information such as images, voice and the like of the children can be acquired or not is confirmed by the guardian. In addition, the scheme can repeatedly perform privacy notification and consent management every time when the fixed link section of the privacy notification is executed, regardless of whether the autonomous system previously acquires the consent of the user. This is inefficient and poor experience for users who frequently and repeatedly use autonomous systems. For example, the user has agreed that the robot acquires his image in the game of the hobbyist, but the robot asks the user whether to allow the user to acquire his image again the next time the game of the hobbyist is played, because the flow of the autonomous system requires that the user is asked whether he agrees to acquire the image as soon as the game of the hobbyist is started.

In view of this, embodiments of the present application provide a service interaction method and an electronic device, where the electronic device sends a privacy notification to a user in a sensing range of the electronic device through voice and performs consent management on the user, so that any user in the sensing range of the electronic device can use a service provided by the electronic device without accessing a management end.

First, terms related to embodiments of the present application are explained.

And the privacy notification means that the electronic equipment sends a prompt to prompt the user to acquire personal data of the user, such as face information, voiceprint and the like.

And (4) agreeing to management, namely judging whether the user agrees to collect the personal data of the user by the electronic equipment. For example, the electronic device displays a privacy notification on the display screen, and displays an "agreement button" and a "cancel" button on the display screen, and determines whether the user agrees to collect personal data according to the selection of the user. For another example, the electronic device recognizes the user's body movement, gesture, voice, etc., and determines whether the user agrees to collect personal data according to the recognition result.

Personal data including biological feature information and non-biological features, wherein the biological features refer to face information of a user, voiceprints contained in voice information and the like; the non-biological characteristic information refers to semantics contained in the voice uttered by the user, such as a name and a relationship map contained in the voice uttered by the user.

A first user, a user currently interacting with the electronic device.

A second user, a user with an age greater than or equal to an age threshold, for example, 10 years old.

And the third user is a user added in the midway during the interaction process between the first user and the electronic equipment.

Second, the system architecture.

Fig. 2A is a system architecture diagram used in the service interaction method according to the embodiment of the present application. Referring to fig. 2A, the system architecture includes an electronic device, such as a smart robot, a smart speaker, and the like, which is illustrated as a smart robot. The electronic equipment is loaded with an autonomous system and has the environment perception capability, the AI capability, the decision-making capability and the like. In the aspect of hardware, the electronic equipment is composed of hardware such as a processor, a microphone, a camera, a display screen, a loudspeaker, a sensor and various communication interfaces, and sensing information and the like are acquired by utilizing the hardware. In the aspect of software, the electronic equipment has AI (artificial intelligence) capability and decision-making capability, wherein the AI capability comprises voiceprint recognition, semantic recognition, natural language generation, face recognition, action recognition and the like; the business decision capability comprises operation processing, storage and the like.

Fig. 2B is another system architecture diagram used in the service interaction method provided in the embodiment of the present application. Referring to fig. 2B, the system architecture includes an electronic device, such as an intelligent robot, an intelligent speaker, and the like, and a server, in which the intelligent robot is shown. The electronic equipment loads an autonomous system, has environment sensing capability and AI capability, and the server has decision making capability, and provides background processing capability including operation processing, storage and the like for application programs running on the electronic equipment. In addition, the electronic device can only have the environment perception capability, and the server has both the AI capability and the decision-making capability.

Finally, a core framework.

Fig. 3 is a schematic frame diagram of a service interaction apparatus for executing the service interaction method provided in the embodiment of the present application. Referring to fig. 3, the basic framework of the service interaction device is divided into three parts, namely a hardware layer, an AI capability layer and a service layer, which are described in detail below.

First, hardware layer.

In the embodiment of the present application, the hardware layer is located in the electronic device, and the hardware layer mainly includes an input device and an output device, for example, see table 1.

TABLE 1

In addition to the enumerated devices, the electronic device may be equipped with a greater variety of input devices and output devices. For example, an electronic device having a limb, such as a robot, may also communicate information to the user through limb movements.

Second, AI capability layer

In the embodiment of the application, the AI capability layer may be located in the electronic device or in the server, and includes modules such as voiceprint recognition, face recognition, semantic recognition, natural language generation, and action recognition, which are called by the service layer to provide capability support for the service layer.

And thirdly, a service layer.

In the embodiment of the application, the service layer can be located on the electronic device side or the server side, and the service layer triggers the electronic device to send out the privacy notification by calling the AI capability layer and the like, and performs consent management on the user according to the user response.

In the following, taking an example that a hardware layer, an AI capability layer, and a service layer are all integrated on an electronic device, a service interaction method described in the embodiment of the present application is described in detail, for example, refer to fig. 4.

Fig. 4 is a flowchart of a service interaction method provided in an embodiment of the present application, where an execution main body of the embodiment is an electronic device, and the embodiment includes:

101. when a first user is in a perception range of electronic equipment, a first prompt is sent to the first user.

Wherein the first prompt is used to consult whether the first user's first personal data is allowed to be collected, the first personal data being personal data required to implement a first business.

For example, when the first user enters the sensing range of the electronic device, the first request may be initiated actively by the first user or the first request may be initiated actively by the electronic device. For example, after a first user enters a sensing range of the electronic device, such as a voice sensing range, wakes up the electronic device, and then sends a first request to the electronic device in a voice manner; for another example, after the first user enters the visual perception range of the electronic device, a first request is sent to the electronic device through gestures, actions, expressions or the like; for another example, after recognizing that the first user enters the sensing range, the electronic device actively initiates a first request to the first user in a voice mode; for another example, after recognizing that the first user enters the sensing range, the electronic device actively initiates a first request to the first user through the display screen.

And then, the electronic equipment sends a first prompt to the first user through a voice mode or a display screen, wherein the prompt is used for inquiring whether the first user is allowed to acquire first personal data of the first user, such as a voiceprint, a human face of the first user or semantics in the voice sent by the first user.

102. Receiving a first indication indicating whether the electronic device is allowed to collect first personal data of the first user.

For example, when the electronic device determines that the first user is a user with an age greater than the age threshold, the electronic device sends a first instruction to the electronic device by means of voice, gesture, motion, or the like after receiving the first prompt.

For another example, when the electronic device determines that the first user is a user whose age is smaller than the age threshold, the first prompt sent by the electronic device is also used to prompt the first user to request the consent of the second user, and the second user sends the first instruction to the electronic device through voice, gesture, motion, and other manners. Wherein the first user is for example a child under the age of 10 and the second user is for example a guardian of the child. When the first user is a child, the electronic device sends 'you are good, your parents can ask him to go next', then when the electronic device captures the voice or the face of the second user, the electronic device sends 'asking for how you are guardians of children', the second user says 'yes, i.e. if you need to collect the face information and the voice information of the children', and asks for whether the people can go.

103. When the first indication is used for indicating that the electronic equipment is allowed to collect the first personal data of the first user, collecting the first personal data of the first user;

104. responding to the first user according to the first personal data.

Illustratively, when the first indication allows the electronic device to collect the first personal data of the first user, then interacting with the first user to implement the first service; and when the first indication does not allow the electronic equipment to collect the first personal data of the first user, refusing to provide the first service for the first user.

According to the service interaction method provided by the embodiment of the application, after receiving a first request sent by a first user, the electronic device sends a first prompt to the first user according to the first request so as to consult whether the first user is allowed to acquire first personal data of the first user. Then, the electronic device receives a first indication for indicating whether the electronic device is allowed to collect the first personal data of the first user, and when the first indication is used for indicating that the electronic device is allowed to collect the first personal data of the first user, the first personal data of the first user is collected, and the electronic device responds to the first user according to the first personal data. In the process, the first user embeds privacy notification and consent management into a business process in the process of interacting with the electronic equipment, so that flexible and complete privacy notification, consent management and the like are realized, and the user can use the business provided by the electronic equipment without accessing a management end. Moreover, when the first user is a user needing to be monitored, such as a child, a special flow is used for the child, namely the personal data of the first user is acquired after the consent of the guardian is solicited, and the use requirement of the child and the requirement of privacy protection are met.

In the above embodiment, when the first service is a service with higher security, for example, a shopping APP is loaded on the electronic device, the APP corresponding to the first service is the shopping APP, and the APP binds to the bank card of the bank card user a. In order to avoid using the bank card of the user A under the condition that the first user is not allowed by the user A, when the first indication allows the electronic equipment to collect the first personal data of the first user, the first personal data of the first user is collected, the identity of the first user is authenticated, and only when the first user is the user A, the first service is provided for the first user; otherwise, even if the first indication is used for indicating that the electronic equipment is allowed to collect the first personal data of the first user, the electronic equipment refuses to provide the first service for the first user if the authentication fails. In addition, the embodiment of the application does not limit the sequence of the identity authentication and the privacy notification, namely the identity authentication can be performed first, and the privacy notification is performed when the identity authentication is passed; or, firstly, carrying out privacy notification and carrying out identity authentication under the condition of permission. It should be noted that the authentication is not a necessary step, and for example, when the first service is picture book reading, a wood game, etc., the authentication is not required.

In the above embodiment, when the electronic device interacts with the first service of the first user, a situation that the third user joins, that is, a situation that a person joins halfway, is likely to occur. At this time, the electronic device issues a second prompt to the third user, the second prompt being used to consult whether the first personal data of the third user is allowed to be collected. Then, the electronic equipment receives a second instruction sent by a third user, wherein the second instruction is used for indicating whether the electronic equipment is allowed to acquire the first personal data of the third user; and responding to the third user according to the second indication. For example, when the electronic device interacts with the first user for a first service, and a third user wants to join the first service, the third user actively sends a voice of "i want to join, can" to the electronic device, and after receiving the voice, the electronic device pauses the first service and sends a second prompt to the third user; for another example, when the electronic device interacts with the first user for the first service, and finds a third user, the first service is suspended, and a second prompt is sent to the third user.

When the second indication allows the electronic device to collect the first personal data of the third user, for example, the third user makes an OK gesture, a nodding action, or agrees, etc., the electronic device continues to interact with the first user and the third user to implement the first service; when the second indication does not allow the electronic device to acquire the first personal data of the third user, for example, the third user makes a hand-shaking motion, a head-shaking motion, or disagreement, the electronic device interacts with the first user to implement the first service and masks the third user, that is, even if the third user appears in a sensing range of the electronic device, the electronic device automatically desensitizes or masks the third user, ignores the presence of the third user, and avoids acquiring the first personal data of the third user.

By adopting the scheme, the electronic equipment achieves the purposes of privacy notification and consent management to a third user, namely a user joining in the midway.

In the above embodiment, the services that can be provided by one electronic device are various, and after a first user interacts with the electronic device to perform a first service, it is likely that the first user wants to switch to a second service, and second personal data required by the second service is likely to be different from first personal data required by the first service. In order to realize service switching, the electronic device receives a second request sent by the first user, wherein the second request is used for interacting with the electronic device to realize a second service, and sends a second prompt to the first user according to the second request, wherein the second prompt is used for consulting whether the acquisition of second personal data of the first user is allowed or not, and then the electronic device receives a third indication sent by the first user, and the third indication is used for indicating whether the acquisition of the second personal data of the first user is allowed or not; responding to the first user according to the third indication.

For example, when the first service is book drawing and reading, the second personal data is voice information of the first user, and when the second service is a wood-head game, the second personal data is face information of the first user. When the first user does not want to read the picture book, the electronic device is sent out that the user plays the wooden game bar, and at the moment, the electronic device judges that the first user needs to be switched to the wooden game from the picture book reading mode, and the wooden game needs to acquire the face information of the first user. The electronic equipment queries historical data according to the voice information of the first user, and sends a prompt to the first user when the face information of the first user does not exist in the historical data, for example, "people need to collect the face information when a game of a wooden person is played"; and when the face information of the first user exists in the historical data, directly switching to the wooden game and providing the wooden game for the first user.

By adopting the scheme, the electronic equipment only judges the personal data related to the stage before the business execution of each stage, if the first user does not agree to collect the personal data, the business of the stage can only be influenced, other businesses of the electronic equipment used by the first user cannot be influenced, and the user experience is good.

In the above embodiment, the number of the first personal data of the first user is at least one, the number of the second personal data of the first user is at least one, and the first personal data and the second personal data have an intersection. That is, a plurality of personal data are needed by one business model, and the electronic device can perform privacy notification and consent management on the user with the business as granularity. For example, the first service is a wooden figure game, which needs to collect face information and voice information, and the second service is drawing book reading, which needs to collect voice information. When a record exists in the historical data, the record shows that the first user agrees that the electronic equipment collects the face information and the voice information of the first user in the wooden head game mode, but whether the record of the electronic equipment for collecting the voice information of the first user is empty is judged in the picture book reading mode. Then, when the first user switches from the wood person game to the book drawing reading, the electronic device needs to prompt the user to: in the picture book reading mode, the electronic device needs to collect voice information of the first user. And only after the agreement of the first user is solicited, the electronic equipment collects the voice information of the first user and provides the first user with the picture book reading service.

By adopting the scheme, the electronic equipment can perform privacy notification and consent management on the user by taking the service as granularity.

In the above embodiment, before the electronic device sends a first prompt to a first user according to a first request, obtaining perception information including first biometric information of the first user, parsing the perception information to obtain the first biometric information of the first user, querying historical data according to the first biometric information of the first user to obtain a first record, where the first record is used to indicate an approval state, a disapproval state, or a null state, where the approval state indicates that the electronic device is allowed to collect first personal data of the first user, the disapproval state indicates that the electronic device is not allowed to collect the first personal data of the first user, the null state indicates that the electronic device does not know whether the first personal data of the first user can be collected, and the first personal data is first personal data required for the electronic device to interact with the first user for a first service, determining that the first record indicates the empty state.

Illustratively, the input device of the hardware layer of the electronic device continuously senses the surrounding environment to detect whether a user appears in the sensing range, and when the first user is sensed to enter the sensing range, the sensing information is generated. For example, a mugger game includes a preparation phase for identifying users who participate in the game and a game phase for playing the game. A preparation stage, capturing an image in a visual field by a camera of the electronic equipment, and generating perception information containing the image of a first user after finding that the first user enters the visual field; or after the microphone of the electronic device captures the sound of the first user, the microphone generates perception information containing the voice of the first user. In the game stage, because the game is set, when a new user appears in the sensing range, the electronic equipment automatically shields the new user.

Then, the AI capability layer of the electronic device parses the perception information to obtain first biometric information, such as face information, voice information, and the like, and queries the historical data according to the first biometric information of the first user to obtain a first record. The first record is used for indicating an agreement state, a disagreement state or an empty state, the agreement state indicates that the electronic equipment is allowed to collect first personal data of the first user, the disagreement state indicates that the electronic equipment is not allowed to collect the first personal data of the first user, the empty state indicates that the electronic equipment does not know whether the first personal data of the first user can be collected, and the first personal data is personal data required by the electronic equipment for interacting with the first user for the first service. The first personal data may be second biometric information or a non-biometric feature of the first user, and when the first personal data is the second biometric information, the second biometric information may be the same as or different from the first biometric information, or when the second biometric information is plural, the first biometric information is one of the plural second biometric information.

And then, when the electronic equipment determines that the first record is in an empty state, sending a first prompt.

In addition, when the first record indicates the consent status, it indicates that the electronic device has been allowed to collect the first personal data of the first user before executing step 101, in this step, the electronic device does not need to issue the first prompt to determine whether it is allowed to collect the first personal data of the first user, but collects the first personal data of the first user, and provides the first service to the first user.

When the first record indicates a disagreement status, it indicates that the electronic device has been denied collecting the first personal data of the first user before performing step 101. In this step, the electronic device automatically desensitizes the first user or automatically skips a process of collecting the first personal data of the first user, so as to refuse to provide the first service to the first user. For example, the electronic device issues "you can't use the first service, please reset if you need to use the first service". For another example, in the mugger game mode, the electronic device sends a first prompt to the first user when the face information needs to be collected, and when the first user disagrees with the collected face information, the electronic device records the disagreement state. Subsequently, the first user sends a prompt of 'playing the wooden person game' to the electronic device or the electronic device actively sends a prompt of 'whether the wooden person game needs to be played' to the first user after detecting that the first user enters a sensing range, and then the electronic device sends a first prompt to the first user to prompt the first user that the face information needs to be collected when playing the wooden person game. Then, when the first user agrees, the electronic equipment modifies the historical data; otherwise, the disagreement state is continuously maintained.

When the first record indicates an empty status, it indicates that there is no record in the history data that allows the electronic device to collect the first personal data of the first user before performing step 101. At this time, the electronic device needs to issue a first request to be allowed to collect the first personal data of the first user. For example, assuming that the electronic device is a robot, a currently-developed service, that is, a first service is a wooden game, first biological characteristic information is voice information, first personal data is face information, and a first user has used a lion book reading service provided by the robot before playing the wooden game, and agrees that the robot acquires voice information of the first user and stores the voice information in historical data. After the robot analyzes the voice information of the first user, the voice information is used for inquiring historical data to obtain a first record, and the first record indicates that: the first user agrees that the robot collects voice information of the first user in the lion king picture book reading mode, but the electronic equipment collects face information of the first user in the wooden head game mode and the face information is null. That is, the robot does not know whether it is allowed to acquire the face information of the first user in the muzzle game mode.

By adopting the scheme, the electronic equipment sends privacy notice to the user in the sensing range of the electronic equipment through voice and the like and performs consent management on the user, so that any user in the sensing range of the electronic equipment can use the service provided by the electronic equipment without accessing a management terminal. Meanwhile, when the electronic equipment needs to collect the first personal data of the first user, the historical consent state of the first user is firstly inquired, so that repeated privacy notification and consent management of the consented users can be avoided.

In the above embodiment, the electronic device generates the second record according to the first indication, the first service, the user identifier of the first user, the first biometric information, and the first personal data, and replaces the updated historical data of the first record with the second record.

By adopting the scheme, the electronic equipment supports the conversion of various forms of privacy notice and consent (text and voice) into a structured form for storage, and has the advantages of simple data structure and small occupied storage space.

In the above embodiments, the electronic device has a hardware layer, an AI capability layer, and a service layer, which are the keys for making service decisions, and may have various implementation manners. The service layer is explained in detail below.

Referring to fig. 3 again, in the embodiment of the present application, the service layer may include a service decision module, an identity recognition module, a personal data determination module, an agreement management module, and a personal data notification and agreement acquisition module.

The business decision module is a general scheduling module and has the capabilities of semantic recognition, limb action recognition or gesture recognition and the like. And the business decision module calls other software modules to jointly make a decision to realize business according to the interaction between the user and the electronic equipment, and in the calling process, calls other software modules to cooperatively finish privacy notification and consent management.

The identity recognition module calls the capability of the AI capability layer, recognizes the identity of the user according to the face and/or voiceprint and the like of the user by using an identity recognition technology, and judges whether the user is an adult or a child. The identification of the user refers to randomly assigning a user Identity (ID) to each user.

And the personal data judging module inquires personal data designed by the business mode according to the business mode. Different services provided by the electronic equipment correspond to different service modes. In a common manner, a look-up table is used, for example, see table 2.

TABLE 2

Business mode	Collected personal data
		English learning	Phonetic information, … …
Drawing book reading	Phonetic information, … …
		Game machine	Speech information, face information … …
……	……

The agreement management module supports fine-grained agreement management and supports indexing/storing of agreement information of a user according to multiple dimensions of the user, a business mode (such as a chat scene and a game scene), a type of personal data to be collected (such as face information and voice information) and the like. Meanwhile, various forms of consent (such as voice, actions such as nodding and shaking the head) and the like are converted into a structured form index and stored in a database, wherein the structured form index comprises the identity (i.e. user ID) of a user, age group (children or adults), business mode (english learning, book drawing and reading, games and the like), personal data type (voice information, face information and the like), and exemplarily, the table 3 can be referred to.

TABLE 3

The personal data notification and consent acquisition module generates various forms of output such as voice, animation and the like of the personal data notification to be output by calling the AI capability layer capability, the specific form is determined according to the output capability of the hardware layer, and finally the personal data notification and consent acquisition module outputs the personal data notification through the generation equipment such as a loudspeaker, a display screen and the like. Then, the opinions returned by the user, such as agreement, cancellation, etc., are obtained.

In the following, how the respective modules in the business layer cooperate will be described in detail. Illustratively, referring to fig. 5, fig. 5 is a collaboration flow diagram of the modules included in the business layer of fig. 3, including:

201. the identity recognition module recognizes the identity of the user.

For example, the identity recognition module determines the identity of the first user from historical data according to the voiceprint of the first user analyzed by the AI capability layer; for another example, the identity recognition module determines the identity of the first user from the historical data according to the face information of the first user analyzed by the AI capability layer. The identity of the first user refers to a user ID randomly allocated to the first user by the electronic equipment when the first user uses the electronic equipment for the first time.

Meanwhile, the identity recognition module can also recognize the age bracket of the first user by utilizing voiceprint/face information and the like, for example, the first user is an adult or a child.

202. And the identity identification module sends the identified identity ID of the new user to the service decision module.

When the user changes, the identity recognition module recognizes the identity of the new user and sends the new user identity to the service decision module. Wherein, the user changing means to replace the user, add a person in the middle, and the like. For example, a first user interacts with an electronic device to perform a first service, the first service requires the electronic device to acquire face information of the first user, after a period of time, the first user leaves a sensing range of a camera of the electronic device, a new user appears in the sensing range of the camera, and the phenomenon is called a midway person changing; for another example, a first user interacts with the electronic device to perform a first service, the first service requires the electronic device to acquire face information of the first user, and after a period of time, a third user appears in a sensing range of the camera, which is called adding people in midway.

In addition, the identity identification module can also send the age bracket and the like of the identified new user to the service decision module.

203. The service decision module determines whether the service mode or the user is changed, if any one of the service mode or the user is changed, step 204 is executed; when neither the traffic pattern nor the user has changed, step 220 is performed.

Illustratively, the service decision module determines whether to switch from the first service to the second service according to the recognition result of the voice/visual information of the AI capability layer and the interaction condition with the user. For example, during the process of the first user interacting with the electronic device to perform the book drawing and reading business, the voice of "do not want to read the book, and we play the wooden game bar" is emitted.

204. And the business decision module sends the business mode to the personal data judgment module.

Illustratively, when the service decision module determines that a switch from the first service to the second service is required, the new service mode, that is, the second service, is sent to the personal data determination module.

In addition, even if the current service mode is not changed, when the user changes, that is, a person is changed or a person is added in the middle of the process, the service decision module needs to send the current service, that is, the first service to the personal data judgment module.

205. The personal data judging module determines the type of the personal data related to the business mode through table lookup.

206. And the personal data judgment module returns the query result to the service decision module.

207. And the business decision module sends a query request to the agreement management module.

Illustratively, the service decision module carries the identity ID, age group, service mode, type of personal data, and the like of the user in the query information and sends the query information to the agreement management module.

208. The consent management module queries the historical data to obtain a first record.

209. And the agreement management module sends the state indicated by the first record to the service decision module.

Illustratively, after receiving information such as the user identity, the age group, the service module, the type of the personal data, and the like, the consent management module obtains the state indicated by the first record by looking up a table, and sends the state indicated by the first record to the service decision module.

210. The business decision module determines the status of the first record indication, and if the first record indication agrees with the status indication, step 218 is executed; when the first record indicates an empty status, step 211 is performed.

Illustratively, when the historical data shows that the user agrees to the electronic equipment to collect the personal data of the user, the service corresponding to the service mode is directly provided for the user; if it is not recorded in the history data whether the user agrees to the electronic device to collect the personal data of the user, 211 is executed.

211. And the business decision module sends a personal data acquisition request to the personal data notification and agreement acquisition module.

Illustratively, the business decision module sends the user identity, age group, business mode, type of personal data, etc. to the personal data notification and consent acquisition module, instructs the personal data notification and consent acquisition module to make privacy notification and obtain consent to the user, and waits for a response to be returned.

212. The personal data notification and consent acquisition module asks the user whether consent is present.

Illustratively, after receiving the user identity, age group, business mode and type of personal data, the personal data notification and consent acquisition module composes a first request presented in a voice manner and/or an animation manner through the AI capability layer and plays the first request to the user.

213. And the privacy notice and the consent acquisition module receive and store the response of the user.

Illustratively, the privacy notification and acquisition module stores the audio/video file of the privacy notification acquisition process after receiving the response from the user.

214. And the personal data notification and agreement acquisition module sends the response of the user and the saving path of the process file to the service decision module.

215. The business decision module determines whether the user agrees to collect the personal data, if the user agrees to collect the personal data, step 216 is executed, and if the user disagrees to collect the personal data, step 219 is executed.

216. The business decision module will send an indication to the consent management module before performing step 218.

The indication information comprises user identity, age group, service mode, type of personal data, state of response indication, saving path of process file and the like, the indication agreement management module generates a second record according to the information, and updates the first record in the historical data by using the second record. After the indication message is issued, step 218 is performed without waiting for a response.

217. The consent management module updates the historical data.

Illustratively, the historical data is updated after receiving the identity, age group, business mode and type of personal data, agreement state, saving path of process file and the like sent by the business decision module.

218. And the service decision module interacts with the user to complete the service.

Illustratively, the business decision module agrees to enter a new business mode when the user agrees to enter the new business mode; or, accepting the new user and interacting with the current user (including the new user) to complete the service; or, when the user has not changed and the service mode is changed, the current service is provided to the current user.

219. The service decision module refuses to provide the service to the user.

Illustratively, the business decision module refuses to develop a new business or refuse to join a new user when the user does not agree to collect personal data.

220. The traffic decision module maintains the current state.

Next, based on fig. 5, the service interaction method according to the embodiment of the present application will be described in detail by taking an electronic device as a robot as an example. For example, see fig. 6A-9B.

Fig. 6A is a flowchart of a voice interaction between a first user and a robot according to a service interaction method provided in an embodiment of the present application, and fig. 6B is a flowchart of a service interaction method corresponding to fig. 6A. In this implementation, the first service is a wooden-head game, and when the robot interacts with the first user for the first service, the face information and the voice information of the first user need to be collected. And before the first user requests the first service to the robot, the first service is never used or the use record is deleted, and whether the first user agrees to collect the record of the face information and the voice information does not exist in the historical data stored by the robot. The embodiment comprises the following steps:

301. the identification module identifies the first user as 1111 and adult.

And after the user awakens the robot through voice, an interactive instruction is sent to the robot.

For example, the first user says "we play a mugger game bar" to the robot, and after the robot senses the sensed information including the voice, the sensed information is analyzed to obtain first personal data of the first user, such as voice information, and based on the voice information, the history data is queried to find that the first personal data does not exist in the history data, and then an identity ID such as "1111" is randomly allocated to the first user. In addition, the robot can also recognize the age bracket of the first user, such as an adult, according to the voice.

For another example, the first user makes a thumb gesture with respect to the robot, the thumb gesture corresponding to a muzzle game. The robot captures the face of a first user by using a camera, after perception information containing the face is obtained, the link information is analyzed, first personal data of the first user is obtained, such as face information, historical data is inquired based on the face information, the first personal data does not exist in the historical data, and then an identity ID '1111' is randomly allocated to the first user. In addition, the robot can also identify the age bracket of the first user, such as an adult, according to the face information.

As another example, the first user makes a thumb gesture with respect to the robot, the thumb gesture corresponding to a muzzle game. The robot captures the face of a first user by using a camera, after perception information containing the face is obtained, the link information is analyzed to obtain first personal data of the first user, such as face information, historical data is inquired based on the face information, the first personal data is found to exist in the historical data, and then the robot finds a user ID corresponding to the first personal data to be '1111' by looking up a table. In addition, the robot can also identify the age bracket of the first user, such as an adult, according to the face information.

302. And the identity recognition module sends the recognition result to the service decision module.

The identification result indicates that the user ID of the first user is 1111 and the age group is adult.

303. And the business decision module judges that the first business is a wooden head game.

304. And the business decision module sends the business mode of the wooden person game to the personal data judgment module.

305. The personal data judgment module inquires the face information and the voice information of the first user, which need to be acquired in the wooden figure game.

Illustratively, the personal data judgment module stores a correspondence between the business model and the type of the personal data. Therefore, the personal data judging module can look up a table according to the business mode, so that the type of the personal data required by the wooden person game, such as face information and voice information, can be determined. For example, see table 4.

TABLE 4

Business mode	Type of personal data
		Wooden figure game	Face information and voice information

306. The personal data judging module returns the types of the personal data required by the wooden head game to the business decision module, namely face information and voice information.

307. The business decision module sends the muzzle game business pattern, the adult, the user ID "1111" and the type of personal data to the consent management module.

308. The consent management module queries the historical data for a first record, the first record indicating an empty status.

The consent management module can query the consent status by means of table lookup and the like to obtain a query result. For example, the consent management module looks up a table to determine whether there is a record in the history data that the robot has been granted to collect face information and voice information of an adult with a user ID "1111" in the mugger game business mode. For example, see table 5.

TABLE 5

From table 5, it can be seen that: the first record in the history data indicates that the robot does not know whether face information and voice information of an adult with a user ID of 1111 can be collected in the mugger game mode.

309. And the agreement management module sends the inquiry result to the service decision module.

310. And the business decision module sends indication information to the personal data notification and agreement acquisition module.

Illustratively, the business decision module carries the contents of the user ID, the adult and wooden business model, the personal data type of face information and voice information in the indication information, and sends the indication information to the personal data notification and consent acquisition module, and the indication personal data notification and consent acquisition module sends a first request to a first user with the user ID of 1111, where the first request is used for requesting to acquire the face information and the biological information of the first user.

311. The personal data notification and consent acquisition module issues a first request to a first user.

For example, is the personal data notification and consent acquisition module synthesized speech through the capabilities of the AI capability layer, etc. "i want to collect your speech information and face information to play a mugger game, ask if can? ", and play the voice through the speaker; and then waits for a reply from the first user.

For another example, the personal data notification and consent acquisition module generates an animation having "i want to collect your voice information and face information to play a wooden game, ask for if can? "and the animation is played through a display screen of the hardware layer, and then the reply of the first user is waited.

312. The personal data notification and consent acquisition module identifies a first response.

For example, the first user says "agree/OK, etc", or the user nods to indicate agreement, or the user makes an OK gesture, the input device of the hardware layer collects the information, and the personal data notification and agreement acquisition module invokes the capability of the AI capability layer to parse the information, thereby recognizing a first response indicating that the robot is allowed to collect the voice information and the face information of the first user.

It should be noted that the robot may record/record the privacy notification and consent acquisition process of steps 311 and 312 for the sake of documentation.

313. And the biological information characteristic notification and agreement acquisition module sends the identification result to the service decision module.

Illustratively, after the bioinformatics feature notification and consent acquisition module recognizes that the first user with user ID 1111 agrees to collect the face information and the voice information, the consent status, the recording privacy notification, and the record/video file link agreeing to the acquisition process are returned to the business decision module.

314. The business decision module sends an indication to the consent management module, after which the business decision module executes 314.

Wherein the indication information comprises user ID "1111", adult, mugger game, consent to collect face information and voice information, link of video/audio file, etc. to indicate the consent status in the consent management module history data. After the indication information is issued, step 316 is executed without waiting for a response;

315. the consent management module updates the historical data.

And after the agreement management module receives the indication information sent by the service decision module, updating the state in the historical data. For example, see table 6.

TABLE 6

316. And the business decision module enters a manned game mode.

In addition, in the above embodiment, when the first response made by the first user indicates that the first user does not agree with the acquisition of the face information and the voice information in step 312, the personal data notification and agreement acquisition module invokes the capability of the AI capability layer, so that the hardware layer sends out "sorry, voice that cannot play a wood game with you", and the like, and after sending the first response to the business decision module, the business decision module sends the indication information to the agreement management module, so that the agreement management module updates the first record.

Fig. 7A is a flowchart of a voice interaction between a first user and a robot according to a service interaction method provided in an embodiment of the present application, and fig. 7B is a flowchart of a service interaction method corresponding to fig. 7A. The present embodiment differs from the embodiment shown in fig. 6A and 6B described above in that: in this embodiment, when a record in the historical data indicates that the first user interacts with the robot for the first service, the robot is operated to acquire face information and voice information of the first user. That is, the first user interacts with the robot, performs the flow of the embodiment shown in fig. 6A and 6B, and exits, and then requests the robot to enter the muzzle game mode again. The embodiment includes steps 401 to 410, wherein the related descriptions of step 401 to step 407 can refer to step 301 to step 307 in fig. 6B, and the following mainly describes step 408 to step 410 in detail.

408. The consent management module queries the historical data for a first record indicating the consent status.

The consent management module can query the consent status by means of table lookup and the like to obtain a query result. For example, the consent management module may look up a table to determine whether the history data stores face information and voice information for the muzzle game business pattern, adult, user ID "1111". For exemplary purposes, see table 7.

TABLE 7

From table 7, it can be seen that: there is a record in the history indicating that in the mugger game mode, the first user with user ID 1111 has agreed to the robot to collect his facial information and voice information.

409. And the agreement management module sends the inquiry result to the service decision module.

410. And the business decision module enters a wooden person game mode.

Fig. 8A is a flowchart of a voice interaction between a child and a robot according to a service interaction method provided in an embodiment of the present application, and fig. 8B is a flowchart of a service interaction method corresponding to fig. 8A. The present embodiment differs from the embodiment shown in fig. 6A and 6B described above in that: in this embodiment, the first user is a child (for example, the age does not exceed 10 years), and there is no record in the history data as to whether the robot is allowed to acquire the face information and the voice information of the first user. The embodiment comprises the following steps:

501. the identification module identifies the first user as 2222 and child age group.

502. And the identity recognition module sends the recognition result to the service decision module.

The identification result indicates that the user ID of the first user is 2222 and the age group is child.

503. And the business decision module judges that the first business is a wooden head game.

504. The business decision module can send the business mode of the wooden person game to the personal data judgment module.

505. The personal data judgment module inquires the face information and the voice information of the first user, which need to be acquired in the wooden figure game.

506. The personal data judgment module returns the personal data required by the wooden head game to the business decision module as face information and voice information.

507. The business decision module sends the muzzle game business pattern, the child, the user ID "2222", and the type of personal data to the consent management module.

508. The consent management module queries the historical data for a first record, the first record indicating an empty status.

The consent management module can query the consent status by means of table lookup and the like to obtain a query result. For example, the consent management module looks up a table to determine whether there is a record in the history data that the robot consents to collect the face information and voice information of the child with the user ID of "2222" in the wooden game business mode. For exemplary purposes, see table 7.

TABLE 7

From table 7, it can be seen that: the first record in the history data indicates that in the mugger game mode, the robot does not know whether the face information and voice information of the child with the user ID 222 can be collected.

509. And the agreement management module sends the inquiry result to the service decision module.

510. And the business decision module sends indication information to the personal data notification and agreement acquisition module.

Illustratively, the business decision module carries the content of the user ID 2222, the child, the wooden head business model, the personal data type of the face information, the voice information and the like in the indication information and sends the indication information to the personal data notification and consent acquisition module, and the indication personal data notification and consent acquisition module sends a first request, wherein the first request is used for guiding the first user to seek the help of the second user.

511. The personal data notification and consent acquisition module triggers a speaker of the hardware layer to emit a voice of "do you, how well your parent is, please get it.

512. The personal data notification and consent acquisition module identifies that an adult enters a perception scope.

513. The personal data notification and consent acquisition module triggers a loudspeaker of the hardware layer to send out a voice of 'hello ask you how you are guardians of children'.

After the personal data notification and consent acquisition module identifies that an adult enters a perception range through the capability of the AI capability layer, the personal data notification and consent acquisition module actively confirms whether the adult is a guardian of a child or not. For example, the business decision module invokes the capabilities of the AI capabilities layer, synthesizes the speech, and triggers the speaker of the hardware layer to play the speech of "you are asking you how well you are for the guardian of a child". If the personal data notification and consent acquisition module identifies that the adult says "yes, i am, kids' guardian" or adult nodding to indicate consent, then step 514 is performed; otherwise, the children are guided to seek help of adults continuously, and after a period of time, if no adults come, the game is ended.

514. The personal data notification and consent acquisition module triggers a loudspeaker of the hardware layer to send out a voice prompt of ' I needs to acquire voice information and face information of children and ask for possible ' or not '.

For example, an adult says "agree/OK, etc", or the user nods to indicate agreement, or makes an OK gesture, the input device of the hardware layer collects the information, and the personal data notification and agreement acquisition module invokes the capability of the AI capability layer to parse the information, thereby recognizing a first response indicating that the robot is allowed to collect the voice information and the face information of the first user. For another example, an adult says "voice can be collected but the relationship map cannot be preserved", the personal data notification and consent acquisition module recognizes the first response, and synthesizes "good click, i now starts chatting with children but my memory ratio is worse, i fails to remember the content of the chat" by calling the AI capability layer capability, and the voice is played by a speaker of the hardware layer.

515. And the biological information characteristic notification and agreement acquisition module sends the identification result to the service decision module.

Illustratively, after the bioinformatics feature notification and consent acquisition module identifies that the adult allows the robot to collect facial information and voice information of the child, the consent status, the recorded privacy notification, and the audio/video file link of the consent acquisition process are returned to the business decision module.

It should be noted that the robot may record/record the privacy notification and consent acquisition process of steps 511 and 514 for the sake of being well documented.

516. The business decision module sends an indication to the consent management module, after which the business decision module executes 518.

Wherein the indication information comprises user ID '2222', children, wooden games, links for agreeing to collect face information and voice information, video/audio files, etc. to indicate the first record in the historical data of the agreement management module. After the indication information is issued, step 518 is executed without waiting for a response;

517. the consent management module updates the historical data.

And after the agreement management module receives the indication information sent by the service decision module, updating the state in the historical data. For example, please refer to table 8.

TABLE 8

518. And the business decision module enters a manned game mode.

Fig. 9A is a flowchart of a voice interaction between a newly added user and a robot in a service interaction method provided in an embodiment of the present application, and fig. 9B is a flowchart of a service interaction method corresponding to fig. 9A. The present embodiment differs from the embodiment shown in fig. 6A and 6B described above in that: in this embodiment, in the process that the first user and the robot interactively play the muzzle game, a new user, that is, a third user appears in the sensing range of the robot. For the robot, privacy notification and consent management are required for the third user. The embodiment comprises the following steps:

601. the business decision module and the first user interactively play the wooden head game.

602. The identity module detects that a third user is present within the sensing range.

603. The identification module identifies the third user as 3333 and the adult.

604. And the identity recognition module sends the recognition result to the service decision module.

The identification result indicates that the user ID of the first user is 3333 and the age group is adult.

605. And the business decision module finds that a new user joins, and suspends the wooden person game.

606. And the business decision module sends the business mode of the wooden person game to the personal data judgment module.

607. The personal data judgment module inquires the face information and the voice information of a third user required to be collected in the wooden figure game.

608. The personal data judgment module returns the personal data required for developing the wooden head game to the business decision module as face information and voice information.

609. The business decision module sends the muzzle game business model, the adult, the user ID "3333", and the type of personal data to the consent management module.

610. The consent management module queries the historical data for a first record, the first record indicating an empty status.

611. And the agreement management module sends the inquiry result to the service decision module.

612. And the business decision module sends indication information to the personal data notification and agreement acquisition module.

Illustratively, the business decision module carries the contents of the user ID "3333", the adult, the wooden person business model, the personal data type being face information and voice information, etc. in the indication information, and sends the indication information to the personal data notification and consent acquisition module, and the indication personal data notification and consent acquisition module sends a first request to a third user with the user ID 3333, where the first request is used for requesting that the third user is allowed to acquire the face information and the biological information.

613. The personal data notification and consent acquisition module issues a first request to a third user.

614. The personal data notification and consent acquisition module identifies a second response.

For example, the third user says "disagree/disallow/NO, etc.," or the user shakes his head to indicate disagreement, the input device of the hardware layer collects the information, and the personal data notification and consent acquisition module invokes the capability of the AI capability layer to parse the information, thereby recognizing a second response indicating that the robot is not allowed to collect the voice information and the face information of the third user.

It should be noted that the robot may record/record the privacy notification and consent acquisition process of steps 612 and 613 for the sake of documentation.

615. And the biological information characteristic notification and agreement acquisition module sends the identification result to the service decision module.

Illustratively, after the biological information feature notification and consent acquisition module recognizes that the third user part with the user ID 333 agrees to collect the face information and the voice information, the agreement state is returned to the business decision module.

616. The traffic decision module sends an indication to the consent management module, after which the traffic decision module executes 618.

Wherein the indication information comprises user ID '3333', adult, wooden game, disagreement collection face information and voice information, etc. to indicate the record in the history data of the consent management module. After the indication information is issued, step 618 is executed without waiting for a response;

617. the consent management module updates the historical data.

And after the agreement management module receives the indication information sent by the service decision module, updating the record in the historical data. For example, see table 9.

TABLE 9

618. Continuing to play the muzzle game with the first user.

Illustratively, the business decision module continues to play a wood-head game with the first user, and during the game, even if a third user appears in the perception range of the robot, the robot automatically desensitizes or shields the third user, ignores the presence of the third user, and avoids collecting personal data of the third user.

Fig. 10 is a schematic structural diagram of a message display device according to an embodiment of the present application. The display device 100 of the message may be implemented by means of software and/or hardware. As shown in fig. 10, the display device 100 of the message includes:

the output unit 11 is configured to send a first prompt to a first user when the first user is in a sensing range of an electronic device, where the first prompt is used to consult whether to allow collection of first personal data of the first user, where the first personal data is personal data required for implementing a first service;

an input unit 12, configured to receive a first indication, where the first indication is used to indicate whether the electronic device is allowed to collect first personal data of the first user;

the processing unit 13 is configured to, when the first indication indicates that the electronic device is allowed to collect the first personal data of the first user, and respond to the first user according to the first personal data.

In a possible design, the output unit 11, before issuing the first prompt to the first user, is further configured to:

alternatively, the first and second electrodes may be,

issuing the first request to the first user.

In a possible design, when the processing unit 13 interacts with the first user to implement the first service, the input unit 12 is further configured to receive a first request sent by a third user, where the first request is used to request the first user to interact with the electronic device to implement the first service;

the output unit 11 is further configured to issue a second prompt to the third user, where the second prompt is used to consult whether the third user's first personal data is allowed to be collected;

the input unit 12 is further configured to receive a second indication sent by the third user, where the second indication is used to indicate whether the electronic device is allowed to acquire the first personal data of the third user;

the processing unit 13 is further configured to respond to the third user according to the second indication.

In a possible design, the processing unit 13, after the input unit 12 receives the first indication, is further configured to interact with the first user to implement the first service and shield the third user when the second indication does not allow the electronic device to collect the first personal data of the third user.

In a possible design, when the processing unit 13 interacts with the first user to implement the first service, the input unit 12 is further configured to receive a second request sent by the first user, where the second request is used to interact with the electronic device to implement a second service; the output unit 11 is further configured to send a second prompt to the first user according to the second request, where the second prompt is used to consult whether the acquisition of the second personal data of the first user is allowed;

the input unit 12 is further configured to receive a third indication, where the third indication is used to indicate whether the electronic device is allowed to acquire second personal data of the first user; the processing unit 13 is further configured to respond to the first user according to the third indication.

In a possible design, the processing unit 13, when responding to the first user according to the first indication, interacts with the first user to implement a first service when the first indication allows the electronic device to collect the first personal data of the first user, and refuses to provide the first service to the first user when the first indication does not allow the electronic device to collect the first personal data of the first user.

In a possible design, before the output unit 11 sends a first prompt to the first user when the first user is in a sensing range of the electronic device, the processing unit 13 is further configured to obtain sensing information including first biometric information of the first user, parse the sensing information to obtain first biometric information of the first user, query historical data according to the first biometric information of the first user to obtain a first record, where the first record is used to indicate an approval state, a disapproval state, or a null state, where the approval state indicates that the electronic device is allowed to collect the first personal data of the first user, where the disapproval state indicates that the electronic device is not allowed to collect the first personal data of the first user, and where the null state indicates that the electronic device is not aware of whether the first personal data of the first user can be collected, the first personal data is first personal data required for the electronic device to interact with the first user for a first service, and it is determined that the first record indicates the empty state.

In a possible design, the processing unit 13, after the input unit 12 receives the first indication, is further configured to generate a second record according to the first indication, the first service, the user identifier of the first user, the first biometric information, and the first personal data, and replace the first record with the second record to update the historical data.

In a possible design, the processing unit 13 is configured to determine a user ID of the first user according to the first biometric information of the first user, and query the historical data according to the user ID to obtain the first record.

In a possible design, the processing unit 13, after querying the historical data according to the first biometric information of the first user to obtain a first record, is further configured to skip the step of sending the first prompt to the first user when the first record indicates an agreement state, and directly interact with the first user to implement the first service.

The service interaction device provided in the embodiment of the present application may perform the actions of the electronic device in the above embodiments, and the implementation principle and the technical effect are similar, which are not described herein again.

It should be noted that the above input unit may be a sound pickup device, such as a microphone, or a display screen; the output unit may be a speaker or the like, or may be a display screen. In addition, for the electronic device capable of making limb movement, the output unit may also output limb movement, and the processing unit may be implemented in a form called by software through a processing element; or may be implemented in hardware. For example, the processing unit may be a processing element separately set up, or may be implemented by being integrated into a chip of the apparatus, or may be stored in a memory of the apparatus in the form of program code, and a function of the processing unit may be called and executed by a processing element of the apparatus. In addition, all or part of the units can be integrated together or can be independently realized. The processing element described herein may be an integrated circuit having signal processing capabilities. In implementation, the steps of the method or the units above may be implemented by hardware integrated logic circuits in a processor element or instructions in software.

For example, the above units may be one or more integrated circuits configured to implement the above methods, such as: one or more Application Specific Integrated Circuits (ASICs), or one or more microprocessors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs), etc. For another example, when some of the above units are implemented in the form of a processing element scheduler code, the processing element may be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor that can call the program code. As another example, these units may be integrated together and implemented in the form of a system-on-a-chip (SOC).

Fig. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present application, and as shown in fig. 11, the electronic device 200 includes:

a processor 21 and a memory 22;

the memory 22 stores computer-executable instructions;

the processor 21 executes computer-executable instructions stored by the memory 22, causing the processor 21 to perform a method as applied to an electronic device.

Optionally, referring to fig. 11 again, the electronic device 200 further includes a communication interface 23. The processor 21, the memory 22, and the communication interface 23 may be connected by a bus 24.

The embodiment of the present application further provides a readable storage medium, in which computer execution instructions are stored, and when the computer execution instructions are executed by a processor, the computer execution instructions are used to implement the service interaction method executed by the electronic device.

The embodiment of the present application further provides a computer program product, which is used for implementing a service interaction method executed by an electronic device when the computer program product runs on the electronic device.

Fig. 12 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present application. As shown in fig. 12, electronic device 300 includes, but is not limited to: radio frequency unit 31, network module 32, audio output unit 33, input unit 34, sensor 35, display unit 36, user input unit 37, interface unit 38, memory 39, processor 340, power supply 341, and the like. Those skilled in the art will appreciate that the electronic device configuration shown in fig. 12 does not constitute a limitation of the electronic device, and that electronic device 300 may include more or fewer components than shown, or some components may be combined, or a different arrangement of components. In the embodiment of the present application, the electronic device includes, but is not limited to, a home robot, a smart speaker, and other interactive devices.

A user input unit 37, configured to receive an input from a user; a display unit 36 for displaying contents according to an input in response to an input received by the user input unit 37.

It should be understood that, in the embodiment of the present application, the radio frequency unit 31 may be used for receiving and transmitting signals during information transmission and reception or during a call. Typically, the radio frequency unit 31 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like. In addition, the radio frequency unit 31 can also communicate with a network and other devices through a wireless communication system.

The electronic device 300 provides the user with wireless broadband internet access via the network module 32, such as helping the user to inquire about music, weather, and the like.

The audio output unit 33 may convert audio data received by the radio frequency unit 31 or the network module 32 or stored in the memory 39 into an audio signal and output as sound. Also, the audio output unit 33 may also provide audio output related to a specific function performed by the electronic apparatus 300 (e.g., a call signal reception sound, a message reception sound, etc.). The audio output unit 33 includes a speaker, a buzzer, a receiver, and the like.

The input unit 34 is for receiving an audio or video signal. The input Unit 34 may include a Graphics Processing Unit (GPU) 343 and a microphone 342, and the Graphics Processing Unit 343 is used to process image data of a picture or video captured by a camera or the like. The processed image frames may be displayed on the display unit 36. The image frames processed by the graphic processor 343 may be stored in the memory 39 (or other readable storage medium) or transmitted via the radio unit 31 or the network module 32. The microphone 342 may receive sound and may be capable of processing such sound into audio data. The processed audio data may be converted into a format output transmittable to a mobile communication base station via the radio frequency unit 31 in case of the phone call mode.

The electronic device 300 also includes at least one sensor 35, such as a light sensor, motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor that adjusts the brightness of the display panel 363 according to the brightness of ambient light, and a proximity sensor that turns off the display panel 363 and/or the backlight when the electronic device 300 moves to the ear. As one type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in each direction (generally three axes), detect the magnitude and direction of gravity when stationary, and can be used to identify the posture of an electronic device (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), and vibration identification related functions (such as pedometer, tapping); the sensors 35 may also include a fingerprint sensor, a pressure sensor, an iris sensor, a molecular sensor, a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, etc., which are not described in detail herein.

The display unit 36 is used to display information input by the user or information provided to the user. The Display unit 36 may include a Display panel 363, and the Display panel 363 may be configured in the form of a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), or the like.

The user input unit 37 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the electronic device. Specifically, the user input unit 37 includes a touch panel 373 and other input devices 372. The touch panel 373, also referred to as a touch screen, may collect touch operations by a user on or near the touch panel 373 (e.g., operations by a user on or near the touch panel 373 using a finger, a stylus, or any other suitable object or attachment). The touch panel 373 may include two parts of a touch detection device and a touch controller. The touch detection device detects the touch direction of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch sensing device, converts the touch information into touch point coordinates, sends the touch point coordinates to the processor 340, receives a command from the processor 340, and executes the command. In addition, the touch panel 373 may be implemented by various types such as resistive, capacitive, infrared, and surface acoustic wave. The user input unit 37 may include other input devices 372 in addition to the touch panel 373. In particular, the other input devices 372 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which are not described herein again.

Further, the touch panel 373 can be overlaid on the display panel 363, and when the touch panel 373 detects a touch operation on or near the touch panel 373, the touch panel 373 is transmitted to the processor 340 to determine the type of the touch event, and then the processor 340 provides a corresponding visual output on the display panel 363 according to the type of the touch event. Although in fig. 12, the touch panel 373 and the display panel 363 are two independent components to implement the input and output functions of the electronic device, in some embodiments, the touch panel 373 and the display panel 363 may be integrated to implement the input and output functions of the electronic device, and this is not limited herein.

The interface unit 38 is an interface for connecting an external device to the electronic apparatus 300. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device having an identification module, an audio input/output (I/O) port, a video I/O port, an earphone port, and the like. The interface unit 38 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the electronic apparatus 300 or may be used to transmit data between the electronic apparatus 300 and the external device.

The memory 39 may be used to store software programs as well as various data. The memory 39 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required for at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the cellular phone, and the like. Further, the memory 39 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.

The processor 340 is a control center of the electronic device, connects various parts of the whole electronic device by using various interfaces and lines, performs various functions of the electronic device and processes data by operating or executing software programs and/or modules stored in the memory 39 and calling data stored in the memory 39, thereby performing overall monitoring of the electronic device. Processor 340 may include one or more processing units; optionally, the processor 340 may integrate an application processor and a modem processor, wherein the application processor mainly handles operating systems, user interfaces, application programs, and the like, and the modem processor mainly handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into processor 340.

Referring to fig. 12, in the embodiment of the present application, a memory 39 stores a computer program, wherein the processor 340 runs the computer program to enable the electronic device to execute the service interaction method.

In the embodiments of the present application, the processor may be a general-purpose processor, a digital signal processor, an application specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and may implement or execute the methods, steps, and logic blocks disclosed in the embodiments of the present application. A general purpose processor may be a microprocessor or any conventional processor or the like. The steps of a method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware processor, or may be implemented by a combination of hardware and software modules in a processor.

In the embodiment of the present application, the memory may be a non-volatile memory, such as a Hard Disk Drive (HDD) or a solid-state drive (SSD), and may also be a volatile memory (e.g., a random-access memory (RAM)). The memory is any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to such. The memory in the embodiments of the present application may also be circuitry or any other device capable of performing a storage function for storing program instructions and/or data.

The methods provided by the embodiments of the present application may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the application to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, a network appliance, a user device, or other programmable apparatus. The computer instructions may be stored on a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website, computer, server, or data center to another website, computer, server, or data center via wire (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., Digital Video Disk (DVD)), or a semiconductor medium (e.g., SSD), among others.

It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.

Claims

1. A service interaction method is characterized by comprising the following steps:

when a first user is in the perception range of electronic equipment, sending a first prompt to the first user, wherein the first prompt is used for consulting whether the first user is allowed to collect first personal data of the first user, and the first personal data is personal data required for realizing a first service;

receiving a first indication indicating whether the electronic device is allowed to collect first personal data of the first user;

when the first indication is used for indicating that the electronic equipment is allowed to collect the first personal data of the first user, collecting the first personal data of the first user;

responding to the first user according to the first personal data.

2. The method of claim 1, wherein prior to issuing the first prompt to the first user, further comprising:

alternatively, the first and second electrodes may be,

issuing the first request to the first user.

3. The method of claim 1, wherein the first user is a user with an age less than an age threshold, wherein the second user is a user with an age greater than or equal to the age threshold, wherein the first prompt is further configured to prompt the first user to solicit consent from the second user, and wherein the first indication comprises a voice, a limb action, a gesture, or an expression of the second user.

4. The method of claim 1, further comprising:

when interacting with the first user to realize the first service, receiving a first request sent by a third user, wherein the first request is used for requesting the first user to interact with the electronic equipment to realize the first service;

sending a second prompt to the third user, the second prompt being used to consult whether the third user's first personal data is allowed to be collected;

receiving a second indication sent by the third user, wherein the second indication is used for indicating whether the electronic equipment is allowed to acquire the first personal data of the third user;

responding to the third user according to the second indication.

5. The method of claim 4, wherein responding to the third user according to the second indication comprises:

when the second indication allows the electronic equipment to collect the first personal data of the third user, interacting with the first user and the third user to realize a first service;

and when the second indication does not allow the electronic equipment to collect the first personal data of the third user, interacting with the first user to realize a first service and shielding the third user.

6. The method of claim 1, further comprising:

when interacting with the first user to realize the first service, receiving a second request sent by the first user, wherein the second request is used for interacting with the electronic equipment to realize a second service;

sending a second prompt to the first user according to the second request, wherein the second prompt is used for consulting whether the second personal data of the first user is allowed to be acquired or not;

receiving a third indication indicating whether the electronic device is allowed to collect second personal data of the first user;

responding to the first user according to the third indication.

7. The method of claim 6,

the first personal data of the first user is at least one, the second personal data of the first user is at least one, and the first personal data and the second personal data have intersection.

8. The method of claim 1, wherein after receiving the first indication, further comprising:

and when the first indication does not allow the electronic equipment to collect the first personal data of the first user, refusing to provide the first service for the first user.

9. The method of any one of claims 1, 3-8, wherein the first response comprises a voice, a limb action, a gesture, or an expression of the first user.

10. The method of any one of claims 1 to 9, wherein the first personal data of the first user comprises biometric information and/or non-biometric information of the first user, and the non-biometric information comprises semantics of a voice uttered by the first user.

11. A service interaction device, comprising:

and the processing unit is used for acquiring the first personal data of the first user when the first indication is used for indicating that the electronic equipment is allowed to acquire the first personal data of the first user, and responding to the first user according to the first personal data.

12. The apparatus of claim 11, wherein the output unit, prior to issuing the first prompt to the first user, is further configured to:

alternatively, the first and second electrodes may be,

issuing the first request to the first user.

13. The apparatus of claim 11, wherein the first user is a user with an age less than an age threshold, wherein the second user is a user with an age greater than or equal to the age threshold, wherein the first prompt is further configured to prompt the first user to solicit consent from the second user, and wherein the first indication comprises a voice, a limb action, a gesture, or an expression of the second user.

14. The apparatus of claim 11,

when the processing unit interacts with the first user to realize the first service, the input unit is further configured to receive a first request sent by a third user, where the first request is used to request the first user to interact with the electronic device to realize the first service;

15. The apparatus of claim 14,

the processing unit, when responding to the third user according to the second indication, interacts with the first user and the third user to implement the first service when the second indication allows the electronic device to collect the first personal data of the third user, and interacts with the first user to implement the first service when the second indication does not allow the electronic device to collect the first personal data of the third user, and shields the third user.

16. The apparatus of claim 11,

when the processing unit interacts with the first user to realize the first service, the input unit is further configured to receive a second request sent by the first user, where the second request is used to interact with the electronic device to realize a second service;

17. The apparatus of claim 16, wherein the first user has at least one first personal data item and the first user has at least one second personal data item, and wherein the first personal data item intersects the second personal data item.

18. The apparatus of claim 11,

the processing unit is further configured to, after the input unit receives the first indication, refuse to provide the first service to the first user when the first indication does not allow the electronic device to collect the first personal data of the first user.

19. The device of any of claims 11, 13-18, wherein the first response comprises a voice, a limb action, a gesture, or an expression of the first user.

20. The apparatus of any of claims 11-19, wherein the first personal data of the first user comprises biometric information and/or non-biometric information of the first user, and wherein the non-biometric information comprises semantics of speech uttered by the first user.

21. An electronic device comprising a memory and a processor, and a computer program stored on the memory and executable on the processor, the processor when executing the program performing the method of any of claims 1 to 10.

22. A readable storage medium having stored therein instructions, which when run on an electronic device, cause the electronic device to perform the method of any of claims 1-10 above.