WO2022134556A1

WO2022134556A1 - Online interaction method and server

Info

Publication number: WO2022134556A1
Application number: PCT/CN2021/106914
Authority: WO
Inventors: 刘洋
Original assignee: 北京达佳互联信息技术有限公司
Priority date: 2020-12-25
Filing date: 2021-07-16
Publication date: 2022-06-30
Also published as: CN114697686A; CN114697686B

Abstract

The present disclosure relates to the technical field of the Internet, and relates to an online interaction method and apparatus, and a server and a storage medium. The method comprises: in response to a trigger operation of a user for a mobile terminal on the basis of a current online activity, acquiring surrounding environment information; detecting whether the surrounding environment information includes a valid target object; when the surrounding environment information includes the valid target object, acquiring an object operation set corresponding to the mobile terminal; determining, on the basis of the valid target object and the object operation set, whether the trigger operation is a valid trigger operation; and when the trigger operation is a valid trigger operation, executing an online interaction instruction corresponding to the trigger operation in the current online activity.

Description

Online interactive method and server

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on the Chinese patent application with the application number of 202011565378.8 and the filing date of December 25, 2020, and claims the priority of the Chinese patent application. The entire content of the Chinese patent application is incorporated herein by reference.

technical field

The present disclosure relates to the field of Internet technologies, and in particular, to an online interaction method, an online interaction device, a server and a storage medium.

Background technique

With the development of Internet technology, the emerging industry of webcasting has emerged. Webcasting absorbs and continues the advantages of the Internet, and uses video to carry out online live broadcasts, such as live shopping, live performances, online training, online surveys, etc. During the webcasting process, users watching the livestreaming can participate in the livestreaming interactive activities in real time.

In the related art, when a user watching a live broadcast wants to participate in a live broadcast interactive activity, it is generally necessary to manually click a relevant area or a relevant button on the live broadcast interface to trigger a corresponding function.

SUMMARY OF THE INVENTION

The present disclosure provides an online interaction method, device, server and storage medium. The technical solutions of the present disclosure are as follows:

According to a first aspect of the embodiments of the present disclosure, an online interaction method is provided, including:

In response to the user's triggering operation on the mobile terminal based on the current online activity, obtain the surrounding environment information;

Detecting whether the surrounding environment information contains a valid target object;

In the case that the effective target object is included in the surrounding environment information, obtain an object operation set corresponding to the mobile terminal;

determining whether the trigger operation is a valid trigger operation based on the valid target object and the object operation set;

In the case that the triggering operation is a valid triggering operation, the online interactive instruction corresponding to the triggering operation in the current online activity is executed.

In an exemplary embodiment, in response to the user's triggering operation on the mobile terminal based on the current online activity, acquiring the surrounding environment information includes:

During the valid time of the current online activity, in the case of detecting the triggering operation of the mobile terminal by the user, the image acquisition device of the mobile terminal is called to collect the surrounding environment information.

In an exemplary embodiment, the detecting whether the surrounding environment information contains a valid target object includes:

Analyzing the surrounding environment information to determine whether the surrounding environment information contains living objects;

If the surrounding environment information includes a living object, determining the face orientation of the living object;

In the case that the face orientation of the living object satisfies the preset face orientation condition, determining that the surrounding environment information includes a candidate target object;

The valid target object is determined from the candidate target objects.

In an exemplary embodiment, the determining the valid target object from the candidate target objects includes:

Identifying one or more of the facial expression of the candidate target object, the distance between the candidate target object and the mobile terminal, and the target that the candidate target object focuses on with both eyes;

The effective target object is determined from the candidate target objects according to the recognition result.

In an exemplary embodiment, the method further includes:

In the case that the surrounding environment information does not include a living object, determining that the surrounding environment information does not include the effective target object;

In the case that the face orientation of the living object does not satisfy the preset face orientation condition, it is determined that the effective target object is not included in the surrounding environment information.

In an exemplary embodiment, the object operation set includes facial information of each preset object and a set of customary operation items corresponding to each preset object;

The determining whether the trigger operation is a valid trigger operation based on the valid target object and the object operation set includes:

Obtain the facial information of the effective target object, and match the facial information of the effective target object with the facial information of each preset object in the object operation set to obtain a matching object; the matching object is in the object operation set. , the preset object corresponding to the facial information matching the facial information of the effective target object;

Matching the trigger operation with the habitual operation items in the habitual operation item set of the matching object to obtain an operation item matching result;

According to the operation item matching result, it is determined whether the trigger operation is a valid trigger operation.

In an exemplary embodiment, the determining whether the trigger operation is a valid trigger operation according to the operation item matching result includes:

When the operation item matching result is that there is a habitual operation item matching the trigger operation in the habitual operation item set of the matching object, it is determined that the trigger operation is a valid trigger operation.

In an exemplary embodiment, the method further includes pre-generating the set of object operations, and the pre-generating the set of object operations includes:

For each preset object operating the mobile terminal, obtain face information of each preset object;

Record at least one habitual operation of each preset object when operating the mobile terminal, and generate and generate data with each preset object based on at least one habitual operation of each preset object when operating the mobile terminal. The set of custom operation items corresponding to the preset object;

The object operation set is generated based on the face information of each preset object and a set of habitual operation items corresponding to each preset object.

In an exemplary embodiment, the object operation set includes facial information of each preset object and a set of habitual operation items corresponding to each preset object, and the trigger operation includes operation parameters of the trigger operation; each The set of habitual operation items includes operation parameters corresponding to each habitual operation item; and

Obtain the facial information of the effective target object, and match the facial information of the effective target object with the facial information of each preset object in the object operation set to obtain a matching object, and the matching object is in the object operation set. , the preset object corresponding to the facial information matching the facial information of the effective target object;

Matching the operation parameter of the trigger operation with the operation parameter corresponding to each habitual operation in the habitual operation item set of the matching object to obtain an operation parameter matching result;

According to the operation parameter matching result, determine whether there is a target operation parameter in the operation parameters corresponding to each habitual operation of the matching object, and the target operation parameter and the operation parameter of the trigger operation satisfy a preset condition;

In response to the determination result being yes, it is determined that the trigger operation is a valid trigger operation.

In an exemplary embodiment, in the case that the triggering operation is a valid triggering operation, the method further includes:

Acquiring in advance the correspondence between each preset operation on the mobile terminal and each online interactive instruction in the current online activity;

In the case that the triggering operation is a valid triggering operation, executing the online interactive instruction corresponding to the triggering operation in the current online activity includes:

Matching the trigger operation with each preset operation on the mobile terminal, and determining a preset operation that matches the trigger operation;

Based on the corresponding relationship, determining a valid online interaction instruction corresponding to the preset operation matching the trigger operation in the current online activity;

Execute the effective online interactive instruction.

According to a second aspect of the embodiments of the present disclosure, an online interactive device is provided, including:

a trigger operation response unit, configured to perform a trigger operation on the mobile terminal in response to the user based on the current online activity, and obtain surrounding environment information;

a valid target object detection unit, configured to perform detection of whether the surrounding environment information contains a valid target object;

an object operation set acquisition unit, configured to acquire an object operation set corresponding to the mobile terminal when the surrounding environment information includes the effective target object;

a valid trigger operation determination unit, configured to determine whether the trigger operation is a valid trigger operation based on the valid target object and the set of object operations;

The online interactive instruction execution unit is configured to execute the online interactive instruction corresponding to the trigger operation in the current online activity when the trigger operation is a valid trigger operation.

In an exemplary embodiment, the trigger operation response unit includes:

The image acquisition device invoking unit is configured to call the image acquisition device of the mobile terminal to acquire the image acquisition device in the case of detecting the triggering operation of the mobile terminal by the user within the valid time of the current online activity. Surrounding environment information.

In an exemplary embodiment, the effective target object detection unit includes:

a living object determination unit configured to perform analysis on the surrounding environment information to determine whether the surrounding environment information contains a living object;

a face orientation determination unit configured to perform, in the case where the surrounding environment information includes a living object, determining the face orientation of the living object;

a candidate target object determination unit, configured to determine that the surrounding environment information includes a candidate target object when the face orientation of the living object satisfies a preset face orientation condition;

A valid target object determination unit configured to perform determining the valid target object from the candidate target objects.

In an exemplary embodiment, the effective target object determination unit includes:

an identification unit, configured to perform identification of one or more of the facial expression of the candidate target object, the distance between the candidate target object and the mobile terminal, and the target that the candidate target object focuses on with both eyes;

The first determining unit is configured to perform determining the valid target object from the candidate target objects according to the recognition result.

In an exemplary embodiment, the apparatus further includes:

a second determining unit configured to perform, in a case where the surrounding environment information does not include a living object, determining that the valid target object is not included in the surrounding environment information;

The third determining unit is configured to perform determining that the valid target object is not included in the surrounding environment information when the face orientation of the living object does not satisfy a preset face orientation condition.

The valid trigger operation determination unit includes:

A face information matching unit, configured to execute the acquisition of the face information of the effective target object, and match the face information of the effective target object with the face information of each preset object in the object operation set to obtain a matching object; the The matching object is the preset object corresponding to the facial information that matches the facial information of the effective target object in the object operation set;

a habitual operation item set matching unit, configured to perform matching between the trigger operation and the habitual operation items in the habitual operation item set of the matching object, and obtain an operation item matching result;

The fourth determination unit is configured to perform determining whether the trigger operation is a valid trigger operation according to the operation item matching result.

In an exemplary embodiment, the fourth determining unit includes:

a fifth determining unit, configured to determine that the triggering operation is valid in the case that there is a customary operation item matching the triggering operation in the customary operation item set of which the matching result of the operation item is the matching object Trigger action.

In an exemplary embodiment, the apparatus further includes a first generating unit, including:

a facial information acquisition unit, configured to perform, for each preset object operating the mobile terminal, acquiring the facial information of each preset object;

A habitual operation item set generating unit configured to perform recording at least one habitual operation of each preset object when operating the mobile terminal, based on at least one habitual operation of each preset object when operating the mobile terminal A habitual operation, generating a set of habitual operation items corresponding to each preset object;

The object operation set generating unit is configured to perform generating the object operation set based on the face information of each preset object and a set of customary operation items corresponding to each preset object.

In an exemplary embodiment, the object operation set includes facial information of each preset object and a set of habitual operation items corresponding to each preset object, the trigger operation includes operation parameters of the trigger operation, each The habitual operation item set includes operation parameters corresponding to each habitual operation;

The effective trigger operation determination unit further includes:

A facial information matching unit, configured to obtain the facial information of the effective target object, and to match the facial information of the effective target object with the facial information of each preset object in the object operation set to obtain a matching object, and the matching The object is the preset object corresponding to the facial information that matches the facial information of the effective target object in the object operation set;

a first matching unit, configured to perform matching between the operation parameter of the trigger operation and the operation parameter corresponding to each habitual operation in the set of habitual operation items of the matching object, to obtain an operation parameter matching result;

A judging unit, configured to perform, according to the operation parameter matching result, whether there is a target operation parameter in the operation parameters corresponding to each habitual operation of the matching object, the target operation parameter and the operation parameter of the trigger operation meet the preset conditions;

A determination unit configured to perform determining that the trigger operation is a valid trigger operation in response to the determination result being yes.

In an exemplary embodiment, the apparatus further includes:

a corresponding relationship acquiring unit, configured to perform pre-obtaining the corresponding relationship between each preset operation on the mobile terminal and each online interactive instruction in the current online activity;

The online interactive instruction execution unit includes:

a preset operation matching unit configured to perform matching the trigger operation with each preset operation on the mobile terminal, and determine a preset operation matching the trigger operation;

an online interactive instruction determining unit, configured to execute, based on the corresponding relationship, determine the valid online interactive instruction corresponding to the preset operation matching the trigger operation in the current online activity;

The first execution unit is configured to execute the effective online interactive instruction.

According to a third aspect of embodiments of the present disclosure, there is provided a server, comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to achieve the above The online interactive method described.

According to a fourth aspect of the embodiments of the present disclosure, a storage medium is provided, when the instructions in the storage medium are executed by the processor of the server, the server can execute the above-mentioned online interaction method.

According to a fifth aspect of embodiments of the present disclosure, there is provided a computer program product, the computer program product comprising a computer program, the computer program being stored in a readable storage medium, and at least one processor of a computer device from the readable storage medium The storage medium reads and executes the computer program, so that the device executes the above-mentioned online interaction method.

It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

Description of drawings

The accompanying drawings, which are incorporated into and constitute a part of this specification, illustrate embodiments consistent with the present disclosure, and together with the description, serve to explain the principles of the present disclosure and do not unduly limit the present disclosure.

FIG. 1 is a schematic diagram of an implementation environment of an online interaction method according to an exemplary embodiment.

Fig. 2 is a flowchart of an online interaction method according to an exemplary embodiment.

Fig. 3 is a flowchart of a method for determining a valid target object in surrounding environment information according to an exemplary embodiment.

Fig. 4 is a flowchart of a method for determining a valid target object from candidate target objects according to an exemplary embodiment.

Fig. 5 is a flowchart of a method for generating an object operation set corresponding to a mobile terminal according to an exemplary embodiment.

Fig. 6 is a flowchart of a method for determining the validity of a trigger operation according to an exemplary embodiment.

FIG. 7 is a schematic flowchart of judging the validity of a triggering operation of a mobile terminal according to an exemplary embodiment.

Fig. 8 is a flowchart of a method for determining a valid trigger operation based on an operation parameter according to an exemplary embodiment.

Fig. 9 is a flowchart showing a processing method for an effective trigger operation according to an exemplary embodiment.

FIG. 10 is a schematic diagram illustrating an interactive process of participating in online live shopping according to an exemplary embodiment.

FIG. 11 is a first exemplary diagram according to an exemplary embodiment.

FIG. 12 is a second exemplary diagram according to an exemplary embodiment.

FIG. 13 is a flow chart illustrating shopping interaction by operating a mobile terminal according to an exemplary embodiment.

FIG. 14 is a third exemplary diagram according to an exemplary embodiment.

FIG. 15 is a fourth example diagram according to an example embodiment.

FIG. 16 is a schematic diagram of an online interactive device according to an exemplary embodiment.

Fig. 17 is a schematic structural diagram of a server according to an exemplary embodiment.

Detailed ways

In order to make those skilled in the art better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

It should be noted that the terms "first", "second" and the like in the description and claims of the present disclosure and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific sequence or sequence. It is to be understood that the data so used may be interchanged under appropriate circumstances such that the embodiments of the disclosure described herein can be practiced in sequences other than those illustrated or described herein. The implementations described in the illustrative examples below are not intended to represent all implementations consistent with this disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as recited in the appended claims.

In the present disclosure, the surrounding environment information is obtained by responding to the user's trigger operation on the mobile terminal based on the current online activity; in the case that the surrounding environment information contains a valid target object, the object operation set corresponding to the mobile terminal is obtained; If the valid target object and the object operation set determine that the trigger operation is an operation with elimination method, the online interactive instruction corresponding to the trigger operation in the current online activity is executed. The present disclosure can directly convert the triggering operation on the mobile terminal into the corresponding online interactive instruction, so that the user can participate in the corresponding online interactive activity by directly performing the corresponding triggering operation on the mobile terminal. Participate in online interactive activities through manual click operations, and can avoid the failure of participating in real-time interaction due to complicated manual click operations, shorten the conversion process of the interaction process, improve the interaction efficiency, and improve the user experience.

Please refer to FIG. 1 , which shows a schematic diagram of an implementation environment provided by an embodiment of the present disclosure. The implementation environment may include: at least one first terminal 110 and a second terminal 120 , and the first terminal 110 and the second terminal 120 may perform operations through a network. data communication.

In some embodiments, the first terminal 110 can obtain from the second terminal 120 the correspondence between each preset operation on the first terminal 110 and each online interactive instruction in the online activity; Execute the trigger operation, the first terminal 110 can respond to the corresponding trigger operation when detecting the trigger operation, and in the circumstance that the surrounding environment information includes a valid target object and the trigger operation to the first terminal 110 is a valid trigger operation, execute the trigger operation. The online interaction command corresponding to the trigger action in the current online activity. The first terminal 110 may communicate with the second terminal 120 based on a browser/server mode (Browser/Server, B/S) or a client/server mode (Client/Server, C/S). The first terminal 110 may be a portable mobile terminal device, for example, may include: a smart phone, a tablet computer, a digital assistant, a smart wearable device, and other types of physical devices. The operating system running on the first terminal 110 in the embodiment of the present disclosure may include, but is not limited to, an Android system, an IOS system, linux, windows, and the like.

The second terminal 120 and the first terminal 110 may establish a communication connection by wire or wirelessly, and the second terminal 120 may include an independently running server, or a distributed server, or a server cluster composed of multiple servers, wherein the server may be a cloud server.

The embodiment of the present disclosure provides an online interaction method, and the execution body of the method may be the above-mentioned first terminal, and the first terminal may be a mobile terminal, which is not limited in the embodiment of the present disclosure, and the first terminal may also be Servers, processors, in-vehicle devices, mobile devices, computing devices, etc. Referring to FIG. 2, the online interaction method may include:

S210. Acquire the surrounding environment information in response to the user's triggering operation on the mobile terminal based on the current online activity.

The triggering operation on the mobile terminal in the embodiment of the present disclosure may be an operation that changes the motion state of the mobile terminal, and the reason for the change in the motion state of the mobile terminal may be caused by human factors or non-human factors. ; For operations that change the movement state of the mobile terminal caused by human factors, such as the user shaking the mobile terminal horizontally, shaking the mobile terminal vertically, etc.; For operations that change the movement state of the mobile terminal caused by non-human factors, such as the mobile terminal falling, or The mobile terminal slides by itself due to the inclination of the placement table.

The response process of the mobile terminal to the trigger operation may include: in the valid time of the current online activity, in the case of detecting the user's trigger operation on the mobile terminal, invoking the image acquisition device of the mobile terminal to capture the surrounding environment information. Here, the user's triggering operation on the mobile terminal will only be responded to within the valid time of the current online activity, so that the mobile terminal can be prevented from frequently responding to user operations and the processing resources of the mobile terminal can be saved.

S220. Detect whether the surrounding environment information includes a valid target object. The surrounding environment information collected by the image acquisition device includes a variety of information, such as person information, object information, etc. In the embodiment of the present disclosure, it is necessary to detect whether the surrounding environment information contains a valid target object, which may be the detection of targeted person information. In some embodiments, please refer to FIG. 3 , which shows a method for determining a valid target object in surrounding environment information. The method may include:

S310. Analyze the surrounding environment information of the mobile terminal, and determine whether the surrounding environment information of the mobile terminal includes a living object.

S320. If the surrounding environment information includes a living object, determine the face orientation of the living object.

S330. In the case that the facial orientation of the living object satisfies a preset facial orientation condition, determine that the surrounding environment information includes a candidate target object.

S340. Determine the effective target object from the candidate target objects.

As can be seen from Figure 3, for the judgment of an effective target object, two conditions need to be met. The first is to determine whether the surrounding environment information contains living objects, and the second is to determine whether the face orientation of the living object satisfies the preset face orientation. conditions; thus, it can be determined that the user is watching the current online activity through the mobile terminal, thereby avoiding a false triggering operation on the mobile terminal that the user is not watching the current online activity. In the actual usage scenario, the surrounding environment information of the mobile terminal may contain multiple candidate target objects. In order to avoid the influence of non-operating users on the judgment result, it is necessary to determine the effective target objects from the multiple candidate objects at this time, so that the subsequent It is only necessary to process and analyze the valid target objects. The non-operating user here refers to a user who does not perform a trigger operation on the mobile terminal but meets the conditions of living objects and face orientation.

Please refer to FIG. 4, which shows a method for determining valid target objects from candidate target objects, including:

S410. Identify one or more of the facial expression of the candidate target object, the distance between the candidate target object and the mobile terminal, and the target that the candidate target object focuses on with both eyes.

S420. Determine the effective target object from the candidate target objects according to the recognition result.

The facial expression of the target object can be captured by the camera of the mobile terminal. The facial expression of the candidate target object can be a focused expression, a happy expression, a nervous expression, etc., and the camera of the mobile terminal can also use the camera of the mobile terminal to measure the binocular focus of the candidate target object. Determine, thereby determining the target that the candidate target object pays attention to on the display screen of the mobile terminal, so that an effective target object can be determined from the candidate target objects according to the identification results of one or more of the above. Here, the effective target object can be determined by the facial expression of the candidate object, the distance from the mobile terminal, the attention target, etc., to further ensure that the effective target object is the object that is paying attention to the relevant content in the mobile terminal, thereby improving the accuracy of the effective target object determination. sex.

In addition, in the case that the living object is not included in the surrounding environment information, it is determined that the effective target object is not included in the surrounding environment information; in the case that the facial orientation of the living object does not meet the preset facial orientation condition , determine that the effective target object is not included in the surrounding environment information; if the surrounding environment information does not include a living object or includes a living object but the face orientation does not meet the preset face orientation conditions, directly determine that the effective target object is not included , which can improve the efficiency of effective target object determination. The valid target object here refers to a living object whose face orientation satisfies the preset face orientation condition, such as a living object whose front face faces a mobile terminal, or a living object whose side faces face the mobile terminal, so it can be determined that a user is passing through the mobile terminal Watch the current online activity; through the detection of valid target objects, in the case that the user is not watching the current online activity but mis-triggered the mobile terminal, it can be determined that the valid target object is not included, so that the follow-up will not be carried out. The triggering operation processing flow avoids waste of processing resources caused by the user's false triggering of the mobile terminal.

In some embodiments, the image acquisition device used to acquire the surrounding environment information in the embodiment of the present disclosure may include a camera, a dot matrix projector, a structured light sensor, and the like. For the collected surrounding environment information, the mobile terminal can directly determine whether it contains a valid target object based on the surrounding environment information, or the collected surrounding environment information can be sent to the second terminal. The second terminal can be a server, and the server Whether a valid target object is included is determined based on the surrounding environment information; the embodiment of the present disclosure does not make specific limitations, and can be selected according to the specific implementation situation.

S230. In the case that the effective target object is included in the surrounding environment information, acquire an object operation set corresponding to the mobile terminal.

In this embodiment of the present disclosure, the object operation set corresponding to the mobile terminal may be established in advance according to the operation habits of each preset object. For each mobile terminal, a corresponding object operation set can be generated respectively, and the object operation set corresponding to the mobile terminal can be: the object operation set corresponding to the device identification of the mobile terminal, or the online activity account logged in the mobile terminal. Therefore, in the case of obtaining the object operation set corresponding to the mobile terminal, the device identification of the mobile terminal can be determined first, and then the object operation set corresponding to the corresponding device identification can be obtained according to the device identification of the mobile terminal; or First, the online activity account logged in the mobile terminal is determined, and then an object operation set corresponding to the corresponding online activity account is obtained according to the online activity account. Wherein, the object operation set corresponding to the device identification of each mobile terminal is respectively generated, and the operation habits of the preset objects that have operated the mobile terminal can be included, so that the description of the object operation set of each mobile terminal can be improved. comprehensiveness; for generating the object operation set corresponding to the online activity account logged in the mobile terminal, since each online activity account generally corresponds to a relatively limited number of user objects, only need to be involved in the generation of the object operation set The operation habits related to these user objects narrow the scope of the operation habits of the user objects, so that the object operation sets of each mobile terminal can be described in a targeted manner, thereby improving the processing efficiency in subsequent data processing based on the object operation sets. .

Please refer to FIG. 5 , which shows a method for generating an object operation set corresponding to a mobile terminal, which may include:

S510. For each preset object operating the mobile terminal, acquire face information of each preset object.

S520. Record at least one habitual operation of each preset object when operating the mobile terminal, and generate and generate a habitual operation for each preset object based on at least one habitual operation of each preset object when operating the mobile terminal A set of custom operation items corresponding to each preset object.

S530. Generate the object operation set based on the face information of each preset object and the habitual operation item set corresponding to each preset object.

For each preset object, an object description information item may be established for it respectively. In the embodiment of the present disclosure, the object description information item may include the face information of the preset object and the set of habitual operation items of the preset object; for the face of the preset object The information can be collected by the relevant sensors on the mobile terminal; for the set of habitual operation items of the preset objects, it can be obtained by analyzing the characteristics of the multiple operations of the mobile terminal and continuously learning according to each preset object, and getting the same as each preset object. The set of customary operation items corresponding to the preset objects. The multiple operations here can include various types of operations, such as shaking the mobile terminal horizontally, shaking the mobile terminal vertically, etc. For each preset object, it can be different Type operations for feature analysis and learning.

The pre-generated object operation set includes the facial information of each preset object and a set of habitual operation items, which facilitates subsequent judgment of an effective trigger operation based on the object operation set, and provides a basis for judging whether an effective trigger operation is performed.

S240. Based on the valid target object and the object operation set, determine whether the trigger operation is a valid trigger operation.

In this embodiment of the present disclosure, only a trigger operation that is determined to be valid will be processed and executed. Please refer to FIG. 6 , which shows a method for determining the validity of a trigger operation. The method may include:

S610. Obtain the facial information of the effective target object, and match the facial information of the effective target object with the facial information of each preset object in the object operation set to obtain a matching object; the matching object is the object operation In the set, the preset object corresponding to the facial information that matches the facial information of the valid target object.

Similarly, the facial information of the valid target object can be collected through the relevant sensors of the mobile terminal, and the mobile terminal obtains the facial information of the valid target object and matches it with the facial information of each preset object in the object operation set.

S620. Match the trigger operation with the customary operation items in the customary operation item set of the matching object to obtain a matching result of the operation items.

S630. Determine whether the trigger operation is a valid trigger operation according to the operation item matching result.

As can be seen from FIG. 6 , when there is a matching object and there is a customary operation item that matches the triggering operation on the mobile terminal in the set of customary operation items of the matching object, it is judged that the current triggering operation on the mobile terminal is valid. Trigger action. Whether it is an effective trigger operation is determined by matching the customary operation item of the matching object with the trigger operation, which can improve the convenience of the effective trigger operation.

In the case of judging whether the triggering operation on the mobile terminal is an effective triggering operation, it can be first determined whether there is a matching object matching the valid target object in the object operation set, so as to determine whether the current valid target object is a certain preset object, and which operation corresponds to; for the sake of security, in the embodiment of the present disclosure, it can be set that only the trigger operation of the preset object on the mobile terminal will be processed, and the operation of the mobile terminal by the non-preset object can be filtered to ensure the operation Security, so as to avoid security problems caused by other non-preset users or users without authority operating the mobile terminal; in addition, by matching preset objects first, the matching scope can be further narrowed, and the matching efficiency of subsequent trigger operations can be improved.

In some embodiments, please refer to FIG. 7 , which shows a schematic flowchart of judging the validity of the trigger operation of the mobile terminal, which may include:

S710. Acquire the facial information of the effective target object, and match the facial information of the effective target object with the facial information of each preset object in the object operation set.

S720. Determine whether there is facial information in the object operation set that matches the facial information of the valid target object; if it exists, execute step S730; if it does not, execute step S780.

S730. Determine the preset object corresponding to the facial information in the object operation set that matches the facial information of the valid target object as a matching object.

S740. Extract a set of customary operation items of the matching object from the object operation set.

S750. Match the triggering operation on the mobile terminal with the habitual operation items in the habitual operation item set of the matching object.

S760. Determine whether there is a habitual operation item matching the triggering operation on the mobile terminal in the habitual operation item set of the matching object; if it exists, execute step S770; if not, execute step S770. Step S780 is executed.

S770. Determine that the triggering operation on the mobile terminal is a valid triggering operation.

S780. Determine that the triggering operation on the mobile terminal is an invalid triggering operation.

In the embodiment of the present disclosure, only the trigger operation that is judged to be valid will be executed. Here, the validity of the trigger operation can be considered from two aspects. On the one hand, the trigger operation is issued by a preset object, On the other hand, the trigger operation matches the habitual operation item of the preset object, so that the safe operation of the mobile terminal can be ensured, and the specific operation corresponding to the current trigger operation can be determined.

Further, the trigger operation on the mobile terminal includes operation parameters on the mobile terminal; each habitual operation item set includes operation parameters corresponding to each habitual operation; in some embodiments, the operation parameters may include multiple Data information of each dimension, such as horizontal and vertical screen information, amplitude, frequency, rotation direction, etc., these parameters can be detected by the sensor of the mobile terminal. Since the mobile terminal detects the triggering operation on the mobile terminal, the relevant parameters are detected by the sensor in the mobile terminal to determine that there is a triggering operation, but which type of operation of which preset object corresponds to, further matching is required. Determine, that is, the operation type of the mobile terminal can be determined according to these parameters detected by the sensor.

Please refer to FIG. 8 , which shows a method for determining an effective trigger operation based on operation parameters, which may include:

S810. Match the operation parameter of the trigger operation with the operation parameter corresponding to each habitual operation in the habitual operation item set of the matching object to obtain an operation parameter matching result.

S820. According to the operation parameter matching result, determine whether there is a target operation parameter in the operation parameters corresponding to each habitual operation of the matching object, and the target operation parameter and the operation parameter of the trigger operation satisfy a preset condition.

S830. In response to the determination result being yes, determine that the triggering operation is a valid triggering operation.

The data of each dimension of the operation parameters of the mobile terminal will be matched with the data of each dimension of the habitual operation of the matching object, and the matching result here meets the preset condition may refer to the data of a certain dimension, the current data of the mobile terminal is matched. If the difference between the data in the operation parameter and the data in the habitual operation of the matching object is smaller than the preset value, it is determined that the matching result meets the preset condition.

In the set of habitual operation items, each habitual operation has corresponding operation parameters. For example, a habitual operation is to shake the mobile terminal horizontally, and the corresponding dimension data are: horizontal and vertical screen information: horizontal screen, amplitude: x, frequency : y; in the case that the current operation parameters of the mobile terminal match the horizontal and vertical screen information: horizontal screen, amplitude: x, frequency: y of each dimension data, it is determined that the current trigger operation on the mobile terminal is horizontal shaking. Through the matching of the operation parameters, the operation parameters of the mobile terminal can be determined as corresponding operations.

The reason why it is necessary to match the valid target object with the preset object is that for the same operation, the operation parameters of different preset objects are different, and it is difficult to directly determine the corresponding operation by directly matching the operation parameters. By matching the preset object first, it can be further reduced. The matching range improves the matching efficiency of subsequent trigger operations; when the matching object is determined, further matching of operation data can be performed to quickly determine the corresponding matching operation. Matching the trigger operation of the mobile terminal is based on the matching of operation parameters. The operation parameters of the trigger operation are matched with the operation parameters corresponding to each habitual operation item of the matching object, so as to determine whether the trigger operation is a valid trigger operation. Operation item matching based on operation parameters can more accurately determine customary operation items that match the trigger operation, avoid matching errors, and improve matching accuracy.

S250. In the case that the trigger operation is a valid trigger operation, execute the online interaction instruction corresponding to the trigger operation in the current online activity.

In the embodiment of the present disclosure, the corresponding relationship between each preset operation on the mobile terminal and each online activity instruction in the current online activity can be pre-established, and when the triggering operation is an effective triggering operation, the triggering operation is executed. Before operating the corresponding online interactive instructions in the current online activity, the corresponding relationship between each preset operation on the mobile terminal and each online interactive instruction in the current online activity can be obtained in advance, so as to Refer to Figure 9 for the processing method of an effective trigger operation, and the method may include:

S910. Match the trigger operation with each preset operation on the mobile terminal, and determine a preset operation matching the trigger operation.

S920. Based on the corresponding relationship, determine a valid online interaction instruction corresponding to the valid trigger operation in the current online activity.

S930. Execute the effective online interactive instruction.

According to the corresponding relationship between the preset operation and the online interactive instruction, the online interactive instruction corresponding to the current triggering operation on the mobile terminal is determined, and the online interactive instruction is executed, thereby realizing the triggering operation on the mobile terminal. Converted to online interactive instructions in the current online activity. The effective online interaction instruction corresponding to the trigger operation is determined by the online interaction instruction corresponding to the preset operation matching the trigger operation, so that the corresponding effective online interaction can be triggered only by performing the trigger operation on the mobile terminal. Instructions, thereby enhancing the convenience of participating in online activities and improving the efficiency of interaction.

In addition, in order to further meet the user's personalized settings and realize the flexibility of online interaction and participation, the user can preset the interaction conditions for participating in online activities. When the second terminal receives an online interaction request, it first determines the current online activity. Whether the online activity complies with the interaction conditions preset by the user, in the case of compliance, the corresponding interaction is realized according to the above-mentioned methods of the embodiments of the present disclosure; in the case of non-compliance, within the valid time of the current online activity, All online interaction requests sent will not be processed. After pre-setting the interaction conditions for participating in online activities, if it is determined that the triggering operation on the mobile terminal is an effective triggering operation, you can directly participate in the online interaction to achieve the purpose of online interaction, without the need to go through complicated procedures. The manual click interaction process to achieve the purpose of online interaction.

It should be noted that the execution order of each step in the above methods of the present disclosure is not limited, and in the specific implementation process, the execution order of the corresponding steps may be adjusted according to specific conditions.

In the present disclosure, the surrounding environment information is obtained by responding to the user's trigger operation on the mobile terminal based on the current online activity; in the case that the surrounding environment information contains a valid target object, the object operation set corresponding to the mobile terminal is obtained; If the valid target object and the object operation set determine that the trigger operation is a valid trigger operation, the online interactive instruction corresponding to the valid trigger operation in the current online activity is executed. The present disclosure can directly convert the trigger operation on the mobile terminal into the corresponding online interactive instruction, so that the user can participate in the corresponding online interactive activity by directly performing the trigger operation on the mobile terminal to change its motion state, It is suitable for scenarios where users cannot participate in online interactive activities through manual click operations, and can avoid the failure to participate in real-time interaction due to complicated manual click operations. Compared with manual click operations, the movement state of the mobile terminal is changed. The triggering operation is simpler and easier to implement; thus, the present disclosure can shorten the conversion process of the interaction process, improve the interaction efficiency, and improve the user experience.

The online interaction method in the present disclosure can be applied to application scenarios such as online live shopping, ticket rush purchases, and instant product shooting. The following takes the online live shopping scenario as an example to illustrate the implementation process of the present disclosure.

Please refer to FIG. 10 , which shows a schematic diagram of the interactive process of participating in online live shopping. The execution subject may be a mobile terminal that displays live events, and may include:

S1010. When detecting that the user enters the shopping live broadcast room, the mobile terminal obtains the current live broadcast content in real time, and analyzes the current live broadcast product according to the obtained current live broadcast content.

For the current live product, it can be specific to SKU (Stock Keeping Unit), where the SKU can be set as simple as possible to make the interaction process as short as possible.

S1020. Determine whether the user is watching a live broadcast based on a sensor of the mobile terminal.

Whether the user is watching the live broadcast in real time is determined by the distance sensor, ambient light sensor and camera of the mobile terminal. In the embodiment of the present disclosure, it can be set that when the user is far away from the mobile terminal, the mobile terminal can automatically turn off the screen to avoid misoperation. It can be realized by a distance sensor and an ambient light sensor; when the user uses the mobile terminal normally, the screen of the mobile terminal is displayed normally. As shown in FIG. 11 , the front camera of the mobile terminal and the data detected by the distance sensor and the ambient light sensor are used to determine whether there is a living object in front of the current mobile terminal, and further determine that the face orientation of the living object is displaying the live broadcast Active mobile terminal, so as to determine that the current user is watching the live broadcast.

S1030. Based on the sensor collection of the mobile terminal, the operation parameters for changing the motion state of the mobile terminal are changed, and the operation behavior of the current user on the mobile terminal is determined according to the operation parameters.

The operation parameters here may be some physical data of the user in the process of watching the live broadcast, that is, the data of the mobile terminal at the physical level. Restore the user's 3D behavior through the acceleration sensor, gravity sensor, and gyroscope: determine the acceleration values in the three directions of x/y/z through the acceleration sensor, determine the horizontal and vertical screen information of the mobile terminal through the gravity sensor, and determine the rotation of the mobile terminal through the gyroscope directions, etc. As shown in Figure 12, when the user performs a trigger operation on the mobile terminal, the gravity sensor can determine that the mobile terminal is in vertical screen, the acceleration value of the shaking operation in the three directions of x/y/z can be determined by the acceleration sensor, and the gyroscope can be used to determine whether the mobile terminal is in a vertical position. The shaking direction of the terminal, etc.

S1040. Determine, according to the correspondence between the user's operation behavior on the mobile terminal and the live broadcast shopping behavior, the live broadcast shopping behavior corresponding to the current user's operation behavior on the mobile terminal.

Identify and classify through algorithms, so as to convert physical movements into logical codes, and then use logical codes to help users complete actions such as ordering, collection, and seconds shooting. For example, the following simple high-frequency behaviors can be set: 1. Shake the mobile terminal horizontally at a low frequency - pay attention; 2. Shake the mobile terminal horizontally at a high frequency - add purchase; 3. Shake the mobile terminal vertically at a low frequency - one-click purchase, For the distinction between low frequency and high frequency, the corresponding value can be set according to the user's operating habits. Please refer to FIG. 13 for the flowchart of the shopping interaction that the user realizes by operating the mobile terminal.

As shown in Figure 14, during the live shopping process, when the shopping cart icon appears on the live broadcast interface, it is considered that the current shopping interaction activity has started, and the corresponding interaction can be realized by triggering the mobile terminal at this time; for example, this When the user wants to add the current product to the shopping cart, he or she can make additional purchases by shaking the mobile terminal horizontally and at high frequency.

In order to ensure the smooth completion of the above interactions, it is necessary to systematically increase fault-tolerant measures, such as confirming the current orientation of the mobile terminal and the current customers watching the live broadcast through the camera. Through such real-time verification, users can ensure their safety and effectiveness while shopping. . If you need to further ensure the validity, you can also obtain the user's face information in real time through the structured light sensor to confirm whether the order is valid.

Applying the present disclosure to online live shopping scenarios can enable users to quickly and accurately purchase favorite products to satisfy various groups of people; in order to achieve this purpose, it is first necessary to confirm whether the object in front of the mobile terminal is a "person". Whether the user is watching the live broadcast and whether the user is himself can be determined through structured light sensors, cameras, and dot matrix projectors; secondly, to determine effective trigger actions, users have their own user characteristics similar to "user portraits". Collect and accumulate the action characteristics of mobile terminal users to determine a number of simple and effective actions as trigger points for the user to place an order in the live broadcast (thousands of people change, and each person's trigger characteristics are different); finally, the necessary fault-tolerant filtering function is required , to ensure that the system can place the order correctly. After the above steps, complex shopping behaviors can be completed through several simple behaviors such as high-frequency lateral shaking, low-frequency lateral shaking, and high-frequency vertical shaking.

Further, a "quick shopping" function can also be set, that is, a series of conditions can be set, including: the upper limit of the product amount, the category to which the product belongs, the current user is the user of the shopping account, and the commonly used shopping address has been filled in, etc.; The commodities that meet the above preset conditions can execute corresponding shopping instructions when the physical behavior of the mobile terminal is detected; by setting these conditions in advance, the shopping process can be made shorter, more effective, and more time-sensitive; To prevent users from purchasing products that do not meet their own requirements, making live shopping more personalized. Among them, in the case where the physical behavior of the mobile terminal is detected, but the current live broadcast product does not meet at least one of the preset conditions, a corresponding prompt window may pop up on the live broadcast interface at this time, indicating whether it is necessary to continue this operation. Different physical behaviors of the mobile terminal can also be defined to correspond to different choices. For example, shaking the mobile terminal in the direction of the user corresponds to agreeing to continue the current operation, and shaking the mobile terminal away from the user corresponds to not agreeing to continue the current operation. Referring to Figure 15, when the user sets the "quick shopping" function, the upper limit of the set product amount is 500 yuan, and the price of the current live broadcast product is 600. When the shopping cart appears on the current live broadcast shopping interface, the user can pass Shake the mobile terminal vertically and low frequency to make a purchase. At this time, a prompt window will pop up on the live shopping interface, which reads "The current commodity price is higher than the set commodity amount upper limit. Do you want to continue the operation? If so, perform operation 1 on the mobile terminal; if No, perform operation 2" on the mobile terminal, so that it can be determined whether the user finally wants to buy the current product according to the user's further operations on the mobile terminal, and if the user performs operation 1 on the mobile terminal, then continue to buy the current live product; When the user performs operation 2 on the mobile terminal, the purchase process ends.

In the specific implementation process, you can set many interesting actions when shopping offline, and then set them as trigger actions for online shopping, so that you can feel the physical feedback of offline shopping while shopping online, so that users have Different experience; set different body movements and set them as online shopping trigger actions; abstract physical data into corresponding body movements through accurate algorithms, and then cooperate with rich face information to accurately place orders .

Through the trigger operation on the mobile terminal, the order, collection, second shot and other operations of the current live broadcast products can be realized. Different from the traditional online shopping system, the trigger shopping is no longer completed by a single manual click, and the ordering method is enriched. Shopping is more suitable; as the sensor technology of the mobile terminal becomes more and more abundant, the physical movement of the user's mobile terminal can be restored with high precision, and the user's behavior can be determined by the physical movement instead of relying on the operation on the screen. to complete some operations to meet the use of different user groups and different life scenarios. Online shopping based on the physical movement of the mobile terminal can reduce the shopping process, realize efficient order placement, facilitate special users, improve the user's physical participation, get rid of the restrictions of use scenarios, and can take care of special groups; avoid complicated interaction processes , will not cause the loss of live broadcast information, which is conducive to the effect of live broadcast shopping.

FIG. 16 shows an online interactive device 1600 according to an exemplary embodiment. Referring to FIG. 16 , the device includes:

The trigger operation response unit 1610 is configured to perform a trigger operation on the mobile terminal in response to the user based on the current online activity, and obtain surrounding environment information;

a valid target object detection unit 1620, configured to perform detection of whether the surrounding environment information contains a valid target object;

an object operation set obtaining unit 1630, configured to obtain an object operation set corresponding to the mobile terminal in the case that the effective target object is included in the surrounding environment information;

A valid trigger operation determination unit 1640, configured to perform, based on the valid target object and the set of object operations, to determine whether the trigger operation is a valid trigger operation;

The online interactive instruction execution unit 1650 is configured to execute the online interactive instruction corresponding to the triggering operation in the current online activity when the triggering operation is a valid triggering operation.

In an exemplary embodiment, the trigger operation response unit 1610 includes:

In an exemplary embodiment, the effective target object detection unit 1620 includes:

In an exemplary embodiment, the apparatus further includes:

The valid trigger operation determination unit 1640 includes:

In an exemplary embodiment, the fourth determining unit includes:

The valid trigger operation determination unit 1640 further includes:

a first matching unit, configured to perform matching between the operation parameter of the trigger operation and the operation parameter corresponding to each habitual operation in the set of habitual operation items of the matching object to obtain an operation parameter matching result;

In an exemplary embodiment, the apparatus further includes:

The online interactive instruction execution unit 1650 includes:

Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and has corresponding functional modules and beneficial effects for performing the method, which will not be described in detail here. .

The present disclosure provides a server, including: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement any of the above-mentioned embodiments of the present disclosure A method; please refer to FIG. 17 for the structural diagram of the server. The server 1700 may vary greatly due to different configurations or performance, and may include one or more central processing units (CPU) 1722 (for example, a or more processors) and memory 1732, one or more storage media 1730 (eg, one or more mass storage devices) that store applications 1742 or data 1744. Among them, the memory 1732 and the storage medium 1730 may be short-lived storage or persistent storage. The program stored in the storage medium 1730 may include one or more modules (not shown), and each module may include a series of instructions to operate on the device. Furthermore, the central processing unit 1722 may be configured to communicate with the storage medium 1730 to execute a series of instruction operations in the storage medium 1730 on the server 1700 . Server 1700 may also include one or more power supplies 1726, one or more wired or wireless network interfaces 1750, one or more input and output interfaces 1758, and/or, one or more operating systems 1741, such as Windows Server™, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM and many more.

The present disclosure provides a computer program product, the computer program product includes a computer program, the computer program is stored in a readable storage medium, and at least one processor of a computer device reads and executes the computer program from the readable storage medium. The computer program described above enables the device to execute any of the methods described above in the embodiments of the present disclosure.

In an exemplary embodiment, a storage medium including instructions is also provided, when the instructions in the storage medium are executed by the processor of the server, the server can execute the online interaction method as described above. In some embodiments of the present disclosure, the storage medium may be a non-transitory computer-readable storage medium, for example, the non-transitory computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape , floppy disks and optical data storage devices.

All the embodiments of the present disclosure can be implemented independently or in combination with other embodiments, which are all regarded as the protection scope required by the present disclosure.

Other embodiments of the present disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common general knowledge or techniques in the technical field not disclosed by this disclosure . The specification and examples are to be regarded as exemplary only, with the true scope and spirit of the disclosure being indicated by the following claims.

It is to be understood that the present disclosure is not limited to the precise structures described above and illustrated in the accompanying drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

An online interaction method, comprising:

In response to the user's triggering operation on the mobile terminal based on the current online activity, obtain the surrounding environment information;

Detecting whether the surrounding environment information contains a valid target object;

In the case that the effective target object is included in the surrounding environment information, obtain an object operation set corresponding to the mobile terminal;

determining whether the trigger operation is a valid trigger operation based on the valid target object and the set of object operations; and

In the case that the triggering operation is a valid triggering operation, the online interactive instruction corresponding to the triggering operation in the current online activity is executed.
The online interaction method according to claim 1, wherein, in response to a user's triggering operation on the mobile terminal based on the current online activity, acquiring the surrounding environment information comprises:

During the valid time of the current online activity, in the case of detecting the triggering operation of the mobile terminal by the user, the image acquisition device of the mobile terminal is called to collect the surrounding environment information.
The online interaction method according to claim 1 or 2, wherein the detecting whether the surrounding environment information includes a valid target object comprises:

Analyzing the surrounding environment information to determine whether the surrounding environment information contains living objects;

If the surrounding environment information includes a living object, determining the face orientation of the living object;

In the case that the face orientation of the living object satisfies the preset face orientation condition, determining the living object as a candidate target object in the surrounding environment information;

determining whether the valid target object exists in the candidate target objects; and

In response to the existence of the valid target object in the candidate target objects, it is determined that the valid target object is included in the surrounding environment information.
The online interaction method according to claim 3, wherein determining whether the valid target object exists in the candidate target objects comprises:

Identifying one or more of the facial expression of the candidate target object, the distance between the candidate target object and the mobile terminal, and the target that the candidate target object focuses on; and

Whether the valid target object exists in the candidate target objects is determined according to the recognition result.
The online interaction method according to claim 3, wherein the detecting whether the surrounding environment information contains a valid target object further comprises:

In the case where the surrounding environment information does not include a living object, determine that the valid target object is not included in the surrounding environment information; or

In the case that the face orientation of the living object does not satisfy the preset face orientation condition, it is determined that the effective target object is not included in the surrounding environment information.
The online interaction method according to claim 1 or 4, wherein the object operation set includes facial information of each preset object and a set of habitual operation items corresponding to each preset object; and

The determining whether the trigger operation is a valid trigger operation based on the valid target object and the object operation set includes:

Obtain the facial information of the effective target object, and match the facial information of the effective target object with the facial information of each preset object in the object operation set to obtain a matching object, and the matching object is in the object operation set. , the preset object corresponding to the facial information matching the facial information of the effective target object;

Matching the trigger operation with the customary operation items in the set of customary operation items of the matching object to obtain the operation item matching result; and

According to the operation item matching result, it is determined whether the trigger operation is a valid trigger operation.
The online interaction method according to claim 6, wherein the determining whether the trigger operation is a valid trigger operation according to the operation item matching result comprises:

When the operation item matching result is that there is a habitual operation item matching the trigger operation in the habitual operation item set of the matching object, it is determined that the trigger operation is a valid trigger operation.
The online interaction method according to claim 6, wherein the method further comprises generating the object operation set in advance, and the pre-generating the object operation set comprises:

For each preset object operating the mobile terminal, obtain face information of each preset object;

Record at least one habitual operation item of each preset object when operating the mobile terminal, and based on the at least one habitual operation item of each preset object when operating the mobile terminal, generate a A set of customary operation items corresponding to the preset object; and

The object operation set is generated based on the face information of each preset object and a set of habitual operation items corresponding to each preset object.
The online interaction method according to claim 1 or 4, wherein the object operation set includes facial information of each preset object and a set of habitual operation items corresponding to each preset object, and the trigger operation includes For the operation parameters of the trigger operation, each habitual operation item set includes an operation parameter corresponding to each habitual operation item; and

The determining whether the trigger operation is a valid trigger operation based on the valid target object and the object operation set includes:

Obtain the facial information of the effective target object, and match the facial information of the effective target object with the facial information of each preset object in the object operation set to obtain a matching object, and the matching object is in the object operation set. , the preset object corresponding to the facial information matching the facial information of the effective target object;

Matching the operation parameter of the trigger operation with the operation parameter corresponding to each habitual operation in the habitual operation item set of the matching object to obtain an operation parameter matching result;

According to the operation parameter matching result, determine whether there is a target operation parameter in the operation parameters corresponding to each habitual operation of the matching object, and the target operation parameter and the operation parameter of the trigger operation satisfy a preset condition; and

In response to the determination result being yes, it is determined that the trigger operation is a valid trigger operation.
The online interaction method according to claim 1 or 9, further comprising:

Acquiring in advance the correspondence between each preset operation on the mobile terminal and each online interactive instruction in the current online activity; and

In the case that the triggering operation is a valid triggering operation, executing the online interactive instruction corresponding to the triggering operation in the current online activity includes:

Matching the trigger operation with each preset operation of the mobile terminal, and determining a preset operation that matches the trigger operation;

Based on the corresponding relationship, determining a valid online interaction instruction corresponding to the preset operation matching the trigger operation in the current online activity; and

Execute the effective online interactive instruction.
An online interactive device, characterized in that it includes:

a trigger operation response unit, configured to acquire surrounding environment information in response to a user's trigger operation on the mobile terminal based on the current online activity;

a valid target object detection unit, configured to detect whether the surrounding environment information contains a valid target object;

an object operation set acquisition unit, configured to acquire an object operation set corresponding to the mobile terminal when the surrounding environment information includes the effective target object;

a valid trigger operation determination unit, configured to determine whether the trigger operation is a valid trigger operation based on the valid target object and the set of object operations; and

The online interactive instruction execution unit is configured to execute the online interactive instruction corresponding to the trigger operation in the current online activity when the trigger operation is a valid trigger operation.
The online interactive device according to claim 11, wherein the trigger operation response unit comprises:

The image capture device invoking unit is configured to call the image capture device of the mobile terminal to capture the surrounding when a triggering operation of the mobile terminal by the user is detected within the valid time of the current online activity. environmental information.
The online interactive device according to claim 11 or 12, wherein the effective target object detection unit comprises:

a living object determination unit, configured to analyze the surrounding environment information to determine whether the surrounding environment information contains a living object;

a face orientation determination unit, configured to determine the face orientation of the living object when the surrounding environment information includes a living object;

a candidate target object determination unit, configured to determine the living object as a candidate target object in the surrounding environment information when the face orientation of the living object satisfies a preset face orientation condition; and

A valid target object determination unit configured to determine whether the valid target object exists in the candidate target objects, and in response to the valid target object existing in the candidate target objects, determine that the surrounding environment information includes the A valid target object.
The online interactive device according to claim 13, wherein the effective target object determination unit comprises:

an identification unit configured to identify one or more of the facial expression of the candidate target object, the distance between the candidate target object and the mobile terminal, and the target that the candidate target object focuses on; and

The first determining unit is configured to determine whether the valid target object exists in the candidate target objects according to the recognition result.
The online interactive device according to claim 13, wherein the device further comprises:

a second determination unit, configured to determine that the effective target object is not included in the surrounding environment information if the surrounding environment information does not include a living object; or

The third determination unit is configured to determine that the effective target object is not included in the surrounding environment information when the face orientation of the living object does not satisfy a preset face orientation condition.
The online interactive device according to claim 11 or 14, wherein the object operation set includes facial information of each preset object and a set of habitual operation items corresponding to each preset object; and

The valid trigger operation determination unit includes:

A facial information matching unit, configured to obtain the facial information of the effective target object, and to match the facial information of the effective target object with the facial information of each preset object in the object operation set to obtain a matching object, and the matching The object is the preset object corresponding to the facial information that matches the facial information of the effective target object in the object operation set;

a habitual operation item set matching unit, configured to match the trigger operation with a habitual operation item in the habitual operation item set of the matching object to obtain an operation item matching result; and

The fourth determination unit is configured to determine whether the trigger operation is a valid trigger operation according to the operation item matching result.
The online interactive device according to claim 16, wherein the fourth determining unit comprises:

a fifth determining unit, configured to determine that the trigger operation is a valid trigger when a custom operation item matching the trigger operation exists in the custom operation item set of the matching object whose matching result is the matching object operate.
The online interactive device according to claim 16, wherein the device further comprises a first generating unit, comprising:

a facial information acquisition unit, configured to acquire facial information of each preset object for each preset object operating the mobile terminal;

A habitual operation item set generating unit, configured to record at least one habitual operation item of each preset object when operating the mobile terminal, based on at least one habitual operation item of each preset object when operating the mobile terminal customary operation items, generating a set of customary operation items corresponding to each preset object; and

The object operation set generating unit is configured to generate the object operation set based on the face information of each preset object and a set of habitual operation items corresponding to each preset object.
The online interactive device according to claim 11 or 14, wherein the object operation set includes facial information of each preset object and a set of habitual operation items corresponding to each preset object, and the trigger operation includes For the operation parameters of the trigger operation, each habitual operation item set includes an operation parameter corresponding to each habitual operation item; and

The valid trigger operation determination unit includes:

A facial information matching unit, configured to obtain the facial information of the effective target object, and to match the facial information of the effective target object with the facial information of each preset object in the object operation set to obtain a matching object, and the matching The object is the preset object corresponding to the facial information that matches the facial information of the effective target object in the object operation set;

a first matching unit, configured to match the operation parameter of the trigger operation with the operation parameter corresponding to each habitual operation in the set of habitual operation items of the matching object to obtain an operation parameter matching result;

A judging unit is configured to, according to the operation parameter matching result, determine whether there is a target operation parameter in the operation parameters corresponding to each habitual operation of the matching object, and the target operation parameter and the operation parameter of the trigger operation satisfy preset conditions; and

The determination unit is configured to, in response to the determination result being yes, determine that the trigger operation is a valid trigger operation.
The online interactive device according to claim 11 or 19, wherein the device further comprises:

a corresponding relationship obtaining unit, configured to obtain in advance the corresponding relationship between each preset operation on the mobile terminal and each online interactive instruction in the current online activity;

The online interactive instruction execution unit includes:

a preset operation matching unit, configured to match the trigger operation with each preset operation on the mobile terminal, and determine a preset operation that matches the trigger operation;

an online interaction instruction determining unit, configured to determine, based on the corresponding relationship, a valid online interaction instruction corresponding to the preset operation matching the trigger operation in the current online activity; and

The first execution unit is configured to execute the effective online interactive instruction.
A server, characterized in that it includes:

processor;

a memory for storing the processor-executable instructions;

Wherein, the processor is configured to execute the instructions to implement an online interaction method comprising the following steps:

In response to the user's triggering operation on the mobile terminal based on the current online activity, obtain the surrounding environment information;

Detecting whether the surrounding environment information contains a valid target object;

In the case that the effective target object is included in the surrounding environment information, obtain an object operation set corresponding to the mobile terminal;

determining whether the trigger operation is a valid trigger operation based on the valid target object and the set of object operations; and

In the case that the triggering operation is a valid triggering operation, the online interactive instruction corresponding to the triggering operation in the current online activity is executed.
The server of claim 21, wherein the processor is configured to execute the instructions to implement the steps of:

During the valid time of the current online activity, in the case of detecting the triggering operation of the mobile terminal by the user, the image acquisition device of the mobile terminal is called to collect the surrounding environment information.
The server of claim 21 or 22, wherein the processor is configured to execute the instructions to implement the following steps:

Analyzing the surrounding environment information to determine whether the surrounding environment information contains living objects;

If the surrounding environment information includes a living object, determining the face orientation of the living object;

In the case that the face orientation of the living object satisfies the preset face orientation condition, determining the living object as a candidate target object in the surrounding environment information;

determining whether the valid target object exists in the candidate target objects; and

In response to the existence of the valid target object in the candidate target objects, it is determined that the valid target object is included in the surrounding environment information.
24. The server of claim 23, wherein the processor is configured to execute the instructions to implement the steps of:

Identifying one or more of the facial expression of the candidate target object, the distance between the candidate target object and the mobile terminal, and the target that the candidate target object focuses on; and

Whether the valid target object exists in the candidate target objects is determined according to the recognition result.
24. The server of claim 23, wherein the processor is configured to execute the instructions to implement the steps of:

In the case where the surrounding environment information does not include a living object, determine that the valid target object is not included in the surrounding environment information; or

In the case that the face orientation of the living object does not satisfy the preset face orientation condition, it is determined that the effective target object is not included in the surrounding environment information.
The server according to claim 21 or 24, wherein the object operation set includes facial information of each preset object and a set of habitual operation items corresponding to each preset object; and

The processor is configured to execute the instructions to implement the following steps:

Obtain the facial information of the effective target object, and match the facial information of the effective target object with the facial information of each preset object in the object operation set to obtain a matching object, and the matching object is in the object operation set. , the preset object corresponding to the facial information matching the facial information of the effective target object;

Matching the trigger operation with the customary operation items in the set of customary operation items of the matching object to obtain the operation item matching result; and

According to the operation item matching result, it is determined whether the trigger operation is a valid trigger operation.
27. The server of claim 26, wherein the processor is configured to execute the instructions to:

When the operation item matching result is that there is a habitual operation item matching the trigger operation in the habitual operation item set of the matching object, it is determined that the trigger operation is a valid trigger operation.
The server of claim 26, wherein the processor is configured to execute the instructions to implement the steps of:

Pre-generating the object operation set, the pre-generating the object operation set includes:

For each preset object operating the mobile terminal, obtain face information of each preset object;

Record at least one habitual operation item of each preset object when operating the mobile terminal, and based on the at least one habitual operation item of each preset object when operating the mobile terminal, generate a A set of customary operation items corresponding to the preset object; and

The object operation set is generated based on the face information of each preset object and a set of habitual operation items corresponding to each preset object.
The server according to claim 21 or 24, wherein the object operation set includes facial information of each preset object and a set of habitual operation items corresponding to each preset object, and the trigger operation includes the trigger operation Operation parameters of the operation, each habitual operation item set includes the operation parameters corresponding to each habitual operation;

The processor is configured to execute the instructions to implement the following steps:

Obtain the facial information of the effective target object, and match the facial information of the effective target object with the facial information of each preset object in the object operation set to obtain a matching object, and the matching object is in the object operation set. , the preset object corresponding to the facial information matching the facial information of the effective target object;

Matching the operation parameter of the trigger operation with the operation parameter corresponding to each habitual operation in the habitual operation item set of the matching object to obtain an operation parameter matching result;

According to the operation parameter matching result, determine whether there is a target operation parameter in the operation parameters corresponding to each habitual operation of the matching object, and the target operation parameter and the operation parameter of the trigger operation satisfy a preset condition; and

In response to the determination result being yes, it is determined that the trigger operation is a valid trigger operation.
The server of claim 21 or 29, wherein the processor is configured to execute the instructions to implement the following steps:

Acquiring in advance the correspondence between each preset operation on the mobile terminal and each online interactive instruction in the current online activity; and

The processor is configured to execute the instructions to implement the following steps:

Matching the trigger operation with each preset operation of the mobile terminal, and determining a preset operation that matches the trigger operation;

Based on the corresponding relationship, determining a valid online interaction instruction corresponding to the preset operation matching the trigger operation in the current online activity; and

Execute the effective online interactive instruction.
A storage medium, characterized in that, when the instructions in the storage medium are executed by a processor of a server, the server is enabled to perform the following steps:

In response to the user's triggering operation on the mobile terminal based on the current online activity, obtain the surrounding environment information;

Detecting whether the surrounding environment information contains a valid target object;

In the case that the effective target object is included in the surrounding environment information, obtain an object operation set corresponding to the mobile terminal;

determining whether the trigger operation is a valid trigger operation based on the valid target object and the set of object operations; and

In the case that the triggering operation is a valid triggering operation, the online interactive instruction corresponding to the triggering operation in the current online activity is executed.