WO2019241920A1

WO2019241920A1 - Terminal control method and device

Info

Publication number: WO2019241920A1
Application number: PCT/CN2018/091927
Authority: WO
Inventors: 张霞
Original assignee: 优视科技新加坡有限公司; 优视科技有限公司
Priority date: 2018-06-20
Filing date: 2018-06-20
Publication date: 2019-12-26
Also published as: CN109496289A

Abstract

Disclosed in the embodiments of the present application are a terminal control method and device. The preferred embodiment for the present method comprises: in some specific scenarios, a user may issue a non-touch instruction, such as a motion, facial expression or voice instruction to a terminal, and after receiving the non-touch instruction issued by the user, the terminal may execute a corresponding operation. By employing the described embodiment, touch interaction between a user and a terminal may be reduced to a certain extent, and the convenience of interaction is increased.

Description

Terminal control method and device

Technical field

The present application relates to the field of computer technology, and specifically to the field of Internet technology, and in particular, to a method and device for controlling a terminal.

Background technique

With the development of information technology, terminals have more and more functions, which are widely used in people's daily life and work. The convenience of human-computer interaction has become an important research topic and development direction.

In the prior art, when a terminal implements a corresponding function, the user often needs to interact with the terminal through contact. For example, a user starts an application after clicking an Application (App) icon on a touch screen, and the user uses a finger to Slide the touch screen to unlock the terminal, etc.

Based on the existing technology, a more convenient way to control the terminal is needed.

Summary of the Invention

The purpose of this application is to provide a terminal control method and device, which are used to solve the problem that it is inconvenient to control a terminal by using contact interaction in some application scenarios.

In a first aspect, an embodiment of the present application provides a terminal control method, including:

The terminal receives the user's contactless instruction in a specific state;

According to the received non-contact instruction, a preset operation corresponding to the non-contact instruction is executed.

In some embodiments, the terminal receiving a contactless instruction from a user in a specific state includes:

The terminal invoking an image acquisition unit in the specific state to collect facial expression characteristics of the user;

The expression feature is determined as a non-contact instruction.

In some embodiments, the terminal invoking an image acquisition unit to collect the expression feature of the user in the specific state includes:

Receiving, by the terminal, an expression collection trigger operation sent by a user on an instant messaging interface;

When the received expression collection trigger operation is received, the image acquisition unit is called to collect the user's expression characteristics.

In some embodiments, performing a preset operation corresponding to the contactless instruction according to the received contactless instruction includes:

Find the virtual expression that matches the expression feature among the existing virtual expressions according to the expression feature;

Displaying the found virtual expression to the user.

In some embodiments, displaying the found virtual emoticon to the user includes:

Displaying the found virtual expression in a designated area of the terminal interface for a user to select.

In a second aspect, an embodiment of the present application further provides a terminal control device, including:

A receiving processing module configured to receive a user's contactless instruction in a specific state;

The execution module is configured to execute a preset operation corresponding to the contactless instruction according to the received contactless instruction.

In some embodiments, the receiving processing module is configured to call an image acquisition unit in the specific state to collect an expression feature of the user, and determine the expression feature as a non-contact instruction.

In some embodiments, the receiving processing module is configured to receive an expression acquisition trigger operation sent by a user on an instant messaging interface, and when the received expression acquisition trigger operation is received, call the image acquisition unit to acquire the User emoticons.

In some embodiments, the execution module is configured to find a virtual expression matching the expression characteristic among the existing virtual expressions based on the expression characteristics, and display the found virtual expression to The user.

In some embodiments, the execution module is configured to display the found virtual emoticon in a designated area of a terminal interface for a user to select.

The method and device for controlling a terminal provided in this application, in some specific scenarios, the user can send non-contact instructions such as actions, expressions, sounds, etc. to the terminal. After receiving the non-contact instructions issued by the user, the terminal then Perform the corresponding operation. Such a control method can reduce the contact interaction between the user and the terminal to a certain extent, thereby improving the convenience of the interaction. In addition, the non-contact instruction method can reduce the corresponding cost without using a wearable smart device bound to the terminal.

Specifically, in a scenario where an expression is sent during a social process, the terminal can collect a user's facial expression, and based on the user's facial expression, find and select a virtual expression that matches the user's facial expression from existing virtual expressions. It is precisely with the above method in this application that the user does not need to scroll through pages in the virtual expression candidate area to find a virtual expression, but can use a more convenient interaction method-make corresponding expressions, and accordingly, The terminal may find a matching virtual expression for the user based on the user's expression. As a result, the interactive operations performed by the user to find some virtual expressions can be reduced to a certain extent, the interaction is more convenient, and the time required to find the virtual expressions can be shortened.

BRIEF DESCRIPTION OF THE DRAWINGS

Other features, objects, and advantages of the present application will become more apparent by reading the detailed description of the non-limiting embodiments with reference to the following drawings:

FIG. 1 is a schematic diagram of interaction between a user and a terminal according to an embodiment of the present application;

2 is a schematic flowchart of a terminal control method according to an embodiment of the present application;

3 is a schematic diagram of an expression candidate region provided by an embodiment of the present application;

4 is a schematic flowchart of a terminal control method in a virtual expression scene query by expression control according to an embodiment of the present application;

5a is a schematic diagram of an expression control provided by an embodiment of the present application;

FIG. 5b is a schematic diagram of interaction in a virtual expression scene query by expression control provided by an embodiment of the present application; FIG.

5c is a schematic diagram of sending an expression according to an embodiment of the present application;

6a is a schematic diagram of a display mode of a virtual expression found based on a user's expression according to an embodiment of the present application;

6b is a schematic diagram of another virtual expression display method based on a user's expression provided by an embodiment of the present application;

7 is a schematic flowchart of a terminal control method in a shooting scene controlled by an expression according to an embodiment of the present application;

FIG. 8 is a schematic structural diagram of a terminal control device according to an embodiment of the present application.

detailed description

The following describes the present application in detail with reference to the accompanying drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the related invention, but not to limit the invention. It should also be noted that, for convenience of description, only the parts related to the related invention are shown in the drawings.

It should be noted that, in the case of no conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The application will be described in detail below with reference to the drawings and embodiments.

As mentioned above, users usually need to control the terminal to perform certain operations through contact interaction methods. The contact interaction methods described herein may include, but are not limited to, the terminal screen, physical buttons, or external devices connected to the terminal. Clicks, presses, drags, swipes, etc. performed.

However, in some cases, the aforementioned contact interaction method is more inconvenient. For example, when a user takes a selfie with a tablet computer, the user needs to use the virtual shutter button displayed on the shooting interface to implement shooting control. However, because the screen size of the tablet computer is large, it is more inconvenient for the user to control the shooting through the virtual shutter button in the shooting interface. , May lead to changes in the camera angle or jitter, affecting the camera effect. Another example: the Instant Messaging (IM) function on a mobile communication terminal provides users with a large number of virtual emoticons (such as emoji). During the chat process by the user through the instant messaging function, the corresponding virtual emoticons may be used. However, since there are many virtual expressions and the virtual expression candidate regions are limited, the user usually needs to slide pages in the virtual expression candidate regions to find the virtual expressions required by the user.

For this reason, a terminal control method is provided in the embodiment of the present application, which allows a user to control a terminal to perform a corresponding operation in a non-contact interaction mode in some cases, thereby further improving the convenience of interaction.

Referring to FIG. 1, an interaction control manner between a user and a terminal according to an embodiment of the present application is shown. As shown in Figure 1, the user can control the terminal through non-contact instructions. It should be understood that in the process of the user issuing a non-contact instruction, the user may not touch the terminal's touch screen, physical keys, or external devices connected to the terminal. Of course, the user should not be understood as a user by controlling the terminal with a non-contact instruction No contact at all with the terminal. Specifically, in some embodiments, the user can hold the terminal and issue non-contact instructions to the terminal through expressions, gestures, sounds, etc .; in other embodiments, the user and the terminal can be completely separated, and the user can send the terminal to the terminal. Sending non-contact instructions to control the terminal. In this mode, the distance between the user and the terminal should not affect the terminal receiving the non-contact instructions from the user.

It should be noted that, in the embodiments of the present application, the non-contact instructions may specifically be non-contact interactive instructions in the form of expressions, body movements, sounds, and the like, and the non-contact instructions do not need to rely on other External devices related to the terminal (such as smart wearable devices that have a binding relationship with the terminal) can control the terminal.

Generally, the terminal has image acquisition and / or sound acquisition functions, and may specifically include, but is not limited to, mobile terminals such as mobile phones, tablet computers, notebook computers, smart watches, and cameras, or has image acquisition and sound Collection of computers, etc., will not be listed here one by one. Of course, if the terminal device does not have an image acquisition or sound acquisition function, but realizes the foregoing acquisition function by connecting with an external device, it should also be understood to be within the coverage of the terminal described in the embodiment of the present application.

In addition, among the above-mentioned mobile terminals, terminals with communication functions, such as: mobile phones, tablet computers, notebook computers, smart watches and other devices with communication functions, may be collectively referred to as mobile communication in subsequent descriptions in the embodiments of the present application. terminal.

Based on the architecture shown in FIG. 1, the technical solutions provided in the embodiments of the present application will be described in detail below.

With continued reference to FIG. 2, a terminal control method provided in an embodiment of the present application is shown, which specifically includes the following steps:

Step S201: The terminal receives a contactless instruction from the user in a specific state.

The specific state may be considered as a state in which the terminal is running certain applications or enabling certain functions, such as a state in which the camera is framing but not shooting, a state when an IM application running on the mobile phone displays an expression candidate area, and the like. It should be noted here that, in the existing implementation manner, in the specific state of the terminal, the interaction between the user and the terminal can usually only be a contact interaction, which is inconvenient to the user to a certain extent.

In the embodiment of the present application, the terminal can receive and identify the user's non-contact instruction in the specific state described above. Specifically, in practical applications, there may be multiple ways for a user to issue a non-contact instruction, such as: a user's facial expression, a limb movement, a sound made, and the like. In other words, the terminal can receive contactless instructions through the terminal's acquisition unit or component, such as a camera or microphone. Of course, the acquisition unit or component mentioned here may also be an external device, which is not specifically limited here.

The terminal's identification of non-contact instructions can be realized by the corresponding identification function, which can be provided by the operating system running by the terminal itself, or by an application (Application, App) installed on the terminal. Generally, The recognition function may specifically include at least one of sound detection, natural semantic recognition, body recognition, motion recognition, facial feature recognition (which may further include expression recognition), which should not be construed as limiting the present application.

Step S203: According to the received contactless instruction, an operation corresponding to the contactless instruction set in advance is performed.

The non-contact instruction is used to instruct the terminal to perform the corresponding operation. It should be understood that the operation corresponding to the non-contact instruction may be preset by a device manufacturer or a function provider, or may be set by a user. In one example: when the mobile phone is in a photographing mode, the user can control the mobile phone to perform photographing by saying an instruction of “photographing”. In another example: the user can record the phrase "click" as an instruction to control the mobile phone to take a picture.

Obviously, through the above steps, in some specific scenarios, the user can send non-contact instructions such as actions, expressions, and sounds to the terminal. After receiving the non-contact instructions issued by the user, the terminal can perform the corresponding operations. . Such a control method can reduce the contact interaction between the user and the terminal to a certain extent, thereby improving the convenience of the interaction. In addition, the non-contact instruction method can reduce the corresponding cost without using a wearable smart device bound to the terminal.

For the above steps, the execution subject may generally be the terminal itself, but in some embodiments, an App running on the terminal may also be used as the execution subject. In addition, during the execution of the above steps, the execution subject may also change. For example, the execution subject of step S201 may be the terminal itself, and the execution subject of step S203 may be an App running on the terminal. Of course, the specific will be determined according to the actual application situation, which should not be understood as a limitation on this application.

The terminal control method in the embodiments of the present application will be described in detail below with reference to specific application scenarios.

scene one

When users are communicating instantly, virtual expressions can increase the fun of chatting. Based on this, the chat function on the mobile communication terminal or the third-party IM application provides rich virtual expressions for users to use. However, the size of the virtual expression candidate area in the chat interface is limited, and it is difficult to fully display the virtual expression with a large number of virtual expressions.

With continued reference to FIG. 3, a schematic diagram of a virtual expression candidate region in an IM application is shown. As shown in FIG. 3, the virtual expression candidate region cannot fully display all virtual expressions. If the existing interaction method is adopted, the user is required to slide and turn pages in the virtual expression candidate area to browse and find the virtual expression that is not displayed. Obviously, in this scenario, it is inconvenient for the user to adopt the existing interaction method, and the user will spend a certain amount of time in the process of finding the virtual expression.

To this end, continuing to refer to FIG. 4, which illustrates a terminal control method in this scenario. As shown in FIG. 4, the method may specifically include the following steps:

Step S401: The terminal collects the user's expression features in the expression candidate state.

It should be noted that the virtual expression may at least include a character expression, a picture expression, or an animation expression. The presentation mode of the virtual expression may be static or dynamic, which is not specifically limited here.

In the embodiment of the present application, the expression candidate state may be considered as a state triggered by the user and the terminal is ready to provide the user with a candidate virtual expression.

It should be understood that, for the foregoing step S401, when the terminal is in an expression candidate state, the image acquisition unit is called to collect the expression characteristics of the user, and the collected expression characteristics are determined as a non-contact instruction. Of course, the image acquisition unit may be carried by the terminal itself, or may be an external camera of the terminal.

Generally speaking, the expression candidate status is triggered by the expression collection trigger operation issued by the user, and there are various expression collection trigger operations issued by the user:

In a possible implementation manner, the expression candidate status is triggered when the user views the virtual expression. Specifically, the user can click the virtual expression candidate control in the IM interface (the operation of the user clicking the virtual expression candidate control can be considered as an expression collection trigger). Operation), the virtual expression candidate area (the area shown in FIG. 3) will be displayed in the IM interface, and the terminal will be triggered to enter the expression candidate state.

In another possible implementation manner, the user may click an expression collection control in the IM interface (similarly, the operation of the user clicking the control may also be considered as an expression collection trigger operation), and the expression collection control is used to awaken the terminal An image acquisition unit (such as a camera) to collect expressions on the user. At this time, the terminal may also be considered to be in an expression candidate state.

Therefore, when the terminal is in the expression candidate state, the image acquisition unit is called to collect the expression characteristics of the user. This process can be as follows: the terminal receives the expression acquisition trigger operation sent by the user on the instant messaging interface, and when the received expression After the acquisition trigger operation, the image acquisition unit is called to collect the facial expression characteristics of the user.

The above two methods may be the triggering modes of the expression candidate state, and the specific conditions depend on the actual situation. For example, the operation of the user holding the handshake and pointing the front camera of the mobile phone towards the user may also be regarded as an expression collection triggering operation; or the user unlocking the terminal may also be regarded as an expression collection triggering operation. This should not constitute a limitation on this application.

In the embodiment of the present application, the facial feature recognition function, especially the expression recognition function, can be used to collect and recognize the user's expression. Of course, the specific expression recognition function can be implemented by technologies such as deep learning algorithms and neural network models, which are not described in detail here.

Step S403: According to the collected facial expression features, find a virtual facial expression that matches the facial expression feature in the virtual facial expression, so as to send it as an instant messaging message.

In this scenario, the facial expressions collected by the user's facial expressions can be considered as a non-contact instruction, and the terminal can find or send the virtual facial expressions that match the facial expression characteristics in the virtual facial expressions. The operation of the expression can be regarded as the operation corresponding to the non-contact instruction.

As a possible implementation manner in the embodiment of the present application, a corresponding virtual expression can be found according to the type of the expression. Specifically, in actual applications, the user's expressions can be divided into different categories based on the facial features, such as: smile, pouting, frowning, closing eyes, etc. Of course, in addition to dividing emotions based on emotions, such as : Angry, happy, etc. It should be understood that which categories are actually divided into expressions usually depends on the specific expression recognition function. In other words, through the expression recognition function, the category to which the user's actual facial expression belongs can be identified.

In addition, virtual emoticons are usually provided with corresponding identification information, such as: "/ smile", "/ frown", "/ glare eyes", etc. (the emoticon identification information can usually be set by the device manufacturer or a third-party IM application provider), of course This is only one possible way of identifying information, and should not be construed as limiting the present application. Obviously, based on the category to which the user's expression belongs and the identification information of the virtual expression, a virtual expression matching the user's expression can also be determined.

As another possible implementation manner in the embodiment of the present application, a corresponding virtual expression can be found according to the similarity of the expression. Specifically, the terminal can extract the user's expression features and calculate the similarity between the user's expression features and the characteristics of the virtual expression, thereby finding the virtual expression most similar to the user's expression. For example, the terminal may find different virtual expressions such as similar smiles, squints, and laughs according to the characteristics of the mouth, eyes, and face when the user laughs.

The found virtual emoticons can be sent directly to other users as instant messaging messages.

For example, as shown in FIG. 5a, when a user uses a mobile phone for chatting and wants to send an emoticon, the user can select an emoticon control in the current chat interface, and then the virtual emoticon candidate area will be displayed in the current chat interface. At this time, the mobile phone is in an expression candidate state, the front camera of the mobile phone is activated, and the user's expression is collected. As shown in FIG. 5b, if the mobile phone collects and recognizes that the user's facial expression is "smile", then the virtual expression labeled "/ smile" can be found and selected in the virtual expression. As shown in FIG. 5c, the found "smile" virtual expression can be directly sent to other users as an instant messaging message.

From the above, it can be seen that in the scene of sending emotions in the social process, the terminal can collect the user's facial expressions, and based on the user's facial expressions, find and select from the existing virtual expressions that match the user's facial expressions. Virtual expression. It is precisely with the above method in this application that the user does not need to scroll through pages in the virtual expression candidate area to find a virtual expression, but can use a more convenient interaction method-make corresponding expressions, and accordingly, The terminal may find a matching virtual expression for the user based on the user's expression. As a result, the interactive operations performed by the user to find some virtual expressions can be reduced to a certain extent, the interaction is more convenient, and the time required to find the virtual expressions can be shortened.

It should be noted that, in the above content, the terminal recognizes the category to which the user's expression belongs. Specifically, the terminal can collect the facial expression image of the user, and extract expression feature data to determine the expression category described in the expression feature data. This determines the category described by the user's expression. Of course, the process of extracting facial feature data can be specifically located and extracted by facial feature extraction algorithms for facial features, texture regions, and predefined feature points. The specific process is not described in detail here.

Once the category to which the user's expression belongs is determined, it is possible to perform a traversal search on the identification information of the virtual expression in order to find a virtual expression of the same category.

The terminal determines the similarity between the user's facial expression and the virtual facial expression. Specifically, the terminal can collect a facial expression image of the user and extract facial expression characteristic data, and calculate based on the extracted facial expression characteristic data and characteristic data corresponding to the virtual facial expression. Similarity. Of course, similarity algorithms such as Euclidean distance and cosine distance can be used, which are not specifically limited here.

In the embodiment of the present application, the number of found virtual expressions may be one or more than one, and the specific processing methods are different:

In a feasible implementation manner, if the number of found virtual emoticons is one, then the virtual emoticons can be directly sent as an instant communication message.

In another feasible implementation manner, the number of selected virtual expressions may be more than one. In this case, each of the found virtual expressions may be displayed to the user, and the user may decide to send a certain virtual expression. Specifically, when the number of found virtual expressions is more than one, a display manner as shown in FIG. 6a may be adopted. In FIG. 6a, each virtual expression matching the actual expression of the user is displayed in a separate line above the virtual expression candidate area. If the user selects a certain virtual expression, the virtual expression selected by the user is sent as an instant messaging message, and the line disappears.

In practical applications, the display method shown in FIG. 6b can also be adopted, that is, in FIG. 6b, each virtual expression matching the user's actual expression is displayed in the original virtual expression candidate area by adding a new layer. on. If the user selects a virtual expression, the newly added layer disappears, and the original virtual expression candidate area continues to be displayed.

Of course, the above-mentioned display methods in Figs. 6a and 6b are only examples. In actual applications, other display methods may be adopted, such as pop-up window display, floating layer display, etc., depending on the needs of the actual application.

Scene two

When a user uses a mobile phone, camera, or tablet with a shooting function to take a photo with other users, it may be inconvenient to use the existing interaction methods:

If a user holds a terminal for shooting, it may be restricted by factors such as the size and shape of the terminal, which is not conducive to the user clicking the virtual shutter button or the physical shutter button, which may affect the shooting effect. If the user uses the timer photo function to shoot, the shooting preparation time provided by the timer photo function is usually fixed, and the user cannot adjust it by himself, which is disadvantageous for the user to take a photo.

Although users can use external devices such as camera sticks and smart watches, the use of external devices will undoubtedly increase shooting costs.

Therefore, in this scenario, an embodiment of the present application provides a terminal control method. As shown in FIG. 7, the method may specifically include the following steps:

Step S701: The terminal collects and monitors a user's expression in a pre-shooting state.

The pre-shooting state may be considered as a state in which the terminal has started framing but has not yet performed shooting. Specifically, if the terminal is a mobile communication terminal such as a mobile phone or tablet computer, the pre-shooting status may be that the terminal has its own camera function activated or a third-party shooting app. At this time, the shooting interface will be displayed on the terminal screen but shooting has not been performed; If the terminal is a camera, the pre-photographing state may be that the camera of the camera has started framing but the user has not pressed the shutter.

It should be noted that, during actual shooting, the user usually adjusts his or her facial expression and maintains a state suitable for shooting. Generally, the user maintains such an expression for a period of time. Therefore, in the embodiment of the present application, the terminal monitoring the user's expression can be considered as monitoring whether the user's expression changes within a specified time.

The monitoring of the user's expression can be achieved by the corresponding monitoring function. In a feasible implementation manner, the user's face can be determined by the face recognition function, and whether the texture, light and shadow, preset anchor points, etc. of the face have changed , And then determine whether the user's expression has changed. This method does not require overly complex recognition algorithms.

Step S703: When it is detected that the expression does not change within a set period of time, a preset shooting operation is performed.

In this scenario, the non-contact instruction can be considered as the expression of the photographed user has not changed within the set time period. Accordingly, the preset shooting operation performed by the terminal can be considered as the execution corresponding to the non-contact instruction. Operation.

The preset shooting operations may include: triggering a shutter, continuous shooting, switching photography, and the like, and may be specifically set according to actual application requirements.

The preset duration can be set to 1 to n seconds, which can be set in advance by the device manufacturer or a third-party shooting APP provider to different time scales, or can be defined by the user. Once the terminal detects that the facial expression of the user has not changed within the set time, the above-mentioned shooting operation may be performed, thereby implementing non-contact interactive shooting control.

Obviously, through the above steps, when taking selfies or group photos, the user can implement shooting control through an interactive manner in which the expression remains unchanged for a certain period of time. This method is not limited by the size of the terminal, and does not require the user to enable the timer shooting function, and only needs to adjust the terminal to the pre-photographing state, thereby effectively improving the convenience of interactive control. At the same time, the above method focuses on monitoring whether the user's expression changes within a certain period of time, and does not need to recognize the specific expression made by the user, and does not need to use an overly complex expression recognition algorithm.

During the actual shooting process using the above method, the number of users being shot may be more than one. In this case, to achieve shooting control, the expressions of each user who is shot and whose face is facing the lens need to be within a certain period of time. constant.

As a feasible implementation manner of the present application, after the terminal performs shooting, the terminal may provide an image collected before the shutter trigger and / or a period of time after the shutter trigger is provided as a shooting result to the user. For example, if the terminal detects that the user ’s expression has not changed within a set period of time (assuming it is 1s), the camera performs a shooting operation (the terminal automatically triggers the shutter to shoot). Here, it is assumed that the terminal automatically triggers the shutter at t, then The shooting result provided to the user may be the image content captured by the terminal from time t-1 to time t, or the image content captured by the terminal from time t to time t + 1, or it may be from time t- From 1 to time t + 1, the image content captured by the terminal.

After that, the user can select the corresponding shooting content for saving in the above shooting results (save as a photo or a video). Of course, this should not be construed as limiting the present application.

It should be noted that, in the above scenario, the expression of the user is used as the non-contact instruction. In fact, the foregoing terminal control method in the embodiment of the present application is not limited to this. In some application modes, the non-contact The instructions can be diversified, for example, the user also makes corresponding limb movements while making facial expressions.

The limb movements described herein include, but are not limited to, gestures, postures, limb movements, etc. of the user. The limb movements of the user may be dynamic or static, which is not specifically limited herein.

Therefore, in the above scenario 1, in addition to the above-mentioned application method for finding a virtual expression based on the user's expression, the above-mentioned method in the embodiment of the present application can also find the corresponding response based on the user's expression and the limb movements made simultaneously by the user. Virtual expression. Specifically, in this application mode, the terminal will collect the user's facial expression and the user's limb movement, and use the collected facial expression features and limb motion characteristics as the non-contact instruction issued by the user.

As for the virtual expressions, part of the virtual expressions are composed of the expressions and actions of the virtual image. Then, according to the user's expression characteristics and motion characteristics, the virtual expressions can be found in the virtual expressions that match the aforementioned characteristics. Of course, the method for finding virtual expressions can be similar to the above method, that is, you can find the virtual expressions of the same category by determining the category to which the user's characteristics (including expression characteristics and action characteristics) belong; or by comparing the user's expressions, The manner of similarity between the action and the virtual expression realizes the search for the virtual expression, which will not be described in detail here.

Of course, in actual applications, the user's limb movements can be collected and identified by various motion recognition models such as corresponding body recognition, gesture recognition, and limb movement recognition.

In the second scenario described above, the user's expression and the user's limb movement can also be used to implement the shooting control of the terminal. Specifically, in a possible application manner, the terminal collects and monitors the user's facial expressions and motion characteristics in a pre-shooting state, and when the monitored facial expressions and motion characteristics of the user are not changed within a set duration ( Or the amplitude of the change is within a preset range), a preset shooting operation is performed.

In another possible application method, the user can control the terminal to trigger the shutter by changing the facial expression for a certain period of time, and at the same time, can control the terminal to switch between different shooting modes through physical actions such as gestures. For example: in the pre-shooting state, if the user holds out his index finger and makes a movement that simulates pressing the camera shutter, the terminal performs a photo shoot; for another example: in the pre-shooting state, the user extends his left hand (or right hand), with his four fingers close together Bend, the terminal performs video shooting. Of course, this is only a possible example, and it will depend on the needs of the actual application.

Of course, the above contents should not be construed as limiting the present application.

The above is the terminal control method provided in the embodiment of the present application. Based on the same idea, the embodiment of the present application also provides a corresponding terminal control device.

Specifically, with continued reference to FIG. 8, a terminal control device provided in an embodiment of the present application is shown. The device includes:

A receiving processing module 801 configured to receive a contactless instruction of a user in a specific state;

The execution module 802 is configured to perform a preset operation corresponding to the contactless instruction according to the received contactless instruction.

The terminal control device shown in FIG. 8 can be applied in different scenarios.

Scenario 1: Terminal control is achieved through expressions during social interaction.

In this scenario, the specific state includes at least an expression candidate state, wherein the expression candidate state is used to provide a virtual expression to a user.

Further, the receiving processing module 801 is configured to call an image acquisition unit to collect the expression feature of the user in the specific state, and determine the expression feature as a non-contact instruction.

Further, the receiving processing module 801 is configured to receive an expression collection trigger operation sent by a user on an instant messaging interface, and after receiving the expression collection trigger operation, call the image acquisition unit to collect the user ’s Emoticons.

Further, the execution module 802 is configured to find a virtual expression matching the expression characteristic among the existing virtual expressions according to the expression characteristics, and display the found virtual expression to the virtual expression. user.

Further, the execution module 802 is configured to display the found virtual emoticon in a designated area of a terminal interface for a user to select.

Scene two, shooting control

In this scenario, the specific state includes at least a pre-shooting state, where the pre-shooting state may be considered as a state in which the terminal has started framing but has not yet performed shooting.

The receiving processing module 801 is configured to collect and monitor a user's expression in a pre-shooting state.

The execution module 802 is configured to execute a preset shooting operation when it is detected that the expression has not changed within a set period of time.

Based on the apparatus shown in FIG. 8, it may be implemented by a physical electronic device in practical applications. Specifically, the terminal includes one or more processors; and a storage device for storing one or more programs;

When the one or more programs are executed by the one or more processors, the one or more processors implement the method described above.

Based on the apparatus shown in FIG. 8, it can be implemented by a computer-readable storage medium in practical applications. Specifically, a computer program is stored on the computer-readable storage medium. When the program is executed by a processor, the method described above is implemented.

Each embodiment in this application is described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the device, device, and media embodiments, since they are basically similar to the method embodiments, the description is relatively simple. For the related parts, refer to the description of the method embodiments, and they will not be described here one by one.

So far, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired result. In addition, the processes depicted in the figures do not necessarily require the particular order shown or sequential order to achieve the desired result. In certain embodiments, multitasking and parallel processing may be advantageous.

In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart may be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product including a computer program carried on a computer-readable medium, the computer program containing program code for performing a method shown in a flowchart. In such an embodiment, the computer program may be downloaded and installed from a network through a communication section, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), the above functions defined in the method of the present application are performed. It should be noted that the computer-readable medium described in this application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable Programming read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the foregoing. In this application, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in combination with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal that is included in baseband or propagated as part of a carrier wave, and which carries computer-readable program code. Such a propagated data signal may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. The computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, and the computer-readable medium may send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. . Program code embodied on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for performing the operations of this application may be written in one or more programming languages, or a combination thereof, including programming languages such as Java, Smalltalk, C ++, and also conventional Procedural programming language—such as "C" or a similar programming language. The program code can be executed entirely on the user's computer, partly on the user's computer, as an independent software package, partly on the user's computer, partly on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or wide area network (WAN), or it can be connected to an external computer (such as through an Internet service provider Internet connection).

The flowchart and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, which contains one or more functions to implement a specified logical function Executable instructions. It should also be noted that in some alternative implementations, the functions labeled in the blocks may also occur in a different order than those labeled in the drawings. For example, two blocks represented one after the other may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified function or operation , Or it can be implemented with a combination of dedicated hardware and computer instructions.

The units described in the embodiments of the present application may be implemented by software or hardware. The described unit may also be provided in a processor, for example, it may be described as: a processor includes a receiving unit, a parsing unit, an information selecting unit, and a generating unit. The names of these units do not in any way constitute a limitation on the unit itself. For example, the receiving unit may also be described as a "unit that receives a user's web browsing request."

The above description is only a preferred embodiment of the present application and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to the technical solution of the specific combination of the above technical features, but should also cover the above technical features or Other technical solutions formed by arbitrarily combining their equivalent features. For example, a technical solution formed by replacing the above features with technical features disclosed in the present application (but not limited to) having similar functions.

Claims

A terminal control method, comprising:

The terminal receives the user's contactless instruction in a specific state;

According to the received non-contact instruction, a preset operation corresponding to the non-contact instruction is executed.
The method according to claim 1, wherein the terminal receiving the contactless instruction of the user in a specific state comprises:

The terminal invoking an image acquisition unit in the specific state to collect facial expression characteristics of the user;

The expression feature is determined as a non-contact instruction.
The method according to claim 2, wherein the terminal calling an image acquisition unit to collect the expression feature of the user in the specific state comprises:

Receiving, by the terminal, an expression collection trigger operation sent by a user on an instant messaging interface;

When the received expression collection trigger operation is received, the image acquisition unit is called to collect the user's expression characteristics.
The method according to claim 2, characterized in that, according to the received contactless instruction, performing a preset operation corresponding to the contactless instruction comprises:

Find the virtual expression that matches the expression feature among the existing virtual expressions according to the expression feature;

Displaying the found virtual expression to the user.
The method according to claim 4, characterized in that displaying the found virtual emoticon to the user comprises:

Displaying the found virtual expression in a designated area of the terminal interface for a user to select.
A terminal control device, comprising:

A receiving processing module configured to receive a user's contactless instruction in a specific state;

The execution module is configured to execute a preset operation corresponding to the contactless instruction according to the received contactless instruction.
The device according to claim 6, wherein the receiving processing module is configured to call an image acquisition unit in the specific state to collect facial expression characteristics of the user, and determine the facial expression characteristics as non-contact instruction.
The device according to claim 7, wherein the receiving processing module is configured to receive a facial expression triggering operation sent by a user on an instant messaging interface, and call the mobile terminal after the received facial expression triggering operation is received. The image acquisition unit collects expression features of the user.
The device according to claim 7, wherein the execution module is configured to find a virtual expression matching the expression characteristic among existing virtual expressions according to the expression characteristics, and find out The virtual emoticon is displayed to the user.
The device according to claim 9, wherein the execution module is configured to display the found virtual emoticon in a designated area of a terminal interface for a user to select.
An electronic device includes: one or more processors;

A storage device for storing one or more programs;

When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5.
A computer-readable storage medium having stored thereon a computer program, characterized in that when the program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.