WO2023197888A1 - Interaction method, device and medium - Google Patents

Abstract

The present application relates to an interaction method, a device, and a medium. The method comprises: an electronic device displaying a first interface of a first application, wherein the first interface comprises a first session window of a first contact; detecting an expression input operation of a user on the first session window of the first interface; displaying, corresponding to determining that a first portrait expression set related to the first contact exists in a plurality of portrait expression sets stored in the electronic device, at least one portrait expression in the first portrait expression set in the first session window. According to the interaction method in the present application, when the user needs to input the expression, the electronic device can display a portrait expression set of a corresponding contact required by the user in response to the expression input operation of the user, the user can operate a screen of the electronic device, the expression required by the user is selected in the portrait expression set of the corresponding contact, the user does not need to manually make and import the expression of the corresponding contact, the user operation is simple, and the experience is high.

Description

Interaction methods, devices and media

This application claims priority to the Chinese patent application submitted to the China Patent Office on April 12, 2022, with the application number 202210383986. in this application.

Technical field

This application relates to the technical field of terminal equipment, and specifically relates to an interaction method, equipment and medium.

Background technique

With the popularization of mobile phones, tablets, computers and other terminal devices and the development of social software, emoticons (including image emoticons and text emoticons) are increasingly used. Users can enliven the atmosphere of communication by turning photos of people they know well into emoticons.

In current social software, after users create portrait emoticons based on the portrait photos of the corresponding contacts, when chatting with the corresponding contacts, they need to manually add the portrait emoticons to the favorite emoticon set in the social software, and then add the facial expressions displayed in the favorite emoticon set. Search among the emoticons to find the desired emoticon of the corresponding contact and send it to the corresponding contact. The emoticon interaction solution requires the user to go through multiple operations to add and select the portrait emoticon corresponding to the contact that the user needs. When there are many expressions in the collection of emoticons, it is difficult for the user to find the portrait emoticon corresponding to the contact, and the experience is poor.

Contents of the invention

The embodiments of the present application provide an interactive method, device and medium, which reduces the user's operations of making and searching for emoticons, and is conducive to improving the user experience.

In a first aspect, embodiments of the present application provide an interaction method applied to an electronic device, including: a first interface of a first application displayed on the electronic device, wherein the first interface includes a first conversation window of a first contact ; Detecting the user's expression input operation in the first conversation window of the first interface; Corresponding to the determination that there is a first portrait expression set related to the first contact in the plurality of portrait expression sets stored in the electronic device, in the first At least one portrait expression in the first portrait expression set is displayed in the session window.

It can be understood that the expression input operation is an operation in which the user triggers the input of an expression in the first session window, which may be, for example, an operation in which the user clicks an expression input button in the first session window. For example, the first session window may include a text input box, and the user may enter text in the text input box of the first session window and click an expression input button.

With the interaction method provided by this application, the electronic device can display the portrait expression set corresponding to the contact that the user needs in response to the user's expression input operation. Furthermore, the user can operate the screen of the electronic device to select the user from the portrait expression set corresponding to the contact. The required emoticons do not require users to manually create and import the emoticons corresponding to the contacts. The user operation is simple and the experience is high.

In a possible implementation of the above first aspect, the electronic device is preset with association information representing a correspondence between at least one contact in the first application and multiple portrait expression sets.

It can be understood that the association information is that the electronic device responds to the user's association operation and associates the portrait expression set in the electronic device with the contact in the first application. In some embodiments, the associated information can be associated with communication contacts in the address book application in the electronic device, for example, each portrait expression set and each contact in the first application are associated with the communication contacts in the address book application. , furthermore, when determining the first portrait expression set, it can be achieved by accessing the address book application. In some embodiments, the associated information can be determined by adding corresponding name tags or other custom tags to each set of portrait expressions and each contact in the first application to confirm the first set of portrait expressions. Certainly.

In a possible implementation of the above first aspect, the first conversation window includes a conversation area and an expression area, and displaying at least one portrait expression in the first portrait expression set in the first conversation window includes: displaying the first portrait expression in the expression area. A portrait expression tag of a portrait expression set, wherein the portrait expression tag is determined by the electronic device based on the portrait expression in the first portrait expression set; at least one portrait expression in the first portrait expression set is stored and displayed in the portrait tag.

In a possible implementation of the above first aspect, the portrait expression label is the portrait avatar of the first contact person.

It can be understood that the portrait avatar can be any image in the first portrait expression set that can represent the facial features of the first contact. Furthermore, when the electronic device displays the portraits of multiple contacts, the user can quickly determine through the portrait avatars. Which contact does the portrait expression set belong to.

In a possible implementation of the above first aspect, the above method further includes: corresponding to determining that there is no first set of portrait expressions related to the first contact, displaying the second set of portrait expressions in the first conversation window At least one portrait expression set, wherein the second portrait expression set is a set of portrait expressions among multiple portrait expression sets that meet preset conditions.

It can be understood that when the electronic device does not contain the portrait expression of the first contact, the electronic device can provide the user with portrait expressions of other contacts for the user to select.

In some embodiments, if there is a portrait expression of the first contact in the electronic device, the electronic device can also respond to the user's query operation and display the portrait expressions of other contacts on the display interface, and the user can also select the portrait expression of the first contact. Interact with the facial expressions of common friends to make chatting more interesting.

In a possible implementation of the above first aspect, the second portrait expression set is determined in the following manner: obtaining historical usage information of multiple portrait expression sets; determining the portrait expression corresponding to the first historical usage information that meets the preset conditions The set is the second portrait expression set.

In a possible implementation of the above first aspect, the historical usage information includes historical usage frequency; determining the portrait expression set corresponding to the first historical usage information that meets the preset conditions as the second portrait expression set includes: determining the historical usage frequency The portrait expression set corresponding to the first historical usage frequency higher than the preset frequency threshold is the second portrait expression set; or at least one portrait expression set with the highest historical usage frequency is determined to be the second portrait expression set.

In a possible implementation of the above first aspect, the first application includes multiple contacts, and the multiple portrait expression sets are generated in the following manner: acquiring multiple portrait images stored in the electronic device; At least one portrait image of the contact is divided into the same portrait image set; image processing is performed on at least one portrait image in each portrait image set to obtain a portrait expression set corresponding to each portrait image set, and each portrait expression set is compared with the first portrait image set Each contact in the application is associated.

In a possible implementation of the above first aspect, image processing is performed on at least one portrait image in each portrait image set to obtain a portrait expression set corresponding to each portrait image set, including: processing at least one portrait image in each portrait image set For portrait images, the expression production model is used to generate the portrait expression set corresponding to each portrait image set.

In a possible implementation of the first aspect above, the expression production model includes: a portrait sub-model that determines the weight of each portrait image as a preset expression; a text sub-model that determines the weight of the preset expression text as the preset expression; According to the output results of the portrait sub-model and the text sub-model, each portrait image and the matching preset expression text are generated to generate an expression generation sub-model corresponding to the portrait expression.

In a possible implementation of the above first aspect, the above method further includes: detecting the user's text input operation in the first conversation window of the first interface, and receiving the text input by the user; Determine at least one target portrait image that matches the text from a set of portrait images; the electronic device displays a second interface of the first application, and the second interface includes at least one target portrait image; detects that the user is in at least one of the second interface The image selection operation of the portrait image generates a target portrait expression based on the text and the portrait image corresponding to the image selection operation; and displays the target portrait expression in the first session window.

It is understandable that when the portrait expressions in the electronic device do not meet the user's needs, the electronic device can adjust the expression according to the text and selection input by the user. It automatically generates facial expressions from a certain portrait image, without the need for users to manually create them, and the operation is simple.

In a possible implementation of the above first aspect, the first application includes at least one of an instant messaging application, a text message, a conferencing application, and a social networking application.

It can be understood that instant messaging applications can be, for example, WeChat ^TM , QQ ^TM, etc., and social networking applications can be, for example, Douyin ^TM , Weibo ^TM , Xiaohongshu ^TM , etc. In some embodiments, the first application can also be other applications that can realize the transmission of image information between users and other users, such as ^TaobaoTM , etc., as long as the chat function between users can be realized, this application does not cover this. limit.

In a second aspect, embodiments of the present application provide an electronic device, including: one or more processors; one or more memories; one or more memories store one or more programs. When one or more programs are When one or more processors are executed, the electronic device is caused to execute the above interaction method.

In a third aspect, embodiments of the present application provide a computer-readable storage medium. Instructions are stored on the storage medium. When the instructions are executed on the computer, they cause the computer to perform the above interactive method.

In the fourth aspect, embodiments of the present application provide a computer program product, which includes a computer program/instruction. When the computer program/instruction is executed by a processor, the above interactive method is implemented.

Description of the drawings

Figures 1a to 1c show schematic diagrams of interface changes of some expression interaction processes provided by embodiments of the present application;

Figure 2 shows a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present application;

Figures 3a to 3e show some schematic diagrams of the interface changes of the mobile phone after adding the "Allow acquisition of gallery to generate emoticon package" switch provided by the embodiment of the present application;

Figure 4 shows a schematic flow chart of an interaction method provided by an embodiment of the present application;

Figures 5a to 5b are schematic diagrams of interface changes of some expression interaction processes provided by embodiments of the present application;

Figure 6 shows a schematic flow chart of an interaction method provided by an embodiment of the present application;

Figure 7 shows a schematic flow chart of an expression generation method provided by an embodiment of the present application;

Figures 8a to 8e are schematic diagrams of interface changes in the generation process of some expressions provided by embodiments of the present application;

Figure 9 shows a schematic flow chart of a method for automatically generating a portrait expression set provided by an embodiment of the present application;

Figure 10 shows a schematic diagram of the software structure of an electronic device provided by an embodiment of the present application.

Detailed ways

In order to solve the above-mentioned problems of cumbersome operations and poor experience when users process images into expressions and send them to chatters, this application provides an interactive method. The method includes: the electronic device divides the portrait images of the same person in multiple portrait images it stores into a portrait image set, and generates a portrait expression set for each portrait image set that can be used for communication applications. After the electronic device generates the portrait expression set , pre-associate each portrait expression set with each chat person in the communication application. Among them, the portrait expression set includes at least one expression. When the electronic device detects the user's expression input operation in the communication application, it obtains the contact information corresponding to the expression input operation. If the electronic device stores a portrait expression set corresponding to the contact, the acquired portrait expression set corresponding to the contact will be displayed on the display interface of the communication application for the user to select.

It can be understood that pre-association means that the electronic device determines the chat person information corresponding to each portrait expression set in the communication application by obtaining the user's input information. For example, you can add the name or other custom tags of the corresponding person to each set of portrait expressions, and at the same time, note the name or other custom tags of each chat person in the communication application. Through the same name or custom tag, you can realize the integration of each set of portrait expressions with each other. Pre-association of chat persons in communication applications. Another example is to correspond each portrait expression set to the communication contacts in the address book application in the electronic device, and at the same time allow The communication application is allowed to access the address book application of the electronic device to determine the corresponding communication contact information in the address book application of the electronic device based on the chat person information in the communication application, so as to realize the matching of each portrait expression set with each chat person in the communication application. Pre-associated.

In some embodiments, the electronic device can use the expression production model to produce multiple portrait images into each portrait expression set. The expression production model may be the corresponding relationship between the portrait image and the preset text obtained by training based on a large number of portrait images, corresponding expression tags, and preset text corresponding to the expression tag by the electronic device, and the portrait image and the corresponding preset text Text is combined to create a model of the resulting expression. The preset text can be text corresponding to commonly used expressions.

With the interactive method provided by this application, the electronic device can automatically create corresponding portrait expressions for the portrait images it stores. When the user needs to input an expression, the electronic device can display the set of portrait expressions corresponding to the contact that the user needs in response to the user's expression input operation. Furthermore, the user can operate the screen of the electronic device to select the user from the set of portrait expressions corresponding to the contact. The required emoticons do not require users to manually create and import the emoticons corresponding to the contacts. The user operation is simple and the experience is high.

In addition, in some embodiments, if the electronic device does not store the portrait expression set corresponding to the contact, the electronic device can display at least one portrait expression set with the highest frequency of use on the electronic device according to the user's frequency of use of each portrait expression. display interface.

In some other embodiments, the portrait expression set generated in the electronic device may be that after the electronic device automatically backs up the multiple portrait images it stores to the cloud server, the cloud server divides the backed up multiple portrait images into portrait images of the same person. to a set of portrait images, generate a set of portrait expressions for each set of portrait images that can be used in communication applications, and send each generated set of portrait expressions to an electronic device. It can be understood that the production of the portrait expression set can be executed by electronic equipment or by a cloud server.

The above interactive method will be introduced below with reference to the accompanying drawings, taking the electronic device 100 as a mobile phone as an example.

For example, Figure 1 shows an interface change diagram of the interaction method provided by the embodiment of the present application. As shown in Figure 1a, the mobile phone 100 displays a chat interface 101, and an expression input button 111 is displayed on the chat interface 101. The expression input button 111 can be a smiley face graphic as shown in Figure 1a, or other graphics representing expression input. When the user operates the mobile phone 100 to select an expression, the user can click the expression input button in the chat interface 101 of the mobile phone 100, that is, perform operation ①. During this process, the mobile phone 100 obtains the user's operation ①, that is, the user's expression input operation. The mobile phone 100 can respond to the user's operation ① and obtain the corresponding chat person information in the chat interface 101. At this time, the corresponding chat person information obtained by the mobile phone (that is, the corresponding contact person above) is Alice. If the mobile phone 100 stores a portrait expression set corresponding to the chatter Alice, the mobile phone 100 can obtain the portrait expression set corresponding to the chatter Alice, and display the expression selection interface 102 as shown in Figure 1b on the display interface of the mobile phone 100.

As shown in Figure 1b, in response to the user's operation ①, the mobile phone 100 displays an expression selection interface 102 as shown in Figure 1b on the display interface. The expression selection interface 102 displays an expression selection area 112 and an expression set selection area 113. The expression selection area 112 is used to display the expressions of the selected expression set, and the expression set selection area 113 displays the selectable expression set. After the user performs operation ①, the mobile phone 100 can display the expression corresponding to the default expression set of the communication application in the expression selection area 112, and at this time the default expression button 114 is triggered. The portrait expression set corresponding to the chat person Alice obtained by the mobile phone 100 in response to the user's operation ① can be displayed in the expression set selection area 113 in the form of a portrait expression button. Among them, the portrait expression button can be made by using the cover of the portrait image set corresponding to the portrait expression set. For example, in Figure 1b, the portrait expression set corresponding to the chatter Alice can be displayed as the chatter emoticon button 115. The user can also click the expression hiding button 116 in Figure 1b to obtain the portrait expression sets of other chatters.

In some embodiments, when the mobile phone 100 displays the portrait expression button, it may be displayed in the expression set selection area 113 according to whether the corresponding expression set is the portrait expression set of the corresponding chat person and the frequency of use of each portrait expression set.

When the user wants to send the expression automatically generated by the mobile phone 100 based on the stored portrait image, for example, when the user wants to send the expression concentrated based on the portrait expression of the corresponding chat person Alice to the corresponding chat person Alice, the user can click the chat person expression button 115, that is, execute Operation ②. In response to the user's operation ②, the mobile phone 100 displays the expression selection interface 103 on the display interface of the mobile phone 100, as shown in Figure 1c.

As shown in Figure 1c, in the expression set selection area 113 of the expression selection interface 103, the chat person's expression button 115 is triggered. At this time The expression selection area 112 may display at least one expression in the portrait expression set corresponding to Alice, the chat person, automatically generated by the mobile phone 100 .

It is understandable that when the user wants to send his or her emoticon to the corresponding chat person, there is no need to manually process the image of the current chat person and make an emoticon package. The mobile phone 100 can automatically generate a portrait expression set of each chatter based on the stored multiple portrait images, and display the portrait expression set on the display interface of the mobile phone 100. The user can select the expression of the corresponding chatter by clicking the chatter expression button 115. The user's expression-based interaction is simple to operate and has a higher sense of experience.

It can be understood that the interactive methods provided by the embodiments of the present application are applicable to electronic devices including but not limited to mobile phones, portable computers, laptop computers, desktop computers, tablet computers, head-mounted displays, and mobile emails. devices, automotive devices, portable game consoles, reader devices, televisions with one or more processors embedded or coupled therein, or other electronic devices capable of accessing a network.

Exemplarily, FIG. 2 shows a schematic diagram of the hardware structure of an electronic device.

As shown in Figure 2, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a sensor module 180, a display screen 190, etc. The sensor module 180 may include a pressure sensor 180A, an acceleration sensor 180E, a touch sensor 180K, etc.

It can be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100 . In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figures, or some components may be combined, some components may be separated, or some components may be arranged differently. The components illustrated may be implemented in hardware, software, or a combination of software and hardware.

The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processing unit (GPU), and an image signal processor. (image signal processor, ISP), controller, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural network processor (neural-network processing unit, NPU), etc. Among them, different processing units can be independent devices or integrated in one or more processors. The controller can generate operation control signals based on the instruction operation code and timing signals to complete the control of fetching and executing instructions. The processor 110 may also be provided with a memory for storing instructions and data. In the embodiment of the present application, the relevant instructions and data for executing the interactive method of the present application can be stored in the memory for the processor 110 to call. The processor 110 can control the execution of each step of the interactive method through the controller. The specific implementation process will be It is described in detail below and will not be repeated here.

In some embodiments, processor 110 may include one or more interfaces. Interfaces may include integrated circuit (inter-integrated circuit, I2C) interface, mobile industry processor interface (MIPI), general-purpose input/output (GPIO) interface, subscriber identity module (subscriber identity) module, SIM) interface, and/or universal serial bus (universal serial bus, USB) interface, etc.

The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (derail clock line, SCL). In some embodiments, processor 110 may include multiple sets of I2C buses. The processor 110 may separately couple the touch sensors 180K and the like through different I2C bus interfaces. For example, the processor 110 can be coupled to the touch sensor 180K through an I2C interface, so that the processor 110 and the touch sensor 180K communicate through the I2C bus interface to implement the touch function of the electronic device 100 .

The MIPI interface can be used to connect the processor 110 and peripheral devices such as the display screen 190 . MIPI interfaces include camera serial interface (CSI), display serial interface (DSI), etc. The processor 110 and the display screen 190 communicate through the DSI interface to implement the display function of the electronic device 100 .

The GPIO interface can be configured through software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface may be used to connect the processor 110 display 190, sensor module 180, etc. The GPIO interface can also be configured Set as I2C interface, MIPI interface, etc.

It can be understood that the interface connection relationships between the modules illustrated in the embodiments of the present application are only schematic illustrations and do not constitute a structural limitation of the electronic device 100 . In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

The electronic device 100 implements display functions through a GPU, a display screen 190, an application processor, and the like. The GPU is an image processing microprocessor and is connected to the display screen 190 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or alter window display information.

The display screen 190 is used to display images, videos, etc. Display 190 includes a display panel. The display panel can use a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light emitting diode or an active matrix organic light emitting diode (active-matrix organic light emitting diode). emitting diode (AMOLED), flexible light-emitting diode (FLED), Mini-LED, Micro-LED, Micro-OLED, quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 190, where N is a positive integer greater than 1.

The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement the data storage function. Such as saving music, videos, etc. files in external memory card.

Internal memory 121 may be used to store computer executable program code, which includes instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the stored program area can store an operating system, at least one application program required for a function (such as a sound playback function, an image playback function, etc.). The storage data area may store data created during use of the electronic device 100 (such as audio data, phone book, etc.). In addition, the internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by executing instructions stored in the internal memory 121 and/or instructions stored in a memory provided in the processor.

In the embodiment of the present application, the internal memory 121 can store execution instructions for implementing the interactive method of the present application for the processor 110 to call to implement the interactive method of the present application, so that the electronic device 100 automatically obtains the expression when the user performs an expression input operation. The portrait expression set corresponding to the contact stored in the electronic device does not require the user to manually import it, improving the user experience.

The pressure sensor 180A is used to sense pressure signals and can convert the pressure signals into electrical signals. In some embodiments, the pressure sensor 180A may be disposed on the display screen 190 . There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. A capacitive pressure sensor may include at least two parallel plates of conductive material. When a force is applied to pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure based on the change in capacitance. When a touch operation is performed on the display screen 190 , the electronic device 100 detects the strength of the touch operation according to the pressure sensor 180A. The electronic device 100 may also calculate the touched position based on the detection signal of the pressure sensor 180A.

The acceleration sensor 180E can detect the acceleration of the electronic device 100 in various directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of electronic devices and be used in horizontal and vertical screen switching, pedometer and other applications.

Touch sensor 180K, also known as "touch device". The touch sensor 180K can be disposed on the display screen 190. The touch sensor 180K and the display screen 190 form a touch screen, which is also called a "touch screen". The touch sensor 180K is used to detect a touch operation on or near the touch sensor 180K. The touch sensor can pass the detected touch operation to the application processor to determine the touch event type. Visual output related to the touch operation may be provided through the display screen 190 . In the embodiment of the present application, for example, the touch screen composed of the touch sensor 180K and the display screen 190 can detect the user's click operation. With the user's click operation, the touch screen can display corresponding interface changes, such as click emoticons. If the switch is obtained, a floating window interface will pop up on the current display interface. For details, please refer to the detailed description below and will not be repeated here. In other embodiments, the touch sensor 180K may also be disposed on the surface of the electronic device 100 at a location different from that of the display screen 190 .

Based on the structure of the electronic device 100 shown in FIG. 2 , the implementation process of the interactive method according to the embodiment of the present application will be described in detail below with reference to the accompanying drawings, taking the electronic device 100 as a mobile phone as an example.

First of all, it should be noted that in some embodiments, in order to facilitate user operations, a switch for obtaining emoticons can be added to the operation interface of the communication application. The user can choose whether to turn on the switch for obtaining emoticons, thereby choosing whether to turn on the communication application to obtain the mobile phone 100 The function of automatically generating emoticons.

For example, as shown in Figure 3a, the communication application is ^WeChatTM . The user operates the mobile phone 100 to open WeChat ^TM , and the WeChat ^TM interface 301 is displayed on the display interface of the mobile phone 100. Then the user can click the setting button 311 to open the WeChat ^TM setting interface 312, as shown in Figure 3b. The setting interface 312 of WeChat ^TM includes various settings of WeChat ^TM , and various settings include added emoticon package acquisition switches, such as the "allow obtaining emoticon packages generated by the gallery" switch 312 in the setting interface 312 . The user can click the "Allow acquisition of emoticon packages generated by the gallery" switch 312 in the setting interface 302 of WeChat ^TM to enable the function of WeChat ^TM to obtain the emoticon packages automatically generated in the mobile phone 100 .

Further, in some embodiments, when the user clicks the "Allow acquisition of emoticon packages generated by the gallery" switch 312 in the setting interface 302 of WeChat ^TM , a floating window interface 303 will pop up above the setting interface 302 of WeChat ^TM , as shown in Figure 3c shown. In the floating window interface 303, the words "Whether the gallery is allowed to automatically generate emoticons" will be displayed, as well as the "OK" button 321 and the "Cancel" button 322. At this time, the user can click the "OK" button 321 to enable the automatic generation of emoticon packages by the gallery, and then WeChat ^TM can obtain the emoticon packages automatically generated by the gallery and use them for emoticon interaction between the user and the corresponding contact. After the user clicks the "OK" button 321, the floating window interface 303 will disappear, and only the WeChat ^TM setting interface 304 will be displayed on the screen of the mobile phone 100, as shown in Figure 3d. At this time, in the display interface of the mobile phone 100, the "Allow acquisition of emoticon packages generated by the gallery" switch 312 is turned on.

The user can also click the "Cancel" button 322 in the floating window interface 303 to abandon the function of allowing the gallery to automatically generate emoticons. At this time, by default, the user of the mobile phone 100 gives up turning on the "Allow acquisition of emoticons generated from the gallery" switch 312, that is, the user can also click the "Cancel" button 322 in the floating window interface 303, and the floating window interface 303 will disappear. At this time, the mobile phone 100 The display interface still displays the setting interface 302 of WeChat ^TM , and the "Allow acquisition of emoticon packages generated by the gallery" switch 312 is turned off.

It can be understood that the "Allow acquisition of emoticon packages generated by the gallery" switch 312 in the embodiment of the present application can be turned on only after the gallery is authorized to automatically generate emoticon packages. In some embodiments, other software other than the gallery may be used to automatically generate emoticon packages. Correspondingly, the emoticon package acquisition switch may also be displayed through other types of switches. This application does not limit this.

In some embodiments, after the user clicks the "Allow acquisition of emoticons generated by the gallery" switch 312 in the settings interface 302 of WeChat ^TM as shown in Figure 3b, the display interface of the mobile phone 100 will jump from the settings interface 302 of WeChat ^TM . To the setting interface 305 of the gallery, that is, after the user clicks the "Allow acquisition of emoticons generated by the gallery" switch 312, the display interface of the mobile phone 100 displays the setting interface 305 of the gallery, as shown in Figure 3e. The user can turn on the "Allow automatic generation of emoticons" switch 331 in the setting interface 305 of the gallery shown in Figure 3e. If the user turns on the "Allow automatic generation of emoticons" switch 331, the display interface of the mobile phone 100 will jump back to ^WeChatTM . Settings interface, and the "Allow acquisition of emoticon packages generated by the gallery" switch 312 is turned on, and the display interface of the mobile phone 100 is as shown in Figure 3d.

It can be understood that in some embodiments, the method for turning on the emoticon package acquisition switch can also adopt other methods to achieve interaction with the user, and this application does not limit this.

After performing the above operation and turning on the emoticon package acquisition switch, the interaction method provided by the embodiment of the present application will be further introduced below in conjunction with Figure 4.

Figure 4 shows a schematic flowchart of an interaction method provided by an embodiment of the present application.

As shown in Figure 4, the interaction method provided by the embodiment of the present application is applied to the mobile phone 100, including:

401: Detect the user's expression input operation in the communication application, and obtain the corresponding contact information for the expression input operation. Among them, hand The display interface of the machine 100 displays the chat interface of the communication application.

It can be understood that communication applications can include instant messaging applications such as ^WeChatTM and ^QQTM , social network platforms with chat functions such as ^DouyinTM , ^WeiboTM , and ^{XiaohongshuTM} , and text messaging and conferencing applications. , and the communication application can support the sending and receiving of image type files, that is, the communication application has the expression interaction function. The image type files can be static images or dynamic images.

It can be understood that the user's expression input operation can generate a trigger event for the user to operate the screen of the mobile phone 100, and the trigger event is related to the location of the user's operation and the user's operation method.

For example, the user's expression input operation can be, for example, the user clicks the expression input button 111 as shown in Figure 1a, that is, the user performs operation ①, and the mobile phone 100 detects that the expression input button 111 in the chat interface of WeChat ^TM is triggered, which can be considered as detecting the user. The emoticon input operation triggers the acquisition of the chatter information corresponding to the emoticon input operation.

It can be understood that the mobile phone 100 obtains the chat person information corresponding to the expression input operation, so that the mobile phone 100 obtains the corresponding chat person information for chatting with the user in the display page where the expression input operation is performed. The obtained corresponding chat person information may include the corresponding chat person's user name, account number, and the user's remarks about the corresponding chat person, etc.

For example, if the communication application is ^WeChatTM , the corresponding chat person information may include the corresponding chat person's nickname, the user's notes about the corresponding chat person, mobile phone number, WeChat ID, etc. It can be understood that the corresponding chat person information obtained by the communication application can be used to match the corresponding chat person with the portrait expression set stored in the mobile phone 100 .

402: Determine whether to store the portrait expression set corresponding to the contact.

It can be understood that the portrait expression set is the portrait photos stored in the mobile phone 100. It automatically clusters the images, divides the portrait images corresponding to the same person into a portrait image set, and performs image processing on each portrait image set to generate a corresponding Portrait expression sets, each generated portrait expression set can be stored in the mobile phone 100 as a portrait expression library automatically generated by the mobile phone 100 . That is, the mobile phone 100 processes the portrait images stored therein and generates a portrait expression library including at least one portrait expression set, and the portrait expression set includes at least one expression of the same person generated by the mobile phone 100 .

In some embodiments, the clustering of images and the generation of portrait image sets and portrait expression sets can be implemented through photo album applications, such as the gallery application in the mobile phone 100, and after generating the portrait expression set, the photo album application needs to The emoticon set is pre-associated with the chat person in the messaging application. Further, in step 402, the mobile phone 100 determines whether the portrait expression set corresponding to the contact is stored, that is, the gallery determines whether its corresponding stored data includes the portrait expression set corresponding to the contact.

In some embodiments, after generating each portrait expression set, the photo album application can obtain the communication contact information in the address book application of the mobile phone 100, and establish a relationship between the portrait expression set and the communication contact information based on the user's input information. For example, the communication contact information can be used as the label of the portrait expression set. The establishment of the portrait expression library and the correspondence between the portrait expression set and communication contact information will be further introduced below and will not be described in detail here.

Furthermore, the communication application may also include an address book access switch. The user can choose whether to enable the function of the communication application to obtain communication contact information stored in the address book by choosing whether to turn on the address book access switch. The user can operate the mobile phone 100 to turn on the address book access switch. Then in step 402, when the mobile phone 100 determines whether to store the portrait expression set of the corresponding contact, it can obtain the communication contact information in the address book for the communication application, and determine the corresponding contact information of the corresponding contact. Communication contact information, and send the confirmed communication contact information to the gallery application. The gallery application determines whether the portrait expression set corresponding to the received communication contact information is stored in the portrait expression library based on the received communication contact information.

In some embodiments, the user can add the name of the person corresponding to the portrait expression set as the label of the portrait expression set automatically generated by the photo album application to the corresponding portrait expression set. The user can also customize the label of each portrait expression set and add to the corresponding portrait expression set. Furthermore, after obtaining the corresponding contact information, the mobile phone 100 can match it with the tags of each portrait expression set to determine the corresponding contact information. Set of portrait expressions.

If the judgment result in step 402 is yes, step 403 is executed.

403: Obtain and display the portrait expression set of the corresponding contact.

It can be understood that when the mobile phone 100 displays the portrait expression set corresponding to the contact, it can add a portrait expression button (ie, the chat person expression button above) that stores the portrait expression set corresponding to the contact in the expression area of the communication application. Furthermore, when the mobile phone 100 detects that the user clicks the portrait expression button, it can display the expression corresponding to the portrait expression set of the contact in the expression display area of the communication application. When the display interface of the mobile phone 100 displays the expression of the corresponding contact, the user can select the expression of the corresponding contact that he needs from the displayed expression of at least one corresponding contact, and send it to the corresponding contact.

In some embodiments, when generating a portrait expression set, the gallery application will select a portrait image in the corresponding portrait image set or an expression in the portrait expression set as the cover of the portrait expression set. Further, when the mobile phone 100 adds a portrait expression button, the cover of the portrait expression set corresponding to the contact can be used as the portrait expression button corresponding to the contact.

For example, as shown in Figure 1c, if the communication application is ^WeChatTM , then the portrait expression set corresponding to the chatter obtained by the mobile phone 100 is Alice's portrait expression set, and the newly added portrait expression button corresponding to the chatter can be, for example, the chatter expression button 115 , when the user clicks the chat person expression button 115, at least one expression corresponding to the portrait expression set of the chat person Alice can be displayed in the expression selection area 112. In some embodiments, the expression selection area 112 displays the expressions in the portrait expression set corresponding to the chatter in descending order according to the number of uses of the expressions in the portrait expression set of the corresponding chatter.

The expression interaction method provided by the embodiment of the present application can display the portrait expression set corresponding to the contact required by the user in response to the user's expression input operation. Furthermore, the user can operate the screen of the electronic device to select from the portrait expression set corresponding to the contact. The emoticons that users need do not require users to manually create and import emoticons for corresponding contacts. The user operation is simple and the experience is high. At the same time, emoticons made from portrait images of corresponding contacts are used as expressions for interaction between the user and the corresponding contact, which can make the chat more interesting. And the expressions made with portrait images corresponding to the contacts can make the expressions more familiar.

Continuing to refer to Figure 4, in some embodiments, when the determination result in step 402 is no, the mobile phone 100 performs the following steps:

404: Obtain all stored portrait expression sets and obtain the user's historical usage information for each portrait expression set.

It can be understood that the historical usage information may include, for example, the user's frequency of use of each portrait expression set, the user's recently used portrait expression sets, etc. The frequency of use of each portrait expression set can represent the user's preference for each portrait expression set, and the user's recently used portrait expression set can represent the user's recently preferred portrait expression set. In some embodiments, the mobile phone 100 can select a category from the user's frequency of use of each portrait expression set, the user's most recently used portrait expression set, etc., as historical usage information. In other embodiments, the mobile phone 100 may use the user's frequency of use of each portrait expression set and the weighted values of at least two categories in the user's recently used portrait expression sets as historical usage information.

405: According to the historical usage information of each portrait expression set, obtain and display the portrait expression set that meets the preset conditions.

It is understood that the preset conditions can be set based on historical usage information. For example, when the historical usage information is the user's frequency of use of each portrait expression set, the preset condition can be that the top three most frequently used portrait expression sets are displayed, and the preset condition can also be that the usage frequency of the portrait expression set exceeds the preset limit. Set threshold. The same applies when the historical usage information is other categories of information.

As shown in Figures 5a to 5b, they are interface diagrams of some interactive methods provided by embodiments of the present application.

In some embodiments, taking the communication application WeChat ^TM as an example, the mobile phone 100 can also provide portrait expressions of other chat persons (corresponding to the contacts mentioned above) for the user to communicate with the corresponding chat persons (ie, corresponding contacts). Expression interaction. For example, when the expression of the corresponding chat person cannot meet the user's expression interaction needs, or the user prefers to select expressions from the portrait expression set of other chat persons to interact with the corresponding chat person, or the user needs portrait images of common friends with the corresponding chat person The made emoticons make chatting with the corresponding chat person more interesting, etc.

Specifically, the mobile phone 100 can provide a query button for the portrait expression set on the display interface. The mobile phone 100 detects that the user is using the communication application. When the query button of the portrait expression set is clicked, the mobile phone 100 can display a portrait expression set selection window on the display interface. Each portrait expression set can be displayed sequentially in the portrait expression set selection window through its corresponding cover or label of the portrait expression set.

In some embodiments, when each portrait expression set is displayed sequentially in the portrait expression set selection window through the label of the portrait expression set, each portrait expression set can be sorted according to the label, and the portrait expression set selection window can be displayed according to the sorting result. to display. For example, if the label of the portrait expression set is the name of the person, the labels of each portrait expression set can be sorted according to the first letter of the name.

As shown in Figure 5a, the display interface of the mobile phone 100 displays the chat interface 501 of WeChat ^TM . The emoticon set selection area 511 of the chat interface 501 of ^WeChatTM includes a chat person emoticon button 521 and a query button 522. The user can click the query button 522. At this time, the mobile phone responds to the user's click operation and displays the portrait expression set selection window 502 in a floating window mode above the chat interface 501 of ^WeChatTM . As shown in Figure 5a, the portrait expression set selection window 502 can display the labels of the portrait expression sets of other chatters except the portrait expression set of the corresponding chatter, and the labels of each portrait expression set can be sorted by first letter. For example, if the labels of the portrait expression set include "Peter Pan" and "Zwc", "Peter Pan" can be ranked first and "Zwc" can be ranked second.

In some embodiments, when each portrait expression set is displayed sequentially in the portrait expression set selection window through the cover of the portrait expression set, the portrait expression set can be sorted according to the historical usage information of each portrait expression set, and the portrait expression set selection can be made based on the sorting results. displayed in the window. The historical usage information is introduced above and will not be described in detail here. For example, the covers of each portrait expression set are sorted according to the frequency of use of each portrait expression set from high to low.

As shown in Figure 5b, the display interface of the mobile phone 100 displays the chat interface 503 of WeChat ^TM . The emoticon set selection area 512 of the chat interface 503 of ^WeChatTM includes a chat person emoticon button 531 and a query button 532. The user can click the query button 532. At this time, the mobile phone responds to the user's click operation and displays the portrait expression set selection window 504 in a floating window mode above the chat interface 503 of ^WeChatTM . As shown in Figure 5b, the portrait expression set selection window 504 can display the covers of portrait expression sets of other chatters except the portrait expression set of the corresponding chatter, and the cover of each portrait expression set can be based on the frequency of use of each portrait expression set. Sort from high to low. For example, if the portrait expression set corresponding to the cover 541 of the portrait expression set is used more frequently than the portrait expression set corresponding to the cover 542 of the portrait expression set, then the cover 541 of the portrait expression set and the cover 542 of the portrait expression set are in the portrait expression set selection window. The display of 504 can be as shown in Figure 5b.

Next, with reference to Figure 6, taking the communication application WeChat ^TM and the gallery application automatically generating emoticons as an example, the interaction method in the embodiment of the present application will be further introduced.

Figure 6 shows a flow chart of an interaction method provided by an embodiment of the present application.

As shown in Figure 6, in some embodiments, the interaction method includes:

601: WeChat ^TM 611 detects the user's expression input operation.

It can be understood that the expression input operation is for the user to click an expression input button on the chat interface of WeChat ^TM 611, such as the expression input button 111 in Figure 1a. ^WeChatTM 611 detects the user's operation of clicking the expression input button 111, that is, detects the user's expression input operation.

602: WeChat ^TM 611 obtains the corresponding chat person information in response to the user's expression input operation.

It can be understood that after WeChat ^TM 611 obtains the emoticon input operation, it indicates that the user needs to input emoticons in the chat interface of WeChat ^TM 611. At this time, WeChat ^TM 611 turns on the emoticon package acquisition switch, such as "Allow acquisition of gallery generation" in Figure 3b "emoticon package" switch, in order to automatically obtain the required portrait emoticon set, WeChat ^TM 611 needs to first obtain the corresponding chat person information to chat with the user.

603: WeChat ^TM 611 sends the corresponding chat person information to the gallery application 612.

It can be understood that after WeChat ^TM 611 obtains the corresponding chat person information, it will send the obtained corresponding chat person information to the gallery application 612, and apply to the gallery application 612 for the portrait expression set of the corresponding chat person automatically generated by the gallery application 612.

604: The gallery application 612 determines whether to store the portrait expression set corresponding to the chat person.

It can be understood that in response to the information from ^WeChatTM 611, the gallery application 612 will match the corresponding chat person information with the tags of each portrait expression set generated by the gallery application 612, and determine whether there is a corresponding chat person in each portrait expression set generated by the gallery application 612. set of portrait expressions. The tags of the portrait expression set may be, for example, the name of the portrait corresponding to the portrait expression set, a user-defined tag, the portrait expression set, and the corresponding communication contact information in the address book application.

If the determination result in step 604 is yes, ^WeChatTM 611 can directly display the portrait expression set corresponding to the chat person, and execute step 605.

605: The gallery application 612 sends the portrait expression set corresponding to the chat person to the WeChat ^TM 611.

It can be understood that if the gallery application 612 matches the portrait expression set of the corresponding chat person, it can respond to the application of WeChat ^TM 611 and send the portrait expression set of the corresponding chat person to WeChat ^TM 611 .

606: WeChat ^TM 611 displays the portrait expression set corresponding to the chat person.

If the judgment result in step 604 is no, the portrait image corresponding to the chatter may not be stored in the gallery application 612, and the portrait image corresponding to the chatter may not be processed to obtain the portrait expression set corresponding to the chatter, then the gallery application 612 The remaining portrait expression sets can be sent to ^WeChatTM 611 and displayed according to the user's preference, that is, steps 607 and 608 are performed.

607: The gallery application 612 obtains the historical usage information of each portrait expression set.

It can be understood that the historical usage information of the portrait expression set can represent the user's usage preference for each portrait expression set. Among them, historical usage information has been introduced in detail previously and will not be described again here.

608: The gallery application 612 sends at least one portrait expression set that meets the preset conditions to WeChat ^TM 611.

It can be understood that the gallery application 612 sends at least part of the portrait expression set generated by it to the WeChat ^TM 611, without the user having to manually create the portrait expressions or manually import the portrait expressions into the WeChat ^TM 611. User operation is simple.

609: ^WeChatTM 611 displays at least one received portrait expression set that meets the preset conditions.

In some embodiments, when the user wants to obtain an emoticon package with specified text, the user can enter the text he wants to specify and the portrait image that needs to be involved in making the emoticon. The mobile phone 100 can automatically generate the corresponding emoticon based on the information input by the user. expression. The process will be further introduced below with reference to Figures 7 to 8d, taking the communication application as WeChat ^TM 611 and making the expression of the corresponding chat person as an example.

Figure 7 shows a schematic flowchart of an expression generation method provided by an embodiment of the present application.

Figures 8a to 8e show interface change diagrams of some expression generation processes provided by embodiments of the present application.

As shown in Figure 7, the method includes:

701: WeChat ^TM 611 obtains the target emoticon text input by the user.

It can be understood that the target emoticon text is the content input by the user in the text input box of WeChat ^TM 611.

For example, the display interface of the mobile phone 100 displays the chat interface 801 of WeChat ^TM 611, as shown in Figure 8a. A text input box 811 of WeChat ^TM 611 is displayed in the chat interface 801 of WeChat ^TM 611. The user can input the text content he wants to send or the target emoticon text corresponding to the target emoticon of the chatter in the text input box 811. The chat interface 801 of ^WeChatTM 611 also includes an expression selection area 810. Among them, the expression selection area 810 displays expressions corresponding to the portrait expression set of the chatter, and an add button 812 .

702: WeChat ^TM 611 detects the user's expression adding operation, and obtains the corresponding chat person information in response to the expression adding operation.

It can be understood that the user can operate the corresponding position on the screen of the mobile phone 100 to trigger the expression adding button. At this time, the WeChat ^TM 611 detects the user's expression adding operation.

For example, as shown in Figure 8a, the user can click the add button 812 in the chat interface 801 of WeChat ^TM 611. When the add button 812 is triggered, the WeChat ⁶¹¹ detects the user's expression adding operation.

703: WeChat ^TM 611 obtains the portrait image set of the corresponding chat person from the gallery application 612.

It can be understood that when WeChat ^TM 611 detects the operation of adding an expression, it needs to obtain a portrait image of the corresponding chat person that can be made into an expression. Therefore, WeChat ^TM 611 will obtain a set of portrait images of the corresponding chat person in the gallery application 612.

704: WeChat ^TM 611 displays the portrait image set corresponding to the chat person.

It is understandable that after the user performs the operation of adding an expression, the portrait image of the corresponding chat person will pop up at the top of the display interface of WeChat ^TM 611 set.

For example, after the user clicks the add button 812 in the chat interface 801 of WeChat ^TM 611 shown in Figure 8a, WeChat ^TM 611 will display the obtained set of portrait images of the corresponding chat person on the chat interface 801 of WeChat ^TM 611 in a floating window mode. above. As shown in Figure 8b. After the user clicks the add button 812, a portrait selection window 802 will be displayed above the display interface 801 of the WeChat ^TM 611. The portrait selection window 802 may include a portrait selection area 813 and a "manual specification button" 814.

705: ^WeChatTM 611 detects the user's portrait designation operation, and determines the portrait image information corresponding to the portrait designation operation.

It can be understood that the portrait designation operation is for the user to select a portrait image that matches the target expression text from the set of portrait images displayed on WeChat ^TM 611.

For example, as shown in FIG. 8b , after the portrait selection window 802 pops up on the mobile phone 100 , the user can click the portrait image that matches the target expression text among the multiple portrait images corresponding to the chat person displayed in the portrait selection area 813 to select the target expression. Emoji text matched to portrait image. At this time, the user can click the "manual designation" button 814 to complete the portrait designation operation.

It can be understood that in some embodiments, the portrait image information corresponding to the portrait specifying operation determined by ^WeChat 611 may be a portrait image. In some embodiments, the portrait image information corresponding to the portrait specifying operation determined by ^WeChatTM 611 may be other types of information that can characterize the portrait image, such as the name of the portrait image, the characteristics of the portrait image, etc. This application does not limit this.

706: WeChat ^TM 611 sends the determined portrait image information and target emoticon text to the gallery application 612.

707: The gallery application 612 generates a target expression based on the received portrait image information and target expression text.

It can be understood that after receiving the portrait image information, the gallery application 612 will determine the corresponding portrait image of the chatting person, and combine it with the target expression text to generate the target expression.

708: The gallery application 612 sends the target expression to WeChat ^TM 611.

709: ^WeChatTM 611 displays the target expression on the display interface.

It can be understood that after receiving the target expression, WeChat ^TM 611 can add the target expression to the portrait expression set of the corresponding chat person in the chat interface of WeChat ^TM 611.

In some embodiments, when performing a portrait designation operation, the user can also specify the font, font size, text color, etc. of the target text. Furthermore, the information sent by ^WeChatTM 611 to the gallery application 612 also includes the font, font size, etc. of the target emoticon text. , text color, etc.

The expression generation method described in the above steps 701 to 709 will be further introduced with specific embodiments in conjunction with Figures 8c to 8e.

As shown in Figure 8c, the display interface of the mobile phone 100 displays the chat interface 803 of WeChat ^TM 611. The chat interface 803 of WeChat ^TM 611 shown in Figure 8c is the same as the chat interface 801 of WeChat ^TM 611 shown in Figure 8a above, including a text input box 811 of WeChat ^TM 611, in which the user can input target emoticon text. "Shock!!". The chat interface 803 of ^WeChatTM 611 also includes an add button 812.

When WeChat ^TM 611 detects the user's operation of clicking the add button 812, WeChat ^TM 611 will obtain the portrait image set of the corresponding chat person ^{from the gallery application 612. At this time, WeChat TM 611} adds an operation in response to the user's expression. Above the chat interface 803, a portrait selection window 804 is displayed in a floating window mode. The portrait selection window 804 may include a portrait selection area 813 and a "manual specification button" 814.

The user can select a portrait image that matches the target emoticon text "Shocked!!" in the portrait selection area 813. At this time, the user selects image 815. After selecting the image 815, the user can click the "manual designation button" 814 to complete the selection of the portrait image. At this time, ^WeChatTM 611 can determine the portrait image information of the image 815, and send the information and the target emoticon text "Shocked!!" to the gallery application 612. The gallery application 612 may determine the portrait image corresponding to the target emoticon text based on the portrait image information, and produce the target emoticon text and the determined portrait image into the target emoticon.

After automatically generating the target expression, the gallery application 612 can send the target expression to WeChat ^TM 611, and WeChat ^TM 611 will display it on its chat interface, as shown in Figure 8e. After acquiring the target expression, WeChat ^TM 611 displays the chat interface 805 of WeChat ^TM 611. Among them ^WeChatTM 611 The newly created target emoticon 821 is displayed in the emoticon selection area of the chat interface 805.

In some embodiments, the target expression generated by the user by specifying the target expression text will be added to the portrait expression set of the corresponding chat person, and when WeChat ^TM 611 displays the portrait expression set of the corresponding chat person, the target expression will be displayed on the portrait The top position in the emoticon set can be understood as the target expression generated by the user by specifying the target emoticon text is more reliable.

It can be understood that in the above embodiment, the expression generation process is introduced by taking the expression of the corresponding chat person as an example. The expression of other portrait expression sets can also be added in the same way, and this application does not limit this.

The following is an introduction to the process of automatically generating a portrait expression set in the embodiment of the present application with reference to Figure 9 .

FIG. 9 is a schematic flowchart of a method for automatically generating a portrait expression set provided in an embodiment of the present application.

As shown in Figure 9, the method includes:

901: Obtain the portrait image stored in the mobile phone 100.

It can be understood that in some embodiments, the automatic generation method of expressions provided in the embodiments of the present application can be executed by a photo album application to complete the production of a portrait expression set, such as a photo gallery application. Furthermore, obtaining the portrait image stored in the mobile phone 100 can be understood as the gallery application obtaining the portrait image data stored in its corresponding storage module.

902: Cluster the portrait images stored in the mobile phone 100 according to the different characters corresponding to the different portrait images.

It can be understood that clustering the portrait images is to extract features from all the portrait images stored in the mobile phone 100 and then classify the portrait images with similar features into the same category.

In some embodiments, the clustering algorithm for clustering portrait images can be, for example, K-Means clustering algorithm, mean shift clustering algorithm, etc., and a matching algorithm can be selected according to the portrait image to be processed, etc. Clustering algorithm, this application does not limit this.

903: Divide portrait images of the same category into a portrait image set.

It can be understood that the portrait images of the same category obtained by using the clustering algorithm can have the same character characteristics, that is, the portrait images of the same category are portrait images of the same task, and the portrait images belonging to the same category can be used as a portrait image set.

904: Use the expression production model to make the portrait images in each portrait image set into expressions to obtain each portrait expression set.

It can be understood that the expression production model is based on a large number of portrait images, the expressions corresponding to the portrait images, and the expression text corresponding to the expression label, and generates a model that represents the correspondence between the portrait image and the expression text, and the expression production model can also be used for portrait images. Analyze and calculate the target position of the emoticon text in the portrait image, and determine the font size and text color of the emoticon text based on the size and color of the target position. You can also set the correspondence between the emoticon text and the font according to the emotional color of the emoticon text. relation.

In some embodiments, the expression production model includes a portrait sub-model, a text sub-model, and an expression generation sub-model.

Among them, the portrait sub-model is a sub-model established using computer vision (Computer Vision, cv) technology. For the input portrait image, the portrait sub-model can calculate the weight of various expressions of the portrait image. Specifically, corresponding expression tags can be set in advance in the portrait sub-model according to different expressions, such as smile, frown, laugh, etc. Then, for the input portrait image, the portrait sub-model can calculate the weights of smile and frown respectively. weight, size weight, etc., and output at least one expression label with the highest weight as the output result of the portrait model.

Among them, the text sub-model is the preset emoticon text obtained through model training and is the weight of various emoticon tags. Specifically, the same multiple expression tags corresponding to the expression tags of the portrait expression can be set in advance in the text sub-model, such as smile, frown, laugh, etc., and then the text sub-model can calculate for the input preset expression text The emoticon text is the weight of each emoticon label, and at least one emoticon label with the highest weight is output as the output result of the text sub-model.

The expression generation sub-model matches the output results of the portrait sub-model and the text sub-model, determines the preset expression text corresponding to the portrait image, and performs image processing on the portrait image by analyzing the portrait image, such as cropping and adding filters. Mirror, etc., and determine the target position, font size, font, text color, etc. of the preset emoticon text at the same time. Specifically, the matching process can be, for the portrait sub-model input The output results of each portrait image and the output results of each preset emoticon text output by the text sub-model are compared with the expression labels and corresponding weights of the portrait images and the preset emoticon text expression labels and corresponding weights. Matching determines whether expression tags and portrait images with similar weights match the preset expression text. After the matching is completed, during the process of generating portrait expressions, face analysis technology can be used to analyze the portrait image, determine the position of the face in the image, and calculate the areas with small differences in color blocks around the face. For example, you can give priority to blank spaces. area, use this area as the location of the preset emoticon text.

Furthermore, in some embodiments, the color of the preset emoticon text can be a contrasting color of the blank area to highlight the emoticon text.

In some embodiments, based on the determined area with small difference in color blocks, the shape and size of the area can be determined to determine the layout and font size of the preset emoticon text in the area.

In some embodiments, the font of the emoticon text can be determined according to the emotional bias of the emoticon corresponding to the preset emoticon text. For example, for the emoticon text corresponding to happy, a round and cheerful font can be used.

In some embodiments, the interception size of the portrait expression on the portrait image can be controlled according to the length of the preset emoticon text. When there are many preset emoticon words, the size of the interception area can be appropriately expanded to place the preset emoticon text.

In some embodiments, in step 901, a video file including human faces may also be obtained as material for making portrait expressions. Specifically, video frames including human faces in the video file can be extracted, and the extracted video frames can be input into the portrait sub-model as the acquired face images, and the weights of each expression label can be calculated.

In some embodiments, after obtaining video files including human faces as materials for making expressions, video frames with similar weights of multiple expression tags can be combined to achieve the dynamic expression effect of portrait expressions. It can be understood that the embodiments of this application The expressions in may include dynamic portrait expressions or static portrait expressions, and this application does not limit this.

In some embodiments, in the process of making the target expression shown in Figure 7, after inputting the target expression text, when obtaining the portrait expression of the corresponding chat person, the output result of each portrait image in the portrait sub-model can be obtained at the same time, and Input the target emoticon text into the text sub-model and output the weight of the corresponding emoticon. Furthermore, when the portrait images in the portrait image set are displayed on the chat interface of WeChat ^TM 611, the portrait images can be displayed in descending order of the matching degree between the target emoticon text and the portrait image, and then the user can display multiple portrait images in the portrait emoticon set. Quickly find the portrait images required by the subset.

In some embodiments, after the mobile phone 100 executes the process of creating the target expression shown in FIG. 7 , the mobile phone 100 can also optimize the expression production model in step 904 based on the generated target expression. Specifically, the mobile phone 100 can update the weight parameters of the portrait image and the expression label in the portrait sub-model, and update the weight parameters of the preset emoticon text and expression label in the text sub-model. In some embodiments, if the target emoticon text input by the user does not belong to the preset emoticon text in the text sub-model, the target emoticon text can be added to the text sub-model, and the target emoticon text can be determined as the weight of each emoticon tag.

Figure 10 shows a software structure block diagram of an electronic device 100 according to an embodiment of the present application.

The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. The embodiment of this application takes the Android system with a layered architecture as an example to illustrate the software structure of the electronic device 100 .

The layered architecture divides the software into several layers, and each layer has clear roles and division of labor. The layers communicate through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: application layer, application framework layer, Android runtime and system libraries, and kernel layer.

The application layer can include a series of application packages.

As shown in Figure 10, the application package can include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and other applications.

The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

As shown in Figure 10, the application framework layer can include window manager, task manager, phone manager, resource manager, notification manager, view system, etc.

A window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, capture the screen, etc. In the embodiment of this application, the window manager can obtain the touch event corresponding to the user's click operation, including the application information corresponding to the window, the touch position, etc., to match the corresponding display task and display the corresponding interface, such as displaying the above steps For the portrait expression set described in step 403, please refer specifically to the relevant descriptions in step 403 and step 405, which will not be described again here.

The task manager is used to cooperate with the window manager to retrieve the task content corresponding to the user's sliding operation, such as display tasks that need to be controlled by the window manager. The task manager retrieves the content of the corresponding display task and sends it to the window manager for execution. , thereby realizing the process of the electronic device 200 displaying the corresponding interface.

Content providers are used to store and retrieve data and make this data accessible to applications. The above data can include videos, images, audio, calls made and received, browsing history and bookmarks, phone book, etc.

The resource manager provides various resources to applications, such as localized strings, icons, pictures, layout files, video files, etc.

The notification manager allows applications to display notification information in the status bar, which can be used to convey notification-type messages and can automatically disappear after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be notifications that appear in the status bar at the top of the system in the form of charts or scroll bar text, such as notifications for applications running in the background, or notifications that appear on the screen in the form of conversation windows. For example, text information is prompted in the status bar, a beep sounds, the electronic device vibrates, the indicator light flashes, etc.

The view system includes visual controls, such as controls that display text, controls that display pictures, etc. A view system can be used to build applications. The display interface can be composed of one or more views. For example, a display interface including a text message notification icon may include a view for displaying text and a view for displaying pictures.

The Android runtime includes core libraries and a virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.

The core library contains two parts: one is the functional functions that need to be called by the Java language, and the other is the core library of Android.

The application layer and application framework layer run in virtual machines. The virtual machine executes the java files of the application layer and application framework layer into binary files. The virtual machine is used to perform object life cycle management, stack management, thread management, security and exception management, and garbage collection and other functions.

System libraries can include multiple functional modules. For example: surface manager (surface manager), media libraries (Media Libraries), 3D graphics processing libraries (for example: OpenGL ES), 2D graphics engines (for example: SGL), etc.

The surface manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple applications.

The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, composition, and layer processing.

2D Graphics Engine is a drawing engine for 2D drawing.

The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, touch driver, and sensor driver.

Reference in the specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one example embodiment or technology disclosed herein. The appearances of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment.

The present disclosure also relates to means for performing the operations described herein. This apparatus may be specially constructed for the required purposes, or it may comprise a general-purpose computer selectively activated or reconfigured by a computer program stored in the computer. Such computer programs may be stored on a computer-readable medium such as, but not limited to, any type of disk, including floppy disks, optical disks, CD-ROMs, magneto-optical disks, Read memory (ROM), random access memory (RAM), EPROM, EEPROM, magnetic or optical card, application specific integrated circuit (ASIC), or any type of medium suitable for storing electronic instructions, and each may be coupled to a computer system bus. Furthermore, the computers referred to in the specification may include a single processor or may employ an architecture involving multiple processors for increased computing power.

Additionally, the language used in this specification has been selected primarily for readability and instructional purposes and may not have been selected to delineate or limit the disclosed subject matter. Accordingly, this disclosure is intended to illustrate, but not to limit, the scope of the concepts discussed herein.

Claims (15)

An interactive method, applied to electronic devices, is characterized by including: A first interface of the first application is displayed on the electronic device, wherein the first interface includes a first conversation window of the first contact; Detecting the user's expression input operation in the first conversation window of the first interface; Corresponding to determining that there is a first portrait expression set related to the first contact among the plurality of portrait expression sets stored in the electronic device, displaying the first portrait expression set in the first conversation window of at least one portrait expression. The interaction method according to claim 1, characterized in that the electronic device is preset with association information used to characterize the correspondence between at least one contact in the first application and the plurality of portrait expression sets. The interaction method according to claim 1, characterized in that the first conversation window includes a conversation area and an expression area, and at least one portrait in the first portrait expression set is displayed in the first conversation window. Expressions, including: Display the portrait expression tags of the first portrait expression set in the expression area, where the portrait expression tags are determined by the electronic device based on the portrait expressions in the first portrait expression set; Store and display at least one portrait expression in the first portrait expression set in the portrait tag. The interaction method according to claim 3, wherein the portrait expression tag is the portrait avatar of the first contact. The interaction method according to claim 1, further comprising: Corresponding to the determination that there is no first portrait expression set related to the first contact, at least one portrait expression set in the second portrait expression set is displayed in the first conversation window, wherein the second portrait expression set A set of portrait expressions meeting preset conditions is collected for the plurality of portrait expressions. The interaction method according to claim 5, characterized in that the second portrait expression set is determined in the following manner: Obtain historical usage information of the multiple portrait expression sets; The portrait expression set corresponding to the first historical usage information that satisfies the preset condition is determined to be the second portrait expression set. The interactive method according to claim 6, wherein the historical usage information includes historical usage frequency; It is determined that the portrait expression set corresponding to the first historical usage information that meets the preset condition is the second portrait expression set, including: Determine that the portrait expression set corresponding to the first historical usage frequency whose historical usage frequency is higher than the preset frequency threshold is the second portrait expression set; or It is determined that at least one portrait expression set with the highest historical usage frequency among the plurality of historical faces is the second portrait expression set. The interaction method according to claim 1, wherein the first application includes multiple contacts, and the multiple portrait expression sets are generated in the following manner: Obtain multiple portrait images stored in the electronic device; Classify at least one portrait image corresponding to the same contact into the same portrait image set; Perform image processing on the at least one portrait image in each portrait image set to obtain a portrait expression set corresponding to each portrait image set, and compare each portrait expression set with each character in the first application. Described contact association. The interactive method according to claim 8, characterized in that image processing is performed on the at least one portrait image in each portrait image set to obtain a portrait expression set corresponding to each portrait image set, including: For the at least one portrait image in each of the portrait image sets, an expression production model is used to generate a portrait expression set corresponding to each of the portrait image sets. The interaction method according to claim 9, characterized in that the expression production model includes: Determine each of the portrait images as a portrait sub-model with a weight of a preset expression; Determine the preset emoticon text to be the weighted text sub-model of the preset emoticon; According to the output results of the portrait sub-model and the text sub-model, each of the portrait images and the matching preset expression text are used to generate a corresponding expression generation sub-model of the portrait expression. The interactive method according to claim 8, further comprising: Detecting the user's text input operation in the first conversation window of the first interface, and receiving the text input by the user; Determine at least one target portrait image matching the text in the portrait image set corresponding to the first portrait expression set; The electronic device displays a second interface of the first application, and the second interface includes the at least one target portrait image; Detecting the image selection operation of the at least one portrait image by the user in the second interface, and generating a target portrait expression based on the text and the portrait image corresponding to the image selection operation; The target portrait expression is displayed in the first conversation window. The interaction method according to claim 1, wherein the first application includes at least one of an instant messaging application, a text message, a conferencing application, and a social networking application. An electronic device, characterized by including: memory for storing instructions for execution by one or more processors of the electronic device, and A processor, when the instructions are executed by one or more processors, the processor is configured to execute the interactive method according to any one of claims 1 to 12. A computer-readable storage medium, characterized in that instructions are stored on the computer-readable storage medium, and when executed on an electronic device, the instructions cause the electronic device to perform the interactive method described in any one of claims 1 to 12 . A computer program product, characterized in that the computer program product includes instructions, and the instructions are used to implement the interactive method according to any one of claims 1 to 12.