WO2021004114A1 - Automatic meme generation method and apparatus, computer device and storage medium - Google Patents

Abstract

Disclosed are an automatic meme generation method and apparatus, a computer device and a storage medium. The method comprises: extracting a facial micro-expression from a facial image, and acquiring an expression tag of the facial image according to the facial micro-expression (S20); according to the expression tag of the facial image, matching a meme picture from a preset meme library, and determining positions of five sense organs in the matched meme picture (S30); and extracting the facial features in the facial image, and overlapping the facial features with the positions of the five sense organs in the meme picture which is matched from the preset meme library, so as to generate a personalized meme (S40). The method is easily operated, and since the facial image in the generated personalized meme is consistent with the original meme picture, the effect and consistency of fusing the facial features of the facial image with the original meme picture are better, thus improving the user experience, and also improving the degree of user activity and degree of engagement.

Description

Method, device, computer equipment and storage medium for automatically generating emoticons

This application claims the priority of the Chinese patent application filed with the Chinese Patent Office on July 5, 2019, the application number is 201910602401.7, and the invention title is "method, device, computer equipment and storage medium for automatically generating emoticons", all of which are approved The reference is incorporated in this application.

Technical field

This application relates to the field of micro-expression recognition in artificial intelligence, and in particular to a method, device, computer equipment and storage medium for automatically generating an expression pack.

Background technique

With the development of communication technology, mobile phones are used more and more widely, which greatly expands people's social scope. Due to the expansion of the social network, users use mobile instant messaging software to communicate more and more. In order to communicate and communicate between users, many social applications have chat functions. Users can talk through the chat box or send various emoticons to each other to express emotions that are difficult to express in words.

In real applications, most emoticons sent by users are obtained from third parties who specialize in making emoticons, that is, the third party generates emoticons based on the materials collected by them, and then publishes them to the network. The emoticons provided by users from third parties Get the emoticon package you are interested in to use. However, the inventor realizes that in this case, if the user passively accepts or passively selects the emoticon package, it may often happen that the desired effect cannot be achieved.

Summary of the invention

The embodiments of this application provide a method, device, computer equipment and storage medium for automatically generating emoticons. The operation of this application is simple, the fusion effect and consistency of the generated personalized emoticons are better, the user experience is improved, and the user is also improved. Activity and participation.

A method for automatically generating emoticons, including:

Obtain face images;

Extracting facial micro-expressions from the facial image, and obtaining the facial expression tag of the facial image according to the facial micro-expressions;

According to the facial expression tag of the face image, the emoticon package picture is matched from the preset emoticon package library, and the location of the facial features of the matched emoticon package picture is determined; wherein, Each of the emoticon package pictures has at least one facial features, and each of the emoticon package pictures is associated with at least one emoticon label;

Extract the facial features in the face image, and overlay the facial features on the location of the facial features of the emoticon package picture matched from the preset emoticon package library to generate a personalized emoticon package.

A device for automatically generating emoticons, including:

The acquisition module is used to acquire a face image;

An extraction module for extracting facial micro-expressions from the facial image, and obtaining the facial expression tag of the facial image according to the facial micro-expressions;

The matching module is configured to match an emoticon package picture from a preset emoticon package library according to the emoticon tag of the face image, and determine the location of the facial features of the matched emoticon package picture; wherein, the preset The emoticon package pictures in the emoticon package library all have at least one facial features, and each emoticon package picture is associated with at least one emoticon label;

An overlay module for extracting facial features in the face image, and overlaying the facial features to the location of the facial features of the emoticon package picture matched from the preset emoticon package library , Generate personalized emoticons.

A computer device comprising a memory, a processor, and computer-readable instructions stored in the memory and capable of running on the processor. The processor executes the computer-readable instructions to realize the above-mentioned emoticon automatic generation method.

A computer-readable storage medium, which stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, realizes the aforementioned emoticon package automatic generation method.

Description of the drawings

FIG. 1 is a schematic diagram of an application environment of a method for automatically generating emoticons in an embodiment of the present application;

2 is a flowchart of a method for automatically generating emoticons in an embodiment of the present application;

3 is a flowchart of step S20 of the method for automatically generating emoticons in an embodiment of the present application;

4 is a flowchart of step S30 of the method for automatically generating emoticons in an embodiment of the present application;

5 is a flowchart of step S40 of the method for automatically generating emoticons in an embodiment of the present application;

FIG. 6 is a flowchart of step S407 of the method for automatically generating emoticons in an embodiment of the present application;

FIG. 7 is a schematic block diagram of a device for automatically generating emoticons in an embodiment of the present application;

FIG. 8 is a schematic block diagram of the extraction module of the emoticon automatic generation device in an embodiment of the present application;

Fig. 9 is a schematic diagram of a computer device in an embodiment of the present application.

Detailed ways

The method for automatically generating emoticons provided by this application can be applied in the application environment as shown in Fig. 1, where the client (computer equipment) communicates with the server through the network. Among them, the client (computer equipment) includes, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, cameras, and portable wearable devices. The server can be implemented as an independent server or a server cluster composed of multiple servers.

In one embodiment, as shown in FIG. 2, a method for automatically generating emoticons is provided. Taking the method applied to the server in FIG. 1 as an example, the method includes the following steps S10-S40:

S10, acquiring a face image;

The face image refers to an image containing part or all of the facial features. The face image can be taken by the user through a camera device and uploaded to the value server, or it can be pre-stored in the database, and the server can retrieve it from the database at any time as required. Retrieval.

S20, extracting facial micro-expressions from the facial image, and acquiring the facial expression tag of the facial image according to the facial micro-expressions;

That is, in this embodiment, it is necessary to first extract the facial micro-expression in the facial image, and determine the corresponding facial expression label based on the facial micro-expression, and determine the facial expression label as the human face The emoticon tag of the image.

In an embodiment, as shown in FIG. 3, the step S20, that is, the extraction of facial micro-expressions from the facial image, and obtaining the facial expression of the facial image according to the facial micro-expressions Labels, including:

S201, extracting all action unit types of the facial micro-expression from the facial image;

Wherein, the type of the action unit may include, but is not limited to, the internationally used action units (AU) and eye movements in Table 1 below. The eye movements are specifically different movements and perspectives of the eye, such as the left eye, Look right, upward, downward, upper right, etc., and the action units corresponding to different eye movements and viewing angles may also include judging the magnitude of eye movements.

S202: Confirm the micro-expression type of the face image based on extracting all the action unit types from the face image.

That is, the database pre-stores the action unit types corresponding to various micro-expression types (such as crying, laughing, or angry), and each micro-expression type corresponds to a combination of multiple action unit types, such as , The micro expression type is laugh, this micro expression type corresponds to a combination of at least the following action unit types: mouth up (AU12 in Table 1), mouth up (AU12 in Table 1) + outer eyebrows up (in Table 1 AU2), the corners of the mouth are raised (AU12 in Table 1) + lip extension (AU20 in Table 1) + the upper and lower lips are separated (AU25 in Table 1), etc.; therefore, it is only necessary to remove all the words extracted in step S201 The action unit type is compared with the corresponding action unit type of each micro-expression type stored in the database to confirm the type of the micro-expression. Understandably, in one aspect of this embodiment, as long as all the action unit types extracted in step S201 include all the action unit types corresponding to a micro-expression type stored in the database (that is, in All the action unit types extracted in the step S201 may also include other action units), that is, the micro-expression type can be regarded as the micro-expression type. In another aspect of this embodiment, all the action unit types extracted in the step S201 can also correspond one-to-one with the action unit types and sequences of a micro-expression type stored in the database (no more or less Any action unit) is considered to be the micro-expression type of the monitored person.

Table 1 Part AU

[Table 1]

AU label	AU description
AU1	Inner eyebrows raised
AU2	Raised eyebrows
AU4	Eyebrow depression
AU5	Face up
AU6	Raised cheeks
AU7	Eyelid tightening
AU9	Wrinkled nose
AU10	Upper lip up
AU12	Mouth up
AU14	Tighten the corners of the mouth
AU15	Mouth drop
AU16	Lower lip
AU17	Chin tightening
AU18	Lip folds
AU20	Lip stretch
AU23	Lips shrink
AU24	Lip compression
AU25	Separate upper and lower lips
AU26	Jaw drop

S203: Acquire all the emoticon tags associated with the micro-expression type, and simultaneously acquire a characteristic action unit associated with each emoticon tag;

That is, the database has pre-stored expression tags associated with each of the micro expression types, and each micro expression type corresponds to multiple expression tags. For example, if the micro expression type is laugh, the associated expression tags may include: Laugh, smile, smirk, wry smile, smirk, etc. Understandably, each emoticon tag associated with a micro-emoji type has at least one characteristic action unit corresponding to it.

S204. Matching all the action unit types extracted from the face image with the characteristic action units associated with each expression tag, and extracting all the action unit types from the face image When all the characteristic action units associated with one expression label are included, the expression label is recorded as the expression label of the face image.

In this embodiment, only when all the action unit types extracted from the face image in step S201 include all the feature action units associated with an emoticon tag (that is, in step S201) All the extracted action unit types may also include other action units other than the characteristic action unit corresponding to the emoticon tag, that is, the emoticon tag can be regarded as the emoticon tag of the face image. In the above embodiment, the type of micro-expression is first confirmed according to the type of action unit extracted from the face image (the number of micro-expression types is much smaller than the number of emoticon tags), and then the type of action unit extracted from the face image is compared with the micro-expression The feature action unit of the type-associated emoticon tag is matched. In this way, the action unit type extracted from the face image does not need to be compared with all the emoticon tags, only the feature action unit of the emoticon tag corresponding to a few types of micro-expression types is required. By comparison, when the number of emoticons is huge, the amount of calculation is greatly reduced and the server load is reduced.

Understandably, in an embodiment, after extracting all the action unit types of the facial micro-expression from the facial image in step S201, all the facial expression tags and the corresponding expression tags can be directly obtained. Associated feature action unit; then in step S204, all the action unit types extracted from the face image are matched with the feature action unit associated with each expression tag, and then the face image When extracting all the characteristic action units associated with one expression tag among all the action unit types in the extraction, the expression label is recorded as the expression label of the face image. In this embodiment, there is no need to first confirm the micro-expression type according to the type of the action unit extracted from the face image, but directly match the type of the action unit extracted from the face image with the feature action unit of the emoticon tag, which simplifies the comparison. Step: When the number of emoticons is relatively small, this embodiment can be preferentially selected for use.

S30, according to the facial expression tag of the face image, match an emoticon package picture from a preset emoticon package library, and determine the location of the facial features of the matched emoticon package picture; wherein, the preset emoticon package library Each of the emoticon package pictures in has at least one facial features, and each emoticon package picture is associated with at least one emoticon label;

That is, in this embodiment, it is first necessary to obtain the emoticon package picture associated with the emoticon tag in the preset emoticon package library according to the emoticon tag of the face image acquired in step S20 (a emoticon package picture may correspond to For one or more emoticon tags), at this time, the number of emoticon package pictures that may be obtained corresponding to the emoticon label is greater than one. At this time, one of the emoticon package pictures needs to be selected according to the needs, and the selected emoticon The package picture record is the emoticon package picture matched from a preset emoticon package library. And after selecting the only emoticon package picture matching the emoticon tag of the face image from the preset emoticon package library, it is necessary to determine the location of the facial features in the emoticon package picture, and then use the face The facial features extracted from the image cover the facial features, and the image corresponding to the facial features in the emoticon package picture is replaced to generate a new personalized emoticon package.

In one embodiment, as shown in FIG. 4, the step S30, that is, the matching emoticon package pictures from a preset emoticon package library according to the emoticon tags of the facial images, and determining the matched emoticons The location of the facial features of the package picture, including:

S301. Acquire a face contour extracted from the face image; in this embodiment, the face contour is also the edge contour of the face in the face image.

S302: Select all emoticon package pictures with the same emoticon tag and the face image from a preset emoticon package library;

S303: Determine the location of the facial features and the contour of the location of the facial features of each selected emoticon package picture; the location of the facial features refers to all the objects with facial features in one emoticon package picture The location of the five sense organs (such as a person, an animal, an animation character, etc.), and the location contour refers to the contour of the corresponding position of the five sense organs (such as the facial contour of a human face).

S304. Obtain the similarity between the facial contour of the human face and the contour of the position; the similarity can be compared according to similar parameters such as the area occupied by the two, the curvature of the two contours, and other similar parameters. Different weights can be set for each similar parameter, and then the similar parameters are normalized and then multiplied by the corresponding weights respectively, and then the sum of the product of the normalized offset parameter and the weight is taken as the said The criterion of similarity, the greater the sum of products, the higher the similarity.

S305. Record the emoticon package picture corresponding to the location contour with the highest similarity as the emoticon package picture that uniquely matches the face image, and at the same time obtain the unique match with the face image The location of the facial features of the emoticon package picture.

That is, in this embodiment, it may refer to selecting the facial contour and the facial image of the facial image from all the facial expression package pictures corresponding to the facial expression tag of the facial image in the preset facial expression package library. The emoticon picture with the highest contour similarity is recorded as the emoticon picture matched from the preset emoticon library, so that it is convenient to extract the largest facial features from the face image later It is adapted to the extent to cover the facial features, replacing the image corresponding to the facial features in the emoticon package picture, and generating a new personalized emoticon package.

In an embodiment, the matching an emoticon package picture from a preset emoticon package library according to the emoticon tag of the face image, and determining the location of the facial features of the matched emoticon package picture includes:

All the emoticon pictures with the same emoticon label and the face image are selected from the preset emoticon package library; the emoticon package pictures in the preset emoticon package library may be legally obtained from a third party that specializes in making emoticon packages The emoticon package, and the emoticon package pictures in the preset emoticon package library all need to have facial features (that is, the face, which may be part of the face or all).

According to the preset screening rules, from all the selected emoticon pack pictures, the emoticon pack pictures that uniquely match the face images are determined; understandably, the screening rules can be selected randomly or refer to The selection is based on the frequency of use. For example, the emoticon picture with the highest personal use frequency of the user can be selected, that is, the more frequently the user uses the emoticon picture, the greater the probability that it will be selected. In the same way, the filtering rule may also be to first count the total usage times of all the emoticon package pictures corresponding to the emoticon tag in the preset emoticon package library by all users, and select the emoticon package with the highest total usage count. Further, the screening rule can also pass the total usage count through a preset conversion rule (a conversion rule contains the association relationship between the total usage count in different ranges and different popularity, and a total usage count can only Corresponding to a popularity) is converted to popularity. In this case, the higher the popularity, the higher the probability of being selected.

The emoticon package picture determined according to the screening rule is recorded as the emoticon package picture that uniquely matches the face image, and at the same time it is extracted from the emoticon package picture that uniquely matches the face image The location of the facial features.

In this embodiment, the emoticon package picture that uniquely matches the face image can be confirmed according to the screening rule. The screening rule can be confirmed according to the number of times the user has used it. In this way, the user can be selected according to user preferences. The generated personalized emoticon package template, in this way, the generated personalized emoticon package will be more in line with the user's usage habits and bring a better user experience.

S40. Extract facial features in the face image, and overlay the facial features on the positions of the facial features of the emoticon package picture matched from the preset emoticon package library to generate a personality Emoji package.

In this embodiment, after selecting the only emoticon package picture matching the emoticon tag of the face image from the preset emoticon package library, the location of the facial features in the emoticon package picture needs to be determined, and then The facial features extracted from the face image are replaced with the original image content in the contour of the position of the facial features (referring to the contour of the corresponding position of the facial features), and then the facial features of the facial image will be replaced The above-mentioned emoticon package pictures are synthesized into a new personalized emoticon package (for example, the main body of the emoticon package picture is a cute kitten. In this case, replace the cat’s face with the facial features of the face image to generate a personalized expression The bag is a cute kitten with facial features of a face image).

In one embodiment, as shown in FIG. 5, in the step S40, the extracting facial features in the face image, and overlaying the facial features to match from the preset emoticon library The location of the facial features of the emoticon package picture includes:

S401. Acquire the location contour of the facial features, the overall placement angle of the facial features, and the contour area of the facial features of the facial expression package picture matched in the preset facial expression package library; The location contour of the facial features refers to the edge contour of the area occupied by the facial features in the emoticon package picture; the overall placement angle of the facial features refers to the size, upright or inverted angle of the facial features , The overall placement angle can be determined based on one or more of the five senses, for example, the angle between the straight line and the horizontal line between the opposite corners of an eye is used to determine its inclination angle, and then the nose Or whether the mouth is under the eyes (similarly, it can also be determined by whether the mouth is under the nose, etc.) to determine whether it is upright or upside down (the nose or mouth under the eyes is upright, otherwise it is upside down); The contour area of the facial features refers to the total area of the contour of the location.

S402: Extract all facial features within the contour of the face in the face image, and determine the positional relationship between the center points of the facial features and the linear distance between the center points of the facial features ; Wherein, the facial features include but are not limited to ears, eyebrows, eyes, nose, mouth, etc. The positional relationship between the center points of the facial features refers to the distance, relative orientation, etc. between the center points of the facial features.

S403: Create a new canvas, where the contour of the canvas is consistent with the contour of the position of the facial features; preprocess the facial features according to a preset image processing method; that is, the contour of the canvas is consistent with the position of the facial features. The location contours of the five sense organs can be completely overlapped. The preset image processing methods include, but are not limited to, performing transparency adjustment and color toning processing on the facial features, so as to make the generated personalized emoticon package more natural and beautiful.

S404, under the condition that the positional relationship between the center points of the facial features remains unchanged, placing all the facial features after preprocessing in the canvas contour of the canvas according to the overall placement angle; also That is, when all facial features are placed into the outline of the canvas, the relative positional relationship between them needs to be kept unchanged, and then the expression of the face image remains unchanged (if the relative positional relationship between facial features changes, then The facial image composed of the changed facial features may change with the facial expression corresponding to the previous facial image). Understandably, the center position of all the facial features and the center position of the canvas outline can be used as the counterpoint to place all the facial features into the canvas outline; and the composition between all the facial features When the overall placement angle is inconsistent with the overall placement angle, it needs to be adjusted to be consistent with the overall placement angle before placing all facial features into the canvas outline.

S405. Adjust the linear distance between the center points of the facial features at the same ratio, and adjust the figure area surrounded by the outermost facial feature among the facial features after adjusting the same ratio, and The ratio between the contour areas of the facial features is within a preset ratio range; that is, the linear distance between the center points of the facial features is uniformly adjusted by the same ratio to adjust the size of each facial feature as a whole; and When the ratio of the figure area enclosed by the outermost facial feature among the adjusted facial features to the contour area of the facial features is within a preset ratio range (which can be set according to requirements), each The size of the facial features on the five sense organs will be relatively coordinated, otherwise, the facial features may be too large or too small in the canvas contour. That is, the above-mentioned same ratio needs to be based on "the ratio of the figure area enclosed by the outermost facial feature among the facial features after adjustment to the contour area of the facial features is within the preset ratio range" Conditions are selected (it can be arranged in advance and stored in the database according to the priority level, and the server automatically selects the same ratio from the database that meets the above conditions), if the ratio is within the preset ratio range, how many of the same ratios Choose one, you can choose one randomly or according to the order of priority.

S406: Cover the canvas containing the facial features on the contour of the location of the facial features of the emoticon package picture matched from the preset emoticon package library; that is, due to The canvas outline of the canvas and the location outline of the facial features can completely overlap, so in this step, the canvas can directly replace the original image content in the outline of the facial features.

S407: Perform image synthesis processing on the emoticon package picture covered with the canvas outline to generate the personalized emoticon package. The image synthesis processing includes, but is not limited to, merging the emoticon package pictures covered with the facial features into the same picture and performing unified exposure and toning processing to make it more natural.

In an embodiment, as shown in FIG. 6, in step S407, generating the personalized emoticon package includes:

S4071: Receive a text adding instruction, and obtain the emoticon text entered by the user and the text box number selected by the user; wherein, the text adding instruction refers to the image synthesis process performed in step S407, if the user also wants to personalize When the emoticon text is independently entered in the emoticon package, at this time, the preset button can be triggered by clicking, sliding, etc., and then the text adding instruction is sent to the server; the emoticon text refers to what the user wants to configure for the personalized emoticon package Text; the text box number refers to the unique identifier of the text box that can be added to the personalized emoticon package, and each text box number corresponds to the style of a text box.

S4072. Obtain the text box size and default text format associated with the text box number; that is, each text box number has a text box size that can be filled with emoticon text, and each text box corresponds to a default Text format, if the user does not modify the default text format, the emoticon text will be filled in the text box in the default text format.

S4073: Obtain the number of characters in the emoticon text, and adjust the character size in the default text format according to the number of characters and the size of the text box; that is, the number of characters in the emoticon text (ie, the character length ), the character size is automatically adjusted; understandably, other text format items other than the character size in the default text format can also be adjusted according to requirements.

S4074: Generate a text box corresponding to the text box number at a preset position in the emoticon package picture or a position selected by the user, and fill in emoticon text in the text box according to the adjusted default text format ; That is, after the user adjusts the default text format, the emoticon text will be filled into the text box in the adjusted default text format.

S4075: After assembling the emoticon package picture and the text box, generate the personalized emoticon package. The assembling process refers to merging the text box and the emoticon package picture after image synthesis processing into the same personalized emoticon package.

That is, in the above-mentioned embodiment, it also supports user-defined input of emoticon text, and automatically adjusts the character size and so on by judging the number of characters of emoticon text (that is, the character length), and fills in the emoticon characters. The text box and the emoticon package picture are automatically assembled into a personalized emoticon package. Understandably, it is also possible to add prop effects to the personalized emoticon package, such as adding props with effects such as hearts, hats, and stars.

In one embodiment, as shown in FIG. 7, an apparatus for automatically generating emoticons is provided, and the apparatus for automatically generating emoticons corresponds to the method for automatically generating emoticons in the above-mentioned embodiment one-to-one. The device for automatically generating emoticons includes:

The obtaining module 11 is used to obtain a face image;

The extraction module 12 is configured to extract facial micro-expressions from the facial images, and obtain the facial expression tags of the facial images according to the facial micro-expressions;

The matching module 13 is configured to match an emoticon package picture from a preset emoticon package library according to the emoticon tag of the face image, and determine the location of the facial features of the matched emoticon package picture; wherein, the preset It is assumed that the emoticon package pictures in the emoticon package library have at least one facial features, and each emoticon package picture is associated with at least one emoticon label;

The covering module 14 is used to extract the facial features in the face image, and cover the facial features to the facial features of the emoticon package picture matched from the preset emoticon package library. Location, generate personalized emoticons.

In an embodiment, as shown in FIG. 8, the extraction module 12 includes:

The extraction unit 121 is configured to extract all the action unit types of the facial micro-expression from the facial image;

The confirming unit 122 is configured to confirm the micro-expression type of the face image based on all the action unit types extracted from the face image;

The acquiring unit 123 is configured to acquire all the emoticon tags associated with the micro-expression type, and at the same time acquire the characteristic action unit associated with each emoticon tag;

The matching unit 124 is configured to match all the action unit types extracted from the face image with the characteristic action units associated with each expression tag, and extract all the action units from the face image. When the action unit type includes all the characteristic action units associated with the expression tag, the expression tag is recorded as the expression tag of the face image.

For the specific limitation of the emoticon package automatic generating device, please refer to the above limitation on the emoticon package automatic generating method, which will not be repeated here. Each module in the above-mentioned emoticon package automatic generating device can be implemented in whole or in part by software, hardware, and a combination thereof. The foregoing modules may be embedded in the form of hardware or independent of the processor in the computer device, or may be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the foregoing modules.

In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. 9. The computer equipment includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide calculation and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer readable instructions, and a database. The internal memory provides an environment for the operation of the operating system and computer-readable instructions in the non-volatile storage medium. . When the computer-readable instructions are executed by the processor, a method for automatically generating emoticons is realized.

In one embodiment, a computer device is provided, including a memory, a processor, and computer-readable instructions stored on the memory and capable of running on the processor, and the processor implements at least the following steps when executing the computer-readable instructions:

Obtain face images;

Extracting facial micro-expressions from the facial image, and obtaining the facial expression tag of the facial image according to the facial micro-expressions;

According to the facial expression tag of the face image, the emoticon package picture is matched from the preset emoticon package library, and the location of the facial features of the matched emoticon package picture is determined; wherein, Each of the emoticon package pictures has at least one facial features, and each of the emoticon package pictures is associated with at least one emoticon label;

Extract the facial features in the face image, and overlay the facial features on the location of the facial features of the emoticon package picture matched from the preset emoticon package library to generate a personalized emoticon package.

In one embodiment, a computer-readable storage medium is provided. The computer-readable storage medium is a volatile storage medium or a non-volatile storage medium, and computer-readable instructions are stored thereon. At least the following steps are implemented when executed by the processor:

Obtain face images;

Extracting facial micro-expressions from the facial image, and obtaining the facial expression tag of the facial image according to the facial micro-expressions;

According to the facial expression tag of the face image, the emoticon package picture is matched from the preset emoticon package library, and the location of the facial features of the matched emoticon package picture is determined; wherein, Each of the emoticon package pictures has at least one facial features, and each of the emoticon package pictures is associated with at least one emoticon label;

Extract the facial features in the face image, and overlay the facial features on the location of the facial features of the emoticon package picture matched from the preset emoticon package library to generate a personalized emoticon package.

A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing relevant hardware through computer-readable instructions, which can be stored in a non-volatile computer. In a readable storage medium, when the computer-readable instructions are executed, they may include the processes of the above-mentioned method embodiments. Wherein, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and/or volatile memory. Non-volatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not a limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous chain Road DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

Summary of the invention

technical problem

The solution to the problem

The beneficial effects of the invention

Claims (20)

1. A method for automatically generating emoticons, which includes:

   Obtain face images;

   Extracting facial micro-expressions from the facial image, and obtaining the facial expression tag of the facial image according to the facial micro-expressions;

   According to the facial expression tag of the face image, the emoticon package picture is matched from the preset emoticon package library, and the location of the facial features of the matched emoticon package picture is determined; wherein, Each of the emoticon package pictures has at least one facial features, and each of the emoticon package pictures is associated with at least one emoticon label;

   Extract the facial features in the face image, and overlay the facial features on the location of the facial features of the emoticon package picture matched from the preset emoticon package library to generate a personalized emoticon package.
2. The method for automatically generating an emoticon package according to claim 1, wherein the extracting the facial micro-expression from the facial image and obtaining the facial expression tag of the facial image according to the facial micro-expression comprises:

   Extract all the action unit types of the face micro-expression from the face image;

   Confirm the micro-expression type of the face image according to the types of all the action units extracted from the face image;

   Acquiring all the emoticon tags associated with the micro-expression type, and simultaneously acquiring the characteristic action unit associated with each emoticon tag;

   All the action unit types extracted from the face image are matched with the characteristic action units associated with each expression tag, and one is included in all the action unit types extracted from the face image When all the characteristic action units associated with the expression tag are associated, the expression tag is recorded as the expression tag of the face image.
3. The method for automatically generating an emoticon package according to claim 1, wherein the emoticon package picture is matched from a preset emoticon package library according to the emoticon tag of the face image, and the facial features of the matched emoticon package picture are determined The location of the site, including:

   Acquiring a face contour extracted from the face image;

   Select all emoticon package pictures with the same emoticon tag and the face image from a preset emoticon package library;

   Determining the location of the facial features of each selected emoticon package picture and the contour of the location of the facial features;

   Acquiring the similarity between the contour of the human face and the contour of the location;

   Record the emoticon package picture corresponding to the location contour with the highest similarity as the emoticon package picture that uniquely matches the face image, and at the same time obtain all the emoticon package pictures that uniquely match the face image State the location of the facial features of the emoticon picture.
4. The method for automatically generating an emoticon package according to claim 1, wherein the emoticon package picture is matched from a preset emoticon package library according to the emoticon tag of the face image, and the facial features of the matched emoticon package picture are determined The location of the site, including:

   Select all emoticon package pictures with the same emoticon tag and the face image from a preset emoticon package library;

   According to preset screening rules, determine the emoticon package picture uniquely matching the face image from all the selected emoticon package pictures;

   The emoticon package picture determined according to the screening rule is recorded as the emoticon package picture that uniquely matches the face image, and at the same time it is extracted from the emoticon package picture that uniquely matches the face image The location of the facial features.
5. The method for automatically generating emoticons according to claim 1, wherein said extracting facial features in said face image, and overlaying said facial features on said matching from said preset emoticon library The location of the facial features of the emoticon picture includes:

   Obtained from the location contour of the facial features, the overall placement angle of the facial features, and the contour area of the facial features of the facial expression package picture matched from the preset facial expression package library;

   Extracting all facial features located within the contour of the face in the face image, and determining the positional relationship between the center points of the facial features and the linear distance between the center points of the facial features;

   Create a new canvas, the canvas contour of the canvas is consistent with the contour of the position of the facial features; preprocessing the facial features according to a preset image processing method;

   While keeping the positional relationship between the center points of the facial features unchanged, placing all the facial features after preprocessing in the canvas outline of the canvas according to the overall placement angle;

   Adjust the linear distance between the center points of the facial features at the same ratio, and adjust the area of the figure enclosed by the outermost facial features of the facial features after adjusting the same ratio with the The ratio between the contour areas of the facial features is within the preset ratio range;

   Correspondingly overlay the canvas containing the facial features on the contour of the position of the facial features of the emoticon package picture matched from the preset emoticon package library;

   Perform image synthesis processing on the emoticon package picture covered with the canvas outline to generate the personalized emoticon package.
6. 8. The method for automatically generating emoticons according to claim 1, wherein said generating said personalized emoticons comprises:

   Receiving a text adding instruction, obtaining the emoticon text entered by the user and the text box number selected by the user;

   Obtaining the text box size and the default text format associated with the text box number;

   Acquiring the number of characters of the emoticon text, and adjusting the character size in the default text format according to the number of characters and the size of the text box;

   Generating a text box corresponding to the text box number at a preset position in the emoticon package picture or a position selected by the user, and filling in emoticon text in the text box according to the adjusted default text format;

   After assembling the emoticon package picture and the text box, the personalized emoticon package is generated.
7. A device for automatically generating emoticons, which includes:

   The acquisition module is used to acquire a face image;

   An extraction module for extracting facial micro-expressions from the facial image, and obtaining the facial expression tag of the facial image according to the facial micro-expressions;

   The matching module is configured to match an emoticon package picture from a preset emoticon package library according to the emoticon tag of the face image, and determine the location of the facial features of the matched emoticon package picture; wherein, the preset The emoticon package pictures in the emoticon package library all have at least one facial features, and each emoticon package picture is associated with at least one emoticon label;

   An overlay module for extracting facial features in the face image, and overlaying the facial features to the location of the facial features of the emoticon package picture matched from the preset emoticon package library , Generate personalized emoticons.
8. 8. The device for automatically generating emoticons according to claim 7, wherein the extraction module comprises:

   An extraction unit for extracting all the action unit types of the facial micro-expression from the facial image;

   A confirming unit, configured to confirm the micro-expression type of the face image based on all the action unit types extracted from the face image;

   An acquiring unit, configured to acquire all the emoticon tags associated with the micro-expression type, and at the same time acquire a characteristic action unit associated with each emoticon tag;

   The matching unit is configured to match all the action unit types extracted from the face image with the characteristic action units associated with each expression tag, and extract all the actions from the face image When the unit type includes all the characteristic action units associated with the expression tag, the expression tag is recorded as the expression tag of the face image.
9. A computer device including a memory, a processor, and computer-readable instructions stored in the memory and running on the processor, wherein the processor implements an expression when the computer-readable instructions are executed A method for automatically generating a package, wherein the method for automatically generating an emoticon package includes:

   Obtain face images;

   Extracting facial micro-expressions from the facial image, and obtaining the facial expression tag of the facial image according to the facial micro-expressions;

   According to the facial expression tag of the face image, the emoticon package picture is matched from the preset emoticon package library, and the location of the facial features of the matched emoticon package picture is determined; wherein, Each of the emoticon package pictures has at least one facial features, and each of the emoticon package pictures is associated with at least one emoticon label;

   Extract the facial features in the face image, and overlay the facial features on the location of the facial features of the emoticon package picture matched from the preset emoticon package library to generate a personalized emoticon package.
10. 9. The computer device according to claim 9, wherein said extracting the facial micro-expression from the facial image and obtaining the facial expression tag of the facial image according to the facial micro-expression comprises:

    Extract all the action unit types of the face micro-expression from the face image;

    Confirm the micro-expression type of the face image according to the types of all the action units extracted from the face image;

    Acquiring all the emoticon tags associated with the micro-expression type, and simultaneously acquiring the characteristic action unit associated with each emoticon tag;

    All the action unit types extracted from the face image are matched with the characteristic action units associated with each expression tag, and one is included in all the action unit types extracted from the face image When all the characteristic action units associated with the expression tag are associated, the expression tag is recorded as the expression tag of the face image.
11. The computer device according to claim 9, wherein the emoticon package picture is matched from a preset emoticon package library according to the emoticon tag of the face image, and all the facial features of the matched emoticon package picture are determined Locations, including:

    Acquiring a face contour extracted from the face image;

    Select all emoticon package pictures with the same emoticon tag and the face image from a preset emoticon package library;

    Determining the location of the facial features of each selected emoticon package picture and the contour of the location of the facial features;

    Acquiring the similarity between the contour of the human face and the contour of the location;

    Record the emoticon package picture corresponding to the location contour with the highest similarity as the emoticon package picture that uniquely matches the face image, and at the same time obtain all the emoticon package pictures that uniquely match the face image State the location of the facial features of the emoticon picture.
12. The computer device according to claim 9, wherein the emoticon package picture is matched from a preset emoticon package library according to the emoticon tag of the face image, and all the facial features of the matched emoticon package picture are determined Locations, including:

    Select all emoticon package pictures with the same emoticon tag and the face image from a preset emoticon package library;

    According to preset screening rules, determine the emoticon package picture uniquely matching the face image from all the selected emoticon package pictures;

    The emoticon package picture determined according to the screening rule is recorded as the emoticon package picture that uniquely matches the face image, and at the same time it is extracted from the emoticon package picture that uniquely matches the face image The location of the facial features.
13. 8. The computer device according to claim 9, wherein said extracting facial features in said face image, and overlaying said facial features on said emoticon package picture matched from said preset emoticon package library The location of the five sense organs includes:

    Obtained from the location contour of the facial features, the overall placement angle of the facial features, and the contour area of the facial features of the facial expression package picture matched from the preset facial expression package library;

    Extracting all facial features located within the contour of the face in the face image, and determining the positional relationship between the center points of the facial features and the linear distance between the center points of the facial features;

    Create a new canvas, the canvas contour of the canvas is consistent with the contour of the position of the facial features; preprocessing the facial features according to a preset image processing method;

    While keeping the positional relationship between the center points of the facial features unchanged, placing all the facial features after preprocessing in the canvas outline of the canvas according to the overall placement angle;

    Adjust the linear distance between the center points of the facial features at the same ratio, and adjust the area of the figure enclosed by the outermost facial features of the facial features after adjusting the same ratio with the The ratio between the contour areas of the facial features is within the preset ratio range;

    Correspondingly overlay the canvas containing the facial features on the contour of the position of the facial features of the emoticon package picture matched from the preset emoticon package library;

    Perform image synthesis processing on the emoticon package picture covered with the canvas outline to generate the personalized emoticon package.
14. 5. The computer device of claim 1, wherein said generating said personalized emoticon package comprises:

    Receiving a text adding instruction, obtaining the emoticon text entered by the user and the text box number selected by the user;

    Obtaining the text box size and the default text format associated with the text box number;

    Acquiring the number of characters of the emoticon text, and adjusting the character size in the default text format according to the number of characters and the size of the text box;

    Generating a text box corresponding to the text box number at a preset position in the emoticon package picture or a position selected by the user, and filling in emoticon text in the text box according to the adjusted default text format;

    After assembling the emoticon package picture and the text box, the personalized emoticon package is generated.
15. A computer-readable storage medium, the computer-readable storage medium stores computer-readable instructions, wherein when the computer-readable instructions are executed by a processor, a method for automatically generating emoticons is realized, wherein Automatic generation methods include:

    Obtain face images;

    Extracting facial micro-expressions from the facial image, and obtaining the facial expression tag of the facial image according to the facial micro-expressions;

    According to the facial expression tag of the face image, the emoticon package picture is matched from the preset emoticon package library, and the location of the facial features of the matched emoticon package picture is determined; wherein, Each of the emoticon package pictures has at least one facial features, and each of the emoticon package pictures is associated with at least one emoticon label;

    Extract the facial features in the face image, and overlay the facial features on the location of the facial features of the emoticon package picture matched from the preset emoticon package library to generate a personalized emoticon package.
16. 15. The computer-readable storage medium according to claim 15, wherein the extracting the facial micro-expression from the facial image and obtaining the facial expression tag of the facial image according to the facial micro-expression comprises:

    Extract all the action unit types of the face micro-expression from the face image;

    Confirm the micro-expression type of the face image according to the types of all the action units extracted from the face image;

    Acquiring all the emoticon tags associated with the micro-expression type, and simultaneously acquiring the characteristic action unit associated with each emoticon tag;

    All the action unit types extracted from the face image are matched with the characteristic action units associated with each expression tag, and one is included in all the action unit types extracted from the face image When all the characteristic action units associated with the expression tag are associated, the expression tag is recorded as the expression tag of the face image.
17. The computer-readable storage medium of claim 15, wherein the facial image is matched with an emoticon package picture from a preset emoticon package library according to the emoticon tag of the face image, and the facial features of the matched emoticon package picture are determined The location of the site, including:

    Acquiring a face contour extracted from the face image;

    Select all emoticon package pictures with the same emoticon tag and the face image from a preset emoticon package library;

    Determining the location of the facial features of each selected emoticon package picture and the contour of the location of the facial features;

    Acquiring the similarity between the contour of the human face and the contour of the location;

    Record the emoticon package picture corresponding to the location contour with the highest similarity as the emoticon package picture that uniquely matches the face image, and at the same time obtain all the emoticon package pictures that uniquely match the face image State the location of the facial features of the emoticon picture.
18. The computer-readable storage medium of claim 15, wherein the facial image is matched with an emoticon package picture from a preset emoticon package library according to the emoticon tag of the face image, and the facial features of the matched emoticon package picture are determined The location of the site, including:

    Select all emoticon package pictures with the same emoticon tag and the face image from a preset emoticon package library;

    According to a preset screening rule, determine the emoticon package picture that uniquely matches the face image from all the selected emoticon package pictures;

    The emoticon package picture determined according to the screening rule is recorded as the emoticon package picture that uniquely matches the face image, and at the same time it is extracted from the emoticon package picture that uniquely matches the face image The location of the facial features.
19. The computer-readable storage medium of claim 15, wherein the extracting facial features in the face image, and overlaying the facial features on the matching from the preset emoticon library The location of the facial features of the emoticon picture includes:

    Obtained from the location contour of the facial features, the overall placement angle of the facial features, and the contour area of the facial features of the facial expression package picture matched from the preset facial expression package library;

    Extracting all facial features located within the contour of the face in the face image, and determining the positional relationship between the center points of the facial features and the linear distance between the center points of the facial features;

    Create a new canvas, the canvas contour of the canvas is consistent with the contour of the position of the facial features; preprocessing the facial features according to a preset image processing method;

    While keeping the positional relationship between the center points of the facial features unchanged, placing all the facial features after preprocessing in the canvas outline of the canvas according to the overall placement angle;

    Adjust the linear distance between the center points of the facial features at the same ratio, and adjust the area of the figure enclosed by the outermost facial features of the facial features after adjusting the same ratio with the The ratio between the contour areas of the facial features is within the preset ratio range;

    Correspondingly overlay the canvas containing the facial features on the contour of the location of the facial features of the emoticon package picture matched from the preset emoticon package library;

    Perform image synthesis processing on the emoticon package picture covered with the canvas outline to generate the personalized emoticon package.
20. 15. The computer-readable storage medium of claim 15, wherein said generating said personalized emoticon package comprises:

    Receiving a text adding instruction, obtaining the emoticon text entered by the user and the text box number selected by the user;

    Obtaining the text box size and the default text format associated with the text box number;

    Acquiring the number of characters of the emoticon text, and adjusting the character size in the default text format according to the number of characters and the size of the text box;

    Generating a text box corresponding to the text box number at a preset position in the emoticon package picture or a position selected by the user, and filling in emoticon text in the text box according to the adjusted default text format;

    After assembling the emoticon package picture and the text box, the personalized emoticon package is generated.

Images