CN116127102A

CN116127102A - Image generation method, device, apparatus, storage medium, and program product

Info

Publication number: CN116127102A
Application number: CN202111340267.1A
Authority: CN
Inventors: 刘军煜
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2021-11-12
Filing date: 2021-11-12
Publication date: 2023-05-16

Abstract

The application discloses an image generation method, an image generation device, a storage medium and a program product, and relates to the technical field of multimedia. The method comprises the following steps: acquiring a first image corresponding to a target multimedia program; performing face detection on the first image to obtain face areas corresponding to at least two face images respectively; cutting based on the face area to obtain a character shape image; and superposing and displaying the spliced image after the character image is recombined on the appointed background image to obtain a second image. After a first image corresponding to the target multimedia program is obtained, face detection is carried out on the first image, and character images in the first image are obtained by cutting according to the detected face areas, so that a plurality of images corresponding to the target multimedia program are generated based on the matching of different character images, interest points of different users are attached by different character matching dimensions, the manufacturing efficiency of the multimedia program poster is improved, and manpower and material resources in the poster manufacturing process are saved.

Description

Image generation method, device, apparatus, storage medium, and program product

Technical Field

The embodiment of the application relates to the technical field of multimedia, in particular to an image generation method, an image generation device, a storage medium and a program product.

Background

In a multimedia application program, a multimedia program is generally initially displayed in a poster mode, so that a user can know actor information, background information, role information and the like of the multimedia program through the poster, and when the user is interested in the content of the poster, the user clicks to enter the multimedia program to watch the multimedia content, for example: in the first page of the video application, the posters of the television drama and the movie are displayed, and after clicking the interested posters, the user can watch the corresponding television drama or movie.

Because a plurality of roles are usually included in one multimedia program, when a designer draws a poster, different posters are usually required to be drawn respectively aiming at the collocation of different roles in the multimedia program, so that multi-angle display can be carried out aiming at the same multimedia program when the poster is displayed.

However, the above-mentioned manner of making the poster in multiple directions requires a lot of manpower, and the efficiency of making the poster is low, and the requirement of making the poster on the current multimedia program in large quantity cannot be met, and the efficiency of making the poster is low.

Disclosure of Invention

The embodiment of the application provides an image generation method, an image generation device, a storage medium and a program product, which can improve the efficiency of poster manufacturing. The technical scheme is as follows.

In one aspect, there is provided an image generation method, the method including:

acquiring a first image corresponding to a target multimedia program, wherein the first image comprises at least two face images associated with the target multimedia program;

performing face detection on the first image to obtain face areas corresponding to at least two face images in the first image respectively;

cutting out at least two character shape images corresponding to the face area based on the face area, wherein the ith face area corresponds to the ith character shape image, and i is a positive integer;

and superposing and displaying the spliced image after the character image is recombined on the appointed background image to obtain a second image corresponding to the target multimedia program.

In another aspect, there is provided an image generating apparatus, the apparatus including:

the acquisition module is used for acquiring a first image corresponding to a target multimedia program, wherein the first image comprises at least two face images associated with the target multimedia program;

the detection module is used for carrying out face detection on the first image to obtain face areas corresponding to at least two face images in the first image respectively;

The clipping module is used for clipping at least two character shape images corresponding to the face area based on the face area, wherein the ith face area corresponds to the ith character shape image, and i is a positive integer;

and the reorganization module is used for superposing and displaying the spliced image reorganized by the character image on the appointed background image to obtain a second image corresponding to the target multimedia program.

In another aspect, a computer device is provided, the computer device including a processor and a memory, where the memory stores at least one instruction, at least one program, a set of codes, or a set of instructions, the at least one instruction, the at least one program, the set of codes, or the set of instructions being loaded and executed by the processor to implement the image generation method as in any one of the embodiments of the application.

In another aspect, a computer readable storage medium is provided, in which at least one instruction, at least one program, a set of codes, or a set of instructions is stored, the at least one instruction, the at least one program, the set of codes, or the set of instructions being loaded and executed by a processor to implement an image generation method as described in any one of the embodiments of the application.

In another aspect, a computer program product or computer program is provided, the computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions so that the computer device performs the image generation method described in any of the above embodiments.

The beneficial effects that technical scheme that this application embodiment provided include at least:

after a first image corresponding to a target multimedia program (such as a poster of the target multimedia program) is obtained, face detection is carried out on the first image, and character images in the first image are obtained by cutting according to the detected face areas, so that a plurality of images corresponding to the target multimedia program are generated based on the collocation of different character images, interest points of different users are attached by different character collocation dimensions, the manufacturing efficiency of the poster of the multimedia program is improved, and manpower and material resources in the manufacturing process of the poster are saved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a diagram of related art poster generation provided in an exemplary embodiment of the present application;

FIG. 2 is a schematic overall process diagram of an image generation method provided in an exemplary embodiment of the present application;

FIG. 3 is a schematic illustration of an implementation environment provided by an exemplary embodiment of the present application;

FIG. 4 is a flowchart of an image generation method provided by an exemplary embodiment of the present application;

FIG. 5 is a schematic illustration of a crop mode of a character shape graph provided based on the embodiment shown in FIG. 4;

FIG. 6 is a flowchart of an image generation method provided by another exemplary embodiment of the present application;

FIG. 7 is a diagram of a splice requirement for character shape diagrams provided based on the embodiment shown in FIG. 6;

FIG. 8 is a reference line division schematic provided based on the embodiment shown in FIG. 6;

FIG. 9 is a diagram of an image generation method provided by another exemplary embodiment of the present application;

FIG. 10 is a schematic illustration of stitching a candidate stitched image based on the pictorial view provided by the embodiment shown in FIG. 9;

FIG. 11 is a schematic illustration of stretching a candidate stitched image provided based on the embodiment shown in FIG. 9;

FIG. 12 is a schematic drawing of stretching a candidate stitched image provided based on the embodiment shown in FIG. 9;

FIG. 13 is a schematic diagram of the results of a second image provided based on the embodiment shown in FIG. 9;

fig. 14 is a block diagram of an image generating apparatus provided in an exemplary embodiment of the present application;

fig. 15 is a block diagram of an image generating apparatus provided in another exemplary embodiment of the present application;

fig. 16 is a block diagram of a server according to an exemplary embodiment of the present application.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

The poster is widely applied to propaganda of multimedia programs in various forms such as film and television works, musical works, live performances and the like, for example: displaying a propaganda poster of a television play and a movie on a first page or a propaganda page of a film and television application program; displaying a propaganda poster of the musical composition on a first page or propaganda page of the music application program; the live performance promotional posters are displayed on the front page or promotional page of the ticketing application.

In the embodiment of the present application, a poster display in a movie application will be described as an example.

In the related art, the poster is mainly obtained by the following modes:

1. the designer designs and makes according to the plot manual of the drama or video, the process is time-consuming and labor-consuming, the generation efficiency of the poster is low, and the poster matched with various different roles cannot be generated rapidly according to the video roles;

2. Artificial intelligence (Artificial Intelligence, AI) generation, schematically shown in fig. 1, selecting a proper video frame 100 from a video, cutting the video frame 100 into a size required to be put, adding contents such as a logo 110, a search box 120, a text 130 and the like, and generating a pair of posters 140; however, the AI graph selection logic is complex, the graph selection effect cannot be controlled, and the accuracy of the poster generation is low.

Fig. 2 is a schematic overall process diagram of an image generating method according to an exemplary embodiment of the present application, as shown in fig. 2, first, a first image 210 corresponding to a multimedia program is acquired, for example: an edited poster image, the first image 210 is a photo file (Photoshop Document, PSD) format image, or the first image 210 is a large photo file (PSB) format image, or the first image 210 is a file in a common format, such as: JPEG (Joint Photographic Experts Group) format, portable grid graphic (Portable Network Graphics, PNG) format, etc.

In this embodiment, taking the first image 210 as an example of the PSD format, the first image 210 includes a plurality of layers, each layer is used for editing and displaying different image contents, and after all the layers are overlapped, the complete display content of the first image 210 is obtained.

In some embodiments, the layers of the first image 210 include a first type of layer for editing the overall image and a second type of layer for editing the individual character avatar. After face detection is performed on each layer of the first image 210, a second-class layer for editing the image of the single character is obtained from each layer, and a face area corresponding to each layer in the second-class layer is obtained.

The second type of layers including the face region are stitched and superimposed on the background layer 200 to obtain a second image 220, which is used as a newly generated poster for promoting the multimedia program. Schematically, the second type of layers includes a layer a (corresponding to a role a), a layer B (corresponding to a role B), and a layer C (corresponding to a role C), and if the layer a and the layer B are spliced to form a poster matched with the roles a and B, the layer B and the layer C are spliced to form a poster matched with the roles B and C, the layer a and the layer C are spliced to form a poster matched with the roles a and C, and the layer a, the layer B, or the layer C may be independently used as a recombined spliced image.

The background layer 200 may be a background layer of the first image 210, or may be an image uploaded by a user, or a randomly selected image, which is not limited in this embodiment.

It should be noted that, in the above embodiment, the first image 210 is taken as a PSD format as an example, and in some embodiments, the first image 210 may be implemented as a normal format image including only one image layer, and then the character shape images corresponding to the respective characters are cut from the first image 210 by a matting technique to perform stitching.

In addition, in the above embodiments, the first image 210 includes a plurality of character images as an example, in some embodiments, when the first image 210 includes one character image, the character image is obtained (for example, an image layer corresponding to the character image is obtained; or, a character image is scratched), and the character image is adjusted and then superimposed on a specified background image, so as to obtain an updated poster, for example: cutting the diagonal image and then superposing the cut diagonal image on the appointed background image; or, the color of the diagonal image is mixed and then superimposed on the appointed background image; or, the image is sharpened and then superimposed on the designated background image. In this embodiment, a plurality of character images included in the first image 210 will be described as an example.

Next, an implementation environment according to an embodiment of the present application will be described, schematically, with reference to fig. 3, where a terminal 310 and a server 320 are involved, and the terminal 310 and the server 320 are connected through a communication network 330.

In some embodiments, the terminal 310 is configured to send a poster of the target multimedia program to the server 320. In some embodiments, the terminal 310 uploads the poster of the target multimedia program to the server 320 through the poster generation platform; alternatively, the terminal 310 uploads the poster of the target multimedia program to the server 320 through the installed target application, wherein the target application includes a video playing application, a music playing application, a ticket program, an instant messaging application, and other types of programs.

The server 320 is configured to receive the poster uploaded by the terminal 310 and reorganize the roles in the poster, where the reorganization mode of the server 320 includes: 1. when the poster is in a PSD format, obtaining the layers corresponding to each role from the poster, and carrying out recombination and splicing; 2. when the poster is in any format, cutting the images corresponding to each role from the poster, and recombining and splicing the cut role pictograms.

The server 320 superimposes the spliced image after the recombination and splicing on the appointed background image to obtain a newly generated poster image, and feeds back the newly generated poster image to the terminal 310 for display.

The terminal may be a mobile phone, a tablet computer, a desktop computer, a portable notebook computer, an intelligent television, an intelligent vehicle-mounted terminal, or other terminal devices in various forms, which is not limited in this embodiment of the present application.

It should be noted that the server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, a content delivery network (Content Delivery Network, CDN), and basic cloud computing services such as big data and an artificial intelligence platform.

Cloud technology (Cloud technology) refers to a hosting technology that unifies serial resources such as hardware, software, networks and the like in a wide area network or a local area network to realize calculation, storage, processing and sharing of data. The cloud technology is based on the general names of network technology, information technology, integration technology, management platform technology, application technology and the like applied by the cloud computing business mode, can form a resource pool, and is flexible and convenient as required. Cloud computing technology will become an important support. Background services of technical networking systems require a large amount of computing, storage resources, such as video websites, picture-like websites, and more portals. Along with the high development and application of the internet industry, each article possibly has an own identification mark in the future, the identification mark needs to be transmitted to a background system for logic processing, data with different levels can be processed separately, and various industry data needs strong system rear shield support and can be realized only through cloud computing.

In some embodiments, the servers described above may also be implemented as nodes in a blockchain system. Blockchain (Blockchain) is a new application mode of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanisms, encryption algorithms, and the like. The blockchain is essentially a decentralised database, and is a series of data blocks which are generated by association by using a cryptography method, and each data block contains information of a batch of network transactions and is used for verifying the validity (anti-counterfeiting) of the information and generating a next block. The blockchain may include a blockchain underlying platform, a platform product services layer, and an application services layer.

The image generation method provided by the application is described by combining the description and the application scenario, and the method can be executed by a server or a terminal, or can be executed by the server and the terminal together.

Step 401, a first image corresponding to a target multimedia program is acquired, wherein the first image comprises at least two face images associated with the target multimedia program.

In some embodiments, the manner in which the first image is acquired includes at least one of:

Firstly, a first image uploaded by a terminal is received, namely, the terminal uploads the first image to a server through a platform providing a poster generation function for poster generation;

secondly, downloading the published poster corresponding to the target multimedia program from the application program as a first image;

thirdly, the poster of the target multimedia program is acquired from the stored database as a first image.

It should be noted that the above manner of acquiring the first image is merely an illustrative example, which is not limited in the embodiments of the present application.

The first image is a PSD format image, or a PSB format image, or a file with a common format, such as: JPEG format, PNG format, etc.

When the first image is a PSD format or PSB format image, at least two face images included in the first image are distributed in the same or different layers.

Illustratively, the first image includes a layer a, a layer B, and a layer C, where a character a is displayed on the layer a, a character B is displayed on the layer B, and a character C is displayed on the layer C. Or, the first image includes a layer a, a layer B and a layer C, where the layer a is used to edit the overall tone of the first image, the layer B includes roles a and B and a role C, and the layer C is used to edit the background portion of the first image.

The association relationship between the at least two face characters included in the first image and the multimedia program includes at least one of the following cases:

first, at least two face figures are face figures corresponding to a participant of a multimedia program;

secondly, at least two face images are face images corresponding to a planner of the multimedia program;

thirdly, at least two face images are face images corresponding to a host of the multimedia program;

fourth, at least two face avatars are cartoon avatars in the multimedia program.

It should be noted that the above-mentioned case is merely an illustrative example, and the embodiment of the present application is not limited thereto.

Step 402, performing face detection on the first image to obtain face areas corresponding to at least two face images in the first image.

In some embodiments, when the first image includes at least two image layers, face detection may be performed on each image layer in the first image, or face detection may be performed on the first image as a whole.

Taking face detection of each image layer of the first image as an example for explanation, namely carrying out face detection on at least two image layers of the first image to obtain face detection results corresponding to the at least two image layers respectively, wherein the face detection results are used for indicating the face inclusion condition of the image layers; at least two face regions in the image layer are acquired in response to the face contained in the image layer.

If the first image does not comprise at least two image layers, the whole first image is directly subjected to face detection.

In some embodiments, the first image is face detected by a face detector, illustratively by a Dilb face detector.

Optionally, after face detection is performed on the first image, a face region with a face detection result meeting detection requirements is obtained. Schematically, a face region with a face probability greater than a probability threshold is obtained in a face detection result; or, in the face detection result, a face area with the area of the face area being larger than an area threshold value is obtained; or, in the face detection result, the face area corresponding to the identity corresponding to the face is obtained, and the embodiment is not limited to this.

In some embodiments, at least two image layers including a face region are acquired, and the face region in the at least two image layers is acquired. Optionally, at least two image layers including one face region are acquired, and the face region in each image layer is acquired, wherein each image layer includes one face region. And clipping the ith character image corresponding to the ith face area from the image layer where the ith face area is located.

Step 403, clipping based on the face area to obtain at least two character shape charts corresponding to the face area.

Wherein the ith face area corresponds to the ith character shape image, i being a positive integer.

In some embodiments, cropping the graphical image includes at least one of:

1. cutting the periphery of the character corresponding to the face area along the minimum bounding rectangle to obtain a character pictogram;

2. carrying out picture matting and cutting along the edges of the characters corresponding to the face area to obtain character shape pictures;

3. cutting a rectangle with a preset size by taking a person corresponding to the face area as a center to obtain a character shape image;

4. cutting to obtain a character image by taking the appointed position of the character corresponding to the face area in the character image as a reference;

5. the image layer where the face area is located is determined as a character shape image.

It should be noted that the foregoing manner is merely illustrative, and the embodiments of the present application are not limited thereto.

In some embodiments, when the character image is cut, the whole body image corresponding to the face area is cut into the half body image, so as to obtain the character image. That is, the upper body is cut after the position of the face region is acquired. Assuming that the upper edge of the face area is face_up, the lower edge of the face area is face_down, that is, the size of the face frame is face_down-face_up. The length of the upper body of a human body is 3 times of the head length, and the size of the upper body is 3× (face_down-face_up), namely, the top of the image can be cut to face_up+3× (face_down-face_up). Typically, if it is desired to intercept the lumbar region, an alpha-fold head length may be chosen, i.e., crop the top of the image to face_up+alpha× (face_down-face_up).

In some embodiments, after the body figure is cut, the white area of the body figure is also cut.

Illustratively, as shown in FIG. 5, the whole-body illumination 500 may be cut into top-to-solid line 510 portions. Since the area of the right region 520 is too large, if combined with other pictorial images, the left region is too large, and thus the left region is cut for half body, i.e., the upper right region 520 is cut, and the pictorial image 530 is finally obtained.

The clipping mode traverses the area of the image from left to right, and the area with all white colors is clipped. Illustratively, if the picture size is 1000 x 600, the column-wise traversal is performed to calculate whether each column is all white. If 0 to 200, 300 to 400, 700 to 1000 are all white areas, here the 200 to 700 parts are obtained after clipping. This is because 0 to 200 are leftmost regions, indicating that there is no specific image to the left, and 700 to 1000 are rightmost regions, indicating that the right is blank, and 300 to 400 are blank, possibly resulting from certain effects of the human body image.

And step 404, superposing and displaying the spliced image after the character image graph reorganization on the appointed background graph to obtain a second image corresponding to the target multimedia program.

In some embodiments, the character image is spliced according to a preset splicing rule to obtain a spliced image; or, after the character image is spliced randomly, a spliced image is obtained.

When the character image is spliced according to a preset splicing rule, the character image is illustratively spliced in the horizontal direction, that is, the eye heights of the face areas corresponding to the character image are kept relatively consistent, and the character image is transversely spliced; or after the character image is cut to the same height, splicing the character image in a preset direction according to a preset interval. This embodiment is not limited thereto.

Optionally, when the diagonal color image map is recombined and spliced, the diagonal color image map is spliced according to a preset splicing number, for example: the preset splicing number is 2, and two character shape images are spliced each time; or, according to the possible collocation situation, polling to traverse each character shape image, and respectively collocating different situations. Illustratively, the character image 1, the character image 2, the character image 3 and the character image 4 are obtained by clipping from the first image; when matching and splicing the diagonal pictograms, the matching conditions are as follows:

The character shape is like that of fig. 1; the character shape is like fig. 2; the character shape is like fig. 3; character shape figure 4 (single character image);

character figure 1 and character figure 2; character figure 1 and character figure 3; character figure 1 and character figure 4; character figure 2 and character figure 3; character image fig. 2 and character image fig. 4; character figure 3 and character figure 4 (two character figures collocated);

character image fig. 1, character image fig. 2, and character image fig. 3; character image fig. 1, character image fig. 2, and character image fig. 4; character pictogram 2, character pictogram 3, and character pictogram 4 (three character persona collocations).

In some embodiments, outputting the second image after all the matching and stitching; or selecting matching conforming to the alignment rule from the matching and splicing to splice, thereby obtaining a second image.

It should be noted that, in the above embodiment, the first image includes at least two face images, and in other embodiments, the first image may also include only one face image, and then, in the PSD image, after performing face detection on an image layer in the first image, an image layer where a face image is located is obtained from the first image, a character image of the face image is obtained based on the image layer, and an image of the character image is edited, for example: and after the processes of cutting, toning, sharpening, blurring and the like, superposing the role image on the appointed background image to obtain a second image.

In summary, in the image generation method provided by the embodiment of the application, after the first image corresponding to the target multimedia program (such as the poster of the target multimedia program) is obtained, face detection is performed on the first image, and the character image in the first image is obtained by clipping according to the detected face area, so that a plurality of images corresponding to the target multimedia program are generated based on the collocation of different character images, interest points of different users are attached by different character collocation dimensions, the manufacturing efficiency of the poster of the multimedia program is improved, and manpower and material resources in the manufacturing process of the poster are saved.

According to the method provided by the embodiment, for the image with the PSD or PSB format of the first image, face detection is carried out on each image layer of the first image, so that each image layer with a face area is obtained through identification, the face areas of the image layers are obtained, the character image graphs corresponding to the face areas of the image layers are spliced, and the obtaining efficiency of the character image graphs is improved.

In an alternative embodiment, taking an example that at least two face images of the first image include the first face image and the second face image as an illustration, fig. 6 is a flowchart of an image generating method according to another exemplary embodiment of the present application, where the method may be executed by a server or a terminal, or may be executed by the server and the terminal together, and in the embodiment of the present application, the method is executed by the server as an illustration, as shown in fig. 6, and the method includes the following steps.

Step 601, a first image corresponding to a target multimedia program is acquired, wherein the first image comprises at least two face images associated with the target multimedia program.

Wherein the at least two face avatars include a first face avatar and a second face avatar.

Optionally, each layer of the PSD poster file is analyzed, a PSD or PSB file is input, the hierarchical relation analysis file is removed, and all pictures are saved. This step can be implemented with the python psd_tools packet, the output being a number of pictures with PNG channels. These saved pictures are then parsed, so for convenience, these pictures are renamed to prevent them from being found or processed under the linux system. Renaming modifies mainly chinese, or "/" representing hierarchy. After modification, each picture can be saved in the same directory.

Step 602, performing face detection on the first image to obtain face areas corresponding to at least two face images in the first image.

In some embodiments, face detection is performed on at least two image layers of the first image to obtain face detection results corresponding to the at least two image layers respectively, wherein the face detection results are used for indicating the face inclusion condition of the image layers; at least two face regions in the image layer are acquired in response to the face contained in the image layer.

The face area comprises a first face area corresponding to the first face image and a second face area corresponding to the second face image.

Optionally, the face region is recorded by recording the position ratio of the upper edge and the lower edge of the face region, for example: the image is 1280×720, that is, the image width 1280, the image height 720, if the upper left corner height of the face frame is 120 and the lower right corner height is 144, the upper edge of the face frame is 0.167 (120/720), and the lower edge is 0.2 (144/720).

In some embodiments, face detection is performed directly on the first image; alternatively, the first image is first compressed by a fixed factor, such as: and compressing the first image to 1/3 of the original image, and then performing face detection, so that the face detection speed is improved.

Step 603, clipping the face area to obtain a first image corresponding to the first face area and a second image corresponding to the second face area.

Optionally, clipping from an image layer where the first face area is located to obtain a first pictogram corresponding to the first face area; and similarly, clipping from the image layer where the second face area is located to obtain a second pictogram corresponding to the second face area.

In some embodiments, the first and second images are first scaled to a high degree of coincidence, thereby achieving a coincidence in the relative horizontal directions of view, and the first and second images are stitched based on the high degree of coincidence.

In some embodiments, in response to the first face region and the second face region conforming to a preset alignment rule, the first face region and the second face region are formed into a character stitching pair, where the character stitching pair is used to indicate that a combination of character image stitching is required, that is, when the alignment rule conforming to the character stitching between the first face region and the second face region conforms to the character stitching, it is determined that character image images corresponding to the first face region and the second face region need to be stitched.

Optionally, the preset alignment rule includes at least one of the following rules:

first of all, the first one,acquiring upper edge height information and lower edge height information of the face region corresponding to the first image, wherein the upper edge height information is used for indicating the proportion of the upper edge of the face region to the height of the first image, and the lower edge height information is used for indicating the proportion of the lower edge of the face region to the height of the first image; responsive to a first upper edge height of the first facial regionThe information and the second upper edge height information of the second face area meet the second height difference requirement, and the first face area and the second face area form an image stitching pair.

Illustratively, after the heights of the first image and the second image are converted to be consistent, the first image and the second image are spliced in the horizontal direction, when the difference of the heights of faces of the first image and the second image is large, the first image and the second image are not suitable for splicing, and illustratively, as shown in fig. 7, the difference of the heights of faces between the first image 710 and the second image 720 is large, and the first image and the second image are not suitable for splicing. In this embodiment, when the height difference of the upper edges of the two character images is less than 0.08 and the absolute value of the lower edge height difference is less than 0.08, the two character images can be spliced.

Second, the first one is a first one,acquiring line-of-sight data corresponding to the face region, wherein the line-of-sight data is used for indicating eye line angles in the face region; and responding to the first sight line data of the first face area and the second sight line data of the second face area to meet the sight line difference requirement, and forming an image stitching pair by the first face area and the second face area.

In some embodiments, two face regions with the same or similar line of sight are formed into a visual mosaic pair; alternatively, two face regions with opposite vision are formed into a visual mosaic pair. The line-of-sight data is used for representing the angle between the eye line-of-sight direction and the shooting direction in the face area, and the line-of-sight direction of each character in the first image is identified through the machine learning model.

That is, in response to a line-of-sight angle difference between the first line-of-sight data of the first face region and the second line-of-sight data of the second face region being within a preset angle difference threshold, forming an image stitching pair of the first face region and the second two-step region; or in response to the angle of view difference between the first line of sight data of the first face region and the second line of sight data of the second face region being greater than the required angle difference, forming the first face region and the second two-step region into a visual mosaic pair.

Third, the third step of, in the case of a vehicle,dividing reference lines into a first image and a second image based on a preset dividing rule; determining the firstAnd determining a first position relation between the face area and the reference line, determining a second position relation between the second face area and the reference line, and forming an image stitching pair by the first face area and the second face area in response to the position relation meeting the position requirement.

Schematically, after the first image and the second image are adjusted to be consistent in height, the reference lines are divided in the height direction of the first image and the second image according to a preset interval, referring to fig. 8, and after the reference lines are divided for the first image 810 and the second image 820, the positional relationship between each part of the five sense organs in the first face area and the reference lines is determined, for example: and if the first position relation between the first face area and the reference line is consistent with the second position relation between the second face area and the reference line, determining that the first face area and the second face area form an image stitching pair.

It should be noted that the above manner of determining the image stitching pair is merely an illustrative example, and the embodiments of the present application may determine whether to stitch the first image and the second image in other manners, which is not limited.

In addition, the above image stitching is described taking the stitching of the two character images of the first image and the second image as an example, and in some embodiments, the image stitching may be applied to the stitching of three or more character images, which is not limited in this embodiment.

And step 604, overlapping and displaying the spliced image of the first pictogram and the second pictogram on the appointed background image to obtain a second image corresponding to the target multimedia program.

In some embodiments, after stitching the first and second images, the stitched image is superimposed over the designated background image, resulting in a second image. The first image map and the second image map are spliced to obtain spliced images, and the spliced images are displayed on a specified background image in a superimposed mode to obtain a second image corresponding to the target multimedia program.

The specified background image may be a background image uploaded by the user, or may be a background image of the first image, or may be a stored random image, or may be a stored selected image. The embodiments of the present application are not limited in this regard.

According to the method provided by the embodiment, the splicing among the character image graphs is filtered through the preset alignment rule, so that the situation of poor splicing effect caused by large character image graph difference is avoided, and the splicing accuracy of the character image graphs and the accuracy of poster generation are improved.

The difference of the size and the position of the face area between the two character pictograms is judged through the upper edge and the lower edge of the face area, so that the large difference of the positions of the faces of the two spliced character pictograms is avoided, and the splicing effect of the character pictograms is improved.

And judging the difference of the sight directions between the two character image graphs through sight data, and splicing the character image graphs of the opponent characters with different sight directions according to requirements, or splicing the character image graphs of the teammate characters with consistent sight directions, so that the splicing effect of the character image graphs is improved.

By cutting the character pictograms of the characters into the half-body images, the splicing basis of the character pictograms is unified, the problem of dislocation caused by splicing the whole-body images and the half-body images is avoided, and the splicing accuracy of the character pictograms is improved.

In some embodiments, the stitching of the first and second images may also require adjustment. Fig. 9 is a flowchart of an image generation method provided in another exemplary embodiment of the present application. The method may be executed by a server or a terminal, or may be executed by both the server and the terminal, and in this embodiment of the present application, the method is described as being executed by the server, and as shown in fig. 9, the method includes the following steps.

Step 901, a first image corresponding to a target multimedia program is acquired, wherein the first image comprises at least two face images associated with the target multimedia program.

And 902, performing face detection on the first image to obtain face areas corresponding to at least two face images in the first image.

Step 903, clipping a first image corresponding to the first face region and a second image corresponding to the second face region based on the face region.

And step 904, splicing the first image graph and the second image graph to obtain candidate spliced images.

Optionally, stitching the first image graph and the second image graph with the adjusted heights in a horizontal linking mode to obtain candidate stitched images.

In some embodiments, since the last output image is generally used for delivery, the width and height of the image are often selected to be in a common ratio, such as 16:9, 5:3, etc. For ease of handling, the output image is stored at 1280×720px here, so that it is also necessary to stretch the image.

In some embodiments, as shown in fig. 10, after the first image 1010 and the second image 1020 are stitched, a candidate stitched image is obtained, as shown in fig. 10, in the candidate stitched image, the top of the head of the character of the first image 1010 and the second image 1020 is relatively attached to the upper edge of the candidate stitched image, and the distance between the characters of the first image 1010 and the second image 1020 is relatively short.

Step 905, stretching the candidate stitched image with a preset stretching rule to obtain a stitched image, where the preset stretching rule is used to adjust the positional relationship between the first pictogram and the second pictogram.

In some embodiments, the manner in which the candidate stitched image is stretched comprises at least one of:

First kind: and expanding the candidate spliced image to a specified stretching direction at a first stretching ratio.

Illustratively, considering that the human body position in the candidate stitched image starts at the uppermost part of the image and the aesthetic degree is poor, an overparameter is adopted to represent the proportion of the upper expansion, and assuming that the width and the height of the existing image are w×h respectively, the upper expansion area can be represented by β, that is, the image becomes w width and the height is (1+β) ×h, and the effect of β=0.3 is better through experiments. Schematically, as shown in fig. 11, after the first image 1110 and the second image 1120 are spliced, stretching is performed in the height direction, i.e., stretching up to a height of 0.3 h.

In some embodiments, in addition to stretching upward, stretching may also be performed to the left and right, respectively, by a corresponding ratio, such as: stretching 0.3w to the left and right, respectively.

Second kind: and expanding the distance between the first image and the second image in the candidate spliced image at a second stretching ratio.

Schematically, considering that the distance between the human body positions in the candidate stitched image is relatively short and the stitching effect is relatively poor, a super parameter is adopted to represent the distance between the two roles, and assuming that the width and the height of the existing image are w×h respectively, two intermediate expansion areas can be represented by gamma, namely, the image becomes (1+gamma) ×w wide and the height is h, and the effect of gamma=0.1 is relatively good through experiments. Schematically, as shown in fig. 12, after the first image 1210 and the second image 1220 are spliced, stretching is performed between the first image 1210 and the second image 1220, that is, stretching up to a width of 0.1 w.

It should be noted that the stretching manner described above is merely an illustrative example, and the stretching adjustment manner is not limited in the embodiments of the present application.

In some embodiments, the new image aspect ratio is generally not compliant with the aspect ratio requirements after the stretching operation is performed on the candidate stitched image, and therefore expansion is also required. The expansion includes two kinds, the first is high-direction expansion, and the expansion is performed at the top until the new image aspect ratio is the required aspect ratio; the second is a wide-direction expansion, where expansion is on both sides until the aspect ratio is the desired aspect ratio. Illustratively, an aspect ratio of 16:9 is required, if the current image is 1000:900, then 1000 needs to be increased 600 to 1600 more, so the image is 1600:900, i.e. 16:9, where the augmented 600 may be augmented 300 on the left side of the artwork and 300 on the right side, ensuring that the artwork is located in the middle region of the new artwork.

And step 906, superposing and displaying the spliced image on the appointed background image to obtain a second image corresponding to the target multimedia program.

In some embodiments, receiving a specified background image, and displaying a spliced image with recombined character image images superimposed on the received specified background image; or, responding to the first image including at least two image layers, acquiring a background image layer in the first image, and displaying the spliced image with the recombined character image on the background image layer in a superposition manner.

In other embodiments, the background image may also be obtained by cropping and scaling portions of the background of the original first image; or by cropping a portion of the background of the original first image and repeating the stitching.

Optionally, after obtaining the stitched image and the background image, the stitched image and the background image may also be synchronously scaled to the same required ratio, for example: the stitched image with the ratio of 16:9 is scaled up to 1280:720, and the background image is also scaled up to 1280:720, thereby overlaying the stitched image over the background image.

Illustratively, taking the first image including five character images as an example, after performing the matching of the preset alignment rule and the screening, matching to obtain four groups of character matching pairs from the five character images, wherein each group of character matching pairs includes two characters, as shown in fig. 13, the first image 1300 includes a first character 1310, a second character 1320, a third character 1330, a fourth character 1340 and a fifth character 1350, and after performing the matching of the preset alignment rule and the screening, matching to obtain an image 1360, an image 1370, an image 1380 and an image 1390.

According to the method provided by the embodiment, after the first pictogram and the second pictogram are spliced, the candidate spliced image is obtained, and after the candidate spliced image is stretched according to the preset stretching rule and the stretching proportion, the spliced image is obtained, the problem that the coordination is poor because the pictograms are on the top edge of the second image or the distance between the pictograms is too close is avoided, and the effect of generating the poster is improved.

Fig. 14 is a block diagram of an image generating apparatus according to an exemplary embodiment of the present application, and as shown in fig. 14, the apparatus includes:

an obtaining module 1410, configured to obtain a first image corresponding to a target multimedia program, where the first image includes at least two face images associated with the target multimedia program;

the detection module 1420 is configured to perform face detection on the first image to obtain face areas corresponding to at least two face images in the first image respectively;

a clipping module 1430 configured to clip the face region to obtain at least two character shape images corresponding to the face region, where the ith face region corresponds to the ith character shape image, and i is a positive integer;

and the reorganization module 1440 is configured to superimpose and display the spliced image reorganized by the character image on the specified background image to obtain a second image corresponding to the target multimedia program.

In an alternative embodiment, the first image includes at least two image layers therein;

as shown in fig. 15, the detection module 1420 includes:

a detection unit 1421, configured to perform face detection on at least two image layers of the first image to obtain face detection results corresponding to the at least two image layers, where the face detection results are used to indicate a face containing condition of the image layers;

an acquiring unit 1422, configured to acquire at least two face areas in the image layer in response to the image layer including a face.

In an alternative embodiment, the acquiring unit 1422 is further configured to acquire at least two image layers including the face region, and acquire the face region in the at least two image layers.

In an optional embodiment, the clipping module 1430 is further configured to clip the ith character shape image corresponding to the ith face area from the image layer where the ith face area is located.

In an alternative embodiment, the at least two face figures include a first face figure and a second face figure, and the face region includes a first face region of the first face figure and a second face region of the second face figure;

The clipping module 1430 is further configured to clip, based on the face region, a first pictogram corresponding to the first face region and a second pictogram corresponding to the second face region;

the reorganization module 1440 is further configured to superimpose and display a stitched image of the first pictogram and the second pictogram on a specified background image, so as to obtain the second image corresponding to the target multimedia program.

In an alternative embodiment, the apparatus further comprises:

a determining module 1450, configured to, in response to the first face area and the second face area conforming to a preset alignment rule, form an avatar stitching pair with the first face area and the second face area, where the avatar stitching pair is used to indicate a combination of avatar graph stitching needs to be performed.

In an optional embodiment, the determining module 1450 is further configured to obtain upper edge height information and lower edge height information of the face region corresponding to the first image, where the upper edge height information is used to indicate a ratio of an upper edge of the face region to the first image height, and the lower edge height information is used to indicate a ratio of a lower edge of the face region to the first image height; and responding to the condition that the first upper edge height information of the first face area and the second upper edge height information of the second face area meet a first height difference requirement, and the first lower edge height information of the first face area and the second lower edge height information of the second face area meet a second height difference requirement, and forming the first face area and the second face area into the image stitching pair.

In an optional embodiment, the determining module 1450 is further configured to obtain line of sight data corresponding to the face area, where the line of sight data is used to indicate an eye line of sight angle in the face area; and responding to the condition that the first sight line data of the first face area and the second sight line data of the second face area meet the sight line difference requirement, and forming the image stitching pair by the first face area and the second face area.

In an alternative embodiment, the determining module 1450 is further configured to divide the reference line into the first image and the second image based on a preset dividing rule; determining a first positional relationship of the first face region and the reference line; determining a second positional relationship of the second face region with the reference line; and responding to the first position relation and the second position relation to meet the position requirement, and forming the image stitching pair by the first face area and the second face area.

In an alternative embodiment, the reorganizing module 1440 is further configured to stitch the first pictogram and the second pictogram to obtain the stitched image; and superposing and displaying the spliced image on the appointed background image to obtain the second image corresponding to the target multimedia program.

In an alternative embodiment, the reorganizing module 1440 is further configured to stitch the first pictogram and the second pictogram to obtain candidate stitched images; and stretching the candidate spliced image according to a preset stretching rule to obtain the spliced image, wherein the preset stretching rule is used for adjusting the position relationship between the first pictogram and the second pictogram.

In an alternative embodiment, the reorganizing module 1440 is further configured to expand the candidate stitched image in a specified stretching direction at a first stretching ratio;

or alternatively, the process may be performed,

the reorganization module 1440 is further configured to expand a distance between the first pictogram and the second pictogram in the candidate stitched image at a second stretch ratio.

In an alternative embodiment, the reorganizing module 1440 is further configured to receive the specified background image; superposing and displaying the spliced image after the character image is recombined on the appointed background image;

or alternatively, the process may be performed,

the reorganization module 1440 is further configured to, in response to the first image including at least two image layers, obtain a background image layer in the first image, and superimpose and display the stitched image after reorganizing the character shape image on the background image layer.

In summary, the image generating device provided in the embodiment of the present application performs face detection on a first image (e.g., a poster of a target multimedia program) after obtaining the first image corresponding to the target multimedia program, and cuts out a character image in the first image according to the detected face area, so as to generate a plurality of images corresponding to the target multimedia program based on the matching of different character images, and attach interest points of different users with different character matching dimensions, thereby improving the efficiency of manufacturing the poster of the multimedia program and saving manpower and material resources in the manufacturing process of the poster.

It should be noted that: the image generating apparatus provided in the above embodiment is only exemplified by the division of the above functional modules, and in practical application, the above functional allocation may be performed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules, so as to perform all or part of the functions described above. In addition, the image generating apparatus and the image generating method provided in the foregoing embodiments belong to the same concept, and specific implementation processes thereof are detailed in the method embodiments, which are not described herein again.

Fig. 16 shows a schematic structural diagram of a server according to an exemplary embodiment of the present application. The server may be a server as shown in fig. 2.

Specifically, the present invention relates to a method for manufacturing a semiconductor device. The server 1600 includes a central processing unit (Central Processing Unit, CPU) 1601, a system Memory 1604 including a random access Memory (Random Access Memory, RAM) 1602 and a Read Only Memory (ROM) 1603, and a system bus 1605 connecting the system Memory 1604 and the central processing unit 1601. The server 1600 also includes a mass storage device 1606 for storing an operating system 1613, application programs 1614, and other program modules 1615.

The mass storage device 1606 is connected to the central processing unit 1601 through a mass storage controller (not shown) connected to the system bus 1605. The mass storage device 1606 and its associated computer-readable media provide non-volatile storage for the server 1600. That is, the mass storage device 1606 may include a computer-readable medium (not shown) such as a hard disk or compact disk read-only memory (Compact Disc Read Only Memory, CD-ROM) drive.

Computer readable media may include computer storage media and communication media without loss of generality. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes RAM, ROM, erasable programmable read-only memory (Erasable Programmable Read Only Memory, EPROM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read Only Memory, EEPROM), flash memory or other solid state memory technology, CD-ROM, digital versatile disks (Digital Versatile Disc, DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices. Of course, those skilled in the art will recognize that computer storage media are not limited to the ones described above. The system memory 1604 and mass storage device 1606 described above may be collectively referred to as memory.

According to various embodiments of the application, the server 1600 may also operate by a remote computer connected to the network through a network, such as the Internet. That is, the server 1600 may be connected to the network 1612 through a network interface unit 1611 coupled to the system bus 1605, or the network interface unit 1611 may be used to connect to other types of networks or remote computer systems (not shown).

The memory also includes one or more programs, one or more programs stored in the memory and configured to be executed by the CPU.

Embodiments of the present application also provide a computer device that may be implemented as a terminal or server as shown in fig. 2. The computer device includes a processor and a memory in which at least one instruction, at least one program, a set of codes, or a set of instructions is stored, the at least one instruction, the at least one program, the set of codes, or the set of instructions being loaded and executed by the processor to implement the image generation method provided by the above-described method embodiments.

Embodiments of the present application also provide a computer readable storage medium having stored thereon at least one instruction, at least one program, a code set, or an instruction set, where the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the image generating method provided by the above-mentioned method embodiments.

Embodiments of the present application also provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions so that the computer device performs the image generation method described in any of the above embodiments.

Alternatively, the computer-readable storage medium may include: read Only Memory (ROM), random access Memory (RAM, random Access Memory), solid state disk (SSD, solid State Drives), or optical disk, etc. The random access memory may include resistive random access memory (ReRAM, resistance Random Access Memory) and dynamic random access memory (DRAM, dynamic Random Access Memory), among others. The foregoing embodiment numbers of the present application are merely for describing, and do not represent advantages or disadvantages of the embodiments.

It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program for instructing relevant hardware, where the program may be stored in a computer readable storage medium, and the storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.

The foregoing description of the preferred embodiments is merely exemplary in nature and is in no way intended to limit the invention, since it is intended that all modifications, equivalents, improvements, etc. that fall within the spirit and scope of the invention.

Claims

1. An image generation method, the method comprising:

2. The method of claim 1, wherein the first image comprises at least two image layers therein;

the step of performing face detection on the first image to obtain face areas corresponding to at least two face images in the first image, includes:

Face detection is carried out on at least two image layers of the first image to obtain face detection results corresponding to the at least two image layers respectively, wherein the face detection results are used for indicating the face containing condition of the image layers;

and acquiring at least two face areas in the image layer in response to the fact that the image layer contains the faces.

3. The method of claim 2, wherein the acquiring at least two face regions in the image layer in response to the image layer containing a face comprises:

at least two image layers including a face region are acquired, and a face region in the at least two image layers is acquired.

4. A method according to claim 3, wherein said clipping based on said face region results in at least two character shape charts corresponding to said face region, comprising:

and clipping the ith character image corresponding to the ith face area from the image layer where the ith face area is located.

5. The method of any one of claims 1 to 4, wherein the at least two face avatars include a first face avatar and a second face avatar, and wherein the face region includes a first face region of the first face avatar and a second face region of the second face avatar;

The clipping based on the face area to obtain at least two character shape images corresponding to the face area comprises the following steps:

clipping the face area to obtain a first pictogram corresponding to the first face area and a second pictogram corresponding to the second face area;

and superposing and displaying the spliced image after the character image graph recombination on the appointed background image to obtain a second image corresponding to the target multimedia program, wherein the method comprises the following steps:

and superposing and displaying the spliced images of the first pictogram and the second pictogram on the appointed background image to obtain the second image corresponding to the target multimedia program.

6. The method of claim 5, wherein displaying the stitched image of the first pictogram and the second pictogram superimposed on the specified background image, before obtaining the second image corresponding to the target multimedia program, further comprises:

and responding to the first face area and the second face area to accord with a preset alignment rule, forming an image stitching pair by the first face area and the second face area, wherein the image stitching pair is used for indicating the combination of image stitching.

7. The method of claim 6, wherein the grouping the first face region and the second face region into a stitching pair responsive to the first face region and the second face region meeting a preset alignment rule comprises:

acquiring upper edge height information and lower edge height information of the face region corresponding to the first image, wherein the upper edge height information is used for indicating the proportion of the upper edge of the face region to the height of the first image, and the lower edge height information is used for indicating the proportion of the lower edge of the face region to the height of the first image;

and responding to the condition that the first upper edge height information of the first face area and the second upper edge height information of the second face area meet a first height difference requirement, and the first lower edge height information of the first face area and the second lower edge height information of the second face area meet a second height difference requirement, and forming the first face area and the second face area into the image stitching pair.

8. The method of claim 6, wherein the grouping the first face region and the second face region into a stitching pair responsive to the first face region and the second face region meeting a preset alignment rule comprises:

Acquiring line-of-sight data corresponding to the face region, wherein the line-of-sight data is used for indicating eye line-of-sight angles in the face region;

and responding to the condition that the first sight line data of the first face area and the second sight line data of the second face area meet the sight line difference requirement, and forming the image stitching pair by the first face area and the second face area.

9. The method of claim 6, wherein the grouping the first face region and the second face region into a stitching pair responsive to the first face region and the second face region meeting a preset alignment rule comprises:

dividing reference lines to the first image map and the second image map based on a preset dividing rule;

determining a first positional relationship of the first face region and the reference line;

determining a second positional relationship of the second face region with the reference line;

and responding to the first position relation and the second position relation to meet the position requirement, and forming the image stitching pair by the first face area and the second face area.

10. The method of claim 5, wherein displaying the stitched image of the first pictogram and the second pictogram superimposed on the specified background image to obtain the second image corresponding to the target multimedia program comprises:

Splicing the first image graph and the second image graph to obtain the spliced image;

and superposing and displaying the spliced image on the appointed background image to obtain the second image corresponding to the target multimedia program.

11. The method of claim 10, wherein stitching the first and second image maps to obtain the stitched image comprises:

splicing the first image graph and the second image graph to obtain candidate spliced images;

and stretching the candidate spliced image according to a preset stretching rule to obtain the spliced image, wherein the preset stretching rule is used for adjusting the position relationship between the first pictogram and the second pictogram.

12. The method of claim 11, wherein stretching the candidate stitched image with a preset stretching rule comprises:

expanding the candidate spliced image to a specified stretching direction according to a first stretching proportion;

or alternatively, the process may be performed,

and expanding the distance between the first pictogram and the second pictogram in the candidate spliced image at a second stretching proportion.

13. The method according to any one of claims 1 to 4, wherein displaying the recombined stitched image of the character image superimposed on the specified background image comprises:

Receiving the specified background image; superposing and displaying the spliced image after the character image is recombined on the appointed background image;

or alternatively, the process may be performed,

and responding to the first image comprising at least two image layers, acquiring a background image layer in the first image, and displaying the spliced image after the character shape image recombination on the background image layer in a superposition way.

14. An image generation apparatus, the apparatus comprising:

15. A computer device comprising a processor and a memory having stored therein at least one instruction, at least one program, code set or instruction set, the at least one instruction, at least one program, code set or instruction set being loaded and executed by the processor to implement the image generation method of any of claims 1 to 13.

16. A computer readable storage medium having stored therein at least one instruction, at least one program, code set, or instruction set, the at least one instruction, the at least one program, the code set, or instruction set being loaded and executed by a processor to implement the image generation method of any of claims 1 to 13.

17. A computer program product comprising a computer program or instructions which, when executed by a processor, implement the image generation method of any of claims 1 to 13.