WO2014169653A1

WO2014169653A1 - Method and device for optimizing image synthesis

Info

Publication number: WO2014169653A1
Application number: PCT/CN2013/087578
Authority: WO
Inventors: 李玲玲; 李伟
Original assignee: 中兴通讯股份有限公司
Priority date: 2013-08-28
Filing date: 2013-11-21
Publication date: 2014-10-23
Also published as: CN104424624B; CN104424624A

Abstract

Provided are a method and device for optimizing an image synthesis. The method comprises the following steps: determining an image to be synthesized in a source image and location information about the image to be synthesized in the source image (S201); determining a size ratio of a target image to the source image (S202); and according to the location information and the size ratio, synthesizing the image to be synthesized into the target image (S203).

Description

Method and device for optimizing image synthesis

The present invention relates to the field of image processing, and more particularly to a method and apparatus for optimizing image synthesis.

Background technique

The existing image synthesis technology extracts some images in the source image (here, the character image is taken as an example), does not perform any processing on them, and then synthesizes the extracted characters directly to the set background according to the original scale. In the image. As shown in Fig. 1, the image 1 in Fig. 1 is a source image, and the size of the source image is: 15 cm in length and 15 cm in width. The position of the head image A of the person image 11 in the source image can be represented by a coordinate, and the coordinate of the point A is (7, 9) (the coordinate origin is the lower left corner of the source image). The character image 11 is extracted from the source image to obtain the extracted person image 12, and the extracted person image 12 and the target image 22 are combined to obtain a composite image 2 having a size of 10 cm in length and 10 cm in width. . Since the extracted person image 12 is not processed, and the size of the target image 22 and the source image 1 are different, the ratio of the portrait image to the background image in the synthesized image 2 is very uncoordinated. Summary of the invention

The embodiment of the invention provides an image synthesis algorithm, which solves the problem that the ratio of the image to be combined and the target image in the composite image is not coordinated due to the inconsistency between the source image size and the target image size during image synthesis.

An optimization method for image synthesis provided by an embodiment of the present invention includes:

Determining an image to be synthesized in the source image and position information of the image to be synthesized in the source image; determining a size ratio of the target image to the source image;

The image to be synthesized is synthesized into the target image based on the position information and the size ratio.

In an embodiment of the invention, the length ratio and the width of the target image and the source image Degree ratio.

In an embodiment of the invention, the location information is coordinate information of at least one pixel of the image to be composited.

In an embodiment of the present invention, before the image to be synthesized is synthesized into the target image, the method further includes acquiring a color value displayed in the target image of the image to be composited;

When the location information is the coordinate information of each pixel of the image to be combined, the synthesizing the image to be synthesized into the target image according to the location information and the size ratio includes: Coordinate values of the length direction and the width direction of each pixel are multiplied by the length ratio and the width ratio, respectively, to obtain coordinate values of corresponding pixel points in the target image; and the target image is set according to the color value The color parameter of the corresponding pixel;

When the location information is the coordinate information of each pixel on the boundary of the image to be combined, the synthesizing the image to be synthesized into the target image according to the location information and the size ratio includes: Coordinates of each pixel point on the boundary are multiplied by the length ratio and the broadband ratio, respectively, to obtain coordinate values of corresponding pixel points of each pixel on the boundary in the target image; the target image The area enclosed by the boundary pixel is the area to be synthesized; and the color parameter of each pixel in the area to be combined is set according to the color value.

In an embodiment of the invention, the target image and the source image are pictures or videos. In an embodiment of the invention, the target image is a preset background image.

In an embodiment of the invention, the image to be composited is a character image.

An embodiment of the present invention further provides an apparatus for image synthesis optimization, the apparatus comprising: a data acquisition module configured to determine an image to be synthesized in a source image and position information of an image to be synthesized in a source image;

a calculation module configured to determine a size ratio of the source image to the target image; an image synthesis module configured to synthesize the image to be composite into the target image according to the position information and the size ratio .

In an embodiment of the invention, the size ratio includes: a length ratio and a width ratio of the target image to the source image. In an embodiment of the invention, the location information is coordinate information of at least one pixel of the image to be composited.

In an embodiment of the present invention, the device further includes a color acquiring module, configured to acquire the image to be composited before the image synthesizing module synthesizes the image to be synthesized into the target image. The color value displayed in the target image;

The image synthesis module includes a conversion sub-module and a color setting sub-module;

The conversion sub-module is configured to: when the position information is coordinate information of each pixel point of the image to be combined, respectively, the coordinate values of the length direction and the width direction of each pixel point are respectively compared with the length ratio and the Multiplying a width ratio to obtain a coordinate value of a corresponding pixel in the target image; the color setting sub-module is configured to set a color parameter of a corresponding pixel in the target image according to the color value;

The conversion sub-module is further configured to: when the position information is coordinate information of each pixel point on the boundary of the image to be composited, respectively, coordinate the coordinates of each pixel point on the boundary with the length ratio and the Multiplying the broadband ratio to obtain a coordinate value of a corresponding boundary pixel point of each pixel on the boundary in the target image; an area enclosed by the boundary pixel point in the target image is an area to be synthesized; the color setting The sub-module is further configured to set a color parameter of each pixel in the area to be combined according to the color value.

The method and device for optimizing image synthesis provided by the embodiment of the present invention, by determining the image to be synthesized in the source image and the position information of the image to be synthesized in the source image, according to the size ratio of the target image and the source image, will be described The composite image is synthesized into the target image, instead of directly synthesizing the image to be synthesized into the set target image according to the original scale, thereby coordinating the image to be synthesized in the composite image with the target image. BRIEF abstract

Figure 1 is an image synthesis diagram;

2 is a flowchart of a method for optimizing an image to be synthesized according to Embodiment 1 of the present invention; FIG. 3 is a structural diagram of an apparatus for optimizing an image to be synthesized according to Embodiment 2 of the present invention; Another structural diagram of an apparatus for optimizing a composite image to be provided;

FIG. 5 is a schematic diagram of a method for optimizing an image to be synthesized according to Embodiment 3 of the present invention; FIG. FIG. 6 is a flowchart of a method for optimizing a composite image according to Embodiment 3 of the present invention; FIG. 7 is a schematic diagram of another method for optimizing a composite image according to Embodiment 3 of the present invention;

FIG. 8 is a flowchart of applying the optimization method to a background of a video call switching according to Embodiment 4 of the present invention.

Preferred embodiment of the invention

The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings, and the features in the embodiments and the embodiments in the present application can be arbitrarily combined with each other without conflict.

Embodiment 1:

FIG. 2 is a flowchart of a method for optimizing an image to be synthesized according to an embodiment, where the optimization method includes the following steps:

Step 201: Determine the image to be synthesized and the position information of the image to be combined in the source image; the source image referred to herein may be a picture or a video, and the image to be synthesized is extracted from the source image and needs to be synthesized. image. The image to be composited may be any image in the source image, such as a character image, an animal image, etc., taking the character image as an example. Generally, the image of the person in the source image can be determined by the face recognition algorithm, and the image of the person can be extracted by other computer software (such as Photoshop, Meitu Xiuxiu), and the extraction of other images to be synthesized can also be realized by computer software. The position information of the image to be combined, that is, the position of the image to be synthesized in the source image, may be determined by determining the distance of the image to be combined with respect to each side of the source image, or by calibrating the image to be combined. The specific point is determined by the coordinates in the source image (usually the origin of the source image as the origin of the source image). Since each image information has pixels, we can use at least one pixel of the image to be composited. Coordinates to characterize location information. Preferably, it can be determined by determining the coordinates of each pixel of the image to be combined.

Step 202: Calculate a ratio of a source image size to a target image size;

For two-dimensional images, the size can be characterized by "length * width", so the size ratio can be the length ratio and the width ratio. For three-dimensional images, the size has a depth value in addition to the length and width, so the size ratio is the length ratio, the width ratio, and the depth ratio. The same, for more The dimension image, the ratio also includes the ratio of other dimension parameters.

Step 203: synthesize the image to be synthesized into the target image according to the obtained position information and the size ratio;

The target image may be any picture or video, and may also be a preset background. The position information of the image to be synthesized is generally data representing the relative relationship between the image to be synthesized and the source image, so the position information can be processed according to the obtained size ratio, so that the newly obtained position information is adapted to the ratio of the target image and the source image. While processing the position of the composite image, the size of the composite image is also processed according to the ratio, so that the size of the image to be combined is adapted to the target image.

Before the step 203 above, the method further includes the step of acquiring the color value, including at least two ways: Method 1: acquiring the color value in the source image as the color value displayed in the target image; the color value in the source image is to be synthesized The color value of each pixel of the image in the source image (the color value refers to a value that can represent the color information contained in the pixel, such as RGB value, luminance value, gray value, etc.).

Method 2: Directly set the color value displayed in the target image. The predetermined color value can be realized by setting the RGB value, the luminance value, the gradation value, and the like of the pixel in the target image. The predetermined color value may be related to a color value of each pixel in the image to be synthesized (for example, a transformation of the original color value), or may be a color value of each pixel in the source image of the image to be composited. Irrelevant, completely reset.

In step 201, when the position information of the image to be synthesized in the source image is the coordinate information of each pixel of the image to be combined, correspondingly, the position of the image to be synthesized in step 203 should also be made according to the size ratio. The coordinates of each pixel point adjusted are determined. In this embodiment, the coordinates of each pixel point are multiplied by the length ratio and the width ratio, respectively, to obtain the coordinate value of the corresponding pixel point in the target image. The color parameter of the corresponding pixel in the target image is set according to the obtained color value of the image to be synthesized displayed in the target image, thereby realizing the moving of the color value of the image to be synthesized from the source image to the target image. After each of the pixels of the composite image is subjected to the above processing, the synthesis of the image to be synthesized and the target image is completed.

In step 201, when the position information of the image to be synthesized in the source image is the coordinate information of each pixel on the boundary of the image to be combined, correspondingly, the position of the image to be synthesized in step 203 should also be This is determined by the coordinates of the pixel points on the boundary adjusted according to the size ratio. In this embodiment, the coordinates of each pixel on the boundary of the image to be combined are respectively multiplied by the length ratio and the width ratio to obtain the coordinate value of the corresponding pixel in the target image. In the target image, the newly determined coordinate values determine an area enclosed by boundary pixel points, which is the area to be synthesized. Setting the color parameter of the area to be combined according to the obtained color value of the image to be synthesized displayed in the target image (the color parameter referred to in the present invention refers to a quantization value of the color, a bit value, and the like, which can represent the color), thereby achieving Synthesis of a composite image and a target image.

In the process of synthesis, when the size of the source image is relatively large, and the size of the replacement background is relatively small, there will be a case where the image to be synthesized has more pixels in the source image than in the target image. To discard some pixels, or to merge some pixels; when the size of the source image is small, and the size of the target image is large, the pixel ratio of the image to be composited in the source image will appear. In the case where there are few pixels in the target image, the pixels are added at this time.

When the source image and the replacement target are video, the acquisition of the composite image information is performed at a certain frequency, and the processing of the target image and the image to be synthesized is also performed at the same frequency. The synthesis of the target image and the image to be synthesized changes correspondingly according to the information of the acquired image to be combined and the change of the position information.

Embodiment 2:

FIG. 3 is a structural diagram of an apparatus for optimizing an image to be synthesized according to the embodiment, where the apparatus includes a data acquisition module, a calculation module, and an image synthesis module.

The data acquisition module is configured to determine the image to be synthesized in the source image, the position information of the image to be synthesized in the source image, and store the acquired information. The source image referred to herein may be a picture or a video, and the image to be synthesized is extracted from the source image. The image to be composited may be any image in the source image, such as a character image, an animal image, etc., taking the character image as an example. Generally, the person image in the source image can be determined by the face recognition algorithm, and the character image can also be extracted by computer software (such as Photoshop, Meitu Xiuxiu), and the extraction of other images to be synthesized can also be realized by computer software. The position information of the image to be combined, that is, the position of the image to be synthesized in the source image, can be determined by determining the distance of the image to be combined with respect to each side of the source image. The position of the image can also be determined by calibrating the coordinates of several specific points in the image to be combined in the source image (usually the origin of the source image as the origin of the source image), since each image information has pixels. So we can characterize the position information with the coordinates of the pixels of the image to be composited. Preferably, it can be determined by determining the coordinates of each pixel of the image to be combined.

The calculation module is configured to determine a size ratio of the target image to the source image and save the size ratio. For two-dimensional images, the dimensions are characterized by "length * width", so the size ratio is the ratio of length to width. For 3D images, the size has a depth value in addition to the length and width, so the size ratio is the length ratio, the width ratio, and the depth ratio. Similarly, for multi-dimensional images, the ratio also includes the ratio of other size parameters.

The image synthesis module is configured to synthesize the image to be composite into the target image based on the position information and the size ratio. The target image can be any picture or video, and can also be a pre-set background. The position information of the image to be synthesized is generally data representing the relative relationship between the image to be synthesized and the source image, so the position information can be processed according to the obtained size ratio, so that the newly obtained position information is adapted to the ratio of the target image and the source image. While processing the position of the composite image, the size of the composite image is also processed according to the ratio, so that the size of the image to be combined is adapted to the target image.

The apparatus further includes a color acquisition module, as shown in Fig. 4, the color module is configured to acquire a color value displayed in the target image of the image to be composited before the image synthesis module synthesizes the image to be composited into the target image. The image synthesis module also includes a conversion sub-module and a color setting sub-module. The conversion sub-module is set to multiply the coordinate values of the length direction and the width direction of each pixel by the length ratio and the width ratio when the position information is the coordinate information of each pixel of the image to be combined, to obtain the corresponding pixel in the target image. The coordinate value of the point; the color setting sub-module is set to set the color parameter of the corresponding pixel in the target image according to the color value;

The conversion sub-module is further configured to multiply the coordinates of each pixel on the boundary by the length ratio and the broadband ratio when the position information is the coordinate information of each pixel on the boundary of the synthesized image, to obtain each pixel on the boundary. The coordinate value of the corresponding boundary pixel in the target image; in the target image, the newly determined coordinate value determines an area enclosed by the boundary pixel, the area is the area to be synthesized; the color setting sub-module is also set to The color parameters of each pixel in the area to be synthesized are set according to the acquired color values of the image to be synthesized displayed in the target image, thereby realizing the synthesis of the image to be combined and the target image. Embodiment 3:

FIG. 6 is a flowchart of a method for optimizing an image to be synthesized provided by the embodiment; the steps of the method are:

Step 401: determining a position of each pixel of the character image in the source image, represented by coordinates; Step 402: calculating a length ratio and a width ratio of the target image and the source image;

Step 403: multiplying the coordinate values of the length direction and the width direction of each pixel by the length ratio and the width ratio, respectively, to obtain coordinate values of corresponding pixel points in the target image;

Step 404: Set color parameters of corresponding pixel points in the target image according to the acquired color values of the pixels in the image to be synthesized.

For a better understanding of the present invention, the application scenario shown in FIG. 5 is taken as an example for explanation: Step 301: Determine the coordinates of each pixel in the image of the person in the source image 1, and only three points are marked here. A, B, C, the coordinates of other pixels are not marked here. The point A is coordinates (8, 9), the point B is (11, 5), and the point C is (8, 4); the character image 12 is the image of the person extracted from the source image 1.

Step 302: Calculate a size ratio of the target image to the source image, divide the length dimension of the target image 22 and the length dimension of the source image 1, and obtain a length ratio of M=10/16=5/8, and the width of the target image 22 The size is divided from the width dimension of the source image to obtain a width ratio of N=10/15=2/3.

Step 303: Calculate the coordinate value. Multiply the abscissas of the three points of B, C and M, multiply the ordinate by N, and round up to get the new coordinates, ie A, B, C, A, and the point coordinate value is (5,6). ), B' point coordinate value is (7,3), C' point coordinate value is (5,3).

Step 304: Combine the character image 12 with the target image 22 according to the obtained new coordinates. Specifically, the color parameter of the pixel at the A′ in the target image is set as the color value of the pixel at the point A in the source image, and the color parameter of the pixel at the position B′ in the target image is set as the source image. The color value of the pixel at the B point is processed in the pixel of the person image. The processed person image 12 is combined with the target image to obtain a composite image. The person image in the composite image is represented by the person image 13.

When the size of the source image is larger than the size of the target image, there may be a case where the pixel of the person image 11 is larger than the pixel of the person image 13. At this time, discarding or merging processing is performed; when the source image is used When the size of the target image is smaller than the size of the target image, there may be a case where the pixel of the person image 11 is smaller than the pixel of the person image 13, and the supplementary processing is performed at this time.

The target image and the source image in the above embodiment may also be multi-dimensional, and the three-dimensional example will be described here. When the target image and the source image are three-dimensional, when calculating the size ratio, the depth ratio of the target image and the source image is calculated. When you need to process each pixel, because the coordinates of its pixel have three parameters, such as (x, y, z), you must not only process x, y, but also the depth and depth of z and depth. Multiply the ratio to get a new ζ' to determine the new coordinates.

7 is another method for optimizing image synthesis, the method being different from the above method in that, when determining the position of the image to be combined in the source image, only the pixels on the boundary of the person image are determined in the source image 1. The coordinates of the point, here only three points D, E, F are marked, the coordinates of other pixels are not marked here. The coordinates of point D are (8,9), the point of E is (11,5), and the point of point F is (8,4). The coordinate values of the corresponding boundary pixel points of the respective pixel points on the boundary in the target image are obtained in the same manner as the above method, that is, D', E', F', D' point coordinate values are (5, 6), E 'The point coordinate value is (7,3), and the F' point coordinate value is (5,3). According to the obtained new coordinate value, an area to be combined is determined in the target image, that is, an area defined by a broken line of the character image 14 in the figure, and each of the areas to be combined is set according to the acquired color value or the preset color value. The color parameters of the pixels, thereby realizing the synthesis of the image, to obtain a composite image 4.

The above embodiment is an optimization of the image synthesis algorithm, and the following embodiment applies the optimization method to real life.

Embodiment 4:

FIG. 8 is a flow chart of applying the optimization method to a background of a video call switching according to an embodiment of the present disclosure. In this embodiment, the optimized image synthesis algorithm is applied to a video call of mobile communication, and the specific steps are as follows:

Step 701: Presetting a plurality of static solid color backgrounds in the mobile communication terminal, such as white, red, blue, etc., the user may select a preset solid color background, or may customize other static or dynamic background images, where the preset solid colors are preset. The background, the static background, and the dynamic background are the target images to be selected; Step 702: Shooting a background environment video by using the mobile communication terminal, the background environment video Is to replace the background, the background environment video is preset in the mobile communication terminal; and the segment of the environment video can also be other video, where the video selected by the mobile communication terminal is used to make the video synthesis effect better. The video here is also the target image to be selected;

Step 703: Set whether to enable the background filtering function before starting the video call or during the call missed state or video call. Before the start of the video call and during the video call, the call setting can be set; when the call is not answered, the call interface provides a shortcut switch for the background filter function;

Step 704: Before the background filtering function is enabled, before starting the video call and during the video call, the solid color background or the customized background video may be selected as the video background in the call setting menu; when the incoming call is not answered, the background filtering is started from the incoming call interface. After the function, the background selection box is popped up for the user to select; Step 705: The background filtering function is enabled and the video call is started after the background setting is completed; Step 706: After the video call is established, the camera starts to collect the local video data in real time, in the local video data stream. The image is the source image in the above embodiment;

Step 707: According to the local video data collected by the camera, after the portrait recognition module, the portrait image is extracted by the portrait recognition algorithm, and the actual background environment is filtered out.

Step 708: Combine the extracted character image with the preset background in step 704, and optimize the synthesized image by using the optimization method proposed in the present application in the process of synthesis;

Step 709: Pass the synthesized image information to the peer end through the wireless network;

If the background filtering function is enabled, the preset background is video. When the video call duration is greater than the preset background video, the background video is recycled until the call ends.

Step 710: If the background filtering function is not enabled, the video call is initiated according to the normal video call process, or the video call is received;

Step 711: The background filtering function is not enabled. During the video call, the local end passes the actual background environment to the peer.

It can be seen from the above embodiment that the optimization method combines the image to be synthesized in the source image, the position information of the image to be synthesized in the source image, and synthesizes the image to be synthesized according to the size ratio of the target image and the source image. In the target image, the image to be synthesized in the composite image is coordinated with the target image. Applying the method to the video call switching background can also make the video after switching the background more realistic, and realize real-time adjustment of the synthesized video. The above is a further detailed description of the present invention in connection with the specific embodiments, and the specific implementation of the invention is not limited to the description. It will be apparent to those skilled in the art that the present invention may be made without departing from the spirit and scope of the invention.

One of ordinary skill in the art will appreciate that all or a portion of the above steps may be accomplished by a program instructing the associated hardware, such as a read-only memory, a magnetic disk, or an optical disk. Alternatively, all or part of the steps of the above embodiments may also be implemented using one or more integrated circuits. Correspondingly, each module/unit in the above embodiment may be implemented in the form of hardware or in the form of a software function module. The invention is not limited to any specific form of combination of hardware and software.

Industrial applicability

The optimization method provided by the embodiment of the present invention adjusts the size of the image to be synthesized according to the size ratio of the target image and the source image, so that the size of the image to be synthesized in the composite image is coordinated with the target image ratio, so that the composite image is more realistic.

Claims

claims

1. An optimization method for image synthesis, including:

Determine the image to be synthesized in the source image and the position information of the image to be synthesized in the source image; determine the size ratio of the target image and the source image;

The image to be synthesized is synthesized into the target image according to the position information and the size ratio.

2. The optimization method of image synthesis according to claim 1, wherein the size ratio includes: a length ratio and a width ratio of the target image and the source image.

3. The optimization method of image synthesis according to claim 2, wherein the position information is the coordinate information of at least one pixel of the image to be synthesized.

4. The optimization method of image synthesis according to claim 3, wherein before synthesizing the image to be synthesized into the target image, it further includes obtaining the color of the image to be synthesized that is displayed in the target image. value;

When the position information is the coordinate information of each pixel of the image to be synthesized, synthesizing the image to be synthesized into the target image according to the position information and the size ratio includes: The coordinate value of each pixel in the length direction and width direction is multiplied by the length ratio and the width ratio respectively to obtain the coordinate value of the corresponding pixel in the target image; setting the target image according to the color value The color parameters of the corresponding pixels in

When the position information is the coordinate information of each pixel on the boundary of the image to be synthesized, synthesizing the image to be synthesized into the target image according to the position information and the size ratio includes: The coordinates of each pixel point on the boundary are multiplied by the length ratio and the broadband ratio respectively to obtain the coordinate value of the corresponding boundary pixel point of each pixel point on the boundary in the target image; the target image The area enclosed by the boundary pixels in is the area to be synthesized; and the color parameters of each pixel in the area to be synthesized are set according to the color value.

5. The optimization method of image synthesis according to any one of claims 1 to 4, wherein the source image and the target image are pictures or videos.

6. The optimization method of image synthesis according to any one of claims 1 to 4, wherein the target image is a preset background image.

7. The optimization method of image synthesis according to any one of claims 1 to 4, wherein the image to be synthesized is a human image.

8. A device for image synthesis and optimization, including:

a data acquisition module, which is configured to determine the image to be synthesized in the source image and the position information of the image to be synthesized in the source image;

a calculation module, which is configured to determine the size ratio of the target image and the source image; and an image synthesis module, which is configured to synthesize the image to be synthesized into the target image according to the position information and the size ratio.

9. The device for image synthesis optimization according to claim 8, wherein the size ratio includes: a length ratio and a width ratio of the target image and the source image.

10. The device for image synthesis optimization according to claim 9, wherein the position information is the coordinate information of at least one pixel of the image to be synthesized.

11. The device for image synthesis optimization according to claim 10, wherein the device further includes a color acquisition module configured to acquire the color of the image to be synthesized before the image synthesis module synthesizes the image to be synthesized into the target image. The color value of the image to be synthesized that is displayed in the target image;

The conversion sub-module is configured to: when the position information is the coordinate information of each pixel of the image to be synthesized, compare the coordinate value of each pixel in the length direction and width direction with the length ratio and the Multiply the width ratios to obtain the coordinate values of the corresponding pixels in the target image; the color setting sub-module is configured to set the color parameters of the corresponding pixels in the target image according to the color values;

The conversion sub-module is further configured to: when the position information is the coordinate information of each pixel point on the boundary of the image to be synthesized, compare the coordinate information of each pixel point on the boundary with the length ratio and the Multiply the broadband ratios to obtain the coordinate values of the corresponding boundary pixels in the target image for each pixel on the boundary; the area enclosed by the boundary pixels in the target image is the area to be synthesized; the color setting The stator module is further configured to: set color parameters of each pixel in the area to be synthesized according to the color value.