CN114332118A

CN114332118A - Image processing method, device, equipment and storage medium

Info

Publication number: CN114332118A
Application number: CN202111592671.8A
Authority: CN
Inventors: 张亚杰
Original assignee: Beijing Dajia Internet Information Technology Co Ltd
Current assignee: Beijing Dajia Internet Information Technology Co Ltd
Priority date: 2021-12-23
Filing date: 2021-12-23
Publication date: 2022-04-12

Abstract

The present disclosure relates to an image processing method, apparatus, device, and storage medium, which can reduce the amount of computation of processing an image by a semantic segmentation algorithm to be applied to a lightweight device such as a terminal. The specific scheme comprises the following steps: a first image is acquired. And if the key area image exists in the first image, segmenting the first image to obtain a second image, wherein the second image comprises the key area image, and the size of the second image is smaller than that of the first image. And obtaining a key area image according to the second image and a preset segmentation algorithm.

Description

Image processing method, device, equipment and storage medium

Technical Field

The present disclosure relates to the field of network technologies, and in particular, to an image processing method, an image processing apparatus, an image processing device, and a storage medium.

Background

With the development of network technology, various methods for processing resources such as images and videos have been developed. For example, the image may be segmented to obtain key region images in the image (e.g., face region images, hand region images, etc. in the image).

At present, the image can be processed by an image segmentation algorithm (e.g. a semantic segmentation algorithm) to obtain a key area image in the image. However, the semantic segmentation algorithm is obtained based on the deep learning neural network, which causes the requirement of the semantic segmentation algorithm on hardware equipment to be too high, and the semantic segmentation algorithm is difficult to be applied to light-weight equipment such as terminals (for example, mobile phones, tablet computers and notebook computers).

Disclosure of Invention

The present disclosure provides an image processing method, apparatus, device and storage medium, which can reduce the amount of computation of processing an image by a semantic segmentation algorithm to be applied to lightweight devices such as terminals. The technical scheme of the disclosure is as follows:

according to a first aspect of the present disclosure, there is provided an image processing method, the method comprising:

a first image is acquired. And if the key area image exists in the first image, segmenting the first image to obtain a second image, wherein the second image comprises the key area image, and the size of the second image is smaller than that of the first image. And obtaining a key area image according to the second image and a preset segmentation algorithm.

Optionally, the method for obtaining the second image by performing segmentation processing on the first image if it is detected that the key area image exists in the first image includes: and according to the size of the key area image and the position of the key area image in the first image, carrying out segmentation processing on the first image to obtain a second image.

Optionally, the image processing method further includes: if the key area image is not detected to exist in the first image, a third image is obtained according to the first image and a preset segmentation algorithm, wherein the third image is a mask image comprising the key area image, or the third image is a mask image not comprising the key area image.

Optionally, a target position is determined, where the target position is a position of the key area image in the first image. And rendering the key area image to obtain the rendered key area image. And splicing the rendered key area image to the first image according to the target position to obtain a fourth image, wherein the fourth image comprises the rendered key area image.

Optionally, the method for acquiring the first image includes: and acquiring a target video, and acquiring each frame of image from the target video as a first image.

According to a second aspect of the present disclosure, there is provided an image processing apparatus comprising: an acquisition unit and a processing unit.

An acquisition unit configured to perform acquisition of a first image. And the processing unit is configured to execute segmentation processing on the first image to obtain a second image if the key area image is detected to exist in the first image, wherein the second image comprises the key area image, and the size of the second image is smaller than that of the first image. And the processing unit is also configured to execute the second image and a preset segmentation algorithm to obtain a key area image.

Optionally, the processing unit is specifically configured to perform segmentation processing on the first image according to the size of the key area image and the position of the key area image in the first image, so as to obtain a second image.

Optionally, the processing unit is further configured to execute, if it is not detected that the key area image exists in the first image, obtaining a third image according to the first image and a preset segmentation algorithm, where the third image is a mask image including the key area image, or the third image is a mask image not including the key area image.

Optionally, the processing unit is further configured to determine a target position, where the target position is a position of the key area image in the first image. And the processing unit is also configured to perform rendering processing on the key area image to obtain a rendered key area image. And the processing unit is also configured to execute splicing the rendered key area image to the first image according to the target position to obtain a fourth image, wherein the fourth image comprises the rendered key area image.

Optionally, the obtaining unit is specifically configured to perform obtaining a target video, and obtain each frame of image from the target video as the first image.

According to a third aspect of the present disclosure, there is provided a terminal comprising:

a processor. A memory for storing processor-executable instructions. Wherein the processor is configured to execute the instructions to implement any of the above-described alternative image processing methods of the first aspect.

According to a fourth aspect of the present disclosure, there is provided a computer-readable storage medium having instructions stored thereon, which, when executed by a processor of a terminal, enable the terminal to perform any one of the above-mentioned first aspect optional image processing methods.

According to a fifth aspect of the present disclosure, there is provided a computer program product containing instructions which, when run on a computer, cause the computer to perform the optional image processing method of any of the first aspects.

According to a sixth aspect of the present disclosure, there is provided a chip comprising a processor and a communication interface, the communication interface being coupled to the processor, the processor being configured to execute a computer program or instructions to implement the image processing method as described in the first aspect and any one of the possible implementations of the first aspect.

The technical scheme provided by the disclosure at least brings the following beneficial effects: a first image is acquired, wherein the first image comprises a key area image. And then, segmenting the first image to obtain a second image, wherein the second image comprises a key area image, and the display area of the second image is smaller than that of the first image. Since the display area of the second image is smaller than the first image. Therefore, the amount of calculation for processing the image by the preset segmentation algorithm can be reduced. And then, obtaining a key area image according to the second image and a preset segmentation algorithm. Due to the fact that the calculated amount of the preset segmentation algorithm for processing the image is reduced, the requirement of the preset segmentation algorithm on hardware equipment can be reduced, and the preset segmentation algorithm is applied to light-weight equipment such as a terminal. Moreover, the image processing efficiency can be improved by reducing the calculation amount of the preset segmentation algorithm for processing the image.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure and are not to be construed as limiting the disclosure.

Fig. 1 is a schematic diagram illustrating the structure of a terminal according to an exemplary embodiment;

FIG. 2 is a flow diagram illustrating a method of image processing according to an exemplary embodiment;

FIG. 3 is a diagram illustrating an example of an image in accordance with an illustrative embodiment;

FIG. 4 is a schematic diagram illustrating an image positional relationship in accordance with an exemplary embodiment;

FIG. 5 is an example diagram illustrating a masking map in accordance with one illustrative embodiment;

FIG. 6 is a flow diagram illustrating another method of image processing according to an exemplary embodiment;

FIG. 7 is a flow diagram illustrating another method of image processing according to an exemplary embodiment;

FIG. 8 is a schematic diagram illustrating a configuration of an image processing apparatus according to an exemplary embodiment;

fig. 9 is a schematic configuration diagram illustrating another image processing apparatus according to an exemplary embodiment.

Detailed Description

In order to make the technical solutions of the present disclosure better understood by those of ordinary skill in the art, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the disclosure described herein are capable of operation in sequences other than those illustrated or otherwise described herein. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

It should be noted that the user information (including but not limited to user device information, user personal information, etc.) referred to in the present disclosure is information authorized by the user or sufficiently authorized by each party.

First, an application scenario of the embodiment of the present disclosure is described.

The image processing method of the embodiment of the disclosure is applied to a scene of processing an image. In the related art, in order to reduce a condition for using an image segmentation algorithm (e.g., a semantic segmentation algorithm) so that the semantic segmentation algorithm can be applied to a lightweight device, a size (i.e., resolution) of an image may be reduced when the terminal segments the image by the semantic segmentation algorithm. Then, the reduced image is segmented by an image segmentation algorithm. In this way, the amount of computation for segmenting the image by the semantic segmentation algorithm can be reduced. However, the occupation ratio of the key area image in the whole image is usually low, that is, the size of the key area image itself is already small, so the size of the key area image after the image is reduced becomes smaller, and the effect of extracting the key area image through the image segmentation algorithm is further affected.

In order to solve the above problem, an embodiment of the present disclosure provides an image processing method, which acquires an image to be processed (which may also be referred to as a first image) including a key area image. Then, an image (referred to as a second image) having a display area smaller than the first image and a key area is cut out from the first image. And then, processing the second image through a preset segmentation algorithm to obtain a key area image. Thus, the display area of the second image is smaller than that of the first image. Therefore, the calculation amount of processing the image through the preset segmentation algorithm can be reduced, the requirement of the semantic segmentation algorithm on hardware equipment is further reduced, and the semantic segmentation algorithm is applied to light-weight equipment such as terminals (for example, mobile phones, tablet computers and notebook computers). Moreover, the calculation amount of processing the image through the preset segmentation algorithm is reduced, and the efficiency of processing the image can be improved.

Fig. 1 is a schematic structural diagram of a terminal to which the method provided by the present disclosure is applied according to an embodiment of the present disclosure. The terminal 10 includes a processor 101 and a memory 102.

The processor 101 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor 101 may include an Application Processor (AP), a modem processor, a Graphics Processing Unit (GPU), an Image Signal Processor (ISP), a controller, a memory, a video codec, a Digital Signal Processor (DSP), a baseband processor, and/or a neural-Network Processing Unit (NPU), etc. The different processing units may be separate devices or may be integrated into one or more processors.

Memory 102 may include one or more computer-readable storage media, which may be non-transitory. Memory 102 may also include high speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in memory 102 is used to store at least one instruction for execution by processor 101 to implement the group communication method provided by the disclosed method embodiments.

In some embodiments, the terminal 10 may further include: a peripheral interface 103 and at least one peripheral. The processor 101, memory 102 and peripheral interface 103 may be connected by bus or signal lines. Each peripheral may be connected to peripheral interface 103 via a bus, signal line, or circuit board. Specifically, the peripheral device includes: at least one of radio frequency circuitry 104, display screen 105, camera assembly 106, audio circuitry 107, positioning assembly 108, and power supply 109.

The peripheral interface 103 may be used to connect at least one peripheral related to I/O (Input/Output) to the processor 101 and the memory 102. In some embodiments, processor 101, memory 102, and peripheral interface 103 are integrated on the same chip or circuit board; in some other embodiments, any one or both of the processor 101, the memory 102, and the peripheral interface 103 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

The Radio Frequency circuit 104 is used for receiving and transmitting RF (Radio Frequency) signals, also called electromagnetic signals. The radio frequency circuitry 104 communicates with communication networks and other communication devices via electromagnetic signals. The rf circuit 104 converts an electrical signal into an electromagnetic signal for transmission, or converts a received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 104 comprises: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a subscriber identity module card, and so forth. The radio frequency circuitry 104 may communicate with other servers via at least one wireless communication protocol. The wireless communication protocols include, but are not limited to: metropolitan area networks, various generations of mobile communication networks (2G, 3G, 4G, and 5G), Wireless local area networks, and/or Wi-Fi (Wireless Fidelity) networks. In some embodiments, the rf circuit 104 may further include NFC (Near Field Communication) related circuits, which are not limited by this disclosure.

The display screen 105 is used to display a UI (User Interface). The UI may include graphics, text, icons, video, and any combination thereof. When the display screen 105 is a touch display screen, the display screen 105 also has the ability to capture touch signals on or over the surface of the display screen 105. The touch signal may be input to the processor 101 as a control signal for processing. At this point, the display screen 105 may also be used to provide virtual buttons and/or a virtual keyboard, also referred to as soft buttons and/or a soft keyboard. In some embodiments, the display 105 may be one, providing the front panel of the terminal 10; the Display screen 105 may be made of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), and the like.

The camera assembly 106 is used to capture images or video. Optionally, the camera assembly 106 includes a front camera and a rear camera. Generally, the front camera is disposed on the front panel of the server, and the rear camera is disposed on the back of the server. Audio circuitry 107 may include a microphone and a speaker. The microphone is used for collecting sound waves of a user and the environment, converting the sound waves into electric signals, and inputting the electric signals to the processor 101 for processing or inputting the electric signals to the radio frequency circuit 104 to realize voice communication. The microphones may be provided in plural numbers, respectively, at different portions of the terminal 10 for the purpose of stereo sound collection or noise reduction. The microphone may also be an array microphone or an omni-directional pick-up microphone. The speaker is used to convert electrical signals from the processor 101 or the radio frequency circuit 104 into sound waves. The loudspeaker can be a traditional film loudspeaker or a piezoelectric ceramic loudspeaker. When the speaker is a piezoelectric ceramic speaker, the speaker can be used for purposes such as converting an electric signal into a sound wave audible to a human being, or converting an electric signal into a sound wave inaudible to a human being to measure a distance. In some embodiments, audio circuitry 107 may also include a headphone jack.

The positioning component 108 is used to locate the current geographic Location of the terminal 10 for navigation or LBS (Location Based Service). The Positioning component 108 may be a Positioning component based on the united states GPS (Global Positioning System), the chinese beidou System, the russian graves System, or the european union's galileo System.

The power supply 109 is used to supply power to the various components in the terminal 10. The power source 109 may be alternating current, direct current, disposable or rechargeable. When power source 109 comprises a rechargeable battery, the rechargeable battery may support wired or wireless charging. The rechargeable battery may also be used to support fast charge technology.

In some embodiments, the terminal 10 also includes one or more sensors 1010. The one or more sensors 1010 include, but are not limited to: acceleration sensors, gyroscope sensors, pressure sensors, fingerprint sensors, optical sensors, and proximity sensors.

The acceleration sensor can detect the magnitude of acceleration in three coordinate axes of a coordinate system established with the terminal 10. The gyro sensor can detect the body direction and the rotation angle of the terminal 10, and the gyro sensor can cooperate with the acceleration sensor to acquire the 3D motion of the user to the terminal 10. The pressure sensors may be disposed on the side frames of the terminal 10 and/or underlying the display screen 105. When the pressure sensor is provided at the side frame of the terminal 10, a user's holding signal of the terminal 10 can be detected. The fingerprint sensor is used for collecting fingerprints of users. The optical sensor is used for collecting the intensity of ambient light. A proximity sensor, also called a distance sensor, is generally provided at the front panel of the terminal 10. The proximity sensor is used to collect the distance between the user and the front surface of the terminal 10.

An executing body of an image processing method provided by the present disclosure may be an image processing apparatus, and the executing apparatus may be a terminal shown in fig. 1. Meanwhile, the execution device may also be a Central Processing Unit (CPU) of the terminal, or a control module for Processing an image in the terminal. In the embodiment of the present application, a terminal executes an image processing method as an example, and the image processing method provided in the embodiment of the present application is described.

Optionally, the terminal may also be a server. That is, the server may also serve as the execution subject of the embodiments of the present disclosure.

In one implementable manner, the terminal is used to provide voice and/or data connectivity services to the user. A terminal may be referred to by different names, such as UE side, terminal unit, terminal station, mobile station, remote terminal, mobile device, wireless communication device, vehicular user equipment, terminal agent, or terminal equipment.

Optionally, the terminal may be various handheld devices, vehicle-mounted devices, wearable devices, and computers with communication functions, which is not limited in this disclosure. For example, the handheld device may be a smartphone. The in-vehicle device may be an in-vehicle navigation system. The wearable device may be a smart bracelet. The computer may be a Personal Digital Assistant (PDA) computer, a tablet computer, and a laptop computer.

After the application scenario and the implementation environment of the embodiment of the present disclosure are introduced, the following describes in detail the image processing method provided by the embodiment of the present disclosure with reference to the implementation environment shown in fig. 1.

FIG. 2 is a flow diagram illustrating an image processing method according to an exemplary embodiment. As shown in fig. 2, the method may include steps 201-203.

201. A first image is acquired.

As a possible implementation, a first image stored in the terminal is acquired.

Illustratively, a plurality of images are stored in the terminal. And responding to the selection operation of the plurality of images, and acquiring a first image, wherein the first image is any one of the plurality of images stored in the terminal.

202. And carrying out segmentation processing on the first image to obtain a second image.

Wherein the second image comprises a key area image.

It should be noted that the key area image is not limited in the embodiments of the present disclosure. For example, the key area image is a hand area image. For another example, the key area image is a face area image. For another example, the key area image is a sky area image.

In the disclosed embodiments, the size of the second image is smaller than the size of the first image.

The size of the second image is smaller than the size of the first image, and may be referred to as the display area of the second image being smaller than the display area of the first image. The display area of the second image being smaller than the display area of the first image means that the display resolution of the second image is smaller than the display resolution of the first image. For example, if the resolution of the first image is 6000 × 3000, the resolution of the second image may be 2000 × 1000, or the resolution of the second image may be 2000 × 500, or the resolution of the second image may be 300 × 500, which is not limited in the embodiment of the present disclosure.

In the embodiment of the present disclosure, the second image is a partial area image of the first image.

Illustratively, as shown in fig. 3, the first image 301 includes a key region image 302 and a first region image 303, and the resolution of the first image is 6000 × 2500. The second image 304 is a partial area image of the first image 301, the second image 304 includes a key area image 302, and the resolution of the second image is 1000 × 300.

As a possible implementation manner, the first image is segmented according to the size of the key area image and the position of the key area image in the first image, so as to obtain a second image. Specifically, the position of the key area image in the first image is determined through a key area detection algorithm. And then, according to the position of the key area image in the first image and the size of the key area image, carrying out segmentation processing on the first image to obtain a matrix-shaped second image.

Illustratively, the coordinate position of each pixel point in the key area image at the first image is obtained. And then, determining a first image according to the coordinates of the edge pixel points in the key area image. For example, the origin of the coordinate system of the first image may be any corner (e.g., upper left corner or lower left corner) in the first image, and the x-axis and the y-axis are two adjacent edges. As shown in FIG. 4, point o is the origin of coordinates, the x-axis is the lower side of the first image 401, and the y-axis is the left side of the first image 401. The two-dimensional coordinates of the leftmost pixel point a1 in the key area image 402 are a1(x1, y1), the two-dimensional coordinates of the uppermost pixel point a2 in the key area image 402 are a2(x2, y2), the two-dimensional coordinates of the rightmost pixel point A3 in the key area image 402 are A3(x3, y3), and the two-dimensional coordinates of the lowermost pixel point a4 in the key area image 402 are a4(x4, y 4). Then, according to the two-dimensional coordinates of the pixel point a1, the pixel point a2, the pixel point A3, and the pixel point a4, any two opposite corners of the second image (e.g., the upper left corner a5 and the lower right corner a6) are determined. Wherein the two-dimensional coordinates of A5 are A5(x5, y5), and the two-dimensional coordinates of A6 are A6(x6, y 6). Wherein x5 is less than or equal to x1, y5 is greater than or equal to y2, x6 is greater than or equal to x3, and y6 is less than or equal to y 4. The size of the key area image is the number of pixel points in the key area image.

The technical scheme provided by the embodiment at least has the following beneficial effects: and according to the size of the key area image and the position of the key area image in the first image, carrying out segmentation processing on the first image to obtain a second image. Therefore, the first image can be segmented more accurately to obtain the second image with smaller size, and the image processing calculation amount is further reduced.

203. And obtaining a key area image according to the second image and a preset segmentation algorithm.

Optionally, the preset segmentation algorithm is obtained based on a deep learning neural network.

It should be noted that, the preset segmentation algorithm is not limited in the embodiment of the present disclosure. For example, the preset segmentation algorithm may be a semantic segmentation algorithm (unet network). For another example, the preset segmentation algorithm may be deplabv 3. As another example, the preset segmentation algorithm may be hrnet.

As a possible implementation manner, the second image is processed through a preset segmentation algorithm, and the key area image is segmented from the second image. Specifically, each pixel point in the second image can be extracted through a preset segmentation algorithm, and whether each pixel point is a pixel point in the key area image is identified. And then, determining all pixel points in the key area image to obtain the key area image.

Optionally, a mask map including the key region image is obtained according to the second image and a preset segmentation algorithm. Illustratively, as shown in fig. 5, the mask map 501 includes a key area image 502, wherein the images of the areas of the mask map 501 except the key area image 502 are all black.

It should be noted that, specifically, for the way of processing the second image by using the preset segmentation algorithm, reference may be made to a method for processing an image by using a semantic segmentation algorithm in the conventional technology, which is not described in detail in the embodiments of the present disclosure.

As another possible implementation manner, the second image is input into a segmentation model constructed based on a preset segmentation algorithm to obtain a key region image.

The technical scheme provided by the embodiment at least has the following beneficial effects: a first image is acquired. And then, segmenting the first image to obtain a second image, wherein the second image comprises a key area image, and the size of the second image is smaller than that of the first image. Since the second image is smaller in size than the first image. Therefore, the amount of calculation for processing the image by the preset segmentation algorithm can be reduced. And then, obtaining a key area image according to the second image and a preset segmentation algorithm. Due to the fact that the calculated amount of the preset segmentation algorithm for processing the image is reduced, the requirement of the preset segmentation algorithm on hardware equipment can be reduced, and the preset segmentation algorithm is applied to light-weight equipment such as a terminal. Moreover, the image processing efficiency can be improved by reducing the calculation amount of the preset segmentation algorithm for processing the image.

In a practical manner, as shown in fig. 6, after step 201, the image processing method further includes steps 601 to 602.

601. Whether a key area image exists in the first image is detected.

As a possible implementation manner, whether the key area image exists in the first image is detected according to a preset detection algorithm.

It should be noted that, the preset detection algorithm is not limited in the embodiment of the present disclosure. For example, the preset detection algorithm is yolov5 algorithm. As another example, the predetermined detection algorithm is the fastercnnn algorithm.

Illustratively, it is detected whether an image of a hand region is present in the first image.

In one implementation, if it is detected that the key area image exists in the first image, step 202 is executed.

It is understood that whether the key area image exists in the first image may be determined by detecting whether the key area image exists in the first image. And then, under the condition that the key area image exists in the first image, the first image is subjected to segmentation processing to obtain a second image. Thus, the calculation amount of the preset segmentation algorithm for processing the image can be reduced.

In one implementation, if it is detected that the key area image exists in the first image, step 202 is executed. If it is not detected that the key area image exists in the first image, step 602 is executed.

602. And obtaining a third image according to the first image and a preset segmentation algorithm.

It should be noted that, when detecting whether the key area image exists in the first image through the preset detection algorithm, the key area image may exist in the first image, but the key area image does not exist in the first image.

In one possible design, the third image is a mask including the key region image. And processing the first image by a preset segmentation algorithm under the condition that the key area image is not detected to exist in the first image but the first image comprises the key area image to obtain a mask image comprising the key area image.

It should be noted that, when the first image is processed by the preset segmentation algorithm, the resolution (i.e., the display area) of the obtained mask map is greater than the resolution of the mask map obtained when the second image is processed by the preset segmentation algorithm. That is, with reference to fig. 5, the first image is processed by the preset segmentation algorithm, and the resolution of the obtained mask map is greater than that of the mask map 501.

In another possible design, the third image is a mask map that does not include the key region image. And under the condition that the key area image does not exist in the first image and the key area image does not actually exist in the first image, processing the first image through a preset segmentation algorithm to obtain a mask image without the key area image.

Illustratively, when the key area image is not detected to exist in the first image and the key area image does not actually exist in the first image, the first image is processed by a preset segmentation algorithm to obtain a mask image without the key area image.

The technical scheme provided by the embodiment at least has the following beneficial effects: and if the key area image is not detected to exist in the first image, obtaining a third image according to the first image and a preset segmentation algorithm. It can be understood that, in order to avoid the situation that the key area image exists in the first image but is not detected, in the situation that the key area image does not exist in the first image, the third image is obtained according to the first image and the preset segmentation algorithm. In this way, the key area image can be extracted again. And if the key area image exists in the first image, obtaining a mask image comprising the key area image. And if the key area image does not exist in the first image, obtaining a mask image without the key area image.

In the embodiment of the present disclosure, the first calculation amount is smaller than the second calculation amount, and the first calculation amount is: the sum of the calculated amount required for carrying out segmentation processing on the first image and the calculated amount required for carrying out processing on the second image through a preset segmentation algorithm; the second calculated amount is: the amount of computation required to process the first image by a preset segmentation algorithm.

Illustratively, the amount of computation required to perform the segmentation processing on the first image is 100, and the amount of computation required to perform the processing on the second image by the preset segmentation algorithm is 50, then the first amount of computation is 150. The amount of calculation required to process the first image by the preset segmentation algorithm is 300.

It can be understood that, because the calculation amount of the preset segmentation algorithm for processing the image is reduced, the requirement of the preset segmentation algorithm on hardware equipment can be reduced, and the preset segmentation algorithm is applied to light-weight equipment such as a terminal. Moreover, the image processing efficiency can be improved by reducing the calculation amount of the preset segmentation algorithm for processing the image.

In a practical manner, as shown in fig. 7, the image processing method further includes steps 701 to 703.

701. The target location is determined.

And the target position is the position of the key area image in the first image.

As a possible implementation manner, the target position is determined according to the position of the pixel point of the key area image in the first image. Illustratively, the target position may be represented by two-dimensional coordinates of the key area image in the first image (see fig. 4).

702. And rendering the key area image to obtain the rendered key area image.

It should be noted that the rendering process is not limited in the embodiment of the present disclosure. For example, if the key area image is a hand area image, the rendering process may add a glove image to the hand area image. For another example, if the key region map is a sky region image, the rendering process may add a special effect to the sky region image (e.g., change a blue sky to a sunset sky). For another example, if the key area image is a face area image, the rendering process may add a mask image to the face area image.

703. And splicing the rendered key area image to the first image according to the target position to obtain a fourth image.

And the fourth image comprises a rendered key area image.

In the embodiment of the present disclosure, the display area of the fourth image is the same as the display area of the first image.

The technical scheme provided by the embodiment at least has the following beneficial effects: and after the target position is determined, rendering the key area image to obtain a rendered key area image. And splicing the rendered key area image to the first image according to the target position to obtain a fourth image, wherein the fourth image comprises the rendered key area image. Due to the fact that the calculated amount of the preset segmentation algorithm for processing the image is reduced, the time for obtaining the key area image can be reduced. Therefore, the speed of obtaining the fourth image including the rendered key area image can be further improved, and the image rendering efficiency is improved.

In one practical manner, acquiring the first image, i.e. step 201, may include: and acquiring a target video, and acquiring each frame of image from the target video as a first image.

That is, after the target video is acquired, the method of the embodiment of the disclosure (e.g., step 201-step 203) may be performed on each frame of image in the target video.

Optionally, after obtaining the key area image of each frame of image in the target video, the key area image in each frame of image may be rendered, so as to obtain a rendered key area image corresponding to each frame of image. And then, splicing the rendered key area image corresponding to each frame of image to the corresponding image.

Illustratively, suppose the target video includes: image frame a, image frame B, and image frame C. Steps 201 to 203 may be performed for the image frame a, the image frame B and the image frame C, respectively, to obtain a key area image a of the image frame a, a key area image B of the image frame B and a key area image C of the image frame C. And then, rendering the key area image A, the key area image B and the key area image C respectively to obtain a rendered key area image A, a rendered key area image B and a rendered key area image C. And then splicing the rendered key area image A to an image frame A, splicing the rendered key area image B to an image frame B, and splicing the rendered key area image C to an image frame C.

The technical scheme provided by the embodiment at least has the following beneficial effects: and acquiring a target video, wherein the first image is each frame of image in the target video. That is, in the process of processing the target video by the preset segmentation algorithm, the display area of each frame of image in the target video may be reduced, and then the image may be processed by the preset segmentation algorithm. Therefore, the calculation amount of the preset segmentation algorithm for processing the video can be reduced, the requirement of the preset segmentation algorithm on hardware equipment is lowered, and the preset segmentation algorithm is applied to light-weight equipment such as a terminal.

It is to be understood that the above method may be implemented by an image processing apparatus. The image processing apparatus includes a hardware configuration and/or a software module for performing each function in order to realize the above functions. Those of skill in the art will readily appreciate that the various illustrative modules and algorithm steps described in connection with the embodiments disclosed herein may be implemented as hardware or combinations of hardware and computer software. Whether a function is performed as hardware or computer software drives hardware depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the disclosed embodiments.

The image processing apparatus and the like may be divided into functional modules according to the method example, for example, each functional module may be divided according to each function, or two or more functions may be integrated into one processing module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. It should be noted that, the division of the modules in the embodiments of the present disclosure is illustrative, and is only one division of logic functions, and there may be another division in actual implementation.

Fig. 8 is a block diagram illustrating a configuration of an image processing apparatus according to an exemplary embodiment. Referring to fig. 8, the image processing apparatus 80 includes an acquisition unit 81 and a processing unit 82.

An acquisition unit 81 configured to perform acquisition of a first image. And the processing unit 82 is configured to execute segmentation processing on the first image to obtain a second image if the key area image is detected to exist in the first image, wherein the second image comprises the key area image, and the size of the second image is smaller than that of the first image. And the processing unit 82 is further configured to execute the second image and a preset segmentation algorithm to obtain a key area image.

Optionally, the processing unit 82 is specifically configured to perform segmentation processing on the first image according to the size of the key area image and the position of the key area image in the first image, so as to obtain a second image.

Optionally, the processing unit 82 is further configured to execute, if it is not detected that the key area image exists in the first image, obtaining a third image according to the first image and a preset segmentation algorithm, where the third image is a mask image including the key area image, or the third image is a mask image not including the key area image.

Optionally, the processing unit 82 is further configured to determine a target position, where the target position is a position of the key area image in the first image. The processing unit 82 is further configured to perform rendering processing on the key area image, so as to obtain a rendered key area image. The processing unit 82 is further configured to perform stitching of the rendered key area image to the first image according to the target position, so as to obtain a fourth image, where the fourth image includes the rendered key area image.

Optionally, the obtaining unit 81 is specifically configured to perform obtaining a target video, and obtain each frame of image from the target video as the first image.

With regard to the image processing apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Fig. 9 is a schematic structural diagram of an image processing apparatus 90 provided by the present disclosure. As shown in fig. 9, the image processing apparatus 90 may include at least one processor 901 and a memory 903 for storing instructions executable by the processor 901. Wherein the processor 901 is configured to execute instructions in the memory 903 to implement the image processing method in the above-described embodiments.

In addition, the image processing apparatus 90 may further include a communication bus 902 and at least one communication interface 904.

Processor 901 may be a GPU, a microprocessor unit, an ASIC, or one or more integrated circuits for controlling the execution of programs in accordance with the disclosed aspects.

Communication bus 902 may include a path that transfers information between the aforementioned components.

Communication interface 904 may be implemented using any device, such as a transceiver, for communicating with other devices or communication networks, such as an ethernet, a Radio Access Network (RAN), a Wireless Local Area Network (WLAN), etc.

The memory 903 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a Random Access Memory (RAM) or other type of dynamic storage device that can store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disk storage, optical disk storage (including compact disc, laser disc, optical disc, digital versatile disc, blu-ray disc, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to these. The memory may be self-contained and connected to the processing unit by a bus. The memory may also be integrated with the processing unit as a volatile storage medium in the GPU.

The memory 903 is used for storing instructions for executing the disclosed solution, and is controlled by the processor 901 to execute. The processor 901 is configured to execute instructions stored in the memory 903, thereby implementing functions in the disclosed methods.

In particular implementations, processor 901 may include one or more GPUs, such as GPU0 and GPU1 in fig. 9, as one embodiment.

In particular implementations, image processing device 90 may include multiple processors, such as processor 901 and processor 907 in fig. 9, as one embodiment. Each of these processors may be a single-Core (CPU) processor or a multi-core (multi-GPU) processor. A processor herein may refer to one or more devices, circuits, and/or processing cores for processing data (e.g., computer program instructions).

In particular implementations, image processing apparatus 90 may also include an output device 905 and an input device 906, as one embodiment. An output device 905, which is in communication with the processor 901, may display information in a variety of ways. For example, the output device 905 may be a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display device, a Cathode Ray Tube (CRT) display device, a projector (projector), or the like. The input device 906, which is in communication with the processor 901, may accept input from a user in a variety of ways. For example, the input device 906 may be a mouse, keyboard, touch screen device, or sensing device, among others.

Those skilled in the art will appreciate that the configuration shown in fig. 9 does not constitute a limitation of the image processing apparatus 90, and may include more or fewer components than those shown, or combine certain components, or employ a different arrangement of components.

The present disclosure also provides a computer-readable storage medium having instructions stored thereon, which, when executed by a processor of a server, enable the server to perform the group communication method provided by the embodiments of the present disclosure.

The embodiment of the present disclosure also provides a computer program product containing instructions, which when run on a server, causes the server to execute the image processing method provided by the embodiment of the present disclosure.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An image processing method, characterized in that the method comprises:

acquiring a first image;

if the fact that the first image has the key area image is detected, segmenting the first image to obtain a second image, wherein the second image comprises the key area image, and the size of the second image is smaller than that of the first image;

and obtaining the key area image according to the second image and a preset segmentation algorithm.

2. The method according to claim 1, wherein if it is detected that the first image has the key area image, performing segmentation processing on the first image to obtain a second image, comprises:

and according to the size of the key area image and the position of the key area image in the first image, carrying out segmentation processing on the first image to obtain the second image.

3. The method of claim 1, further comprising:

if the key area image is not detected to exist in the first image, obtaining a third image according to the first image and the preset segmentation algorithm, wherein the third image is a mask image including the key area image, or the third image is a mask image not including the key area image.

4. The method according to any one of claims 1-3, further comprising:

determining a target position, wherein the target position is the position of the key area image in the first image;

rendering the key area image to obtain the rendered key area image;

and splicing the rendered key area image to the first image according to the target position to obtain a fourth image, wherein the fourth image comprises the rendered key area image.

5. The method of claim 4, wherein said acquiring a first image comprises:

and acquiring a target video, and acquiring each frame of image from the target video as the first image.

6. An image processing apparatus characterized by comprising:

an acquisition unit configured to perform acquisition of a first image;

the processing unit is configured to execute segmentation processing on the first image to obtain a second image if the first image is detected to have a key area image, wherein the second image comprises the key area image, and the size of the second image is smaller than that of the first image;

and the processing unit is also configured to execute the second image and a preset segmentation algorithm to obtain the key area image.

7. The image processing apparatus according to claim 6,

the processing unit is specifically configured to perform segmentation processing on the first image according to the size of the key area image and the position of the key area image in the first image, so as to obtain the second image.

8. A terminal, characterized in that the terminal comprises:

a processor;

a memory for storing the processor-executable instructions;

wherein the processor is configured to execute the instructions to implement the image processing method of any one of claims 1-5.

9. A computer-readable storage medium having instructions stored thereon, wherein the instructions in the computer-readable storage medium, when executed by a processor of a terminal, enable the terminal to perform the image processing method according to any one of claims 1 to 5.

10. A computer program product comprising computer programs/instructions, characterized in that the computer programs/instructions, when executed by a processor, implement the image processing method of any of claims 1-5.