WO2021136224A1

WO2021136224A1 - Image segmentation method and device

Info

Publication number: WO2021136224A1
Application number: PCT/CN2020/140570
Authority: WO
Inventors: 赵光耀
Original assignee: 华为技术有限公司
Priority date: 2019-12-31
Filing date: 2020-12-29
Publication date: 2021-07-08
Also published as: CN113129307A

Abstract

An image segmentation method and device (800, 900, 1000). The image segmentation method is used for a terminal device having a display screen, and comprises: detecting a first operation that a user manually marks anchor points in a first image, wherein the anchor points comprise a starting anchor point and a target anchor point (210); detecting a second operation that the user instructs automatic segmentation of the first image (220); and in response to the second operation, displaying a second image on the display screen, wherein the second image is an image obtained by performing the second operation on the first image and comprises a segmentation line between the starting anchor point and the target anchor point, the segmentation line being obtained by moving a demarcation point in the first image with the starting anchor point as a starting position and the target anchor point as a target position (230). The described technical solution can obtain a segmentation result matching the natural boundary of the image while saving manpower, thereby improving the accuracy of the image segmentation result.

Description

Image segmentation method and image segmentation device

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office, the application number is 201911411574.7, and the application name is "Image Segmentation Method and Image Segmentation Device" on December 31, 2019, the entire content of which is incorporated into this application by reference .

Technical field

This application relates to the field of artificial intelligence, and more specifically, to an image segmentation method and an image segmentation device in the computer vision field.

Background technique

Computer vision is an inseparable part of various intelligent/autonomous systems in various application fields, such as manufacturing, inspection, document analysis, medical diagnosis, and military. It is about how to use cameras/video cameras and computers to obtain What we need is the knowledge of the data and information of the subject. To put it vividly, it is to install eyes (camera/camcorder) and brain (algorithm) on the computer to replace the human eye to identify, track and measure the target, so that the computer can perceive the environment. Precise segmentation tasks have a wide range of applications in the field of computer vision. For example, image blurring, background replacement, e-commerce advertising production and live broadcast, movie (animation) production, etc., where precise segmentation can refer to the segmentation line between the target object and the background on the acquired image or video.

At present, the polygon method or curve fitting method is usually used for accurate image segmentation; among them, the difference between the image segmentation line obtained by the polygon method and the actual image segmentation line is large and the accuracy is low; the curve fitting method requires manual selection Multiple demarcation points result in the need to consume a lot of manpower.

Therefore, in the case of saving manpower, how to improve the accuracy of the precise segmentation result of the image becomes an urgent problem to be solved.

Summary of the invention

The present application provides an image segmentation method and an image segmentation device, which can obtain a segmentation result that matches the natural boundary of the image while saving manpower, thereby improving the accuracy of the image segmentation result.

In a first aspect, an image segmentation method, applied to a terminal device with a display screen, includes: detecting a first operation of manually marking an anchor point in a first image by a user, wherein the anchor point includes a start anchor point and Target anchor point; detecting the second operation instructed by the user to automatically segment the first image; in response to the second operation, displaying a second image on the display screen, wherein the second image is all The first image is an image obtained after the second operation, and the second image includes a dividing line between the start anchor point and the target anchor point, and the dividing line passes through the dividing point in the The first image is obtained by moving the start anchor point as the start position and the target anchor point as the target position.

Wherein, the above-mentioned dividing line may be obtained by automatically moving the dividing point in the first image with the starting anchor point as the starting position and the target anchor point as the target position; wherein, automatic moving may refer to The user only needs to manually mark the start anchor point and the target anchor point in the first image, or manually mark a small number of marked points including the start anchor point and the target anchor point, and the start anchor point and the target point in the first image can be automatically obtained. The dividing line between anchor points.

Wherein, the dividing point may refer to a point on the dividing line between the target area and the background area in the image, where the target area may refer to the area including the target object.

The above-mentioned first image may refer to an image that has an image segmentation requirement; the second image may refer to an image obtained after automatic segmentation processing is performed on the first image.

It should be understood that the starting anchor point and the target anchor point may be positions in the image to be processed where the position does not change. By moving the dividing point from the starting anchor point to the target anchor point, it can be obtained that the first image is at the starting anchor point and the target anchor point. A dividing line between the anchor points, which coincides with the natural boundary of different objects in the first image.

In the embodiment of the present application, a small number of anchor points including the starting anchor point and the target anchor point are manually marked on the first image, and the starting anchor point is used as the starting point in the pixel gradient map of the first image through the dividing point. Position and automatically move the target anchor point as the target position to obtain the segmentation line of the first image, which can obtain the segmentation result consistent with the natural boundary of the image while saving manpower, thereby improving the accuracy of the image segmentation result.

In a possible implementation manner, the above-mentioned second operation used by the user to instruct the automatic segmentation may include the user clicking the button for automatically segmenting the image in the image processing tool, or may include the behavior of the user instructing the automatic segmentation through voice, or, It can include other actions of the user to instruct automatic segmentation.

With reference to the first aspect, in some implementations of the first aspect, the dividing line coincides with the dividing line in the ridge direction of the pixel gradient map of the first image.

It should be noted that the ridge direction in the pixel gradient map of the first image may refer to a curve formed by the local maximum value of the gradient in the pixel gradient map.

In the embodiment of the present application, the obtained dividing line of the first image may coincide with the dividing line of the ridge direction in the pixel gradient map of the first image, that is, the dividing line of the first image may be based on the pixel gradient of the first image. The image is obtained, so as to avoid the labor-consuming problem caused by manually marking a large number of anchor points in the first image manually; by manually marking a small number of anchor points, the accuracy of the image segmentation result can be improved while saving manpower .

With reference to the first aspect, in some implementations of the first aspect, the displaying the second image on the display screen in response to the second operation includes: searching for the The mask image and the secant image corresponding to the first image, wherein the mask image is used to represent different objects in the first image, and the secant image is used to represent different objects in the first image. Boundary; displaying the superimposed image of the first image, the mask image, and the secant image on the display screen.

In a possible implementation manner, the mask image and/or the secant image corresponding to the first image may be searched under the preset file path according to the first image.

Exemplarily, each image file can be saved as a bitmap file (bitmap, BMP), a losslessly compressed bitmap graphics format (portable network graphics, PNG), or other image file formats; image files, mask files, and secant files The corresponding relationship can be described by icon or file name naming rules, or by packaging three images in a file or a folder.

With reference to the first aspect, in some implementations of the first aspect, the method further includes: detecting that the user instructs to process the second image through the mask image or the secant image.

In a possible implementation manner, the second image may refer to the superimposed image of the first image, the mask image, and the secant image displayed on the display screen, and the third operation may be used to instruct to pass the mask image or cut the image. The line image performs automatic segmentation processing on the superimposed image.

In the embodiment of the present application, the user can choose to perform automatic segmentation processing on the first image according to the mask image or the secant image. For example, the user can manually adjust the position of the anchor point according to the mask image or the secant image.

With reference to the first aspect, in some implementations of the first aspect, the second image is obtained according to a pixel gradient map of the first image and an anchor point pulling model, and the anchor point pulling model is used to indicate State the direction of movement of the dividing point.

In the embodiment of the present application, the second image may be obtained according to the pixel gradient map of the first image and the anchor point pulling model. The anchor point pulling model may indicate that the dividing point automatically moves from the starting anchor point to the target anchor point every time. One-step movement direction, thereby reducing manual participation.

With reference to the first aspect, in some implementations of the first aspect, when the boundary point is currently located in an area without ridges in the pixel gradient map, the moving direction of the boundary point is based on the first line and The angle between the second line is determined, wherein the first straight line refers to the straight line where each of the eight directions is located, and the second straight line refers to the current position of the dividing point and the The line between the target anchor points; or

When the dividing point is currently located in an area with ridges and bifurcations in the pixel gradient map, the moving direction of the dividing point is based on the angle between the first line and the second line and the distance parameter Determined, wherein the first straight line refers to a straight line where each of the eight azimuths is located, and the second straight line refers to a line between the current position of the dividing point and the target anchor point, The distance parameter refers to the distance between the current position of the boundary point and the target anchor point; or

When the boundary point is currently located in the pixel gradient map with a ridge and no bifurcation area, the moving direction of the boundary point is determined according to the ridge direction of the pixel gradient map.

With reference to the first aspect, in some implementations of the first aspect, when the moving direction of the dividing point at the previous moment is the same as the moving direction of the dividing point at the current moment, the moving direction of the dividing point is Determined according to the absolute value of the gradient in different directions.

In the embodiment of the present application, when the movement direction of the demarcation point at the previous moment is the same as the movement direction of the current moment, the demarcation point can be moved in a preset direction, that is, the movement direction of the demarcation point can be based on different The absolute value of the gradient of the moving direction is determined, so that the error of the dividing point when moving along the ridge direction of the gradient map can be compensated.

With reference to the first aspect, in some implementations of the first aspect, the anchor point is obtained by optimizing an initial anchor point, where the initial anchor point is an anchor point manually marked by the user in the first image .

In the embodiment of the present application, since the anchor points in the image to be processed are manually marked by the user in the image to be processed, the position of the anchor point may have a certain deviation; in order to make the position of the anchor point more accurate, it improves The accuracy of the segmentation line of the image to be processed can optimize the position of the anchor points manually marked by the user.

In a second aspect, an image segmentation method is provided, including: acquiring a first image and position information of an anchor point in the first image, wherein the anchor point includes a start anchor point and a target anchor point; An image and the anchor point to obtain a second image, where the second image is an image obtained after the first image is subjected to image segmentation processing, and the second image includes the starting anchor point and the A dividing line between target anchor points, the dividing line is to move through the dividing point in the pixel gradient map of the first image with the starting anchor point as the starting position and the target anchor point as the target position owned.

In a possible implementation manner, the first image, the pixel gradient map of the first image, and the position information of the anchor points in the first image may be obtained, and the anchor points may include a start anchor point and a target anchor point.

Wherein, the above-mentioned dividing line may be obtained by automatically moving the dividing point in the first image with the starting anchor point as the starting position and the target anchor point as the target position; wherein, the automatic moving may be obtained The start anchor point and the target anchor point in the first image, or a small number of marked points including the start anchor point and the target anchor point, can automatically obtain the segmentation between the start anchor point and the target anchor point in the first image line.

With reference to the second aspect, in some implementations of the second aspect, the dividing line coincides with the dividing line in the ridge direction in the pixel gradient map.

It should be noted that the ridge direction in the pixel gradient map may refer to a curve formed by the local maximum value of the gradient in the pixel gradient map.

With reference to the second aspect, in some implementations of the second aspect, the obtaining a second image according to the first image and the anchor point includes: according to the pixel gradient map, the anchor point, and the anchor point. Point pulling model to obtain the second image, wherein the anchor point pulling model is used to indicate the moving direction of the dividing point.

In the embodiment of the present application, the secant image may be obtained according to the pixel gradient map of the first image and the anchor point pulling model. The anchor point pulling model may indicate that the dividing point automatically moves from the starting anchor point to the target anchor point every time. One-step movement direction, thereby reducing manual participation.

With reference to the second aspect, in some implementations of the second aspect, if the boundary point is currently located in an area without ridges in the pixel gradient map, the moving direction of the boundary point is based on the first line and The angle between the second line is determined, wherein the first straight line refers to the straight line where each of the eight directions is located, and the second straight line refers to the current position of the dividing point and the The line between the target anchor points; or,

If the dividing point is currently located in an area with ridges and bifurcations in the pixel gradient map, the moving direction of the dividing point is determined according to the angle between the first line and the second line and the distance parameter , Wherein the first straight line refers to the straight line where each of the eight azimuths is located, and the second straight line refers to the line between the current position of the dividing point and the target anchor point, so The distance parameter refers to the distance between the current position of the boundary point and the target anchor point; or,

If the boundary point is currently located in the pixel gradient map with a ridge and no bifurcation area, the moving direction of the boundary point is determined according to the ridge direction of the pixel gradient map.

With reference to the second aspect, in some implementations of the second aspect, the method further includes: if the moving direction at a time on the boundary point is the same as the moving direction at the current time, determining the absolute value of the gradient according to the different moving directions The direction of movement of the demarcation point.

In the embodiment of the present application, when the movement direction of the demarcation point at the previous moment is the same as the movement direction of the current moment, the demarcation point can be made to move in a preset direction, that is, the movement direction of the demarcation point can be based on different The absolute value of the gradient of the moving direction is determined, so that the error of the dividing point when moving along the ridge direction of the gradient map can be compensated.

With reference to the second aspect, in some implementations of the second aspect, the anchor point is obtained by optimizing an initial anchor point, where the initial anchor point is an anchor point manually marked by the user in the first image .

In the embodiment of the present application, because the anchor points in the image to be processed are manually marked by the user in the image to be processed, the position of the anchor point may have a certain deviation; in order to make the position of the anchor point more accurate, it improves The accuracy of the segmentation line of the image to be processed can optimize the position of the anchor points manually marked by the user.

In a third aspect, an image segmentation device is provided. The image segmentation device has a terminal device with a display screen, and includes: a detection unit configured to detect a first operation of a user to manually mark an anchor point in a first image, wherein: The anchor point includes a start anchor point and a target anchor point; the second operation of automatically segmenting the first image instructed by the user is detected;

The processing unit is configured to display a second image on the display screen in response to the second operation, the second image being an image obtained by the first image after the second operation, and the second image The image includes a dividing line between the starting anchor point and the target anchor point, wherein the dividing line is a dividing line in the first image with the starting anchor point as the starting position and The target anchor point is obtained by moving the target position.

Wherein, the above-mentioned dividing line may be obtained by automatically moving the dividing point in the first image with the starting anchor point as the starting position and the target anchor point as the target position; wherein, automatic moving may refer to The user only needs to manually mark the start anchor point and the target anchor point in the first image, or a small number of marked points including the start anchor point and the target anchor point, and then automatically obtain the difference between the start anchor point and the target anchor point in the first image. The dividing line between.

With reference to the third aspect, in some implementation manners of the third aspect, the dividing line coincides with the dividing line in the ridge direction of the pixel gradient map of the first image.

With reference to the third aspect, in some implementations of the third aspect, the processing unit is specifically configured to: search for a mask image and a secant image corresponding to the first image according to the first image, wherein the The mask image is used to represent different objects in the first image, the secant image is used to represent the boundaries of different objects in the first image; the first image, the An image obtained by superimposing the mask image and the secant image.

With reference to the third aspect, in some implementations of the third aspect, the detection unit is further configured to detect that the user instructs to process the second image through the mask image or the secant image The third operation.

With reference to the third aspect, in some implementations of the third aspect, the second image is obtained according to a pixel gradient map of the first image and an anchor point pulling model, and the anchor point pulling model is used to indicate State the direction of movement of the dividing point.

With reference to the third aspect, in some implementations of the third aspect, when the boundary point is currently located in an area without ridges in the pixel gradient map, the moving direction of the boundary point is based on the first line and The angle between the second line is determined, wherein the first straight line refers to the straight line where each of the eight directions is located, and the second straight line refers to the current position of the dividing point and the The line between the target anchor points; or

With reference to the third aspect, in some implementations of the third aspect, when the moving direction of the dividing point at the previous moment is the same as the moving direction of the dividing point at the current moment, the moving direction of the dividing point is Determined according to the absolute value of the gradient in different directions.

With reference to the third aspect, in some implementations of the third aspect, the anchor point is obtained by optimizing an initial anchor point, wherein the initial anchor point is an anchor point manually marked by the user in the first image .

In a fourth aspect, an image segmentation device is provided, including: an acquiring unit for acquiring a first image and position information of an anchor point in the first image, wherein the anchor point includes a start anchor point and a target anchor Point; a processing unit for obtaining a second image according to the first image and the anchor point, where the second image is an image obtained after the first image is subjected to image segmentation processing, and the second The image includes a dividing line between the starting anchor point and the target anchor point, where the dividing line passes through the dividing point in the pixel gradient map of the first image with the starting anchor point as the starting position And it is obtained by moving the target anchor point as the target position.

Wherein, the above-mentioned dividing line may be obtained by automatically moving the excessive demarcation point in the first image with the starting anchor point as the starting position and the target anchor point as the target position; wherein, the automatic movement may be obtained The start anchor point and the target anchor point in the first image, or a small number of marked points including the start anchor point and the target anchor point, can automatically obtain the segmentation between the start anchor point and the target anchor point in the first image line.

With reference to the fourth aspect, in some implementation manners of the fourth aspect, the dividing line coincides with the dividing line in the ridge direction of the pixel gradient map.

With reference to the fourth aspect, in some implementation manners of the fourth aspect, the processing unit is specifically configured to: obtain the second image according to the pixel gradient map, the anchor point, and the anchor point pulling model, wherein: The anchor point traction model is used to indicate the moving direction of the boundary point.

With reference to the fourth aspect, in some implementations of the fourth aspect, if the boundary point is currently located in an area without ridges in the pixel gradient map, the moving direction of the boundary point is based on the first line and The angle between the second line is determined, wherein the first straight line refers to a straight line in each of the eight directions, and the second straight line refers to the current position of the dividing point and the target The line between anchor points; or,

If the dividing point is currently located in an area with ridges and bifurcations in the pixel gradient map, the moving direction of the dividing point is determined according to the angle between the first line and the second line and the distance parameter , Wherein the first straight line refers to a straight line in each of the eight directions, the second straight line refers to the line between the current position of the dividing point and the target anchor point, and the The distance parameter refers to the distance between the current position of the boundary point and the target anchor point; or,

With reference to the fourth aspect, in some implementation manners of the fourth aspect, the processing unit is further configured to: if the moving direction at the previous moment of the dividing point is the same as the moving direction at the current moment, according to the gradient of different moving directions The absolute value determines the direction of movement of the dividing point.

With reference to the fourth aspect, in some implementations of the fourth aspect, the anchor point is obtained by optimizing an initial anchor point, where the initial anchor point is an anchor point manually marked by the user in the first image .

In a fifth aspect, an image segmentation device is provided. The image segmentation device has a display screen, including: a memory for storing a program; a processor for executing the program stored in the memory, and when the program stored in the memory is executed , The processor is configured to perform: detecting a first operation of manually marking an anchor point in the first image by the user, wherein the anchor point includes a starting anchor point and a target anchor point; detecting that the user instructs to automatically segment the The second operation of the first image; in response to the second operation, a second image is displayed on the display screen, the second image is an image obtained by the first image after the second operation, so The second image includes a dividing line between the starting anchor point and the target anchor point, wherein the dividing line starts from the starting anchor point in the first image through a dividing point The position is obtained by moving the target anchor point as the target position.

In a possible implementation manner, the processor included in the foregoing image segmentation apparatus is further configured to execute the first aspect and the image segmentation method in any one of the implementation manners of the first aspect.

It should be understood that the expansion, limitation, explanation and description of the related content in the above-mentioned first aspect are also applicable to the same content in the fifth aspect.

In a sixth aspect, an image segmentation device is provided. The image segmentation device includes: a memory for storing a program; a processor for executing the program stored in the memory, and when the program stored in the memory is executed, the processor uses In execution: acquiring the first image and the position information of the anchor points in the first image, wherein the anchor points include a starting anchor point and a target anchor point; according to the pixel gradient map and the anchor point, the first image is obtained Two images, wherein the second image is an image obtained after the first image is subjected to image segmentation processing, and the second image includes a dividing line between the starting anchor point and the target anchor point, so The dividing line is obtained by moving the dividing point in the pixel gradient map of the first image with the starting anchor point as the starting position and the target anchor point as the target position.

It should be understood that the expansion, limitation, explanation and description of the related content in the above-mentioned first aspect are also applicable to the same content in the sixth aspect.

In a seventh aspect, a computer-readable medium is provided, and the computer-readable medium stores program code for device execution, and the program code includes the program code for executing the first aspect and any one of the implementation manners of the first aspect. Image segmentation method.

In an eighth aspect, a computer-readable medium is provided. The computer-readable medium stores program code for device execution. The program code includes Image segmentation method.

In a ninth aspect, a computer program product containing instructions is provided. When the computer program product runs on a computer, the computer executes the first aspect and the image segmentation method in any one of the first aspects.

In a tenth aspect, a computer program product containing instructions is provided. When the computer program product runs on a computer, the computer executes the second aspect and the image segmentation method in any one of the second aspects.

In an eleventh aspect, a chip is provided. The chip includes a processor and a data interface. The processor reads instructions stored in a memory through the data interface, and executes any one of the first aspect and the first aspect. An image segmentation method in one implementation.

Optionally, as an implementation manner, the chip may further include a memory in which instructions are stored, and the processor is configured to execute instructions stored on the memory. When the instructions are executed, the The processor is configured to execute the above-mentioned first aspect and the image segmentation method in any one of the implementation manners of the first aspect.

In a twelfth aspect, a chip is provided. The chip includes a processor and a data interface. The processor reads instructions stored in a memory through the data interface, and executes any one of the second aspect and the second aspect. An image segmentation method in an implementation.

Optionally, as an implementation manner, the chip may further include a memory in which instructions are stored, and the processor is configured to execute instructions stored on the memory. When the instructions are executed, the The processor is configured to execute the above-mentioned second aspect and the image segmentation method in any one of the implementation manners of the second aspect.

Description of the drawings

FIG. 1 is a schematic diagram of a system architecture provided by an embodiment of the present application;

FIG. 2 is a schematic flowchart of an image segmentation method provided by an embodiment of the present application;

FIG. 3 is a schematic flowchart of an image segmentation method provided by an embodiment of the present application;

FIG. 4 is a schematic diagram of an N-map adjustment mask image provided by an embodiment of the present application;

FIG. 5 is a schematic diagram of an image to be processed and a pixel gradient map provided by an embodiment of the present application;

FIG. 6 is a schematic flowchart of a method for optimizing anchor point positions provided by an embodiment of the present application;

FIG. 7 is a schematic diagram of a four-way convolution kernel provided by an embodiment of the present application;

FIG. 8 is a schematic flowchart of an image segmentation method provided by an embodiment of the present application;

FIG. 9 is a schematic flowchart of an image segmentation method provided by an embodiment of the present application;

FIG. 10 is a schematic flowchart of a ridge running algorithm based on anchor point traction gradient provided by an embodiment of the present application;

FIG. 11 is a schematic diagram of a convolution kernel for calculating gradients of different orientations according to an embodiment of the present application;

FIG. 12 is a schematic diagram of performing lateral drift according to an embodiment of the present application;

FIG. 13 is a schematic diagram of an anchor point traction model provided by an embodiment of the present application;

FIG. 14 is a schematic block diagram of an image segmentation device provided by an embodiment of the present application;

FIG. 15 is a schematic block diagram of an image segmentation device provided by an embodiment of the present application;

FIG. 16 is a schematic diagram of the hardware structure of an image segmentation device provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.

In view of this, the embodiments of the present application provide an image segmentation method and device. The user manually marks the starting anchor point and the target anchor point on the image to be processed, so that the starting anchor point is the starting position of the dividing point and the The target anchor point is the target position and automatically moves according to the pixel gradient map of the image to be processed to obtain the segmentation line of the image to be processed; that is, a segmentation line can be automatically run out on the pixel gradient map along the gradient ridge direction boundary point to make the segmentation The line coincides with the natural boundary of the image, so that the accuracy of the image segmentation result is improved while saving manpower.

Fig. 1 is a schematic diagram of a system architecture provided by an embodiment of the present application.

As shown in FIG. 1, the system 100 may include an application server 110, a data server 120, and multiple clients (for example, a client 131, a client 132, and a client 133); among them, the client may communicate with the application server through a communication network. 110 is connected to a data server 120; the data server 110 can be used to store a large number of image files and annotation files; the application server 120 can be used to provide image annotation or image editing services; the client can be used to provide a human-computer interaction interface.

For example, the client can be a mobile or fixed terminal; for example, the client can be a mobile phone with image processing functions, a tablet personal computer (TPC), a media player, a smart TV, or a laptop computer. , LC), personal digital assistant (personal digital assistant, PDA), personal computer (PC), camera, video camera, smart watch, wearable device (WD) or self-driving vehicle, etc., this application The embodiment does not limit this.

Among them, each client can interact with the data server 110 or the application server 120 through a communication network of any communication mechanism/communication standard. The communication network can be a wide area network, a local area network, a point-to-point connection, etc., or any combination thereof.

It should be understood that the system 100 shown in FIG. 1 is a multi-user system, and the image segmentation method provided in this application is also applicable to a single-user system; for a single-user system, that is, a system including one client, the image segmentation method provided in the embodiment of the application is It can be deployed on a client; for a multi-user system, that is, the system including multiple clients as shown in FIG. 1, the image segmentation method provided in this embodiment of the present application can be deployed on a server, such as an application server or a data server.

Exemplarily, the image segmentation method provided in the embodiments of the present application can be applied to an image segmentation labeling tool or a retouching tool, and the image segmentation method can realize the background replacement of the image or the background blur of the image.

For example, the user turns on the video call function in the smart terminal. During the shooting process, the image can be segmented in real time, and only the target object area is reserved to realize the replacement of the background area of the video call.

For example, the user turns on the shooting function on the smart terminal. During the shooting process, the image can be segmented in real time, so that the foreground area of the photographed target object is clear and the background area is blurred, realizing the image effect of the large aperture of the SLR camera.

It should be understood that the background blurring of the image and the replacement of the background of the image described above are only two specific scenarios applied by the image processing method of the embodiment of the present application, and the image processing method of the embodiment of the present application is not limited to the above two when applied. For each scene, the image processing method of the embodiment of the present application can be applied to any scene that requires image segmentation.

The image segmentation method of the embodiment of the present application will be described in detail below in conjunction with FIG. 2 to FIG. 13.

Fig. 2 is a schematic flowchart of an image segmentation method provided by an embodiment of the present application. The method may be executed by the server or the client shown in FIG. 1, and the method shown in FIG. 2 includes step 210 to step 230, and these steps are respectively described in detail below.

Step 210: A first operation of manually marking an anchor point in the first image by the user is detected.

Wherein, the first image may refer to an image to be processed with an image segmentation requirement, and the above-mentioned anchor points may include a starting anchor point and a target anchor point.

Exemplarily, before step 210, the user can open the image processing tool by operating and import the first image into the image processing tool, and the first image that needs to be segmented can be displayed on the interface of the image processing tool.

Step 220: A second operation instructed by the user to automatically segment the first image is detected.

Exemplarily, the second operation used by the user to instruct to automatically segment the first image may include the user clicking the button for automatically segmenting the image in the image processing tool, or may include the behavior of the user instructing the automatic segmentation through voice, or may also include the user Other actions that indicate automatic segmentation; the above are examples and do not limit this application in any way.

Step 230: In response to the second operation, display a second image on the display screen;

Wherein, the second image is an image obtained after the first image is subjected to the second operation, and the second image includes a dividing line between the starting anchor point and the target anchor point, and the The dividing line is obtained by moving the dividing point in the first image with the starting anchor point as the starting position and the target anchor point as the target position.

For example, the above-mentioned dividing line may be obtained by automatically moving an excessive demarcation point in the first image with the starting anchor point as the starting position and the target anchor point as the target position; wherein, automatic movement may refer to The user only needs to manually mark the start anchor point and the target anchor point in the first image, or a small number of mark points including the start anchor point and the target anchor point, and then automatically obtain the difference between the start anchor point and the target anchor point in the first image. The dividing line between.

It should be understood that the starting anchor point and the target anchor point may be positions in the image to be processed where the position does not change, and the starting anchor point and the target in the first image can be obtained by moving the dividing point from the starting anchor point to the target anchor point. A dividing line between the anchor points, which coincides with the natural boundary of different objects in the first image.

Exemplarily, the aforementioned dividing point may be based on the starting anchor point as the starting position, and the target anchor point as the target position. A dividing line is run along the ridge of the pixel gradient map of the first image, that is, the dividing line and the first image The division lines in the ridge direction of the pixel gradient map of the overlapped; wherein, the pixel gradient map of the first image may refer to an image composed of changes in the brightness of pixels in different rows or different columns in the first image.

In an example, after the second operation for instructing automatic segmentation by the user, the terminal device can search for the mask image and the secant image corresponding to the first image according to the first image, where the mask image can be used to represent the to-be The object area in the first image, the secant image can be used to represent the boundary of different objects in the first image; the image to be processed, the mask image, and the secant image are superimposed and displayed on the interface of the image processing tool .

For example, after searching, the mask image corresponding to the first image and the secant image corresponding to the first image are found.

In one example, if the secant image and the mask image corresponding to the first image are not found after searching, it is necessary to generate the secant image and the mask image corresponding to the first image.

For example, using Mask-RCNN, DeepLabV3 or other algorithms to perform automatic image segmentation through the first image to generate a mask image corresponding to the first image; further, a preset boundary threshold can be used according to the first image and the mask image , Use the N-map-based method to automatically adjust the mask range, and then automatically generate the secant image according to the mask boundary, so that the secant image and the mask image are automatically aligned.

Among them, the N-map method can also be called boundary drift based on N-map; suppose that there are N image segmentation areas in the mask layer file, and the value of each pixel is the area number K to which the pixel belongs, K∈[ 0, N-1]; Set the pixels of the unknown classification on both sides of the secant, and change the value of the mask layer to N; Then, use the pre-trained deep neural network to process the corresponding image layer files, and classify the unknown Pixels and background pixels are allocated to N divided regions.

In one example, after searching, the secant image corresponding to the first image is found, and the mask image corresponding to the first image is not found, the mask image corresponding to the first image needs to be generated.

In an example, after searching, the mask image corresponding to the first image is found. If the secant image corresponding to the first image is not found, the secant image corresponding to the first image needs to be generated; the specifics of generating the secant image Refer to the schematic flowcharts shown in Figure 9 and Figure 10 below for the process.

Further, the user may manually select to perform segmentation processing on the first image through the above-mentioned mask image or secant image.

It should be noted that the secant image needs to be updated synchronously after the user adjusts the mask image, that is, the updated mask image is synchronized with the secant image; similarly, the user needs to adjust the secant image after the user adjusts the secant image. The film image is updated synchronously, that is, the updated secant image is synchronized with the mask image.

Optionally, in the embodiment of the present application, the image segmentation method further includes: detecting a third operation instructed by the user to process the second image through the mask image or the secant image.

In a possible implementation manner, the above-mentioned second image may refer to the superimposed image of the first image, the mask image, and the secant image displayed on the display screen, and the third operation may be used to instruct to pass the mask image or The secant image performs automatic segmentation processing on the superimposed image.

Optionally, in the embodiment of the present application, the second image may be obtained according to the pixel gradient map of the first image and the anchor point pulling model, and the anchor point pulling model may be used to indicate the moving direction of the boundary point, that is, The preset anchor point pulling model can be used to make the anchor point obtain the segmentation line of the first image along the ridge direction of the pixel gradient map, so that the segmentation line of the first image coincides with the natural boundary of the image.

Exemplarily, the anchor point traction model may provide a moving direction for the dividing point in the dividing line. For example, when the dividing point is currently located in an area without ridges in the pixel gradient map, the moving direction of the dividing point is Determined according to the angle between the first line and the second line, where the first line refers to the line where each of the eight directions is located, and the second line refers to the boundary The line between the current position of the point and the target anchor point;

Alternatively, when the dividing point is currently located in an area with ridges and bifurcations in the pixel gradient map, the moving direction of the dividing point is based on the angle between the first line and the second line and The distance parameter is determined, wherein the first straight line refers to the straight line where each of the eight azimuths is located, and the second line refers to the distance between the current position of the dividing point and the target anchor point. Line, the distance parameter refers to the distance between the current position of the boundary point and the target anchor point;

Alternatively, when the boundary point is currently located in the pixel gradient map with a ridge and no bifurcation area, the moving direction of the boundary point is determined according to the ridge direction of the pixel gradient map.

It should be noted that the first straight line is the direct line where each of the eight directions is located, as shown in (a) in Figure 11, which can respectively refer to the straight line oa, the straight line ob, the straight line oc, the straight line od, and the straight line oe. , Straight of, straight og, straight oh.

Exemplarily, the gravitational model of the target anchor point (for example, anchor point B) to the running point (for example, the demarcation point) can be regarded as composed of a strong gravitational model and a weak gravitational model; suppose the distance between the running point and the target anchor point is The distance is dis, the distance between anchor point A and anchor point B is d0, and the angle between the 8 azimuths and anchor point AB is θ _i , and the strength of each azimuth can be calculated according to the gravity model shown in Figure 13 The gravity weight w _i and the weak gravity offset b _i . For the strong gravitational model and the weak gravitational model, the smaller the angle between the eight azimuths and the anchor point AB, the greater the gravitational force; for example, the angle between the eight azimuths and the anchor point AB is 0° The azimuth gravitational force is the largest, and the azimuth gravitational force is the smallest when the angle between the anchor point AB is 180°. The difference between the strong gravitational model and the weak gravitational model is that the weak gravitational model has nothing to do with the distance dis; the strong gravitational model has the slowest gravitational decay rate with the increase of the angle when dis=d0, and the faster the decay rate of dis deviates from d0.

Further, in order to compensate for the error when the dividing point moves along the ridge direction of the gradient map, when the movement direction of the dividing point at the previous moment is the same as the movement direction at the current moment, the dividing point can be made to move toward the preset Directional movement, that is, the movement direction of the dividing point is determined according to the absolute value of the gradient of different moving directions.

In the embodiment of the present application, because the position of the anchor point is generated by the user clicking on the image with the mouse, the position of the anchor point may be deviated and not located on the gradient map corresponding to the image, in order to make the anchor point be able to The gradient map accurately obtains the segmentation line of the image, and the anchor point position can be optimized.

Exemplarily, in the embodiment of the present application, the position of the anchor point can be optimized by the four-way gradient convolution kernel as shown in FIG. 6; for example, the anchor point can be adjusted according to the absolute value of the gradient of the anchor point in different directions. The position is optimized, and the optimized position information of the anchor point is obtained.

FIG. 3 shows a schematic flowchart of an image segmentation method provided by an embodiment of the present application. The method can be implemented by an image processing tool (for example, an image segmentation labeling tool, or a retouching tool) in the server or client shown in FIG. carried out. The method shown in FIG. 3 includes steps 301 to 311, and these steps are respectively described in detail below.

First, an explanation will be given of the layers included in the image processing tools; the image processing tools may include image layers, mask layers, and cutline layers.

Wherein, each pixel value in the image layer can represent the gray level of the image, and the pixel coordinates (x, y) can be a positive integer, and the image layer is displayed on the screen as a normal image, such as a color image.

Exemplarily, each pixel value in the mask layer can represent the serial number of the mask; for example, when the mask layer is displayed on the screen, the same area of the mask serial number can be colored with the same color, for example, the background The mask serial number is 0, and the coloring can be blue when displaying; the mask serial number of TV is 1, and the coloring is red when displaying; other mask serial numbers can be deduced by analogy. The user can assign a different display color to each mask serial number by operating a menu or button or palette on a graphical user interface (GUI).

Exemplarily, the secant layer may be composed of a dividing line consisting of an anchor point and a dividing point; wherein the anchor point may be the end point of the dividing line; the value of the anchor point may represent the serial number of the anchor point (for example, 129 to 255) The value of the dividing point may represent the serial number of the dividing line (for example, 1 to 127), and the point with a value of 0 may not be an anchor point or a dividing point.

For example, the anchor point and the segmentation point may be between pixels and not coincide with the pixel position, and their two-dimensional coordinates (u, v) and the coordinates (x, y) of nearby pixels are offset by a distance of 0.5 pixels.

For example, the split layer can be displayed as a dotted line on the screen, and the anchor point can be displayed as a small circle on the screen; the user can move any anchor point by clicking or dragging, and the split line also moves with the anchor point.

It should be noted that the user can operate the menu or button on the GUI of the annotation tool to enable each layer to be turned off (not displayed) or turned on (displayed) independently.

Step 301: Read the image layer file.

For example, it may be that the user imports an image layer file through an operation.

Step 302: Search for the mask layer file and the secant layer file corresponding to the image layer file.

The above-mentioned image layer file may refer to an image file with specific image segmentation requirements, and the file corresponding to the image layer file can be searched through a preset file path.

Exemplarily, each layer file can be saved as a bitmap file (bitmap, BMP), a losslessly compressed bitmap graphics format (portable network graphics, PNG), or other image file formats; the correspondence between different layer files , Can be described by icon or file name naming rules, or by packaging the image layer file, the mask layer file corresponding to the image layer file, and the secant file corresponding to the image layer file in one file or a file Folder, as shown in Table 1.

Table 1

图像层文件Image layer file	掩膜层文件Mask file	割线层文件Secant layer file
P0001_img.jpgP0001_img.jpg	P0001_msk.pngP0001_msk.png	P0001_ctl.pngP0001_ctl.png
P0002_img.jpgP0002_img.jpg	P0002_msk.pngP0002_msk.png	P0002_ctl.pngP0002_ctl.png
P0003_img.jpgP0003_img.jpg	P0003_msk.pngP0003_msk.png	P0003_ctl.pngP0003_ctl.png

Exemplarily, the corresponding relationship between the serial number of the dividing line and the serial number of the anchor point can be adopted, as shown in Table 2.

Table 2

分割线Dividing line	锚点AAnchor point A	锚点BAnchor point B
11	129129	130130
22	130130	132132
33	130130	134134

In an example, after searching and determining to save the mask layer file and the secant layer file corresponding to the image layer file, step 306 may be executed.

In an example, after searching and determining that there is no secant layer file and mask layer file corresponding to the image layer file, step 303 is executed to automatically generate the mask layer file and the secant layer file corresponding to the image layer file.

For example, use Mask-RCNN or DeepLabV3 or other algorithms to perform automatic image segmentation through the image layer file, thereby generating the mask layer file; further, according to the image layer file and the mask layer file, the preset boundary threshold is used, and the N-based The -map method automatically adjusts the mask range, and then automatically generates the secant layer file according to the mask boundary, so that the secant layer and the mask layer are automatically aligned.

The above-mentioned method of adjusting the mask layer based on N-map can also be called boundary drift based on N-map; suppose that there are N image segmentation areas in the mask layer file, and the value of each pixel is the area to which the pixel belongs Number K, K ∈ [0, N-1]; set the pixels of unknown classification on both sides of the secant, and change the value of the mask layer to N; then, use the pre-trained deep neural network to process the corresponding image layer File, the pixels of unknown classification and background pixels are allocated to N segmented areas.

For example, as shown in Figure 4, the value of the mask layer can be changed to N for consecutive T pixels on both sides of the vertical secant at the midpoint of the secant line; the farther from the midpoint of the secant line, the mask layer The number of pixels whose value is changed to N is less; at the anchor point, the number of pixels whose mask layer value is changed to N is 0; if the secant is a closed curve that does not include the anchor point, it can be at any of the secant Position Change the value of the mask layer on both sides of the secant line to N and the number of pixels is T; where T can be a preset threshold, which can be manually adjusted in the user interface.

In an example, after searching, it is determined to save the mask layer file corresponding to the image layer file, but there is no secant layer file corresponding to the image layer file, then step 304 is executed to automatically generate the secant layer file.

For example, according to the image layer file and its corresponding mask layer file, you can use the N-map-based method to automatically adjust the mask range according to the preset boundary threshold, and then automatically generate the secant layer file according to the mask boundary to make the cut The line layer and the mask layer are automatically aligned.

In an example, after searching, it is determined that the secant layer file corresponding to the image layer file is saved, but there is no mask layer file corresponding to the image layer file, then step 305 is executed to automatically generate the mask layer file.

For example, according to the image layer file and its corresponding segmentation layer file, the user can observe the edge of the image, and the user can manually adjust the anchor point of the secant layer, and use the anchor point pulling gradient ridge run method to automatically adjust the secant position of the secant layer; , The mask layer file is automatically generated, so that the mask layer and the secant layer are automatically aligned.

Step 306: The three layer files can be superimposed and displayed to allow the user to edit the image segmentation.

Step 307: The user can select the image for image segmentation editing; for example, if the user selects the mask layer file to adjust the image segmentation, step 308 and step 309 are executed; if the user selects the secant layer file to adjust the image segmentation, then Step 310 and step 311 are performed.

Step 308: Adjust the mask layer file.

For example, the mask layer file can be adjusted by the N-map method shown in FIG. 4 above.

Step 309: Perform a secant layer file update on the updated mask layer file, that is, make the updated mask layer file and the secant layer file synchronously aligned.

Step 310: Adjust the secant layer file.

For example, the user can manually adjust the anchor point of the secant layer file by observing the edge of the image; or use the anchor point pulling gradient ridge run algorithm shown in Figure 9 and Figure 10 to automatically adjust the secant position of the secant layer file.

Step 311: Perform a mask layer file update on the updated secant layer file, that is, the updated secant layer file and the mask layer file are synchronized and aligned.

Exemplarily, as shown in FIG. 5, where (a) in FIG. 5 is the original image, and (b) in FIG. 5 is the image corresponding to the original image shown in (a) in FIG. 5 Pixel gradient map; the user can observe the original image (for example, color map), click 6 anchor points (for example, 1-6) in the original image with the mouse, and map the positions of the 6 anchor points to the pixel gradient map; On the pixel gradient map, you can draw a dividing line between two adjacent anchor points along the ridge direction of the pixel gradient map by pulling the anchor point. 6 dividing lines can form a closed curve, and the inside of the curve can be the foreground , The outside of the curve can be the background.

It should be understood that generally, only the original image may be displayed on the interface of the annotation tool, and the pixel gradient map may not be displayed.

Further, in the embodiment of the present application, since the anchor points in the image to be processed are manually marked by the user in the image to be processed, the position of the anchor point may have a certain deviation; in order to make the position of the anchor point more accurate, Thereby, the accuracy of the segmentation line of the image to be processed is improved, and the position of the anchor points manually marked by the user can be optimized.

For example, FIG. 6 is a schematic flowchart of a method for optimizing anchor point positions provided by an embodiment of the present application. The method shown in FIG. 6 includes steps 401 to 406, and these steps are respectively described in detail below.

Step 401: The image layer and the secant layer are displayed in a graphical user interface (GUI).

Step 402: The user edits the anchor point position.

For example, the user can generate an anchor point by clicking on the image, or the user can drag an existing anchor point to generate an updated anchor point.

Step 403: Obtain the original coordinates of the anchor point.

For example, the original coordinates of anchor point A are (x1, y1).

Step 404: Search the ridge according to the four-way gradient to obtain the optimized anchor point position.

Exemplarily, the four-way convolution kernel provided in this embodiment of the application is shown in FIG. 7, where (a) in FIG. 7 is the V-direction gradient convolution kernel; (b) in FIG. 7 Shown is the H-direction gradient convolution kernel; Fig. 7 (c) shows the L-direction gradient convolution kernel; Fig. 7 (d) shows the R-direction gradient convolution kernel.

For example, the V-direction gradient ridge search position shown in (e) in Fig. 7 and the (f) R-direction gradient ridge search position shown in Fig. 7 are examples to illustrate the method of optimizing anchor points. In the four-direction gradient Search for the gradient ridge in the direction with the largest absolute value; for example, calculate the same at the positions of A-1, A-2, A-3, A-4, and A+1, A+2, A+3, and A+4. Gradient, if there is a position opposite to the sign of the same gradient at anchor point A, stop searching immediately; then at the position with the same sign of the gradient at anchor point A, take the position with the largest absolute value of the gradient as the optimized anchor Point position A.

Step 405: Adjust the anchor point data in the secant line.

Step 406: Display the optimized anchor point (x2, y2) in the graphical user interface.

FIG. 8 is a schematic flowchart of an image segmentation method provided by an embodiment of the present application. The method can be executed by the server or the client shown in FIG. 1. The method shown in FIG. 8 includes step 510 and step 520, and these steps are respectively described in detail below.

Step 510: Acquire the first image and the position information of the anchor points in the first image, where the anchor points may include a starting anchor point and a target anchor point.

Wherein, the first image may refer to an image that has an image segmentation requirement.

Exemplarily, in step 510, a pixel gradient map of the first image may also be obtained.

Step 520: Obtain a second image according to the first image and the anchor point.

Wherein, the second image is an image obtained after image segmentation of the first image, the second image may include a dividing line between the starting anchor point and the target anchor point, and the dividing line may be a pixel in the first image passing through the dividing point. The gradient graph is obtained by moving the starting anchor point as the starting position and the target anchor point as the target position.

It should be understood that the starting anchor point and the target anchor point may be positions in the first image where the position does not change, and the starting anchor point and the target anchor point in the first image can be obtained by moving the boundary point from the starting anchor point to the target anchor point. A dividing line between the anchor points, which coincides with the natural boundary of different objects in the first image.

Exemplarily, the aforementioned dividing point may be based on the starting anchor point as the starting position, and the target anchor point as the target position. A dividing line is run along the ridge of the pixel gradient map of the first image, that is, the dividing line and the first image The dividing lines in the ridge direction of the pixel gradient map coincide.

Optionally, in the embodiment of the present application, obtaining the second image according to the first image and the anchor point may include: obtaining the second image according to the pixel gradient map, the anchor point, and the anchor point pulling model of the first image. Image, where the anchor point traction model can be used to indicate the direction of movement of the demarcation point.

Exemplarily, if the boundary point is currently located in an area without ridges in the pixel gradient map, the moving direction of the boundary point is determined according to the angle between the first line and the second line, Wherein, the first straight line refers to the straight line where each of the eight directions is located, and the second straight line refers to the line between the current position of the dividing point and the target anchor point; or,

For example, as shown in Figure 13, the strong gravity weight w _i and the weak gravity offset b _i for each azimuth can be calculated according to the gravity model. For the strong gravitational model and the weak gravitational model, the smaller the angle between the eight azimuths and the anchor point AB, the greater the gravitational force; for example, the angle between the eight azimuths and the anchor point AB is 0° The azimuth gravitational force is the largest, and the azimuth gravitational force is the smallest when the angle between the anchor point AB is 180°. The difference between the strong gravitational model and the weak gravitational model is that the weak gravitational model has nothing to do with the distance dis; the strong gravitational model has the slowest gravitational decay rate as the angle increases when dis=d0, and the faster dis deviates from d0.

Further, in order to compensate for the error when the dividing point moves along the ridge direction of the gradient map, when the moving direction of the dividing point at the previous moment is the same as the moving direction at the current moment, the dividing point can be made to move toward the preset Directional movement, that is, the movement direction of the dividing point is determined according to the absolute value of the gradient of different moving directions.

FIG. 9 shows a schematic flowchart of an image segmentation method provided by an embodiment of the present application. The method may be executed by the server or client shown in FIG. 1. The method shown in FIG. 9 includes steps 601 to 609, and these steps are respectively described in detail below.

Step 601: Obtain the anchor point position A.

Wherein, the anchor point position A may be an anchor point selected by the user in the image, or a new anchor point generated by the user by dragging an existing anchor point.

For example, it may be an anchor point generated by the user clicking on the image with the mouse, or a new anchor point generated by the user dragging an existing anchor point with the mouse.

Further, the position information of the anchor point A on the picture can be obtained through the annotation tool.

Step 602: Optimize the anchor point position A.

It should be understood that because the anchor point position A is generated by the user clicking on the image with the mouse, the anchor point position A may have a deviation and is not located on the gradient map corresponding to the image, so that the anchor point can be accurately obtained according to the gradient map. The dividing line of the image can optimize the anchor point position A.

Exemplarily, in the embodiment of the present application, the anchor point position A may be optimized through the four-way gradient convolution kernel as shown in FIG. 6.

For example, by convolving the image with the convolution kernel shown in Figure 7, the four-way gradient of anchor point A (for example, V, H, L, and R) can be obtained, where the absolute value of the four-way gradient is the largest Search for the gradient ridge in the direction of. If there is a position opposite to the same gradient sign at anchor point A, stop searching immediately; then at the same position as the gradient sign at anchor point A, take the position with the largest absolute value of the gradient, As the optimized anchor position A.

Step 603: Obtain the anchor point position B.

Wherein, the anchor point position A may refer to the starting point of the image segmentation, and the anchor point position B may refer to the target point of the image segmentation. The image segmentation line can be obtained by the connection between the anchor point position A and the anchor point position B.

Similarly, the anchor point position B can be the anchor point selected by the user in the image, or the user can drag an existing anchor point to generate a new anchor point.

For example, it may be that the user clicks the image with the mouse to generate the anchor point B, or the user drags the existing anchor point with the mouse to generate a new anchor point.

Step 604: Optimize the anchor point position B.

It should be noted that the specific process of optimizing the anchor point position B is similar to the foregoing step 602, and will not be repeated here.

It should be understood that step 601 and step 603 can be performed at the same time, or step 601 can be performed first and then step 603; similarly, step 602 and step 604 can be performed at the same time, or step 602 can be performed first and then step 602 is performed. 604. This application does not make any limitation on this.

Step 605: Generate multiple candidate segmentation lines according to the optimized anchor point position A and the optimized anchor point position B.

According to the optimized anchor point position A and anchor point position B, the process of ridge running based on the anchor point traction gradient is executed. For the specific process, refer to the flowchart shown in FIG. 10.

In one example, the optimized anchor point position A and the anchor point position B perform a two-way run, that is, the optimized anchor point position A takes position A as the starting point, and the optimized anchor point position B is the target point to execute the running point; optimized anchor point position A Point position B takes position B as the starting point, and optimizes anchor point position A as the target point to execute the run; obtain two dividing lines L1 and L2, where the number of breakpoints is N1 and N2, respectively, and the breakpoint can refer to a certain The gradient value of the running point is lower than the preset threshold.

In an example, from the optimized anchor point position B to the optimized anchor point position A, the dividing line L3 is obtained, where the number of breakpoints is N3.

In an example, from the optimized anchor point position A to the optimized anchor point position B, the dividing line L4 is obtained, where the number of breakpoints is N4.

Step 606: Preferably, the dividing line with the fewest breakpoints is selected.

For example, based on the division lines L1 to L4 and the number of break points N1 to N4 obtained above, a division line with the least number of break points is selected from a plurality of candidate division lines.

Step 607: It is judged whether the selection of the dividing line is successful; if it succeeds, step 608 is executed to end the process; if it fails, step 609 is executed manually by the user.

Exemplarily, the number of breakpoints for each dividing line is counted, and the dividing line with the least number of breakpoints is preferred; if the dividing line with the least number of breakpoints has little difference in the number of breakpoints from other dividing lines, multiple dividing lines can be presented Give the user manual selection.

Fig. 10 is a schematic flowchart of an anchor-point traction gradient ridge run provided by an embodiment of the present application. This method can be executed by the server or the client shown in FIG. 1. The method shown in FIG. 10 includes steps 701 to 710, and these steps are respectively described in detail below.

Step 701: Start from anchor point position A.

For example, the anchor point position A may be an anchor point selected by the user in the image, or the user may drag an existing anchor point to generate a new anchor point.

Exemplarily, a preset step length can be set before starting from anchor point position A, that is, it can refer to moving from anchor point position A to target anchor point B, and the distance shifted by each step.

It should be understood that the aforementioned anchor point position A may be the initial anchor point manually marked by the user in the image, or may also refer to the anchor point position in the process of moving from the initial anchor point to the target anchor point, which is not limited in this application .

Step 702: Update the starting point coordinates.

Wherein, updating the starting point coordinates may refer to optimizing the anchor point position A, thereby improving the accuracy of the dividing line; the specific process of optimizing the anchor point position A can be seen in FIG. 7 and will not be repeated here.

It should be understood that, in the embodiment of the present application, the dividing line of the image may be obtained by moving the dividing point in the pixel gradient map corresponding to the image with the starting anchor point as the starting position and the target anchor point as the target position, where The running point can refer to the point on the dividing line obtained by moving the above-mentioned dividing point according to the preset step length through the algorithm. The running point can start from the position of the starting anchor point according to the preset step length and the selected running direction Move to the position of the target anchor point to obtain the dividing line of the image between the starting anchor point and the target anchor point.

Step 703: Choose a running direction.

It should be understood that the above-mentioned running direction may refer to the moving direction of each step of the running point according to the preset step length.

In the embodiment of the present application, the running direction of the running point can be selected according to the preset step length in the following ways: Method 1: The running direction of the running point is determined based on the calculation of the gradient value of different directions.

For example, Figure 11 (a) shows the schematic diagram of the 8-direction azimuth; Figure 11 (b) shows the oa azimuth gradient convolution kernel; Figure 11 (c) shows the ob azimuth gradient Convolution kernel; Figure 11 (d) shows the oab azimuth gradient convolution kernel; Figure 11 (e) shows the ocd azimuth gradient convolution kernel; if the run-in direction is in the fob direction Below, you can use the oab gradient convolution kernel; if the running direction of the running point is above the fob direction, you can use the ocb gradient convolution kernel; if the running direction of the running point is fo, you can use the oab gradient and The larger absolute value of the ocb gradient.

In the embodiment of the present application, it is not allowed to run in the reverse direction when the anchor point runs along the ridge of the gradient map; for example, if the previous step is running in from the ao direction, the next step is not allowed to run from the oa direction Out; so in order to determine the next running direction, you need to calculate the gradients of the remaining 7 directions that are different from the running direction. For example, if the previous step is running from the ao direction, calculate ob, oc, od, oe, of, og , Oh and other 7-way gradients, choose the next step to run from the direction with the largest absolute value of the 7-way gradient.

Method 2: Determine the running direction of the running point based on the anchor point traction model.

Exemplarily, first, the distance between the running point and the anchor point position B can be calculated as dis, for example, the distance between the anchor point position A and the anchor point position B is d0; then, the traction weight wi and the anchor point A can be calculated bi, get the anchor traction model.

The above anchor point traction model can be regarded as composed of a strong gravitational model and a weak gravitational model; for example, suppose the distance between the running point and the target anchor point is dis, and the distance between anchor point A and anchor point B is d0,8 AB azimuth angle with the anchor connection as θ _i, can be calculated according to the gravity model 13 shown in FIG strong gravitational weight of each weight w _i and orientation offsets weak gravitational b _i. For the strong gravitational model and the weak gravitational model, the smaller the angle between the eight azimuths and the anchor point AB, the greater the gravitational force; for example, the angle between the eight azimuths and the anchor point AB is 0° The azimuth gravitational force is the largest, and the azimuth gravitational force is the smallest when the angle between the anchor point AB is 180°. The difference between the strong gravitational model and the weak gravitational model is that the weak gravitational model has nothing to do with the distance dis; the strong gravitational model has the slowest gravitational decay rate with the increase of the angle when dis=d0, and the faster the decay rate of dis deviates from d0.

In the embodiment of the present application, the weighted calculation of the gradient can be performed by the following equation:

D' _i =w _i *(D _i +b _i );

Among them, D' _i represents the weighted 7-way gradient of _{the i-th orientation; D i} represents the unweighted 7-way gradient of _{the i-th orientation; w i} represents the strong gravity weight of the i-th orientation; b _i represents the i-th orientation. Weak gravitational weight of each direction.

For example, when the running point is located in an area without obvious ridges, the weak gravity model can provide a clear direction for the running point; when the running point is at the bifurcation of the ridge, the strong gravity model can provide a clear direction for the running point. Direction guidance; when the running point is located on an obvious ridge and there is no bifurcation position, the path of the running point can be mainly determined by the gradient map.

For example, as shown in Figure 13, the strong gravitational weight wi and the weak gravitational offset bi can be calculated for each azimuth; among them, Figure 13 (a) shows different anchor point directions; Figure 13 (b) ) Shows the schematic diagram of the strong gravitational model; Figure 13(c) shows the schematic diagram of the weak gravitational model. For example, the schematic image shown in Figure 13(b) includes straight line 1, straight line 2, and straight line 3. Among them, straight line 1 can represent the strong gravitational model when dis=0 or dis=2d0; straight line 2 can represent The strong gravitational model when dis=0.4d0 or dis=1.6d0; the straight line 3 can represent the strong gravitational model when dis=d0; from the three straight lines shown in Figure 13 (b), it can be seen that the greater the slope of the straight line Then the traction to the running point is stronger; the angle between the strong gravitational model and the line between the anchor point AB and the different azimuths and the distance between the running point and the target anchor point are related to dis; the weak gravitational model is related to the different azimuths and the anchor point AB The connection angle is related.

In the embodiment of the present application, the movement direction of each step of the running point can be calculated at the preset step length according to the above-mentioned method 1 and method 2, so as to obtain the running direction of the next step of the running point.

Step 704: Determine whether the running direction and the running direction of the running point are the same; if they are the same, perform step 705; if they are not the same, perform step 709 to determine whether the anchor point B has been reached.

If the selected running direction is different from the running direction, and the running point reaches the anchor point position B, step 710 is executed to end the running point process, that is, the moving process of the dividing point is ended; if the selected running direction is different from the running direction, And if the running point does not reach the anchor point position B, return to step 702.

Further, in the embodiment of the present application, in order to compensate for the error of the running point in the process of moving along the pixel gradient map, step 705 may be performed.

Step 705: Determine the drift direction.

For example, if the running point runs in from the eo direction and runs out from the oa direction, the three positions a, a1, and a2 are used as alternate lateral drift directions. For example, as shown in Figure 12(a), you can calculate the o1a1 direction gradient and the o2a2 direction gradient as shown in Figure 12(b) to exclude the position opposite to the oa direction gradient, and choose from the remaining positions The position where the absolute value of the gradient is the largest is the drift direction.

Step 706: Determine whether it is a continuous same-directional drift; if it is a continuous same-directional drift, execute step 708 to cancel the drift; if it is not a continuous same-directional drift, execute step 707 to execute the drift.

Exemplarily, the aforementioned continuous drift in the same direction means that when the running point executes the aforementioned multiple preset step lengths, the moving direction of each step length moves in the aforementioned drift direction. At this time, in order to ensure that the running point can reach the target anchor Point position (ie anchor point position B), then drift can be cancelled, and the running direction of the running point can be re-determined through the above-mentioned method one or the above-mentioned method two.

It should be understood that the above examples are intended to help those skilled in the art understand the embodiments of the present application, and are not intended to limit the embodiments of the present application to the specific numerical values or specific scenarios illustrated. Those skilled in the art can obviously make various equivalent modifications or changes based on the above examples given, and such modifications or changes also fall within the scope of the embodiments of the present application.

The image segmentation method of the embodiment of the present application is described in detail above with reference to Figs. 1 to 13, and the device embodiment of the present application will be described in detail below with reference to Figs. 14 to 16. It should be understood that the image segmentation device in the embodiment of the present application can execute the various image segmentation methods of the foregoing embodiment of the present application, that is, the specific working process of the following various products, and the corresponding process in the foregoing method embodiment may be referred to.

FIG. 14 is a schematic block diagram of an image segmentation device provided by an embodiment of the present application. It should be understood that the image segmentation apparatus 800 can execute the image segmentation methods shown in FIG. 6 and FIG. 8 to FIG. 13. The image segmentation device 800 includes a detection unit 810 and a processing unit 820.

Wherein, the detecting unit 810 is configured to detect the first operation of the user to manually mark the anchor point in the first image, wherein the anchor point includes a start anchor point and a target anchor point; and the user indicates that the user is detected to automatically segment the anchor point. The second operation of the first image; the processing unit 820 is configured to display a second image on the display screen in response to the second operation, the second image being the first image after the second operation Obtained image, the second image includes a dividing line between the starting anchor point and the target anchor point, wherein the dividing line is a dividing line in the first image with the starting anchor point and the starting anchor point. The anchor point is the starting position and the target anchor point is moved to the target position.

Optionally, as an embodiment, the dividing line coincides with the dividing line in the ridge direction in the pixel gradient map of the first image.

Optionally, as an embodiment, the processing unit 820 is specifically configured to:

Search for a mask image and a secant image corresponding to the first image according to the first image, wherein the mask image is used to represent different objects in the first image, and the secant image is used to represent Boundaries of different objects in the first image; displaying the superimposed image of the first image, the mask image, and the secant image on the display screen.

Optionally, as an embodiment, the detection unit 810 is further configured to:

A third operation instructed by the user to perform segmentation processing on the second image through the mask image or the secant image is detected.

Optionally, as an embodiment, the second image is obtained according to a pixel gradient map of the first image and an anchor point pulling model, and the anchor point pulling model is used to indicate the moving direction of the boundary point.

Optionally, as an embodiment, when the boundary point is currently located in an area without ridges in the pixel gradient map, the moving direction of the boundary point is based on the difference between the first line and the second line The included angle is determined, wherein the first straight line refers to the straight line where each of the eight azimuths is located, and the second line refers to the line between the current position of the dividing point and the target anchor point Connect; or

When the dividing point is currently located in an area with ridges and bifurcations in the pixel gradient map, the moving direction of the dividing point is based on the angle between the first line and the second line and the distance parameter Determined, wherein the first straight line refers to the straight line where each of the eight directions is located, and the second line refers to the line between the current position of the dividing point and the target anchor point , The distance parameter refers to the distance between the current position of the boundary point and the target anchor point; or

Optionally, as an embodiment, in the case that the moving direction of the dividing point at the previous moment is the same as the moving direction of the dividing point at the current moment, the moving direction of the dividing point is based on the absolute value of the gradient of different orientations. definite.

Optionally, as an embodiment, the anchor point is obtained by optimizing an initial anchor point, where the initial anchor point is an anchor point manually marked by the user in the first image.

FIG. 15 is a schematic block diagram of an image segmentation device provided by an embodiment of the present application. It should be understood that the image segmentation device 900 can perform the image segmentation methods shown in FIGS. 2 to 13. The image segmentation device 900 includes: an acquisition unit 910 and a processing unit 920.

Wherein, the acquiring unit 910 is configured to acquire the first image and the position information of the anchor points in the first image, wherein the anchor points include a starting anchor point and a target anchor point; the processing unit 920 is configured to obtain information according to the first image Image and the anchor point to obtain a second image, where the second image is an image obtained after image segmentation of the first image, and the second image includes the starting anchor point and the target anchor A dividing line between points, the dividing line is obtained by moving a dividing point in the pixel gradient map of the first image with the starting anchor point as the starting position and the target anchor point as the target position .

Optionally, as an embodiment, the dividing line coincides with the dividing line in the ridge direction in the pixel gradient map.

Optionally, as an embodiment, the processing unit 920 is specifically configured to:

The second image is obtained according to the pixel gradient map, the anchor point, and the anchor point pulling model, wherein the anchor point pulling model is used to indicate the moving direction of the boundary point.

Optionally, as an embodiment, if the boundary point is currently located in an area without ridges in the pixel gradient map, the moving direction of the boundary point is based on the difference between the first line and the second line The included angle is determined, wherein the first straight line refers to the straight line where each of the eight azimuths is located, and the second line refers to the line between the current position of the dividing point and the target anchor point Connection; or,

If the dividing point is currently located in an area with ridges and bifurcations in the pixel gradient map, the moving direction of the dividing point is determined according to the angle between the first line and the second line and the distance parameter , Wherein the first straight line refers to the straight line where each of the eight directions is located, and the second line refers to the line between the current position of the dividing point and the target anchor point, The distance parameter refers to the distance between the current position of the boundary point and the target anchor point; or,

Optionally, as an embodiment, the processing unit is further configured to:

If the movement direction of the boundary point at the previous moment is the same as the movement direction of the current moment, the movement direction of the boundary point is determined according to the absolute value of the gradient of different moving directions.

It should be noted that the above-mentioned image segmentation device 800 and image segmentation device 900 are embodied in the form of functional units. The term "unit" herein can be implemented in the form of software and/or hardware, which is not specifically limited.

For example, a "unit" may be a software program, a hardware circuit, or a combination of the two that realizes the above-mentioned functions. The hardware circuit may include an application specific integrated circuit (ASIC), an electronic circuit, and a processor for executing one or more software or firmware programs (such as a shared processor, a dedicated processor, or a group processor). Etc.) and memory, merged logic circuits and/or other suitable components that support the described functions.

Therefore, the units of the examples described in the embodiments of the present application can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether these functions are executed by hardware or software depends on the specific application and design constraint conditions of the technical solution. Professionals and technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered beyond the scope of this application.

As shown in FIG. 16, the image segmentation apparatus 1000 (the image segmentation apparatus 1000 may specifically be a computer device) includes a memory 1001, a processor 1002, a communication interface 1003, and a bus 1004. Among them, the memory 1001, the processor 1002, and the communication interface 1003 implement communication connections between each other through the bus 1004.

The memory 1001 may be a read only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1001 may store a program. When the program stored in the memory 1001 is executed by the processor 1002, the processor 1002 is configured to execute each step of the image segmentation method of the embodiment of the present application, for example, execute each of the steps shown in FIGS. 2 to 13 step.

It should be understood that the image segmentation apparatus shown in the embodiment of the present application may be a server, for example, it may be a server in the cloud, or may also be a chip configured in a server in the cloud.

The processor 1002 may adopt a general central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), or one or more integrated circuits for executing related programs to realize the The image segmentation method of the application method embodiment.

The processor 1002 may also be an integrated circuit chip with signal processing capability. In the implementation process, each step of the image segmentation method of the present application can be completed by an integrated logic circuit of hardware in the processor 1002 or instructions in the form of software.

The aforementioned processor 1002 may also be a general-purpose processor, a digital signal processing (digital signal processing, DSP), an application specific integrated circuit (ASIC), a ready-made programmable gate array (field programmable gate array, FPGA) or other programmable logic devices, Discrete gates or transistor logic devices, discrete hardware components. The methods, steps, and logical block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like. The steps of the method disclosed in the embodiments of the present application may be directly embodied as being executed and completed by a hardware decoding processor, or executed and completed by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, registers. The storage medium is located in the memory 1001, and the processor 1002 reads the information in the memory 1001, and combines its hardware to complete the functions required by the units included in the image segmentation device shown in FIG. 14 or FIG. 15 in the implementation of this application, or execute The image segmentation method shown in FIG. 2 to FIG. 13 of the method embodiment of the present application.

The communication interface 1003 uses a transceiver device such as but not limited to a transceiver to implement communication between the image segmentation device 1200 and other devices or a communication network.

The bus 1004 may include a path for transferring information between various components of the image segmentation device 1000 (for example, the memory 1001, the processor 1002, and the communication interface 1003).

It should be noted that although the above-mentioned image segmentation device 1000 only shows a memory, a processor, and a communication interface, in the specific implementation process, those skilled in the art should understand that the image segmentation device 1000 may also include other necessary for normal operation. Device. At the same time, according to specific needs, those skilled in the art should understand that the above-mentioned image segmentation apparatus 1000 may also include hardware devices that implement other additional functions.

In addition, those skilled in the art should understand that the above-mentioned image segmentation device 1000 may also only include the necessary components for implementing the embodiments of the present application, and not necessarily include all the components shown in FIG. 16.

It should be understood that the term "and/or" in this text is only an association relationship describing the associated objects, indicating that there can be three types of relationships, for example, A and/or B, which can mean: A alone exists, and both A and B exist. , There are three cases of B alone. In addition, the character "/" in this text generally indicates that the associated objects before and after are in an "or" relationship.

It should be understood that in the various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not correspond to the embodiments of the present application. The implementation process constitutes any limitation.

A person of ordinary skill in the art may realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether these functions are executed by hardware or software depends on the specific application and design constraint conditions of the technical solution. Professionals and technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered beyond the scope of this application.

Those skilled in the art can clearly understand that, for the convenience and conciseness of description, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed system, device, and method can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components may be combined It can be integrated into another system, or some features can be ignored or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

In addition, the functional modules in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.

If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solution of the present application essentially or the part that contributes to the existing technology or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (read-only memory, ROM), random access memory (random access memory, RAM), magnetic disks or optical disks and other media that can store program codes. .

The above are only specific implementations of this application, but the protection scope of this application is not limited to this. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in this application. Should be covered within the scope of protection of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

An image segmentation method, characterized in that it is applied to a terminal device with a display screen, and includes:

A first operation of manually marking an anchor point in the first image by the user is detected, where the anchor point includes a starting anchor point and a target anchor point;

Detecting a second operation instructed by the user to automatically segment the first image;

In response to the second operation, displaying a second image on the display screen,

Wherein, the second image is an image obtained after the first image is subjected to the second operation, and the second image includes a dividing line between the starting anchor point and the target anchor point, and the The dividing line is obtained by moving the dividing point in the first image with the starting anchor point as the starting position and the target anchor point as the target position.
The image segmentation method according to claim 1, wherein the segmentation line overlaps with the segmentation line in the ridge direction of the pixel gradient map of the first image.
The image segmentation method according to claim 1 or 2, wherein the displaying the second image on the display screen in response to the second operation comprises:

Search for a mask image and a secant image corresponding to the first image according to the first image, wherein the mask image is used to represent different objects in the first image, and the secant image is used to represent The boundaries of different objects in the first image;

The superimposed image of the first image, the mask image, and the secant image is displayed on the display screen.
5. The image segmentation method of claim 3, further comprising:

A third operation instructed by the user to process the second image through the mask image or the secant image is detected.
The image segmentation method according to any one of claims 1 to 4, wherein the second image is obtained according to a pixel gradient map of the first image and an anchor point pulling model, and the anchor point pulling The model is used to indicate the direction of movement of the dividing point.
The image segmentation method according to claim 5, wherein when the boundary point is currently located in an area without ridges in the pixel gradient map, the moving direction of the boundary point is based on the first straight line and the second The angle between the two straight lines is determined, wherein the first straight line refers to the straight line where each of the eight orientations is located, and the second straight line refers to the current position of the boundary point and the target anchor The line between points; or

When the dividing point is currently located in an area with ridges and bifurcations in the pixel gradient map, the moving direction of the dividing point is based on the angle between the first line and the second line and the distance parameter Determined, wherein the first straight line refers to a straight line where each of the eight azimuths is located, and the second straight line refers to a line between the current position of the dividing point and the target anchor point, The distance parameter refers to the distance between the current position of the boundary point and the target anchor point; or

When the boundary point is currently located in the pixel gradient map with a ridge and no bifurcation area, the moving direction of the boundary point is determined according to the ridge direction of the pixel gradient map.
The image segmentation method according to claim 6, wherein when the moving direction of the dividing point at the previous time is the same as the moving direction of the dividing point at the current time, the moving direction of the dividing point is based on The absolute value of the gradient in different directions is determined.
The image segmentation method according to any one of claims 1 to 7, wherein the anchor point is obtained by optimizing an initial anchor point, wherein the initial anchor point is the user's position in the first image Manually marked anchor points.
An image segmentation method, characterized in that it includes:

Acquiring a first image and position information of an anchor point in the first image, wherein the anchor point includes a starting anchor point and a target anchor point;

According to the first image and the anchor point, a second image is obtained, wherein the second image is an image obtained after the first image is subjected to image segmentation processing, and the second image includes the starting anchor A dividing line between a point and the target anchor point, the dividing line is a dividing line in the pixel gradient map of the first image with the starting anchor point as the starting position and the target anchor point It is obtained by moving the target position.
9. The image segmentation method of claim 9, wherein the segmentation line coincides with the segmentation line in the ridge direction of the pixel gradient map.
The image segmentation method according to claim 9 or 10, wherein the obtaining the second image according to the first image and the anchor point comprises:

The second image is obtained according to the pixel gradient map, the anchor point, and the anchor point pulling model, wherein the anchor point pulling model is used to indicate the moving direction of the boundary point.
The image segmentation method of claim 11, wherein if the boundary point is currently located in an area without ridges in the pixel gradient map, the moving direction of the boundary point is based on the first line and the second line. The angle between the two lines is determined, wherein the first straight line refers to the straight line at which each of the eight azimuths is located, and the second straight line refers to the current position of the dividing point and the target The line between anchor points; or,

If the dividing point is currently located in an area with ridges and bifurcations in the pixel gradient map, the moving direction of the dividing point is determined according to the angle between the first line and the second line and the distance parameter , Wherein the first straight line refers to the straight line where each of the eight azimuths is located, and the second straight line refers to the line between the current position of the dividing point and the target anchor point, so The distance parameter refers to the distance between the current position of the boundary point and the target anchor point; or,

If the boundary point is currently located in the pixel gradient map with a ridge and no bifurcation area, the moving direction of the boundary point is determined according to the ridge direction of the pixel gradient map.
The image segmentation method of claim 12, further comprising:

If the movement direction of the boundary point at the previous moment is the same as the movement direction of the current moment, the movement direction of the boundary point is determined according to the absolute value of the gradient of different moving directions.
The image segmentation method according to any one of claims 9 to 13, wherein the anchor point is obtained by optimizing an initial anchor point, wherein the initial anchor point is the user's position in the first image Manually marked anchor points.
An image segmentation device, characterized in that the image segmentation device has a display screen and includes:

The detection unit is configured to detect the first operation of the user to manually mark the anchor point in the first image, wherein the anchor point includes a start anchor point and a target anchor point; and the first operation is automatically divided upon detection of the user's instruction The second operation of the image;

The processing unit is configured to display a second image on the display screen in response to the second operation,

Wherein, the second image is an image obtained after the first image is subjected to the second operation, and the second image includes a dividing line between the starting anchor point and the target anchor point, and the The dividing line is obtained by moving the dividing point in the first image with the starting anchor point as the starting position and the target anchor point as the target position.
The image segmentation device according to claim 15, wherein the segmentation line coincides with the segmentation line in the ridge direction of the pixel gradient map of the first image.
The image segmentation device according to claim 15 or 16, wherein the processing unit is specifically configured to:

Search for a mask image and a secant image corresponding to the first image according to the first image, wherein the mask image is used to represent different objects in the first image, and the secant image is used to represent The boundaries of different objects in the first image;

The superimposed image of the first image, the mask image, and the secant image is displayed on the display screen.
18. The image segmentation device of claim 17, wherein the detection unit is further configured to:

A third operation instructed by the user to process the second image through the mask image or the secant image is detected.
The image segmentation device according to any one of claims 15 to 18, wherein the second image is obtained according to a pixel gradient map of the first image and an anchor point pulling model, and the anchor point pulling The model is used to indicate the direction of movement of the dividing point.
The image segmentation device according to claim 19, wherein when the boundary point is currently located in an area without ridges in the pixel gradient map, the moving direction of the boundary point is based on the first line and the second line The angle between the two lines is determined, wherein the first straight line refers to the straight line at which each of the eight azimuths is located, and the second straight line refers to the current position of the dividing point and the target The line between anchor points; or

When the dividing point is currently located in an area with ridges and bifurcations in the pixel gradient map, the moving direction of the dividing point is based on the angle between the first line and the second line and the distance parameter Determined, wherein the first straight line refers to a straight line where each of the eight azimuths is located, and the second straight line refers to a line between the current position of the dividing point and the target anchor point, The distance parameter refers to the distance between the current position of the boundary point and the target anchor point; or

When the boundary point is currently located in the pixel gradient map with a ridge and no bifurcation area, the moving direction of the boundary point is determined according to the ridge direction of the pixel gradient map.
The image segmentation device according to claim 20, wherein when the moving direction of the dividing point at the previous time is the same as the moving direction of the dividing point at the current time, the moving direction of the dividing point is based on The absolute value of the gradient in different directions is determined.
The image segmentation device according to any one of claims 15 to 21, wherein the anchor point is obtained by optimizing an initial anchor point, wherein the initial anchor point is the user's position in the first image Manually marked anchor points.
An image segmentation device, characterized in that it comprises:

An obtaining unit, configured to obtain a first image and position information of an anchor point in the first image, wherein the anchor point includes a start anchor point and a target anchor point;

The processing unit is configured to obtain a second image according to the first image and the anchor point, where the second image is an image obtained after the first image is subjected to image segmentation processing, and the second image includes The dividing line between the starting anchor point and the target anchor point, the dividing line is a dividing line in the pixel gradient map of the first image with the starting anchor point as the starting position and The target anchor point is obtained by moving the target position.
22. The image segmentation device according to claim 23, wherein the segmentation line coincides with the segmentation line in the ridge direction of the pixel gradient map.
The image segmentation device according to claim 23 or 24, wherein the processing unit is specifically configured to:

The second image is obtained according to the pixel gradient map, the anchor point and the anchor point pulling model, wherein the anchor point pulling model is used to indicate the moving direction of the boundary point.
The image segmentation device of claim 25, wherein if the boundary point is currently located in an area without ridges in the pixel gradient map, the moving direction of the boundary point is based on the first line and the second line The angle between the two lines is determined, wherein the first line refers to the line between the starting anchor point and the target anchor point, and the second line refers to the boundary The line between the current position of the point and the target anchor point; or,

If the dividing point is currently located in an area with ridges and bifurcations in the pixel gradient map, the moving direction of the dividing point is determined according to the angle between the first line and the second line and the distance parameter , Wherein the first straight line refers to the straight line where each of the eight azimuths is located, and the second straight line refers to the line between the current position of the dividing point and the target anchor point, so The distance parameter refers to the distance between the current position of the boundary point and the target anchor point; or,

If the boundary point is currently located in the pixel gradient map with a ridge and no bifurcation area, the moving direction of the boundary point is determined according to the ridge direction of the pixel gradient map.
The image segmentation device according to claim 26, wherein the processing unit is further configured to:

If the movement direction of the boundary point at the previous moment is the same as the movement direction of the current moment, the movement direction of the boundary point is determined according to the absolute value of the gradient of different moving directions.
The image segmentation device according to any one of claims 23 to 27, wherein the anchor point is obtained by optimizing an initial anchor point, wherein the initial anchor point is the user's position in the first image Manually marked anchor points.
A computer-readable storage medium, wherein the computer-readable medium stores program code for device execution, and the program code includes a program code for executing any one of claims 1 to 8 or 9 to 14. Image segmentation method.