WO2022105622A1

WO2022105622A1 - Image segmentation method and apparatus, readable medium, and electronic device

Info

Publication number: WO2022105622A1
Application number: PCT/CN2021/128958
Authority: WO
Inventors: 喻冬东; 王长虎
Original assignee: 北京有竹居网络技术有限公司
Priority date: 2020-11-18
Filing date: 2021-11-05
Publication date: 2022-05-27
Also published as: CN112418232A

Abstract

An image segmentation method and apparatus, a readable medium, and an electronic device. The method comprises: acquiring predicted object center point feature information of each of multiple pixels in an image to be segmented, wherein the predicted object center point feature information indicating a level of reliability that the pixel is the center point of an object; determining center point position information of an object in the image according to the predicted object center point feature information of each pixel; and performing image segmentation on the image according to the center point position information. Since the method takes into consideration central point position information of an object in an image to be segmented, a region where the object is located is emphasized and can be distinctly distinguished from a background region, such that the object in the foreground can be accurately separated when the image is subjected to image segmentation, thereby effectively reducing interference of the background region, and improving the accuracy of image segmentation and segmentation performance.

Description

Image segmentation method, device, readable medium and electronic device

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on the CN application number 202011295274.X and the filing date is Nov. 18, 2020, and claims its priority, and the disclosure content of this CN application is hereby incorporated into this application as a whole.

technical field

The present disclosure relates to the technical field of image processing, and in particular, to an image segmentation method, apparatus, readable medium, and electronic device.

Background technique

Image segmentation has important applications in the field of image processing technology. Image segmentation refers to the process of dividing an image into several regions with similar properties, that is, dividing the image into several disjoint regions.

There are usually more prominent foreground objects in the image, and image segmentation can segment the area where the foreground object is located from the background area.

SUMMARY OF THE INVENTION

This Summary is provided to introduce concepts in a simplified form that are described in detail in the Detailed Description section that follows. This summary section is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to be used to limit the scope of the claimed technical solution.

In a first aspect, the present disclosure provides an image segmentation method, the method comprising: acquiring predicted object center point feature information of each pixel of a plurality of pixels in an image to be segmented, where the predicted object center point feature information is used to represent The pixel point is the reliability of the center point of the object; according to the feature information of the predicted object center point of each pixel point, determine the center point position information of the object in the image to be segmented; The to-be-segmented image is subjected to image segmentation.

In a second aspect, the present disclosure provides an image segmentation device, the device comprising: an acquisition module for acquiring predicted object center point feature information of each pixel of a plurality of pixels in an image to be segmented, the predicted object center point The feature information is used to characterize the reliability that the pixel point is the center point of the object; the determination module is used to determine the center point position information of the object in the image to be segmented according to the feature information of the predicted object center point of each pixel point ; an image segmentation module, configured to perform image segmentation on the to-be-segmented image according to the center point position information.

In a third aspect, the present disclosure provides a non-transitory computer-readable medium on which a computer program is stored, and when the program is executed by a processing apparatus, implements the steps of the method provided in the first aspect of the present disclosure.

In a fourth aspect, the present disclosure provides an electronic device, including: a storage device on which a computer program is stored; and a processing device for executing the computer program in the storage device, so as to implement the computer program provided in the first aspect of the present disclosure. the steps of the method.

In a fifth aspect, the present disclosure provides a computer program comprising: instructions that, when executed by a processor, cause the processor to perform the image segmentation method of the first aspect.

In a sixth aspect, the present disclosure provides a computer program product comprising instructions that, when executed by a processor, cause the processor to perform the image segmentation method of the first aspect.

Other features and advantages of the present disclosure will be described in detail in the detailed description that follows.

Description of drawings

The above and other features, advantages and aspects of various embodiments of the present disclosure will become more apparent when taken in conjunction with the accompanying drawings and with reference to the following detailed description. Throughout the drawings, the same or similar reference numbers refer to the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale. In the attached image:

Fig. 1 is a flowchart of an image segmentation method according to an exemplary embodiment.

Fig. 2 is a flow chart of a method for determining center point position information of an object in an image to be segmented according to an exemplary embodiment.

Fig. 3 is a flowchart of an image segmentation method according to another exemplary embodiment.

Fig. 4 is a block diagram of an image segmentation apparatus according to an exemplary embodiment.

Fig. 5 is a schematic structural diagram of an electronic device according to an exemplary embodiment.

Detailed ways

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but rather are provided for the purpose of A more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes, and are not intended to limit the protection scope of the present disclosure.

It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and/or in parallel. Furthermore, method embodiments may include additional steps and/or omit performing the illustrated steps. The scope of the present disclosure is not limited in this regard.

As used herein, the term "including" and variations thereof are open-ended inclusions, ie, "including but not limited to". The term "based on" is "based at least in part on." The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions of other terms will be given in the description below.

It should be noted that concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or interdependence.

It should be noted that the modifications of "a" and "a plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless the context clearly indicates otherwise, they should be understood as "one or a plurality of". multiple".

The names of messages or information exchanged between multiple devices in the embodiments of the present disclosure are only for illustrative purposes, and are not intended to limit the scope of these messages or information.

The inventor of the present disclosure found that in the related art, when an image is segmented, it is easily interfered by the background area in the image, so that the segmentation effect of the objects in the image is not good, and when there are situations such as objects being partially occluded, the It will make the image segmentation result inaccurate.

In view of this, embodiments of the present disclosure provide an image segmentation method to reduce interference of background regions in an image.

Fig. 1 is a flow chart of an image segmentation method according to an exemplary embodiment. The method can be applied to an electronic device with processing capability, such as a terminal or a server. As shown in Fig. 1 , the method may include step S101 ~ S103.

In step S101, the feature information of the predicted object center point of each pixel point of a plurality of pixel points in the image to be divided is acquired.

The image to be segmented may be a pre-stored image, an image collected in real time, or an image frame in a video, which is not specifically limited in the present disclosure. There may be one or more prominent foreground objects in the image to be segmented. Objects may include human bodies and objects. For example, there are one or more vehicles or one or more people in the image to be segmented. Segmentation can divide the area where the foreground object in the image is located and the background area in the image.

The feature information of the predicted object center point of the pixel point can be used to represent the reliability of the pixel point as the object center point, that is, the possibility or probability that the pixel point is the object center point. For example, an object center point prediction module may be integrated into the electronic device, and the object center point prediction module may generate a heatmap corresponding to the image to be segmented. The information of each pixel in the figure can be used to represent the feature information of the predicted object center point of the corresponding pixel in the image to be segmented. Among them, if the possibility of the pixel point being the center point of the object is high, the feature information of the predicted object center point of the pixel point is relatively high; The center point feature information is relatively low. The present disclosure does not limit the specific representation of the feature information of the predicted object center point, for example, it can be represented by a value between 0 and 1.

In step S102, according to the feature information of the predicted object center point of each pixel point, the center point position information of the object in the image to be segmented is determined.

The position information of the center point of the object may refer to the coordinate information of the center point of the object in the image to be segmented.

It is worth noting that in the present disclosure, when performing image segmentation on the image to be segmented, there is no need to pre-designate a specific object, nor is it only necessary to determine the center point position information of a specific object. The feature information of the predicted object center point of each pixel point can simultaneously determine the respective center point position information of the multiple objects, so that when performing image segmentation, the regions where the multiple objects are located can be accurately segmented.

In step S103, image segmentation is performed on the image to be segmented according to the center point position information.

Considering the position information of the center point of the object, the area where the object is located can be more prominent in the image to be segmented, and the difference between it and the background area is more significant, so it is easier to segment the foreground object when the image to be segmented is segmented. Effectively reduce the interference in the background area. Moreover, even if the object is partially occluded, the center point position information of the object will not be affected by the surface features of the object (such as color, size, etc.). In addition, even if other similar objects appear in the image, the center point position information of the object is not It will be affected by other similar objects. Therefore, image segmentation based on the center point position information of the object can accurately segment the area where the object is located, and improve the accuracy and segmentation effect of image segmentation.

Through the above technical solution, first obtain the feature information of the predicted object center point of each pixel in the image to be segmented, and according to the predicted object center point feature information of each pixel point, the center point position information of the object in the image to be segmented can be determined, and then according to the object center point feature information The position information of the center point is used to segment the image to be segmented. In this way, considering the position information of the center point of the object, the area where the object is located can be made more prominent in the image to be segmented, and the difference between it and the background area is more significant, so that the foreground can be more accurately segmented when the image to be segmented is segmented. The object is segmented to effectively reduce the interference of the background area. Moreover, even if the object is partially occluded or other similar objects appear in the image, the center point position information of the object will not be affected. Accurate segmentation can improve the accuracy and segmentation effect of image segmentation.

In an optional embodiment, in S102, according to the predicted object center point feature information of each pixel point, determining the center point position information of the object in the image to be segmented may include: The position information of the pixel corresponding to the local maximum value in the information is determined as the position information of the center point of the object. Wherein, regardless of whether the image to be segmented is an individual image or an image frame in a video, this embodiment can be used to determine the center point position information of the object.

Among them, the pixel point corresponding to the local maximum value may refer to the pixel point with the largest predicted feature information of the center point of the object among the pixel points in the area where the object is located. If there are multiple objects in the to-be-segmented image, in the pixels of each object's area, there are points with relatively large feature information of the predicted object center point, so multiple local maximum points can be determined, and different local maximum points can be determined. Large value points correspond to the center points of different objects. If there is an object in the image to be segmented, a local maximum point can be determined, and the local maximum point is also the pixel with the largest feature information of the predicted object center point, and the position information of the pixel point can be determined as the point to be The position information of the center point of the object in the segmented image.

In this way, the feature information of the predicted object center point of a pixel point can be used to represent the possibility that the pixel point is the object center point, and the pixel point corresponding to the local maximum value in the predicted object center point feature information of multiple pixel points is determined as the object's center point. For the center point, when there are multiple objects in the image to be segmented, the position information of the respective center points of the multiple objects can be accurately determined.

In another optional implementation manner, when the image to be segmented is an image frame in a video, the position information of the center point of the object in the image to be segmented may be determined in combination with a reference image frame in the video. FIG. 2 is a flow chart of a method for determining center point position information of an object in an image to be segmented according to this embodiment. As shown in FIG. 2 , the above-mentioned step S102 may include steps S1021 to S1023 .

In step S1021, the predicted center point of the object in the image to be segmented is determined according to the center point position information of the object in the reference image frame and the motion track information of the object.

The reference image frame may be an image frame in the video that is different from the image to be segmented. For example, the reference image frame may be the first image frame in the video, or may be the previous frame of the image to be divided in the video, which is not specifically limited in the present disclosure.

The image frames in the video have a certain continuity, and the motion track information of the object can include the object's motion direction information, moving speed, moving acceleration, etc. According to the object's motion track information, the moment in the video from the reference image frame can be determined. The distance and direction of movement of the object at the moment when the image is to be segmented. Wherein, the center point position information of the object in the reference image frame may be predetermined, and the predicted center point of the object in the to-be-segmented image can be determined according to the center point position information of the object in the reference image frame and the motion track information of the object. The predicted center point is the possible center point position of the object that is initially determined according to the motion trajectory of the object.

In step S1022, a preset number of pixels with the largest feature information of the predicted object center point in the region where the object is located in the image to be segmented is determined.

There are multiple pixels in the area where the object is located in the image to be segmented, and the preset number of pixels with the largest predicted object center point feature information among the multiple pixel points can be determined, that is, the predicted object center point feature information is ranked in the top K positions , the preset number of pixels is more likely to be the center point of the object. Wherein K may represent a preset number, and K is greater than or equal to 1, and its value is not specifically limited in the present disclosure.

In S1023, the position information of the pixel point with the closest distance to the prediction center point among the preset number of pixel points is determined as the center point position information.

For example, for each pixel in a preset number of pixels, the distance between the pixel and the prediction center point can be calculated, and the pixel point with the closest distance to the prediction center point is not only the possibility of the object center point. The distance between the prediction center point determined according to the motion trajectory of the object is relatively high, so the position information of the pixel point can be used as the center point position information of the object in the image to be segmented.

It is worth noting that the present disclosure does not specifically limit the execution order of S1021 and S1022, S1022 may be executed first and then S1021 may be executed, or both may be executed in parallel, and FIG. 2 is only an example.

Through the above technical solution, in the case where the image to be segmented is an image frame in a video, the reference image frame in the video and the motion trajectory information of the object can be combined to determine the center point information of the object in the image to be segmented. The motion track information can make the determined position information of the center point of the object more accurate.

Fig. 3 is a flowchart of an image segmentation method according to another exemplary embodiment. As shown in Fig. 3 , the method may include steps S301 to S304, wherein the above step S103 may include S303 and S304.

In step S301 (101), the feature information of the predicted object center point of each pixel point of a plurality of pixel points in the image to be divided is acquired. For the implementation of this step S301, reference may be made to step S101.

In step S302 (102), according to the feature information of the predicted object center point of each pixel point, the center point position information of the object in the image to be segmented is determined. In this step, the method of determining the position information of the center point of the object provided by any embodiment of the present disclosure may be used.

In step S303, according to the center point position information and the predicted object center point feature information, the target object center point feature information of each pixel point of the plurality of pixels in the image to be segmented is determined.

Among them, the center point feature information of the target object of the pixel corresponding to the center point position information is greater than the predicted object center point feature information of the pixel point, and the pixel point corresponding to the center point position information is the center point of the object, that is, to further improve the center point of the object The feature information of the point, for example, the Gaussian algorithm can be used to increase the feature information of the center point of the object.

In step S304, image segmentation is performed on the image to be segmented according to the feature information of the center point of the target object.

Since the feature information of the target object center point of the pixel corresponding to the center point position information is greater than the predicted object center point feature information of the pixel, the feature information of the object center point is higher, so the image segmentation is performed according to the target object center point feature information, The foreground objects can be more accurately segmented, and the accuracy and segmentation effect of image segmentation can be further improved.

This step S304 may further include: acquiring preset feature information of each pixel of the multiple pixels in the image to be segmented; information to perform image segmentation on the image to be segmented.

The preset feature information may include at least one of image semantic feature information and image edge feature information. Among them, each pixel in the image to be segmented can be classified to determine the semantic label to which each pixel belongs, and the image semantic feature information of pixels belonging to the same semantic label can be the same. The edge of an image may refer to the part of the image with the most significant change in brightness or gray level, and the image edge feature information of pixels located in the edge part is relatively large.

Optionally, performing image segmentation on the to-be-segmented image according to the target object center point feature information and preset feature information of each pixel of the plurality of pixels in the to-be-segmented image may include: The target feature information of the target object center point and the preset feature information of each pixel point are used to determine the target feature information of the pixel point; according to the target feature information of each pixel point of the plurality of pixel points, image segmentation is performed on the image to be segmented.

Wherein, when the preset feature information includes one of image semantic feature information and image edge feature information, the feature information of the center point of the target object of the pixel point in the image to be segmented and the preset feature information of the pixel point can be compared. The product is used as the target feature information of the pixel. When the preset feature information includes image semantic feature information and image edge feature information, the feature information of the target object center point of the pixel in the image to be segmented, the image semantic feature information of the pixel, and the image edge of the pixel can be The product of feature information is used as the target feature information of the pixel.

In the embodiment of the present disclosure, the feature information point of the target object center point of a pixel point is multiplied by the preset feature information of the pixel point. The target object center point feature information of the pixel points in the object center point area is relatively low, so multiplying the target object center point feature information of the pixel point with the preset feature information can make the target object feature information of the object center point higher. In this way, when the image is segmented according to the target feature information, the region where the object is located can be made more prominent in the image to be segmented, and the foreground object can be segmented from the image to be segmented more accurately.

Illustratively, an exemplary implementation of performing image segmentation on the image to be segmented according to the target feature information of each pixel of the plurality of pixels may be: dividing the image to be segmented and the target feature of each pixel of the plurality of pixels. The information is input into the image segmentation model to perform image segmentation on the image to be segmented by the image segmentation model.

The image segmentation model can be any network model, such as a fully convolutional network model. The image segmentation model may be pre-trained.

Among them, the target feature information of a pixel is obtained according to the feature information of the center point of the target object of the pixel, and the image semantic feature information and/or the image edge feature information. When the image segmentation model performs image segmentation on the image to be segmented, due to the object The target feature information of the center point is relatively higher, so the difference between the area where the object is located and the background area is more significant. According to the target feature information of each pixel point, the area where the object is located can be more accurately segmented, thereby improving image segmentation. Effects and image segmentation accuracy.

Based on the same inventive concept, the present disclosure also provides an image segmentation device. FIG. 4 is a block diagram of an image segmentation device according to an exemplary embodiment. As shown in FIG. 4 , the device 400 may include:

The obtaining module 401 is used to obtain the predicted object center point feature information of each pixel point of a plurality of pixel points in the image to be divided, the predicted object center point feature information is used to represent the reliability of the pixel point as the object center point ;

A determination module 402, configured to determine the center point position information of the object in the to-be-segmented image according to the feature information of the predicted object center point of each pixel;

The image segmentation module 403 is configured to perform image segmentation on the to-be-segmented image according to the center point position information.

Using the above-mentioned device, first obtain the feature information of the predicted object center point of each pixel point in the image to be segmented, and then determine the center point position information of the object in the image to be segmented according to the feature information of the predicted object center point of each pixel point. , and then perform image segmentation on the image to be segmented according to the position information of the center point of the object. In this way, considering the position information of the center point of the object, the area where the object is located can be made more prominent in the image to be segmented, and the difference between it and the background area is more significant, so that the foreground can be more accurately segmented when the image to be segmented is segmented. The object is segmented to effectively reduce the interference of the background area. Moreover, even if the object is partially occluded or other similar objects appear in the image, the center point position information of the object will not be affected. Accurate segmentation can improve the accuracy and segmentation effect of image segmentation.

Optionally, the image segmentation module 403 may include: a first determination sub-module, configured to determine a plurality of pixel points in the image to be segmented according to the center point position information and the predicted object center point feature information. The feature information of the center point of the target object of each pixel point, wherein the feature information of the center point of the target object of the pixel point corresponding to the position information of the center point is greater than the feature information of the center point of the predicted object of the pixel point; first The image segmentation sub-module is configured to perform image segmentation on the to-be-segmented image according to the feature information of the center point of the target object.

Optionally, the first image segmentation sub-module may include: an acquisition sub-module for acquiring preset feature information of each pixel of a plurality of pixels in the to-be-segmented image, where the preset feature information includes: at least one of image semantic feature information and image edge feature information; a second segmentation sub-module, used for the target object center point feature information of each pixel point in the image to be segmented and the Preset feature information, and perform image segmentation on the to-be-segmented image.

Optionally, the second segmentation sub-module may include: a second determination sub-module, which is configured to, according to the feature information of the center point of the target object and the Preset feature information to determine target feature information of the pixel, wherein, in the case that the preset feature information includes one of the image semantic feature information and the image edge feature information, the to-be-to-be-featured feature information is The product of the target object center point feature information of a pixel in the segmented image and the preset feature information of the pixel is taken as the target feature information of the pixel, where the preset feature information includes the image semantic feature. information and the image edge feature information, the product of the target object center point feature information of the pixel point in the to-be-segmented image, the image semantic feature information of the pixel point, and the image edge feature information of the pixel point , as the target feature information of the pixel point; a third segmentation sub-module, configured to perform image segmentation on the to-be-segmented image according to the target feature information of each pixel point of a plurality of pixel points.

Optionally, the third segmentation sub-module may include: an input sub-module for inputting the to-be-segmented image and the target feature information of each pixel into an image segmentation model, so as to divide the image through the image segmentation. The model performs image segmentation on the image to be segmented.

Optionally, the determining module 402 may include: a third determining sub-module, configured to determine the position information of the pixel corresponding to the local maximum value in the feature information of the predicted object center point of the plurality of pixel points as the location information of the center point.

Optionally, the to-be-segmented image is an image frame in a video; the determining module 402 may include: a fourth determining sub-module, configured to determine the object according to the center point position information of the object in the reference image frame and the object The motion trajectory information of the to-be-segmented image determines the predicted center point of the object in the to-be-segmented image, wherein the reference image frame is an image frame in the video that is different from the to-be-segmented image; the fifth determination sub-module uses In determining the area where the object is located in the to-be-segmented image, the preset number of pixels with the largest feature information of the center point of the predicted object; the sixth determination sub-module is used to determine the number of pixels in the preset number of pixels. The position information of the pixel with the closest distance to the prediction center point is determined as the center point position information.

Regarding the apparatus in the above-mentioned embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments of the related method, and will not be described in detail here.

Referring next to FIG. 5 , it shows a schematic structural diagram of an electronic device 500 suitable for implementing an embodiment of the present disclosure. Terminal devices in the embodiments of the present disclosure may include, but are not limited to, such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablets), PMPs (portable multimedia players), vehicle-mounted terminals (eg, mobile terminals such as in-vehicle navigation terminals), etc., and stationary terminals such as digital TVs, desktop computers, and the like. The electronic device shown in FIG. 5 is only an example, and should not impose any limitation on the function and scope of use of the embodiments of the present disclosure.

As shown in FIG. 5 , an electronic device 500 may include a processing device (eg, a central processing unit, a graphics processor, etc.) 501 that may be loaded into random access according to a program stored in a read only memory (ROM) 502 or from a storage device 508 Various appropriate actions and processes are executed by the programs in the memory (RAM) 503 . In the RAM 503, various programs and data required for the operation of the electronic device 500 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other through a bus 504. An input/output (I/O) interface 505 is also connected to bus 504 .

Typically, the following devices may be connected to the I/O interface 505: input devices 506 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; including, for example, a liquid crystal display (LCD), speakers, vibration An output device 507 such as a computer; a storage device 508 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 509 . Communication means 509 may allow electronic device 500 to communicate wirelessly or by wire with other devices to exchange data. While FIG. 5 shows electronic device 500 having various means, it should be understood that not all of the illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided.

In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer readable medium, the computer program containing program code for performing the method illustrated in the flowchart. In such an embodiment, the computer program may be downloaded and installed from the network via the communication device 509, or from the storage device 508, or from the ROM 502. When the computer program is executed by the processing apparatus 501, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are executed.

It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or a combination of any of the above. More specific examples of computer readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable Programmable read only memory (EPROM or flash memory), fiber optics, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. In this disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present disclosure, however, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave with computer-readable program code embodied thereon. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device . Program code embodied on a computer readable medium may be transmitted using any suitable medium including, but not limited to, electrical wire, optical fiber cable, RF (radio frequency), etc., or any suitable combination of the foregoing.

In some embodiments, the client and server can use any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol) to communicate, and can communicate with digital data in any form or medium Communication (eg, a communication network) interconnects. Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet (eg, the Internet), and peer-to-peer networks (eg, ad hoc peer-to-peer networks), as well as any currently known or future development network of.

The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; or may exist alone without being assembled into the electronic device.

The above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by the electronic device, the electronic device: obtains the predicted object center point feature information of each pixel in the image to be segmented, and the described The feature information of the predicted object center point is used to represent the reliability of the pixel point as the object center point; according to the predicted object center point feature information of each pixel point, the center point position information of the object in the image to be segmented is determined; Image segmentation is performed on the to-be-segmented image according to the center point position information.

Computer program code for performing operations of the present disclosure may be written in one or more programming languages, including but not limited to object-oriented programming languages—such as Java, Smalltalk, C++, and This includes conventional procedural programming languages - such as the "C" language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (eg, using an Internet service provider to via Internet connection).

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code that contains one or more logical functions for implementing the specified functions executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented in dedicated hardware-based systems that perform the specified functions or operations , or can be implemented in a combination of dedicated hardware and computer instructions.

The modules involved in the embodiments of the present disclosure may be implemented in software or hardware. Wherein, the name of the module does not constitute a limitation of the module itself under certain circumstances, for example, the acquisition module may also be described as a "central point feature information acquisition module".

The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), Systems on Chips (SOCs), Complex Programmable Logical Devices (CPLDs), etc.

In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in connection with the instruction execution system, apparatus or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), fiber optics, compact disk read only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination of the foregoing.

According to one or more embodiments of the present disclosure, Example 1 provides an image segmentation method, the method comprising: acquiring predicted object center point feature information of each pixel of a plurality of pixels in an image to be segmented, the prediction The feature information of the object center point is used to represent the reliability of the pixel point as the object center point; according to the predicted object center point feature information of each pixel point, the center point position information of the object in the to-be-segmented image is determined; The center point position information is used to perform image segmentation on the to-be-segmented image.

According to one or more embodiments of the present disclosure, Example 2 provides the method of Example 1, wherein performing image segmentation on the to-be-segmented image according to the center point position information includes: according to the center point position information and The predicted object center point feature information is to determine the target object center point feature information of each pixel of a plurality of pixels in the image to be segmented, wherein the target object center of the pixel corresponding to the center point position information The point feature information is greater than the feature information of the predicted object center point of the pixel point; image segmentation is performed on the to-be-segmented image according to the center point feature information of the target object.

According to one or more embodiments of the present disclosure, Example 3 provides the method of Example 2, wherein performing image segmentation on the image to be segmented according to the feature information of the center point of the target object includes: acquiring the image to be segmented The preset feature information of each pixel point of the multiple pixels in the image, the preset feature information includes at least one of image semantic feature information and image edge feature information; The target object center point feature information and the preset feature information of the pixel points are used to perform image segmentation on the to-be-segmented image.

According to one or more embodiments of the present disclosure, Example 4 provides the method of Example 3, wherein the target object center point feature information of each pixel point of a plurality of pixel points in the to-be-segmented image and the prediction method are provided in Example 4. Assuming feature information, performing image segmentation on the to-be-segmented image includes: determining the target object center point feature information and the preset feature information of each pixel point of a plurality of pixel points in the to-be-segmented image. The target feature information of the pixel point, wherein, in the case where the preset feature information includes one of the image semantic feature information and the image edge feature information, the target of the pixel point in the image to be segmented The product of the feature information of the object center point and the preset feature information of the pixel point is used as the target feature information of the pixel point, and the preset feature information includes the image semantic feature information and the image edge feature. In the case of information, the product of the target object center point feature information of the pixel points in the image to be segmented, the image semantic feature information of the pixel points and the image edge feature information of the pixel points is used as the pixel point. the target feature information; image segmentation is performed on the to-be-segmented image according to the target feature information of each pixel point of a plurality of pixel points.

According to one or more embodiments of the present disclosure, Example 5 provides the method of Example 4, wherein performing image segmentation on the to-be-segmented image according to the target feature information of each pixel includes: dividing the to-be-segmented image The image and the target feature information of each pixel point are input into the image segmentation model, so as to perform image segmentation on the to-be-segmented image through the image segmentation model.

According to one or more embodiments of the present disclosure, Example 6 provides the method of Example 1, wherein according to the feature information of the predicted object center point of each pixel point of the plurality of pixel points, determine the object in the image to be segmented. The center point position information includes: determining the position information of the pixel point corresponding to the local maximum value in the predicted object center point feature information of the plurality of pixel points as the center point position information.

According to one or more embodiments of the present disclosure, Example 7 provides the method of Example 1, wherein the image to be segmented is an image frame in a video; the predicted object center point according to each pixel point of a plurality of pixel points feature information, and determining the center point position information of the object in the image to be segmented includes: determining the center point location information of the object in the reference image frame and the motion track information of the object The predicted center point of the object, wherein the reference image frame is an image frame in the video that is different from the image to be segmented; determine the center point feature of the predicted object in the area where the object is located in the image to be segmented A preset number of pixels with the largest information; the position information of the pixel with the closest distance to the predicted center point among the preset number of pixels is determined as the center point position information.

According to one or more embodiments of the present disclosure, Example 8 provides an apparatus for image segmentation, the apparatus includes: an acquisition module configured to acquire feature information of predicted object center points of each pixel point of a plurality of pixel points in an image to be segmented , the feature information of the predicted object center point is used to represent the reliability of the pixel point as the object center point; the determination module is used to determine the image to be segmented according to the predicted object center point feature information of each pixel point The center point position information of the object in the middle; the image segmentation module is configured to perform image segmentation on the to-be-segmented image according to the center point position information.

According to one or more embodiments of the present disclosure, Example 9 provides a non-transitory computer-readable medium having stored thereon a computer program that, when executed by a processing apparatus, implements the method described in any one of Examples 1-7 A step of.

According to one or more embodiments of the present disclosure, Example 10 provides an electronic device, including: a storage device on which a computer program is stored; and a processing device for executing the computer program in the storage device to achieve The steps of the method of any of Examples 1-7.

According to some embodiments of the present disclosure, there is also provided a computer program comprising: instructions which, when executed by a processor, cause the processor to perform the image segmentation method as previously described.

According to some embodiments of the present disclosure, there is also provided a computer program product comprising instructions that, when executed by a processor, cause the processor to perform the image segmentation method as previously described.

The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, and should also cover, without departing from the above-mentioned disclosed concept, the technical solutions formed by the above-mentioned technical features or Other technical solutions formed by any combination of its equivalent features. For example, a technical solution is formed by replacing the above features with the technical features disclosed in the present disclosure (but not limited to) with similar functions.

Additionally, although operations are depicted in a particular order, this should not be construed as requiring that the operations be performed in the particular order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, although the above discussion contains several implementation-specific details, these should not be construed as limitations on the scope of the present disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.

Although the subject matter has been described in language specific to structural features and/or logical acts of method, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims. Regarding the apparatus in the above-mentioned embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be described in detail here.

Claims

An image segmentation method, comprising:

Obtaining predicted object center point feature information of each pixel point of a plurality of pixel points in the image to be segmented, where the predicted object center point feature information is used to characterize the reliability of the pixel point as an object center point;

According to the feature information of the predicted object center point of each pixel point, determine the center point position information of the object in the to-be-segmented image; and

Image segmentation is performed on the to-be-segmented image according to the center point position information.
The method according to claim 1, wherein the performing image segmentation on the to-be-segmented image according to the center point position information comprises:

According to the center point position information and the predicted object center point feature information, determine the target object center point feature information of each pixel point of the plurality of pixel points in the image to be segmented, wherein the center point position information The target object center point feature information of the corresponding pixel point is greater than the predicted object center point feature information of the pixel point; and

Image segmentation is performed on the to-be-segmented image according to the feature information of the center point of the target object.
The method according to claim 2, wherein the performing image segmentation on the to-be-segmented image according to the feature information of the center point of the target object comprises:

acquiring preset feature information of each pixel of the plurality of pixels in the image to be segmented, the preset feature information including at least one of image semantic feature information and image edge feature information; and

Image segmentation is performed on the to-be-segmented image according to the target object center point feature information and the preset feature information of each pixel of the plurality of pixels in the to-be-segmented image.
The method according to claim 3, wherein, according to the feature information of the target object center point and the preset feature information of each pixel point of the plurality of pixel points in the image to be segmented, the Image segmentation is performed on the image to be segmented, including:

According to the target object center point feature information and the preset feature information of each pixel point of the plurality of pixel points in the image to be divided, determine the target feature information of the pixel point; and

Image segmentation is performed on the image to be segmented according to the target feature information of each pixel point of the plurality of pixel points.
The method of claim 4, wherein,

When the preset feature information includes one of the image semantic feature information and the image edge feature information, compare the target object center point feature information of the pixels in the to-be-segmented image with the pixel points The product of the preset feature information of , as the target feature information of the pixel point; or

In the case that the preset feature information includes the image semantic feature information and the image edge feature information, the target object center point feature information of the pixels in the image to be segmented, the image semantic features of the pixels The product of the information and the image edge feature information of the pixel is taken as the target feature information of the pixel.
The method according to claim 4, wherein the performing image segmentation on the to-be-segmented image according to the target feature information of each pixel point of the plurality of pixel points comprises:

Inputting the image to be segmented and the target feature information of each pixel point of the plurality of pixel points into an image segmentation model, so as to perform image segmentation on the image to be segmented through the image segmentation model.
The method according to claim 1, wherein the determining the center point position information of the object in the to-be-segmented image according to the feature information of the predicted object center point of each pixel point comprises:

The position information of the pixel point corresponding to the local maximum value in the feature information of the predicted object center point of the plurality of pixel points is determined as the center point position information.
The method according to claim 1, wherein the to-be-segmented image is an image frame in a video;

The determining the center point position information of the object in the to-be-segmented image according to the feature information of the predicted object center point of each pixel, including:

Determine the predicted center point of the object in the to-be-segmented image according to the center point position information of the object and the motion track information of the object in the reference image frame, wherein the reference image frame is the same as that in the video. different image frames of the to-be-segmented images;

Determine the maximum preset number of pixels in the region where the object is located in the to-be-segmented image; and

The position information of the pixel point with the closest distance to the prediction center point among the preset number of pixel points is determined as the center point position information.
An image segmentation device, comprising:

an acquisition module, configured to acquire the predicted object center point feature information of each pixel of a plurality of pixels in the image to be segmented, where the predicted object center point feature information is used to characterize the reliability of the pixel as the object center point;

a determining module, configured to determine the center point position information of the object in the image to be segmented according to the feature information of the predicted object center point of each pixel; and

An image segmentation module, configured to perform image segmentation on the to-be-segmented image according to the center point position information.
A non-transitory computer-readable medium having stored thereon a computer program which, when executed by a processing device, implements the steps of the method of any one of claims 1-8.
An electronic device comprising:

a storage device on which a computer program is stored;

A processing device, configured to execute the computer program in the storage device, so as to implement the method of any one of claims 1-8.
A computer program comprising:

Instructions which, when executed by a processor, cause the processor to perform the image segmentation method of any of claims 1-8.
A computer program product comprising instructions which, when executed by a processor, cause the processor to perform the image segmentation method of any of claims 1-8.