WO2018166289A1 - Image generation method and device - Google Patents

Abstract

An image generation method and device. A specific embodiment of the method comprises: acquiring a set of pixel values of pixel points of an image to be processed and a set of label values associated with the image, the label values being used to identify classes to which the pixel points belong; and establishing an energy function according to the set of pixel values and the set of label values, the energy function being used to characterize the consistency between the label values and the pixel values; selecting a label value from the set of label values to be assigned to each pixel point of the image, so that the value of the energy function is minimized; classifying the each pixel point of the image according to the label value assigned to the each pixel point of the image, and modifying the pixel values of pixel points belonging to the same class to the same value to generate a processed image. The embodiment achieves image segmentation processing, and can effectively suppress the influence of image noise on the image segmentation result.

Description

Image generation method and device

Cross-reference to related applications

The present application claims the priority of the Japanese Patent Application Serial No. JP-A---------

Technical field

The present application relates to the field of computer technologies, and in particular, to the field of computer image processing technologies, and in particular, to an image generation method and apparatus.

Background technique

Image segmentation is an image processing technique that divides images into meaningful regions. Segmentation technology has important research value and broad application prospects in the fields of auxiliary medical diagnosis, motion analysis and structural analysis. Image segmentation is the first step in image analysis. The next tasks of image segmentation, such as feature extraction and target recognition, depend on the quality of image segmentation. For example, in medicine, with the increasing role of imaging medical technology in medicine, image segmentation has a special significance in medical applications, and segmentation technology enables people to obtain effective medical image information. The segmented images are widely used in various important aspects such as diagnosis of lesions, preoperative planning, and postoperative monitoring.

Currently widely used image segmentation methods mainly include region-based and edge-based approaches. The disadvantage of the threshold method is that if the gray level difference between the target and the background or the target is not obvious, or the image with a large overlap between the target and the background gray value range, it is difficult to obtain an accurate segmentation result, and the threshold method is The noise is very sensitive. The edge-based segmentation method uses the pixel value discontinuity of the object boundary to complete the image segmentation. When there is noise in the image, the false edge is often generated, which affects the segmentation effect.

In general, the above image segmentation method only starts from the pixel value of the single-point image, ignoring the a priori of the image local smoothing, and is sensitive to image noise.

Summary of the invention

The purpose of the present application is to propose an improved image generating method and apparatus to solve the technical problems mentioned in the background section above.

In a first aspect, an embodiment of the present application provides an image generating method, which includes: acquiring a pixel value set of a pixel point of an image to be processed and a label value set associated with the image, wherein the label value is used to identify the pixel The category to which the point belongs; an energy function is established according to the set of tag values and the set of pixel values, wherein the energy function is used to characterize the consistency of the tag value and the pixel value; for each pixel of the image, the tag value is selected from the set of tag values. Allocating, so that the value of the energy function is minimized; each pixel of the image is classified according to the label value assigned to each pixel of the image, and the pixel values of the pixels belonging to the same class are modified to the same value to generate The processed image.

In some embodiments, the energy function includes a data energy function and a smooth energy function, wherein the data energy function is used to characterize the pixel value of the pixel point and the label value assigned by the pixel point, and the smooth energy function is used to characterize the pixel The pixel value of the point is consistent with the tag value assigned by the pixel adjacent to the pixel.

In some embodiments, after acquiring the set of pixel values of the pixel points of the image to be processed, the method further comprises: normalizing each pixel value in the set of pixel values to obtain a normalized pixel value, and Each pixel value in the set of pixel values is replaced with each normalized pixel value.

In some embodiments, after acquiring the set of pixel values of the pixel points of the image to be processed, the method further comprises: normalizing each pixel value in the set of pixel values to obtain a normalized pixel value, and Each pixel value in the set of pixel values is replaced with each normalized pixel value.

In some embodiments, each pixel point is assigned a tag value such that the value of the energy function is minimized, including: a sub-graph matching algorithm using a progressively non-convex-concave process to solve the energy function with a minimum value per pixel point The assigned tag value.

In a second aspect, an embodiment of the present application provides an image generating apparatus, where the apparatus includes: an acquiring unit, configured to acquire a pixel value set of a pixel of an image to be processed, and a label value set associated with the image, where the label The value is used to identify the category to which the pixel belongs; the establishing unit is configured to establish an energy function according to the set of label values and the set of pixel values, wherein the energy function is used to characterize the consistency of the label value and the pixel value; the allocation unit is configured to Each pixel of the image is selected from a set of tag values to be assigned to minimize the value of the energy function; a generating unit for each pixel of the image based on the tag value assigned to each pixel of the image Classify and modify the pixel values of pixels belonging to the same class to the same value to generate a processed image.

In some embodiments, the energy function includes a data energy function and a smooth energy function, wherein the data energy function is used to characterize the pixel value of the pixel point and the label value assigned by the pixel point, and the smooth energy function is used to characterize the pixel The pixel value of the point is consistent with the tag value assigned by the pixel adjacent to the pixel.

In some embodiments, the apparatus further includes: a normalization unit, configured to normalize each pixel value in the set of pixel values after obtaining the set of pixel values of the pixel points of the image to be processed The pixel values are normalized and each pixel value in the set of pixel values is replaced with each normalized pixel value.

In some embodiments, the apparatus further includes: a receiving unit, configured to receive a number of tags input by the user through the terminal, and determine a set of tag values according to the number of tags, before acquiring the set of pixel values of the pixel points of the image to be processed.

In some embodiments, the allocating unit is further configured to: use a subgraph matching algorithm that uses a progressively non-convex augmentation process to solve for a tag value that should be assigned to each pixel point when the value of the energy function is minimum.

In a third aspect, an embodiment of the present application provides an apparatus, including: one or more processors; a storage device, configured to store one or more programs, when one or more programs are executed by one or more processors, One or more processors are caused to implement the method of any of the first aspects.

In a fourth aspect, an embodiment of the present application provides a computer readable storage medium having stored thereon a computer program, the program being executed by a processor to implement the method of any one of the first aspects.

An image generating method and apparatus provided by an embodiment of the present application associates a pixel value of a pixel of an image to be processed with a label value associated with the image by establishing an energy function, and assigns a label value to each pixel of the image. In order to minimize the value of the energy function, each pixel of the image is classified according to the label value, and then the pixel value of the pixel of each category is modified to obtain a processed image. Since the value of the energy function reflects the consistency of the label value and the pixel value, when the value of the energy function is the smallest, the label value has the highest consistency with the pixel value, and the processed image is smoother, thereby eliminating the problem of error segmentation caused by image noise. .

DRAWINGS

Other features, objects, and advantages of the present application will become more apparent from the detailed description of the accompanying drawings.

1 is an exemplary system architecture diagram to which the present application can be applied;

2 is a flow chart of one embodiment of an image generating method according to the present application;

3 is a diagram showing an adjacency relationship of pixel points of an image of the present application;

4a, 4b are schematic diagrams showing an application scenario of an image generating method according to the present application;

FIG. 5 is a schematic structural diagram of an embodiment of an image generating apparatus according to the present application; FIG.

FIG. 6 is a schematic structural diagram of a computer system suitable for implementing a terminal device or a server of an embodiment of the present application.

detailed description

The present application will be further described in detail below with reference to the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are merely illustrative of the invention, rather than the invention. It is also to be noted that, for the convenience of description, only the parts related to the related invention are shown in the drawings.

It should be noted that the embodiments in the present application and the features in the embodiments may be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings.

FIG. 1 illustrates an exemplary system architecture 100 in which an embodiment of an image generation method or image generation device of the present application may be applied.

As shown in FIG. 1, system architecture 100 can include terminal devices 101, 102, 103, network 104, and server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. Network 104 may include various types of connections, such as wired, wireless communication links, fiber optic cables, and the like.

The user can interact with the server 105 over the network 104 using the terminal devices 101, 102, 103 to receive or transmit messages and the like. Various communication client applications, such as an image browser application, a shopping application, a search application, an instant communication tool, a mailbox client, a social platform software, and the like, may be installed on the terminal devices 101, 102, and 103.

The terminal devices 101, 102, 103 may be various electronic devices having a display screen and supporting image browsing, including but not limited to smart phones, tablets, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III, dynamic The video specialist compresses the standard audio layer 3), MP4 (Moving Picture Experts Group Audio Layer IV) player, laptop portable computer and desktop computer, and the like.

The server 105 may be a server that provides various services, such as a background image server that provides support for images displayed on the terminal devices 101, 102, 103. The background image server may perform processing such as analyzing the received image processing request and the like, and feed back the processing result (for example, the newly generated image after the segmentation) to the terminal device.

It should be noted that the image generating method provided by the embodiment of the present application is generally executed by the server 105. Accordingly, the image generating device is generally disposed in the server 105. The image generating method provided by the embodiment of the present application may be directly executed by the terminal device 101, 102, and 103.

It should be understood that the number of terminal devices, networks, and servers in Figure 1 is merely illustrative. Depending on the implementation needs, there can be any number of terminal devices, networks, and servers.

With continued reference to FIG. 2, a flow 200 of one embodiment of an image generation method in accordance with the present application is illustrated. The image generating method comprises the following steps:

Step 201: Acquire a set of pixel values of pixel points of an image to be processed and a set of tag values associated with the image.

In the present embodiment, the electronic device (for example, the server shown in FIG. 1) on which the image generating method runs can receive the image to be processed from the terminal with which the user performs image browsing by means of a wired connection or a wireless connection. And acquiring, from the image to be processed, a set of pixel values of pixels of the image to be processed. The set of tag values associated with the image may be a preset set of tag values. The tag value is used to identify the category to which the pixel belongs. The number of categories to which a pixel belongs can be a fixed value, for example, divided into only two categories, foreground and background. If the pixel value of the pixel of the image is between 0 and 255, the set of tag values can be set to {0, 255}. When a pixel is assigned to a tag value of 0, then the pixel is divided into a background. When a pixel is assigned to the tag value 255, then the pixel is divided into foreground. The size of the set of tag values can be determined to determine that the segmented image contains several categories.

In some optional implementation manners of the embodiment, after acquiring the pixel value set of the pixel of the image to be processed, the method further includes: normalizing each pixel value in the pixel value set to obtain a return The pixel values are normalized and each pixel value in the set of pixel values is replaced with each normalized pixel value. For example, if the pixel value of the pixel of the image is between 0 and 255, the pixel value obtained after the normalization process is between 0-1, and the set of tag values can be set to {0, 1}. When a pixel is assigned to a tag value of 0, then the pixel is divided into a background. When a pixel is assigned to a tag value of 1, the pixel is divided into foreground.

In some optional implementation manners of the embodiment, before acquiring the pixel value set of the pixel of the image to be processed, the method further includes: receiving the number of labels input by the user through the terminal, and determining the label value set according to the label quantity. . For example, if the number of labels entered by the user is 2, then the label set can be determined to be {0, 1}. If the number of labels entered by the user is 3, it can be determined that the label set is {0, 1, 2}.

Step 202: Establish an energy function according to the set of tag values and the set of pixel values.

In this embodiment, an energy function is established based on the set of tag values and the set of pixel values obtained in step 201. This energy function is used to characterize the consistency of the tag value with the pixel value. The energy function is an objective function in the field of computer vision. It depicts the inconsistency of the image and is the energy formed by an interaction between the pixels of the image.

The image segmentation problem can be expressed as a pixel-labeling problem, which is a tag value for each pixel in the image. The foreground and background in the image are distinguished by different labels in image segmentation.

Definition: P = {p ₁ , p ₂ , p ₃ , .., p _n }, where P is a set of n pixels.

Definition: L = {l ₁ , l ₂ , l ₃ , .., l _k }, where L is a set of k labels. In image segmentation, the tag value indicates that the class pixel tag problem to which the pixel points belong is to assign a tag value l _{i in the} tag set to each element p _i in the pixel set. Then the pixel label problem is to establish a mapping between the set P and the set L: F = {f ₁ , f ₂ , f ₃ , .., f _n }

The complete mapping on the entire set L ⁿ is denoted by F. The image segmentation problem is transformed into the process of solving F. Pixel label problems can be solved by an energy function.

In some optional implementations of this embodiment, the energy function includes a data energy function and a smooth energy function, wherein the data energy function is used to characterize the consistency of the pixel value of the pixel with the label value assigned by the pixel, The smooth energy function is used to characterize the consistency of the pixel value of the pixel point with the label value assigned by the pixel point adjacent to the pixel point.

The energy function of this application is as follows:

E(f)=E _data (f)+λE _prior (f) (Equation 1)

Where E _data (f) is called the data energy function, which is the data constraint. In image segmentation, assuming that an image is observed, we need to assign a label value to each pixel in the image to determine the segmentation category to which the pixel belongs. When the global minimum energy is the optimal solution, the data energy is smaller when the label value can better match the gray value of the pixel. When the tag value does not match the intensity value, the penalty is greater, ie the data energy is greater.

If we only use the data energy as a constraint, the actual result may appear to be much noisy and the image is not smooth enough. However, the visual problem is not random, and the pixel label value has a certain relationship, so we introduce the prior knowledge as the constraint of the energy function.

E _prior (f) is called the smooth energy function, which corresponds to the constraints of prior knowledge. In an actual image, the image tends to be locally smooth, that is, the pixel points are always relatively consistent with the pixels in the neighborhood. In the visual task, if the minimum energy is used as the optimal solution, if the label value corresponding to the pixel point and the label value in the neighborhood are in good agreement, the value of the smooth energy function is small, and vice versa.

The parameter λ controls the relationship between the data and the prior knowledge. The larger the value of λ, the greater the weight of the prior, and the greater the role of prior knowledge in the optimal solution. For example, in image segmentation, if the a priori uses a standard neighborhood (MRF (Markov Random Filed) neighborhood) as its neighborhood system, the larger the λ value, the smoother the segmentation result.

The data energy function E _data (f) penalizes the label value and the actual intensity of the pixel. The better the consistency, the smaller the data energy. Its mathematical expression is as follows:

E _data (f)=∑ _p∈P D _p (f _p ) (Equation 2)

D _p (f _p ) describes the data energy when the pixel p obtains the label f _p . In the visual task, D _p (f _p ) is generally considered to be independent of each other. In general, D _p (f _p ) is non-negative. Data energy is an important constraint in the energy function, which reflects the agreement between the overall tag value and the actual data.

In the actual image segmentation problem, the background and foreground tend to have different intensities, so the present invention uses the following data energy form to constrain the consistency of the tag value with the observed data:

Where k is the tag value, I _p is the pixel value of point p, and max(I) is the maximum observed value of the image (ie, the actual pixel value of the pixel). It can be seen from the form of the data energy function that when the pixel value of the pixel of the image is large, if the label of the pixel is assigned a value of 0, that is, the category identifying the pixel is the background, then the data has larger data. Energy and vice versa. When the data energy is minimized, the effect achieved by the image is consistent with the threshold method.

The smooth energy function is used to characterize the inconsistency of the pixel point tag value with the tag value in its neighborhood. The smooth energy function is the result of the interaction of pixel points with adjacent points. Since the image is always locally smooth, smooth energy is used to constrain the smooth prior. The set of adjacent points defining the pixel point p is denoted by N _P . In this application, N _P satisfies the following two conditions:

1)

2) If p∈Nq, then q∈Np.

That is, the definition map is an undirected graph, and the neighborhood relationship is symmetric.

The mathematical expression of smooth energy is shown below:

E _smooth (f)=∑ _{p,q} ∈N Vpq(fp,fq) (Equation 4)

Where N is the neighborhood system of the image. When N is a standard first-order Markov Random Field, the adjacency is shown in Figure 3:

Standard neighborhood N _P ={t,l,b,r};N _q ={x,z}

The form of the smooth energy defined in this application is as follows:

||I _p -I _q || ² is the square of the neighborhood pixel difference and is used to describe the distance of the pixel in the neighborhood. From the form of the smooth energy function, it can be seen that when the neighborhood pixels take the same label, the value of the smooth energy function is 0, which satisfies the smooth prior of the image. When the pixels in the neighborhood take different label values, the image will be given a certain smooth energy, the size of which depends on the distance of the neighboring pixels. When the difference in the neighborhood of the image is larger, the energy obtained is smaller; the smaller the distance, the larger the energy. By analyzing the smooth energy function, it can be seen that in order to minimize the global smoothing energy, the image tends to be locally smooth, that is, the same label in the neighborhood, and the label change occurs in the region where the pixel value is abrupt in the neighborhood of the image. Smooth energy minimization has some similarities with traditional edge-based methods.

By minimizing the data energy and the smooth energy, the image will be smoothed and have strong consistency with the observed data. Due to the introduction of smooth energy, the segmentation result can effectively suppress the influence of noise points. Because the noise points are often isolated, in order to maintain the consistency of the labels in the neighborhood, they often obtain the same labels as the neighborhoods, thereby achieving the purpose of eliminating noise points.

Smooth energy can also take other forms, as shown in the following equation:

Where cons is a fixed constant, which is independent of the pixel value.

Step 203: For each pixel of the image, select a tag value from the set of tag values for allocation to minimize the value of the energy function.

In the present embodiment, the solution of the energy function is a combinatorial optimization problem in the optimization problem, that is, the problem of finding the extremum in the discrete state. Arranging a discrete object according to a certain constraint, and when a specific arrangement known to be in conformity exists, seeks the specific arrangement between the maximal or minimal solution under an optimization criterion question. There are many alternative solutions for the energy function, including the Iteration Condition Model (ICM), the Belief Propagation (BP), and the Graph Cuts (GC).

In some optional implementations of this embodiment, selecting a tag value from the set of tag values for allocation to minimize the value of the energy function includes: using a sub-graph matching algorithm that uses a progressively non-convex-concave process to solve the energy function The value of the tag that should be assigned to each pixel at the lowest value.

Let there be N pixels in a two-dimensional image, and there are K possibilities for each point tag value. Then solving the energy function E(f)=E _data (f)+λE _prior (f) is a combinatorial optimization problem. There are K values for each pixel, and the complexity of each method by exhausting each group and finding the optimal solution is O(N ^K ), which is obviously unachievable for visual tasks. This is a non-deterministic polynomial problem in mathematics. In the actual task, the problem needs to be approximated to obtain the solution of the energy function.

In the present application, the subgraph matching algorithm of the progressively non-convex augmentation process is used to solve the minimum value of the energy function. The core idea is to relax the discrete combination optimization problem to solve in the continuous domain, perform a convex to concave relaxation process on the objective function in the continuous domain, and solve the minimum value of the energy function in the relaxation process. The specific steps are as follows:

(1) Rewrite the energy function into a matrix form:

E=1/2x ^T Qx+Dx (Equation 7)

Where Q∈R ^nk*nk , D∈R ^{1 *nk} , x∈{0,1} ^nk , n is the total number of pixels in the image, and k is the number of label values. The matrices Q, D satisfy Q(ia, jb) = V _ab (i, j), D (ia) = D (a, i), respectively, and if the pixel a takes the label value i, then x (ia) = 1.

(2) Relaxing the energy function, relaxing the discrete x-vectors into the continuous domain, and performing convex and concave relaxation on the energy function.

(3) Initialization

Initial combination coefficient γ=-1

(4) Calculate the direction d of the energy function;

Down direction d=yx, where

(5) Find the step size α;

In this step, the moving step α of the current point in the descending direction is determined,

(6) updating the vector to be sought x;

If updated

To meet the conditions:

Where ε is a small constant, then it is proved that x has converged, turn to step (7), otherwise go to step (4).

(7) Update the combination coefficient γ:

If γ>1, stop the cycle. Output x.

(8) Convert the output x to a discrete tag value.

The vector x is converted to a matrix of n*k, and the optimal set of tag values f ^* = argmax _k (x) is derived.

At this point, by solving the minimum value of the energy function, the label value of each pixel is obtained, and the segmentation result of the image is obtained according to the label value.

Step 204: classify each pixel of the image according to the label value assigned to each pixel of the image, and modify the pixel values of the pixels belonging to the same class to the same value to generate the processed image.

In the present embodiment, each pixel of the image is classified based on the tag value obtained in step 203. The tag value can be proportional to the pixel value of the pixel of the resulting image. For example, a pixel with a label value of 0 is classified as a background, and a pixel with a label value of 1 is classified as a foreground. The pixel values of the pixels belonging to the background are all changed to 0, and the pixel values of the pixels belonging to the foreground are all changed to 255, that is, the black and white colors are used to distinguish the pixels of different categories. The pixel value of each type of pixel is not limited to 0 or 255 as long as it can be easily recognized by the naked eye. Similarly, if the number of tags is 3, the pixels are divided into three categories, and the three types of pixels are distinguished by three different pixel values. The image of the finally generated processing score is the result of division by category.

4a, 4b, which are schematic diagrams of an application scenario of the image generating method according to the present embodiment, wherein FIG. 4a is an original noise image, and FIG. 4b is an image after segmentation processing. In the application scenario of FIG. 4a, 4b, the user sends the original noise image 4a to the server through the terminal, and the number of the desired segmentation categories input by the user is 3, and the server obtains the pixel value of each pixel of the image after receiving the image in FIG. 4a and acquires For the corresponding tag value, the pixel points of the image are assigned an appropriate tag value to minimize the generated energy function value of Figure 4b. The generated Figure 4b is returned to the user's terminal.

The method provided by the above embodiment of the present application utilizes prior knowledge of image local smoothing to establish an energy function for the image, and achieves the purpose of image segmentation by solving the minimum value of the energy function. Due to the introduction of a priori smoothing, when there are isolated noise points in the image, it can automatically classify the pixels according to the pixel values of the neighboring pixels, so it can effectively deal with the segmentation problem of noisy images and eliminate errors caused by noise. Split the problem.

With reference to FIG. 5, as an implementation of the method shown in the above figures, the present application provides an embodiment of an image generating apparatus, and the apparatus embodiment corresponds to the method embodiment shown in FIG. Used in a variety of electronic devices.

As shown in FIG. 5, the image generating apparatus 500 of the present embodiment includes an obtaining unit 501, an establishing unit 502, an allocating unit 503, and a generating unit 504. The acquiring unit 501 is configured to acquire a pixel value set of a pixel point of the image to be processed and a label value set associated with the image, where the label value is used to identify a category to which the pixel point belongs; the establishing unit 502 is configured to The set of tag values and the set of pixel values establish an energy function, wherein the energy function is used to characterize the consistency of the tag value with the pixel value; the allocating unit 503 is configured to: for each pixel of the image, The set of tag values selects a tag value for allocation to minimize a value of the energy function; the generating unit 504 is configured to: each pixel of the image according to a tag value assigned to each pixel of the image Classify and modify the pixel values of pixels belonging to the same class to the same value to generate a processed image.

In this embodiment, the specific processing of the obtaining unit 501, the establishing unit 502, the allocating unit 503, and the generating unit 504 of the image generating apparatus 500 may refer to step 201, step 202, step 203, and step 204 in the corresponding embodiment of FIG.

In some optional implementations of this embodiment, the energy function includes a data energy function and a smooth energy function, wherein the data energy function is used to characterize the consistency of the pixel value of the pixel with the label value assigned by the pixel, The smooth energy function is used to characterize the consistency of the pixel value of the pixel point with the label value assigned by the pixel point adjacent to the pixel point.

In some optional implementation manners of the embodiment, the apparatus 500 further includes: a normalization unit, configured to: after acquiring the pixel value set of the pixel point of the image to be processed, each pixel value in the pixel value set Normalization is performed to obtain normalized pixel values, and each pixel value in the set of pixel values is replaced with each normalized pixel value.

In some optional implementation manners of the embodiment, the apparatus 500 further includes: a receiving unit, configured to receive, by the terminal, the number of labels input by the user before acquiring the pixel value set of the pixel of the image to be processed, and according to the label The quantity determines the set of tag values.

In some optional implementation manners of the embodiment, the allocating unit 503 is further configured to: use a sub-graph matching algorithm that uses a progressively non-convex-concave process to solve a tag value that should be allocated for each pixel point when the value of the energy function is minimum.

Referring now to Figure 6, a block diagram of a computer system 600 suitable for implementing the terminal device/server of an embodiment of the present application is shown. The terminal device/server shown in FIG. 6 is merely an example, and should not impose any limitation on the function and scope of use of the embodiments of the present application.

As shown in FIG. 6, computer system 600 includes a central processing unit (CPU) 601 that can be loaded into a program in random access memory (RAM) 603 according to a program stored in read only memory (ROM) 602 or from storage portion 608. And perform various appropriate actions and processes. In the RAM 603, various programs and data required for the operation of the system 600 are also stored. The CPU 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. An input/output (I/O) interface 605 is also coupled to bus 604.

The following components are connected to the I/O interface 605: an input portion 606 including a keyboard, a mouse, etc.; an output portion 607 including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), and the like, and a storage portion 608 including a hard disk or the like. And a communication portion 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the Internet. Driver 610 is also coupled to I/O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory or the like, is mounted on the drive 610 as needed so that a computer program read therefrom is installed into the storage portion 608 as needed.

In particular, the processes described above with reference to the flowcharts may be implemented as a computer software program in accordance with an embodiment of the present disclosure. For example, an embodiment of the present disclosure includes a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for executing the method illustrated in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network via communication portion 609, and/or installed from removable media 611. When the computer program is executed by the central processing unit (CPU) 601, the above-described functions defined in the method of the present application are performed. It should be noted that the computer readable medium described herein may be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of computer readable storage media may include, but are not limited to, electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable Programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer readable storage medium may be any tangible medium that can contain or store a program, which can be used by or in connection with an instruction execution system, apparatus or device. In the present application, a computer readable signal medium may include a data signal that is propagated in the baseband or as part of a carrier, carrying computer readable program code. Such propagated data signals can take a variety of forms including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the foregoing. The computer readable signal medium can also be any computer readable medium other than a computer readable storage medium, which can transmit, propagate, or transport a program for use by or in connection with the instruction execution system, apparatus, or device. . Program code embodied on a computer readable medium can be transmitted by any suitable medium, including but not limited to wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality and operation of possible implementations of systems, methods and computer program products in accordance with various embodiments of the present application. In this regard, each block of the flowchart or block diagram can represent a module, a program segment, or a portion of code that includes one or more of the logic functions for implementing the specified. Executable instructions. It should also be noted that in some alternative implementations, the functions noted in the blocks may also occur in a different order than that illustrated in the drawings. For example, two successively represented blocks may in fact be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams and/or flowcharts, and combinations of blocks in the block diagrams and/or flowcharts, can be implemented in a dedicated hardware-based system that performs the specified function or operation. Or it can be implemented by a combination of dedicated hardware and computer instructions.

The units involved in the embodiments of the present application may be implemented by software or by hardware. The described unit may also be provided in the processor, for example, as a processor including an acquisition unit, an establishment unit, an allocation unit, and a generation unit. Wherein, the names of the units do not constitute a limitation on the unit itself in some cases. For example, the obtaining unit may also be described as “acquiring a set of pixel values of pixels of an image to be processed and associating with the image. The unit of the label value collection."

In another aspect, the present application also provides a computer readable medium, which may be included in the apparatus described in the above embodiments, or may be separately present and not incorporated into the apparatus. The computer readable medium carries one or more programs, when the one or more programs are executed by the device, causing the device to: obtain a set of pixel values of pixels of the image to be processed and a tag value associated with the image a set, wherein the tag value is used to identify a category to which the pixel point belongs; an energy function is established according to the set of tag values and the set of pixel values, wherein the energy function is used to characterize the consistency of the tag value and the pixel value; for each pixel of the image Point, select a tag value from the set of tag values for assignment to minimize the value of the energy function; classify each pixel of the image according to the tag value assigned to each pixel of the image, and place pixels belonging to the same class The pixel values are modified to the same value to generate a processed image.

The above description is only a preferred embodiment of the present application and a description of the principles of the applied technology. It should be understood by those skilled in the art that the scope of the invention referred to in the present application is not limited to the specific combination of the above technical features, and should also be covered by the above technical features without departing from the inventive concept. Other technical solutions formed by any combination of their equivalent features. For example, the above features are combined with the technical features disclosed in the present application, but are not limited to the technical features having similar functions.

Claims (12)

1. An image generating method, the method comprising:

   Obtaining a set of pixel values of pixels of the image to be processed and a set of tag values associated with the image, wherein the tag value is used to identify a category to which the pixel points belong;

   Establishing an energy function according to the set of tag values and the set of pixel values, wherein the energy function is used to characterize the consistency of the tag value and the pixel value;

   For each pixel of the image, selecting a tag value from the set of tag values for allocation to minimize a value of the energy function;

   Each pixel of the image is classified according to a tag value assigned to each pixel of the image, and pixel values of pixels belonging to the same class are modified to the same value to generate a processed image.
2. The method of claim 1 wherein said energy function comprises a data energy function and a smooth energy function, wherein said data energy function is used to characterize a pixel value of a pixel point and a label value assigned to the pixel point Consistency, the smooth energy function is used to characterize the consistency of the pixel value of the pixel point with the label value assigned by the pixel point adjacent to the pixel point.
3. The method according to claim 1, wherein after acquiring the set of pixel values of the pixel points of the image to be processed, the method further comprises:

   Each pixel value in the set of pixel values is normalized to obtain a normalized pixel value, and each pixel value in the set of pixel values is replaced with each normalized pixel value.
4. The method according to claim 1, wherein the method further comprises: before acquiring a set of pixel values of pixels of the image to be processed, the method further comprising:

   Receiving the number of tags input by the user through the terminal, and determining a set of tag values according to the number of tags.
5. The method according to any one of claims 1 to 4, wherein the selecting a tag value from the set of tag values for allocation to minimize a value of the energy function comprises:

   The subgraph matching algorithm using the progressively non-convex-concave process is used to solve the label value that should be allocated for each pixel when the value of the energy function is minimum.
6. An image generating apparatus, the apparatus comprising:

   An acquiring unit, configured to obtain a pixel value set of a pixel of the image to be processed, and a label value set associated with the image, where the label value is used to identify a category to which the pixel point belongs;

   Establishing a unit, configured to establish an energy function according to the set of label values and the set of pixel values, wherein the energy function is used to characterize consistency of the label value and the pixel value;

   An allocating unit, configured to, for each pixel of the image, select a tag value from the set of tag values for allocation to minimize a value of the energy function;

   a generating unit, configured to classify each pixel of the image according to a label value assigned to each pixel of the image, and modify pixel values of pixels belonging to the same class to the same value to generate processing After the image.
7. The apparatus according to claim 6, wherein said energy function comprises a data energy function and a smooth energy function, wherein said data energy function is used to characterize a pixel value of a pixel point and a label value assigned to the pixel point Consistency, the smooth energy function is used to characterize the consistency of the pixel value of the pixel point with the label value assigned by the pixel point adjacent to the pixel point.
8. The device according to claim 6, wherein the device further comprises:

   a normalization unit, configured to normalize each pixel value in the pixel value set to obtain a normalized pixel value after acquiring a pixel value set of a pixel point of the image to be processed, and use each The normalized pixel values replace each pixel value in the set of pixel values.
9. The device according to claim 7, wherein the device further comprises:

   The receiving unit is configured to receive the number of tags input by the user through the terminal before acquiring the pixel value set of the pixel of the image to be processed, and determine the tag value set according to the number of the tags.
10. The device according to any one of claims 6-9, wherein the allocating unit is further configured to:

    The subgraph matching algorithm using the progressively non-convex-concave process is used to solve the label value that should be allocated for each pixel when the value of the energy function is minimum.
11. A device that includes:

    One or more processors;

    a storage device for storing one or more programs,

    The one or more programs are executed by the one or more processors such that the one or more processors implement the method of any of claims 1-5.
12. A computer readable storage medium having stored thereon a computer program, wherein the program, when executed by a processor, implements the method of any of claims 1-5.

Images