CN105574866A - Image processing method and apparatus - Google Patents

Abstract

The invention discloses an image processing method and apparatus. The image processing method comprises the steps of performing significance analysis on an original image; distinguishing the original image into significance regions including at least two different significance effects according to the significance analysis; and adopting corresponding image processing methods for processing the significance regions respectively. According to the image processing method, the significance regions are distinguished and determined through the significance analysis; and meanwhile, the significance regions are processed by adopting the corresponding image processing methods, so that the display effect of the image body region is improved, and the image display quality is improved.

Description

Method and device for realizing image processing

Technical Field

The present invention relates to image processing technologies, and in particular, to a method and an apparatus for implementing image processing.

Background

The image saliency is an important visual feature of an image, and represents the degree of importance of human eyes on a partial region of the image. For one image, the user only interests a part of regions in the image, the interested part of regions represents the query intention of the user, and other regions are irrelevant to the query intention of the user. Fig. 1(a) is a photographed original image, and as shown in fig. 1(a), a subject region in the image can be prominently displayed in a visual range; fig. 1(b) is a saliency image of a captured image, and as shown in fig. 1(b), the higher the brightness of a pixel point in the saliency image is, the higher the saliency is, the more the original image of the corresponding region can arouse the visual interest of the user, and the image of the region with high saliency is the region in which the user is interested.

When an image is shot, a user generally focuses on an interested main area, the main area generally becomes a salient area of the shot image, and the evaluation weight of the photo quality is mainly reflected in the main area of the image; when the whole shot image has the problems of focusing blur, incorrect exposure, light shielding, poor saturation, unobvious contrast and the like, the existing image processing algorithm performs global adjustment and processing on the image, namely simultaneously performs the same processing on a main body area and a background area, so that the significance of the main body area is weakened. The display effect of the main body region cannot be improved.

At present, after a main body area and a background area are segmented according to the difference of color and brightness, the segmented main body area and the segmented background area are respectively subjected to image processing, when the main body area and the background area obtained by segmentation according to the difference of color and brightness are subjected to image processing, each area which can only be segmented is respectively processed by the same image processing method, for example, when a shot image is a person image with a background, when the shot image is segmented into two areas of a person and the background according to the color and the brightness, the person is taken as the main body area to process the person image according to a unified image processing method, and the background is taken as the background area to process the background image according to the unified image processing method; local features (such as glasses and cheeks for focusing the shot image) in the human image are not distinguished, and the local features are not optimized in significance in a unified processing mode, so that the display effect is not improved.

Disclosure of Invention

In order to solve the above technical problem, the present invention provides a method and an apparatus for implementing image processing, which can improve the display effect of a main body region.

In order to achieve the object of the present invention, the present invention provides an apparatus for implementing image processing, comprising: the device comprises an analysis unit, a determination unit and a processing unit; wherein,

the analysis unit is used for carrying out significance analysis on the original image;

the determining unit is used for distinguishing the original image into a significant area containing at least two different significant effects according to a significant analysis result;

and the processing unit is used for processing each salient region by adopting a corresponding image processing method.

Furthermore, the device further comprises a segmentation unit, which is used for segmenting the salient regions with different significance effects before the salient regions are respectively processed by adopting the corresponding image processing methods.

Further, the analysis unit is specifically configured to perform image contrast comparison on each pixel point of the original image and the color, brightness, and direction of the peripheral pixel points, to obtain a corresponding saliency value of each pixel point, and perform saliency analysis.

Further, the analysis unit is specifically adapted to,

and performing image contrast analysis on the original image by adopting a regional contrast RC algorithm, and performing significance analysis on the original image through the image contrast analysis.

Further, the dividing means extracts the outline of the significant region having the different significance effect and/or the internal cavity of the region filling the significant region having the different significance effect by using mathematical morphology when dividing the significant region having the different significance effect.

In another aspect, the present application further provides a method for implementing image processing, including:

carrying out significance analysis on the original image;

distinguishing the original image into a significant area containing at least two different significant effects according to a significant analysis result;

and processing each salient region by adopting a corresponding image processing method.

Further, distinguishing the original image as a salient region containing at least two different salient effects specifically includes:

and according to the area significance numerical value in the significance analysis result, combining a preset distinguishing threshold value, and distinguishing the original image into significance areas containing at least two different significance effects.

Further, when the original image is divided into a main area and a background area with different significance effects, the preset distinguishing threshold includes: the significance value range of the main area is greater than 64 and less than or equal to 255, and the significance value range of the background area is greater than or equal to 0 and less than 64.

Further, before each salient region is processed by the corresponding image processing method, the method further includes:

and dividing the significant regions with different significant effects.

Further, the performing of the saliency analysis on the original image specifically includes: and comparing the contrast of the image of each pixel point of the original image with the color, brightness and direction of the surrounding pixel points to obtain a corresponding significance value of each pixel point, and performing significance analysis.

Further, the performing of the saliency analysis of the original image specifically includes:

dividing the original image into N areas according to the set pixel size, and calculating the area r by the formula (1)_kS (r) is a significance value of_k) Comprises the following steps:

$S\left(r_k\right)=\underset{i=1,i\≠k}{\overset{n}{\Σ}}\mathrm{\ω}\left(r_i\right)D_r(r_k,r_i)---\left(1\right)$

wherein r is_iRepresents a difference from r_kRegion of (d), ω (r)_i) Is a region r_iWeighted value of D_r(r_k,r_i) Is the difference in color distance between two regions, D_r(r_k,r_i) The calculation formula is as follows:

$D_r(r_1,r_2)=\underset{i=1}{\overset{n_1}{\Σ}}\underset{j=1}{\overset{n_2}{\Σ}}{f}_1\left(i\right)f_2\left(j\right)d(c_{1,i},c_{2,j})---\left(2\right)$

wherein f is₁(i) Indicating the region r₁All statistical color types n of the ith color in the region₁The probability of occurrence of (a); f. of₂(i) Is a region r₂All statistical color types n of the jth color in the region₂The probability of occurrence of (a); d (c)_1,i,c_2,j) Is r₁The ith color c_1,iAnd r₂Middle j color c_2,jA distance difference of (d);

to S (r)_k) Adding a difference value of the space distance of the region to obtain a region significance value as follows:

$S^{\′}\left(r_k\right)=\underset{i=1,i\≠k}{\overset{n}{\Σ}}\exp (-D_s(r_k,r_i)/{\mathrm{\σ}}_s^2)\mathrm{\ω}\left(r_i\right)D_r(r_k,r_i)---\left(3\right)$

wherein D is_s(r_k,r_i) Euclidean distance, σ, of region barycenter for two regions_sThe adjustment factor is influenced for the spatial distance.

Further, the performing of the saliency analysis on the original image specifically includes:

and performing image contrast analysis on the original image by adopting a regional contrast RC algorithm, and performing significance analysis on the original image through the image contrast analysis.

Further, the image processing method includes: adjusting exposure value, blurring processing, white balance special effect, background replacement, saliency value adjustment, color gradation adjustment, brightness adjustment, hue/saturation adjustment, color replacement, gradient mapping, and/or photo filter.

Further, when the salient regions with different significance effects are divided, the method further comprises the following steps:

the outline of the significant region with different significant effects and/or the internal cavity of the region filling the significant region with different significant effects are extracted by mathematical morphology.

Further, the extracting, by using mathematical morphology, the contour of each significant region having a different significant effect specifically includes:

performing contour extraction on the binary images obtained through calculation by using the binary images of the saliency areas with different saliency effects obtained through calculation through expansion, corrosion, opening and closing operations;

the filling of the internal cavity of the region of the significant region with different significant effects specifically includes:

respectively calculating corresponding binary images for the salient regions with different salient effects, extracting the outline of the interior of the binary image of each salient region to obtain an internal outline, determining the internal outline smaller than a preset area as an internal cavity, and filling pixels in the internal cavity.

Compared with the prior art, the technical scheme of the application comprises the following steps: carrying out significance analysis on the original image; distinguishing the original image into a significant area containing at least two different significant effects according to a significant analysis result; and processing each salient region by adopting a corresponding image processing method. According to the method, each saliency area is distinguished and determined through saliency analysis, and each saliency area is processed by adopting a corresponding image processing method, so that the display effect of the image main body area is improved, and the display quality of the image is improved.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:

FIG. 1(a) is a photographed original image;

fig. 1(b) is a saliency image of a captured image;

FIG. 2 is a schematic hardware configuration of a mobile terminal implementing various embodiments of the present invention;

FIG. 3 is a flow chart of a method of implementing image processing according to the present invention;

FIG. 4 is a block diagram of an apparatus for implementing image processing according to the present invention;

FIG. 5 is a flow chart of a method of an embodiment of the present invention;

FIG. 6(a) is a picture content for a first original image;

FIG. 6(b) is a diagram illustrating a first original image being segmented according to an embodiment of the present invention;

FIG. 6(c) is a schematic diagram of a saliency analysis of a first original image according to an embodiment of the present invention;

fig. 6(d) is a schematic diagram of a saliency analysis result of the first original image in this embodiment;

fig. 6(e) is a schematic diagram illustrating the effect of contrast enhancement on the first original image;

FIG. 6(f) is a schematic diagram illustrating an effect of image processing on a first original image according to an embodiment of the present invention;

FIG. 7(a) is the picture content of the second original image;

fig. 7(b) is a schematic diagram illustrating the effect of performing global white balance processing on the second original image;

FIG. 7(c) is a diagram illustrating the result of performing a saliency analysis on a second original image according to an embodiment of the present invention;

FIG. 7(d) is a schematic diagram illustrating the effect of image processing on the second original image according to the embodiment of the present invention;

fig. 8(a) is the picture content of the third original image;

fig. 8(b) is a diagram illustrating the result of performing a saliency analysis on a third original image according to an embodiment of the present invention;

fig. 8(c) is a schematic diagram illustrating an effect of performing image processing on the second original image according to the embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be noted that the embodiments and features of the embodiments in the present application may be arbitrarily combined with each other without conflict.

Fig. 2 is a schematic hardware configuration of a mobile terminal implementing various embodiments of the present invention, as shown in fig. 2,

the mobile terminal 100 may include a user input unit 130, an output unit 150, a memory 160, a controller 180, a power supply unit 190, and the like. Fig. 2 illustrates a mobile terminal having various components, but it is to be understood that not all illustrated components are required to be implemented. More or fewer components may alternatively be implemented. Elements of the mobile terminal will be described in detail below.

The a/V input unit 120 is for receiving a video signal. The a/V input unit 120 may include a camera 121 and a camera 121 to process image data of still pictures or video obtained by an image capturing apparatus in a video capturing mode or an image capturing mode. The processed image frames may be displayed on the display unit 151. The image frames processed by the camera 121 may be stored in the memory 160 (or other storage medium), and two or more cameras 1210 may be provided according to the construction of the mobile terminal.

The user input unit 130 may generate key input data according to a command input by a user to control various operations of the mobile terminal. The user input unit 130 allows a user to input various types of information, and may include a keyboard, dome sheet, touch pad (e.g., a touch-sensitive member that detects changes in resistance, pressure, capacitance, and the like due to being touched), scroll wheel, joystick, and the like. In particular, when the touch pad is superimposed on the display unit 151 in the form of a layer, a touch screen may be formed.

The output unit 150 may include a display unit 151. The display unit 151 may display information processed in the mobile terminal 100. For example, when the mobile terminal 100 is in a phone call mode, the display unit 151 may display a User Interface (UI) or a Graphical User Interface (GUI) related to a call or other communication (e.g., text messaging, multimedia file downloading, etc.). When the mobile terminal 100 is in a video call mode or an image capturing mode, the display unit 151 may display a captured image and/or a received image, a UI or GUI showing a video or an image and related functions, and the like. Meanwhile, when the display unit 151 and the touch pad are overlapped with each other in the form of a layer to form a touch screen, the display unit 151 may serve as an input device and an output device. The display unit 151 may include at least one of a Liquid Crystal Display (LCD), a thin film transistor LCD (TFT-LCD), an Organic Light Emitting Diode (OLED) display, a flexible display, a three-dimensional (3D) display, and the like. Some of these displays may be configured to be transparent to allow a user to view from the outside, which may be referred to as transparent displays, and a typical transparent display may be, for example, a TOLED (transparent organic light emitting diode) display or the like. Depending on the particular desired implementation, the mobile terminal 100 may include two or more display units (or other display devices), for example, the mobile terminal may include an external display unit (not shown) and an internal display unit (not shown). The touch screen may be used to detect a touch input pressure as well as a touch input position and a touch input area.

The memory 160 may store software programs and the like for processing and controlling operations performed by the controller 180, or may temporarily store data (e.g., a phonebook, messages, still images, videos, and the like) that has been or will be output. Also, the memory 160 may store data regarding various ways of vibration and audio signals output when a touch is applied to the touch screen.

The memory 160 may include at least one type of storage medium including a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, and the like. Also, the mobile terminal 100 may cooperate with a network storage device that performs a storage function of the memory 160 through a network connection.

The controller 180 generally controls the overall operation of the mobile terminal. For example, the controller 180 performs control and processing related to voice calls, data communications, video calls, and the like. The controller 180 may perform a pattern recognition process to recognize a handwriting input or a picture drawing input performed on the touch screen as a character or an image.

The power supply unit 190 receives external power or internal power and provides appropriate power required to operate various elements and components under the control of the controller 180.

The various embodiments described herein may be implemented in a computer-readable medium using, for example, computer software, hardware, or any combination thereof. For a hardware implementation, the embodiments described herein may be implemented using at least one of an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a processor, a controller, a microcontroller, a microprocessor, an electronic unit designed to perform the functions described herein, and in some cases, such embodiments may be implemented in the controller 180. For a software implementation, the implementation such as a process or a function may be implemented with a separate software module that allows performing at least one function or operation. The software codes may be implemented by software applications (or programs) written in any suitable programming language, which may be stored in the memory 160 and executed by the controller 180.

Up to this point, mobile terminals have been described in terms of their functionality. Hereinafter, a slide-type mobile terminal among various types of mobile terminals, such as a folder-type, bar-type, swing-type, slide-type mobile terminal, and the like, will be described as an example for the sake of brevity. Accordingly, the present invention can be applied to any type of mobile terminal, and is not limited to a slide type mobile terminal.

Based on the above mobile terminal hardware structure and communication system, the present invention provides various embodiments of the method.

Fig. 3 is a flowchart of a method for implementing image processing according to the present invention, as shown in fig. 3, including:

step 300, carrying out significance analysis on the original image;

the saliency analysis of the original image specifically includes: and comparing the contrast of the image of each pixel point of the original image with the color, brightness and direction of the surrounding pixel points to obtain a corresponding significance value of each pixel point, and performing significance analysis.

The saliency analysis of the original image specifically includes:

and performing image contrast analysis on the original image by adopting a regional contrast RC algorithm, and performing significance analysis on the original image through the image contrast analysis.

It should be noted that, the saliency of the image is calculated based on the human attention mechanism, so that the image content focused on shooting according to the interest in the image shooting process can be obtained, and the saliency effect of the part of the content is better because the user shoots the interested content through the shooting skill. The method performs significance analysis by using a Region Contrast (RC) algorithm, and obtains the significance of the image by using the calculation of the region contrast according to the thought of the RC algorithm; generating a global significance image by obtaining the correlation between the global contrast and the spatial position of the image; that is, the image has the same scale size as the original image, and the contrast method based on the global area sufficiently considers the contrast difference between the single area and the global area, so that the saliency of the entire area can be effectively highlighted, not only the edge saliency of the area (only the saliency of the image in the local area is considered). In addition, the RC algorithm also considers the spatial relationship of the areas, when the significance effect is calculated, the distance between the areas is set to be larger through the weighting parameters, the weighted value is correspondingly smaller, the positions between the areas are closer, the weighted value is correspondingly larger, and the reasonable processing of the area space is realized. The following is the main content of the RC algorithm:

when the RC algorithm is used to perform significance analysis, the original image needs to be divided into super pixels, assuming that the original image is divided into N regions, the number N of image blocks can be determined according to the resolution of the image, the size of each block is set to the width P of a pixel block, if P is 20, 20 is 20 or 400 pixels, and the range of P is usually [2040], and the number N of image blocks is set to M and N, the larger M is, the larger N is, i.e., the larger the resolution of the image is, the larger the unit area of the image block is, and the greater the number N of image blocks is M/(P).

For region r_kThe RC algorithm defines the region r_kS (r) of_k) Comprises the following steps:

$S\left(r_k\right)=\underset{i=1,i\≠k}{\overset{n}{\Σ}}\mathrm{\ω}\left(r_i\right)D_r(r_k,r_i)---\left(1\right)$

in the formula (1), r_iRepresents a difference from r_kRegion of (d), ω (r)_i) Is a region r_iThe weighted value of (2) is defined as ω (r) as the area of the pixel (number of pixels) within the region increases_i) The larger the weight is, the more the specific weight change rule can be set according to the empirical value of a person skilled in the art; d_r(r_k,r_i) The color distance difference of the two regions is specifically defined as:

$D_r(r_1,r_2)=\underset{i=1}{\overset{n_1}{\Σ}}\underset{j=1}{\overset{n_2}{\Σ}}{f}_1\left(i\right)f_2\left(j\right)d(c_{1,i},c_{2,j})---\left(2\right)$

in the formula (2), f₁(i) Represents: region r₁All statistical color types n of the ith color in the region₁The probability of occurrence of (a); f. of₂(i) Is a region r₂All statistical color types n of the jth color in the region₂The probability of occurrence of (c). d (c)_1,i,c_2,j) Is r₁The ith color c_1,iAnd r₂Middle j color c_2,jD (c) is a distance difference_1,i,c_2,j) Mainly distance measures in CIELAB space (CIELAB is a color system of the CIE, distance measures in CIELAB space are numerical information that determines distance measures for a certain color based on the CIELAB space color system). In order to fully utilize the spatial relationship of the regions, based on the formula (1), the RC algorithm adds a spatial distance difference of the regions, and obtains a region saliency value as:

$S^{\′}\left(r_k\right)=\underset{i=1,i\≠k}{\overset{n}{\Σ}}\exp (-D_s(r_k,r_i)/{\mathrm{\σ}}_s^2)\mathrm{\ω}\left(r_i\right)D_r(r_k,r_i)---\left(3\right)$

wherein D is_s(r_k,r_i) Euclidean distance, σ, of region barycenter for two regions_sThe spatial distance influences the adjustment factor, the larger the distance is, the larger the value of the adjustment factor is, the larger the adjustment factor is, and the smaller the spatial distance influences the calculation of the significance. The value of the adjustment factor can be set according to the experience of the person skilled in the art.

Step 301, according to a saliency analysis result, distinguishing the original image into a saliency region containing at least two different saliency effects;

in this step, the distinguishing the original image as a salient region including at least two different salient effects specifically includes:

and according to the area significance numerical value in the significance analysis result, combining a preset distinguishing threshold value, and distinguishing the original image into significance areas containing at least two different significance effects.

Preferably, when the original image is divided into a main area and a background area with different significance effects, the preset distinguishing threshold includes: the significance value range of the main area is greater than 64 and less than or equal to 255, and the significance value range of the background area is greater than or equal to 0 and less than 64.

It should be noted that the size of the significance value range may be adjusted according to an empirical value of a person skilled in the art, and when the original image needs to be divided into more significance regions, the value range corresponding to each significance region may be determined based on the empirical value of the person skilled in the art.

Before each salient region is processed by adopting the corresponding image processing method, the method further comprises the following steps:

and dividing the significant regions with different significant effects.

It should be noted that, the method of the present invention can be implemented by using the existing image segmentation algorithm to segment the salient regions with different salient effects.

And 302, processing each salient region by adopting a corresponding image processing method.

In this step, the image processing method includes: adjusting exposure value, blurring processing, white balance special effect, background replacement, saliency value adjustment, color gradation adjustment, brightness adjustment, hue/saturation adjustment, color replacement, gradient mapping, and/or photo filter.

It should be noted that, the processing of the saliency areas with different saliency effects by using different image processing methods means that corresponding image processing methods are respectively selected for the saliency areas with different saliency effects to perform image processing, and there is no correlation between the image processing methods of the saliency areas, and the processing of the saliency areas with different saliency effects by using different image processing methods in the present invention may include the following application scenarios:

and after the main body area is segmented according to the significance analysis result, the exposure value of the main body area is up-regulated, and the exposure value of the background area is unchanged. The exposure value up-regulation can be gradually regulated according to a preset unit, and can also be directly regulated by adopting a parameter input mode.

Applying a second scene, namely a portrait scene, keeping the contrast ratio of the main body area and the background area unchanged, and blurring the background area; the adjustment of the blurring process may be performed step by step according to a preset adjustment unit, or the blurring parameter may be directly input by using a parameter input method.

And performing white balance processing on the main area and the background area by respectively adopting different parameters when the third scene is applied and the white balance effect of the image is not good.

Applying scene IV and disordered image backgrounds, keeping a main body area of the object unchanged, and replacing the background area; the background area replacement can select a standby image which is shot by a user independently, and can also select a proper background material from an image library.

And fifthly, the saliency effect is poor in the application scene, image fusion processing is carried out, the saliency weight is increased for the main body region, and the display effect of the main body region is improved.

Further, the image processing method further includes: the specific application scenarios and embodiments belong to the common general knowledge of those skilled in the art, and are not described herein in detail; in addition, the implementation of the superposition of two or more image processing methods also belongs to the conventional technical means of those skilled in the art, and is not described herein again.

When the significant areas with different significant effects are segmented, the method further comprises the following steps:

extracting the outline of the significant region with different significant effects by using mathematical morphology, and/or filling the inner cavity of the significant region with different significant effects.

The extracting of the outline of each significant region having a different significant effect by using mathematical morphology specifically includes:

performing contour extraction on the binary images obtained through calculation by using the binary images of the saliency areas with different saliency effects obtained through calculation through expansion, corrosion, opening and closing operations;

the filling of the internal cavity of the region of the significant region with different significant effects specifically includes:

respectively calculating corresponding binary images for the salient regions with different salient effects, extracting the outline of the interior of the binary image of each salient region to obtain an internal outline, determining the internal outline smaller than a preset area as an internal cavity, and filling pixels in the internal cavity.

It should be noted that expansion, erosion, opening, and closing are basic algorithms in the image processing algorithm, and expansion is the same as expansion in the image processing algorithm and erosion expansion is the same as erosion in the image processing algorithm. The contour extraction specifically comprises: obtaining a binary image of an original image through expansion, corrosion, opening and closing operations, performing binarization segmentation on the binary image into a main body region and a background region (generally, a pixel of the main body region can be set to be 255 (displayed as white), and a background region is set to be 0 (displayed as black)), traversing the binary image, extracting pixel mutation points in the binary image, such as the pixel mutation points from 255 to 0 or from 0 to 255, taking the pixel mutation points as boundary points of the image, and connecting the boundary points to form a contour of the main body region.

According to the method, smooth transition and neat image of each salient region of the segmentation can be ensured through contour extraction; filling of the holes in the regions can ensure the integrity of each segmented saliency region.

According to the method, each saliency area is distinguished and determined through saliency analysis, and each saliency area is processed by adopting a corresponding image processing method, so that the display effect of the image main body area is improved, and the display quality of the image is improved.

Fig. 4 is a block diagram of an apparatus for implementing image processing according to the present invention, as shown in fig. 4, including: the device comprises an analysis unit, a determination unit and a processing unit; wherein,

the analysis unit is used for carrying out significance analysis on the original image;

the analysis unit is specifically configured to perform image contrast comparison on each pixel point of the original image and the color, brightness, and direction of the peripheral pixel points, to obtain a corresponding saliency value of each pixel point, and perform saliency analysis.

The analysis unit is specifically configured to perform image contrast analysis on the original image by using a regional contrast RC algorithm, and perform saliency analysis on the original image by using the image contrast analysis.

The determining unit is used for distinguishing the original image into a significant area containing at least two different significant effects according to a significant analysis result;

and the processing unit is used for processing each salient region by adopting a corresponding image processing method.

The device also comprises a segmentation unit which is used for segmenting the salient regions with different salient effects before the salient regions are respectively processed by adopting the corresponding image processing method.

The dividing means is also configured to extract the outline of each significant region having a different significance effect and/or fill the internal cavity of each significant region having a different significance effect by using mathematical morphology when dividing each significant region having a different significance effect.

The process of the present invention is illustrated in clear detail below by means of specific examples, which are provided only for illustrating the present invention and are not intended to limit the scope of the process of the present invention.

Examples

Fig. 5 is a flowchart of a method according to an embodiment of the present invention, as shown in fig. 5, including:

500, segmenting an original image according to a set pixel size;

fig. 6(a) shows the picture content of the first original image, as shown in fig. 6(a), the picture includes two main portions, namely, a human body and a background; fig. 6(b) is a schematic diagram of segmenting the first original image according to the embodiment of the present invention, and as shown in fig. 6(b), the original image is segmented into n regions.

501, performing significance analysis on an original image segmented according to set pixels;

fig. 6(c) is a schematic diagram of a saliency analysis of a first original image according to an embodiment of the present invention, as shown in fig. 6(c), when the image area 1 is taken as a saliency analysis target for the divided image areas 1 and 2, and the image areas (all image areas except the image area 1) of other numbers in the image are taken as images different from the image area 1, a saliency value is calculated according to the calculation of formula (3) of the RC algorithm; when the image area 2 is taken as a significance analysis object, taking image areas with other numbers in the image as images different from the image area 2, and carrying out significance numerical calculation; the result of the saliency calculation of image region 1 is larger, and the result of the saliency calculation of image region 2 is smaller; fig. 6(d) is a schematic diagram of a saliency analysis result of the first original image in this embodiment, as shown in fig. 6(d), the larger the saliency value of the image region 1 is, the higher the brightness of the pixel point in the saliency image is, the higher the saliency degree is, the region with high saliency is the image region in which the user is interested, and the smaller the saliency value of the image region 2 is.

Step 502, according to the result of the significance analysis, the original image is divided into two or more significant areas with different significance effects.

Here, the saliency analysis result may divide the original image into two or more saliency areas different in saliency effect; the number of the specific divided regions and the size of the significance numerical value for dividing each significance region can be set according to the analysis of a person skilled in the art; for example, a first threshold number of image regions with the highest significance value in the sequence are set as the significance regions of a display effect; setting a second threshold value image area with the sequenced significance numerical values as a significance area with a second display effect; and setting a third threshold image area with the ranked saliency values as a saliency area with a third display effect. The division of the saliency areas can also be performed by setting a proportion or a saliency value, and the specific setting can be set and adjusted by a person skilled in the art according to image analysis. The degree of significance of the human subject is determined through significance analysis, and two significance regions of the human subject and the background are divided according to different significance effects.

In this embodiment, the saliency region may be divided into a main region and a background region, and taking an image saliency value (brightness of a saliency image) range [0255] as an example, it may be assumed that a saliency value range of the main region is set as [64255], and a saliency value range of the background region is set as [064].

Step 503, dividing the salient regions with different significance effects.

Preferably, the present embodiment, when dividing the saliency areas having different saliency effects, further includes: extracting the outline of the significant region with different significant effects by using mathematical morphology, and/or filling the inner cavity of the significant region with different significant effects.

And step 504, processing each salient region by adopting a corresponding image processing method.

In fig. 6(a), the contrast between the human subject and the background in the first original image is low, and there is a problem that the display effect of the human subject is not obvious; fig. 6(e) is a schematic diagram illustrating the effect of contrast enhancement on the first original image, as shown in fig. 6(e), since the human subject and the background are simultaneously subjected to the contrast enhancement processing, the purpose of enhancing the human subject is not achieved; fig. 6(f) is a schematic diagram illustrating the effect of image processing on the first original image according to the embodiment of the present invention, and as shown in fig. 6(f), the image processing for enhancing the contrast of the human subject is performed in the embodiment, and no processing is performed on the background, so that the display effect of the human subject is enhanced.

Fig. 7(a) is the picture content of the second original image, as shown in fig. 7(a), the animal subject and the background in the second original image have a problem of poor white balance effect; fig. 7(b) is a schematic view illustrating the effect of performing the global white balance processing on the second original image, as shown in fig. 7(b), since the animal subject and the background are simultaneously subjected to the white balance processing, the display effect of the animal subject appears distorted due to the white balance processing, and the picture display effect becomes poor; fig. 7(c) is a schematic diagram illustrating a result of performing saliency analysis on a second original image according to an embodiment of the present invention, as shown in fig. 7(c), determining a saliency degree of an animal subject through the saliency analysis, and dividing two saliency areas of the animal subject and a background according to differences of saliency effects; fig. 7(d) is a schematic diagram illustrating the effect of performing image processing on the second original image according to the embodiment of the present invention, as shown in fig. 7(d), the embodiment performs white balance processing on the background, and does not perform any processing on the animal subject, so that the display effect of the picture is enhanced due to the local processing of white balance. After the local white balance, the flowers and the grass turn green, the color of the puppy also turns grey, the local white balance is carried out by utilizing the saliency image, the white balance of the puppy can be kept not to be adjusted, and the white balance effect of the background is improved.

Fig. 8(a) is the picture content of the third original image, as shown in fig. 8(a), the animal subject and the background are equally clear, resulting in a reduction in the display effect of the animal subject; fig. 8(b) is a schematic diagram illustrating a result of performing saliency analysis on a third original image according to an embodiment of the present invention, and fig. 8(b) illustrates that a degree of saliency of an animal subject is determined through the saliency analysis, and two saliency areas of the animal subject and a background are divided according to differences of saliency effects; fig. 8(c) is a schematic view illustrating an effect of performing image processing on the second original image according to the embodiment of the present invention, as shown in fig. 8(c), in this embodiment, blurring processing is performed on a background, and no processing is performed on an animal subject, so that a display effect of the animal subject in the picture is improved by the processing.

Although the embodiments of the present invention have been described above, the above description is only for the convenience of understanding the present invention, and is not intended to limit the present invention. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

Claims (15)

1. An apparatus for implementing image processing, comprising: the device comprises an analysis unit, a determination unit and a processing unit; wherein,

the analysis unit is used for carrying out significance analysis on the original image;

the determining unit is used for distinguishing the original image into a significant area containing at least two different significant effects according to a significant analysis result;

and the processing unit is used for processing each salient region by adopting a corresponding image processing method.

2. The apparatus according to claim 1, further comprising a segmentation unit configured to segment the salient regions with different significance effects before the salient regions are processed by the corresponding image processing methods.

3. The apparatus according to claim 1, wherein the analysis unit is specifically configured to perform saliency analysis by performing image contrast comparison on each pixel of the original image with colors, luminances and directions of surrounding pixels to obtain a corresponding saliency value of each pixel.

4. The device according to claim 1, 2 or 3, characterized in that the analysis unit is in particular adapted to,

and performing image contrast analysis on the original image by adopting a regional contrast RC algorithm, and performing significance analysis on the original image through the image contrast analysis.

5. The apparatus according to claim 2, wherein the segmentation unit is further configured to extract an outline of each significant region having a different significance effect and/or fill a cavity inside each significant region having a different significance effect by using mathematical morphology when segmenting each significant region having a different significance effect.

6. A method of implementing image processing, comprising:

carrying out significance analysis on the original image;

distinguishing the original image into a significant area containing at least two different significant effects according to a significant analysis result;

and processing each salient region by adopting a corresponding image processing method.

7. The method according to claim 6, wherein distinguishing the original image as a salient region containing at least two different salient effects specifically comprises:

and according to the area significance numerical value in the significance analysis result, combining a preset distinguishing threshold value, and distinguishing the original image into significance areas containing at least two different significance effects.

8. The method according to claim 6 or 7, wherein when the original image is divided into a main area and a background area with different significance effects, the preset discrimination threshold comprises: the significance value range of the main area is greater than 64 and less than or equal to 255, and the significance value range of the background area is greater than or equal to 0 and less than 64.

9. The method according to claim 6 or 7, wherein before the processing of each salient region by the corresponding image processing method, the method further comprises:

and dividing the significant regions having different significant effects.

10. The method according to claim 6 or 7, wherein the saliency analysis of the raw image comprises in particular: and comparing the contrast of the image of each pixel point of the original image with the color, brightness and direction of the surrounding pixel points to obtain a corresponding significance value of each pixel point, and performing significance analysis.

11. The method according to claim 6, 7 or 10, wherein the saliency analysis of the original image comprises in particular:

and performing image contrast analysis on the original image by adopting a regional contrast RC algorithm, and performing significance analysis on the original image through the image contrast analysis.

12. The method according to claim 11, wherein said performing a saliency analysis of the raw image comprises in particular:

dividing the original image into N areas according to the set pixel size, and calculating the area r by the formula (1)_kS (r) is a significance value of_k) Comprises the following steps:

$S\left(r_k\right)=\underset{i=1,i\≠k}{\overset{n}{\Σ}}\mathrm{\ω}\left(r_i\right)D_r(r_k,r_i)---\left(1\right)$

wherein r is_iRepresents a difference from r_kRegion of (d), ω (r)_i) Is a region r_iWeighted value of D_r(r_k,r_i) Is the difference in color distance between two regions, D_r(r_k,r_i) The calculation formula is as follows:

$D_r(r_1,r_2)=\underset{i=1}{\overset{n_1}{\Σ}}\underset{j=1}{\overset{n_2}{\Σ}}{f}_1\left(i\right)f_2\left(j\right)d(c_{1,i},c_{2,j})---\left(2\right)$

wherein f is₁(i) Indicating the region r₁All statistical color types n of the ith color in the region₁The probability of occurrence of (a); f. of₂(i) Is a region r₂All statistical color types n of the jth color in the region₂The probability of occurrence of (a); d (c)_1,i,c_2,j) Is r₁The ith color c_1,iAnd r₂Middle j color c_2,jA distance difference of (d);

to S (r)_k) Adding a difference value of the space distance of the region to obtain a region significance value as follows:

$S^{\′}\left(r_k\right)=\underset{i=1,i\≠k}{\overset{n}{\Σ}}\exp (-D_s(r_k,r_i)/{\mathrm{\σ}}_s^2)\mathrm{\ω}\left(r_i\right)D_r(r_k,r_i)---\left(3\right)$

wherein D is_s(r_k,r_i) Euclidean distance, σ, of region barycenter for two regions_sThe adjustment factor is influenced for the spatial distance.

13. The method according to claim 6, 7 or 8, characterized in that the image processing method comprises: adjusting exposure value, blurring processing, white balance special effect, background replacement, saliency value adjustment, color gradation adjustment, brightness adjustment, hue/saturation adjustment, color replacement, gradient mapping, and/or photo filter.

14. The method according to claim 7, wherein when segmenting saliency areas that differ in saliency effect, the method further comprises:

and extracting the outline of the significant region with different significant effects and/or filling the inner cavity of the region of the significant region with different significant effects by using mathematical morphology.

15. The method according to claim 14, wherein the extracting the outline of the significant region with different significant effects by using mathematical morphology specifically comprises:

performing contour extraction on the binary images obtained through calculation by using the binary images of the saliency areas with different saliency effects obtained through calculation through expansion, corrosion, opening and closing operations;

the filling of the internal cavity of the region of the significant region with different significant effects specifically includes:

respectively calculating corresponding binary images for the salient regions with different salient effects, extracting the outline of the interior of the binary image of each salient region to obtain an internal outline, determining the internal outline smaller than a preset area as an internal cavity, and filling pixels in the internal cavity.