CN118096505A - Commodity display picture generation method and system - Google Patents

Commodity display picture generation method and system Download PDF

Info

Publication number
CN118096505A
CN118096505A CN202410518744.6A CN202410518744A CN118096505A CN 118096505 A CN118096505 A CN 118096505A CN 202410518744 A CN202410518744 A CN 202410518744A CN 118096505 A CN118096505 A CN 118096505A
Authority
CN
China
Prior art keywords
processed
image
style
region
area
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202410518744.6A
Other languages
Chinese (zh)
Inventor
吴立军
曲书磊
姚冰玉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xiamen 20000 Li Culture Media Co ltd
Original Assignee
Xiamen 20000 Li Culture Media Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xiamen 20000 Li Culture Media Co ltd filed Critical Xiamen 20000 Li Culture Media Co ltd
Priority to CN202410518744.6A priority Critical patent/CN118096505A/en
Publication of CN118096505A publication Critical patent/CN118096505A/en
Pending legal-status Critical Current

Links

Landscapes

  • Image Analysis (AREA)

Abstract

The invention discloses a commodity display picture generation method and a system, which particularly relate to the technical field of image processing and are used for solving the problem of local processing of commodity pictures. The acquisition of the evaluation qualification coefficients effectively guides the subsequent image processing and optimizing work, and improves the accuracy and effect of the image processing. After the evaluation qualification is determined, the convolutional neural network and the style migration technology are adopted to perform style optimization on the area to be processed, so that the consistency of the overall style of the image is maintained, and the replacement and modification requirements are met. The method is beneficial to meeting the replacement and modification requirements while not damaging the original style, thereby improving the image quality and visual effect and providing powerful support for displaying and popularizing commodity images.

Description

Commodity display picture generation method and system
Technical Field
The invention relates to the technical field of image processing, in particular to a commodity display picture generation method and system.
Background
In the local processing of the current commodity picture, particularly when the position of the commodity image is replaced, the area needing to be replaced is usually selected directly. However, there are some significant disadvantages to this approach. Firstly, the replaced commodity picture is often different from the original picture in structure, content and style, so that the replaced effect is abrupt and uncoordinated, and the natural fusion effect cannot be achieved. Secondly, the conventional local replacement method often lacks flexible judgment on structure, content and style differences between the replacement area and the non-replacement area, and cannot be evaluated autonomously. Therefore, when the style migration technology of the original picture is used for processing the replaced picture, the effect is often not as good as that of the original picture, and the processing result is difficult to accurately control. The traditional image local replacement method has a certain limitation in the aspect of self-evaluation of the replaced image, and a more flexible and intelligent method is needed to improve the processing effect and better adapt to the requirements of different scenes.
In order to solve the above problems, a technical solution is now provided.
Disclosure of Invention
In order to overcome the defects in the prior art, the embodiment of the invention provides a commodity display picture generation method and a commodity display picture generation system, which are used for determining qualification of a non-to-be-processed area for evaluating the to-be-processed area by comprehensively utilizing a fractal dimension algorithm and a local contrast matching algorithm and analyzing structural complexity difference and coordination of the to-be-processed area and the non-to-be-processed area. The acquisition of the evaluation qualification coefficients effectively guides the subsequent image processing and optimizing work, and improves the accuracy and effect of the image processing. After the evaluation qualification is determined, the convolutional neural network and the style migration technology are adopted to perform style optimization on the area to be processed, so that the consistency of the overall style of the image is maintained, and the replacement and modification requirements are met. The method is beneficial to meeting the replacement and modification requirements while not damaging the original style, thereby improving the image quality and visual effect, providing powerful support for displaying and popularizing commodity images and solving the problems in the background technology.
In order to achieve the above purpose, the present invention provides the following technical solutions: step S1, inputting a picture to be processed, and marking the boundary of the picture to be processed to obtain a region to be processed;
step S2, evaluating whether the non-processing area has the qualification of self-evaluating the area to be processed through the structural feature of the area combined with the significance difference analysis;
Step S3, judging whether the non-processing area has qualification of self-evaluation on the area to be processed based on the structure and content difference evaluation result;
Step S4, migrating the style of the non-to-be-processed area to the processed content of the to-be-processed area by using an image style migration technology, and calculating the style difference degree between the migrated image and the original image;
And S5, according to the style difference measurement result, providing an image processing quality signal.
In a preferred embodiment, step S1 comprises the following:
s1-1, loading a picture to be processed into a computer as an input of processing;
S1-2, accurately marking the region to be processed by using an interactive marking tool, wherein the region to be processed obtained after marking is represented as a binary mask, the pixel value of the binary mask is 1, and the pixel value of the binary mask is 0, which represents the region to be processed;
S1-3, storing the pixel value corresponding to the area to be processed as backup data for subsequent processing.
In a preferred embodiment, step S2 comprises the following:
Based on the obtained to-be-processed area and the non-to-be-processed area, obtaining a fractal complexity difference index through a fractal dimension algorithm; and obtaining a cooperative scheduling index through a local contrast matching algorithm.
In a preferred embodiment, the fractal complexity difference index is obtained as follows:
S2-1, different scales are set Each window will cover the image and be used to calculate the local curvature, the window sliding from the upper left corner of the image in the horizontal and vertical directions to cover the entire image;
S2-2, combining pixels in each window into a curve by connecting adjacent pixel points to obtain a continuous curve;
once the pixel curve is obtained, calculating a curvature value at each point on the curve by using a three-point difference method;
Counting the curvature values to obtain local curvature characteristics of the whole window;
S2-2, estimating local fractal dimension by using a fractal geometric theory according to a calculation result of the local curvature, wherein a calculation formula is as follows:
Wherein, Representing local fractal dimension,/>Is the scale/>The minimum number of units covered by the lower curve;
s2-3, respectively calculating fractal dimensions of the area to be treated and the untreated area to describe the complexity of the fractal dimensions;
S2-4, comparing the to-be-processed area with the untreated area, calculating the difference value of the to-be-processed area and the untreated area, and marking the difference value as a fractal complexity difference index.
In a preferred embodiment, the process of obtaining the coordination index is as follows:
s3-1, dividing an image into a plurality of overlapped image blocks;
S3-2, calculating the local contrast of each image block to reflect the texture and detail information in the image block, wherein the local contrast is obtained through the following calculation formula:
Wherein, Representing the upper left corner coordinates of an image block,/>Representing gray values of pixels in an image block,/>Representing the size of an image block,/>Representing pixel positions relative to the upper left corner of the image block;
s3-3, performing contrast matching on each image block, comparing the local contrast with the adjacent image blocks to obtain a contrast difference value, and marking the image blocks as saliency mutation areas if the contrast difference value exceeds a contrast threshold value;
s3-4, comparing each image block in the step S3-3 to determine a salient mutation region;
s3-5, analyzing and extracting the salient mutation region to obtain a salient region;
s3-6, respectively obtaining salient region images of the region to be processed and the region not to be processed according to the processing procedure of the step S3-5;
S3-7, setting a salient region image of the region to be processed as The saliency area image of the non-to-be-processed area isRespectively use/>And/>Representation/>And/>In (3) significant region, use/>And/>Representation/>And/>In the region of the difference in (c) and (d),And/>Respectively express/>And/>The coordination index is calculated according to the following formula:
Wherein, Representing the coordination index.
In a preferred embodiment, S3-5, the region of the significance mutation is analyzed and extracted to obtain the region of significance as follows:
Firstly, performing two-dimensional Fourier transform on an image of a salient mutation region to obtain a spectrum representation
Then, the significance score is calculated by using the amplitude and phase information of the frequency components, and the calculation formula is as follows:;
Wherein, Representing the amplitude of the frequency component,/>Representing the phase of the frequency component,/>Is the circumference ratio;
and finally, carrying out inverse Fourier transform on the frequency domain image according to the saliency score to obtain a saliency region image.
In a preferred embodiment, step S3 comprises the following:
carrying out dimensionless comprehensive processing calculation by utilizing the fractal complexity difference index and the coordination index to obtain an evaluation qualification coefficient;
Comparing the evaluation qualification coefficient with an evaluation qualification threshold value, and judging whether the non-to-be-processed area qualifies for evaluating the to-be-processed area, specifically:
if the evaluation qualification coefficient is greater than or equal to the evaluation possessing threshold value, generating possessing signals;
otherwise, if the evaluation qualification coefficient is smaller than the evaluation qualification threshold, generating a non-qualification signal.
In a preferred embodiment, step S4 comprises the following:
S4-1, marking the region as a processed region after the image replacement of the region to be processed is completed, and respectively loading the processed region and the non-processed region into a computer memory after the signal is obtained;
S4-2, respectively carrying out forward propagation on the images of the processed area and the non-processed area by using a pre-trained convolutional neural network, and extracting corresponding style characteristics and content characteristics;
s4-3, calculating a Gram matrix of the extracted style characteristics;
S4-4, defining style losses and content losses, wherein the style losses measure the difference of styles by comparing the differences between Gram matrixes, and the content losses measure the retention of the content by comparing the differences between feature graphs;
S4-5, optimizing the image of the processed area by minimizing style loss and content loss by using an optimization algorithm such as gradient descent;
S4-6, extracting style characteristics of the migrated image of the processed area and the original image of the area to be processed respectively in a mode of extracting the image through a convolutional neural network of S4-2;
s4-7, using the extracted style characteristic representation, calculating the style difference degree between the migrated image and the original image, namely calculating the Frobenius norm between the style characteristics of the migrated image and the original image to obtain the style difference degree.
In a preferred embodiment, step S5 comprises the following:
Comparing the style difference degree with a difference threshold value, and generating a prompt signal when the style difference degree is greater than or equal to the difference threshold value; and otherwise, when the style difference degree is smaller than the difference threshold value, generating a qualified signal.
A commodity display picture generation system comprises a detection boundary module, a content analysis module, a region assessment module, a style migration module and a processing prompt module;
The boundary detection module inputs the picture to be processed, marks the boundary of the picture to be processed to obtain a region to be processed, and sends the marking result to the content analysis module;
the content analysis module evaluates whether the non-processing area has qualification of self-evaluating the area to be processed through the combination of the structural characteristics of the area and the significance difference analysis, and sends the result of the structural characteristics and the significance difference analysis to the area evaluation module;
the region evaluation module judges whether the non-processing region has qualification of self-evaluation on the region to be processed based on the structure and content difference evaluation result, and sends the evaluation result to the style migration module;
The style migration module utilizes an image style migration technology to migrate the style of the non-to-be-processed area to the processed content of the to-be-processed area, calculates the style difference degree between the migrated image and the original image, and sends the style difference degree to the processing prompt module;
And the processing prompt module provides an image processing quality signal according to the style difference measurement result.
The commodity display picture generation method and the commodity display picture generation system have the technical effects and advantages that:
1. The invention adopts fractal dimension algorithm and local contrast matching algorithm to evaluate the difference and coordination of the structural complexity between the area to be processed and the area not to be processed. Firstly, calculating a fractal complexity difference index through a fractal dimension algorithm, and carrying out fractal dimension analysis on a region to be processed and a region not to be processed, so as to describe the complexity of the region to be processed. Secondly, a local contrast matching algorithm is utilized to obtain a coordination index, and local textures and details of the image are analyzed through steps of window scanning, curvature calculation and the like so as to reflect coordination of the local textures and details. And obtaining an evaluation qualification coefficient by integrating the fractal complexity difference index and the coordination index, wherein the evaluation qualification coefficient is used for judging whether the non-to-be-processed area has qualification for evaluating the to-be-processed area. The evaluation qualification coefficient reflects the evaluation readiness of the non-to-be-processed area to the to-be-processed area, and further influences the subsequent image processing and optimization work. Thereby facilitating the high-efficiency analysis of the self-evaluable degree under the condition of partial commodity image processing, promoting a more accurate image processing scheme and improving the accuracy and effect of image processing.
2. According to the invention, after the non-to-be-processed area is definitely qualified for evaluating the to-be-processed area, the convolution neural network and the style migration technology are adopted, and the quality and effect of image processing can be effectively improved by replacing the image of the to-be-processed area and performing style optimization. Firstly, extracting the content and style characteristics of an image by using a convolutional neural network, and then migrating the style of a non-to-be-processed area to the to-be-processed area by using a style migration technology, so that the image of the to-be-processed area is matched with the style of the non-to-be-processed area. Then, the processing effect can be objectively evaluated by calculating the style difference degree between the original image and the image after style migration. The method is beneficial to maintaining the consistency of the overall style of the image and improving the consistency and consistency of image processing. Meanwhile, through a style migration technology, style optimization of the image of the region to be processed can be realized, so that the image can meet the requirement of replacement and modification while the original style is not damaged. The original style is not destroyed, and the replacement and modification requirements are met, so that the image quality and visual effect are improved, and powerful support is provided for displaying and popularizing commodity images.
Drawings
Fig. 1 is a schematic flow chart of a commodity display picture generation method according to the present invention;
Fig. 2 is a schematic structural diagram of a merchandise display picture generating system according to the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Examples
Fig. 1 shows a commodity display picture generating method according to the present invention, including:
Step S1, inputting a picture to be processed, and marking the boundary of the picture to be processed to obtain a region to be processed;
step S2, evaluating whether the non-processing area has the qualification of self-evaluating the area to be processed through the structural feature of the area combined with the significance difference analysis;
Step S3, judging whether the non-processing area has qualification of self-evaluation on the area to be processed based on the structure and content difference evaluation result;
Step S4, migrating the style of the non-to-be-processed area to the processed content of the to-be-processed area by using an image style migration technology, and calculating the style difference degree between the migrated image and the original image;
And S5, according to the style difference measurement result, providing an image processing quality signal.
In the commodity picture processing process, local modification is often required to be carried out on the picture, and in order to ensure the processing accuracy and the convenience of subsequent analysis, the local to be modified must be backed up. This serves two main purposes: firstly, when unsatisfactory conditions occur in the modification process, the operation of withdrawing can be carried out, so that the flexibility of processing and the controllability of the effect are ensured; secondly, the overall front-rear difference condition of the part to be treated and other non-treated parts after the post analysis treatment is finished is used for better evaluating the treatment effect and the improvement direction. By backing up the part to be processed, the original and modified areas can be compared at any time in the processing process, so that the processing accuracy and consistency are ensured. The backup operation can not only improve the processing efficiency, but also help to ensure the quality and controllability of commodity picture processing.
Step S1 includes the following:
s1-1, loading a picture to be processed into a computer as an input of processing;
S1-2, accurately marking the region to be processed by using an interactive marking tool, wherein the region to be processed obtained after marking is represented as a binary mask, the pixel value of the binary mask is 1, and the pixel value of the binary mask is 0, which represents the region to be processed;
S1-3, storing the pixel value corresponding to the area to be processed as backup data for subsequent processing. The backup process can be represented by the following complex formula:
Wherein, Pixel value representing corresponding position of region to be processed,/>Pixel value representing original picture,/>A binary mask representing the region to be treated.
Step S2 includes the following:
Based on the obtained to-be-processed area and the non-to-be-processed area, obtaining a fractal complexity difference index through a fractal dimension algorithm; and obtaining a cooperative scheduling index through a local contrast matching algorithm.
The fractal complexity difference index is obtained as follows:
S2-1, different scales are set Is scanned over the image. Each window will cover the image and be used to calculate the local curvature, the window sliding from the upper left corner of the image, in both the horizontal and vertical directions, to cover the entire image;
S2-2, combining pixels in each window into a curve by connecting adjacent pixel points to obtain a continuous curve;
once the pixel curve is obtained, calculating a curvature value at each point on the curve by using a three-point difference method;
Counting the curvature values to obtain local curvature characteristics of the whole window;
S2-2, estimating local fractal dimension by using a fractal geometric theory according to a calculation result of the local curvature, wherein a calculation formula is as follows:
Wherein, Representing local fractal dimension,/>Is the scale/>The minimum number of units covered by the lower curve.
In fractal analysis, window sizeIs an important factor affecting the fractal dimension. Along with windowMouth dimensions/>Increased local fractal dimension/>And also increases accordingly. This is because larger window dimensions can better capture detail and structural information in the image, resulting in an increase in local fractal dimension.
S2-3, respectively calculating fractal dimensions of the area to be treated and the untreated area to describe the complexity of the fractal dimensions;
S2-4, comparing the to-be-processed area with the untreated area, calculating the difference value of the to-be-processed area and the untreated area, and marking the difference value as a fractal complexity difference index.
The fractal complexity difference index is used for representing and reflecting the structural complexity difference between the area to be treated and the untreated area. The larger the value of this index, the greater the structural difference between the area to be treated and the untreated area, the more obvious the difference in complexity; conversely, a smaller index indicates a smaller structural difference between the two, and a relatively close complexity.
The acquisition process of the cooperative scheduling index is as follows:
s3-1, dividing an image into a plurality of overlapped image blocks;
S3-2, calculating the local contrast of each image block to reflect the texture and detail information in the image block, wherein the local contrast is obtained through the following calculation formula:
Wherein, Representing the upper left corner coordinates of an image block,/>Representing gray values of pixels in an image block,/>Representing the size of an image block,/>Representing the pixel position relative to the upper left corner of the image block.
S3-3, performing contrast matching on each image block, comparing the local contrast with the adjacent image blocks to obtain a contrast difference value, and marking the image blocks as saliency mutation areas if the contrast difference value exceeds a contrast threshold value;
s3-4, comparing each image block in the step S3-3 to determine a salient mutation region;
For images that have been obtained with regions of significant mutation, further analysis and extraction of regions of significant interest is performed in order to more accurately identify and locate important features and structures in the image. Although the region of significance mutation can provide some information, further processing is still required to obtain a more specific, meaningful region of significance. Specifically, the purposes of analyzing and extracting the salient regions include: refinement of region boundaries, attention weighting, feature extraction and recognition, and application requirements. In summary, although the salient mutation region has been obtained, analysis and extraction of the salient region are required to further optimize the feature extraction and recognition process of the image, and improve the accuracy and effect of image processing.
S3-5, analyzing and extracting the salient mutation region to obtain the salient region, wherein the process is as follows:
Firstly, performing two-dimensional Fourier transform on an image of a salient mutation region to obtain a spectrum representation
Then, the significance score is calculated by using the amplitude and phase information of the frequency components, and the calculation formula is as follows:;
Wherein, Representing the amplitude of the frequency component,/>Representing the phase of the frequency component,/>Is the circumference ratio;
and finally, carrying out inverse Fourier transform on the frequency domain image according to the saliency score to obtain a saliency region image.
The inverse fourier transform of the frequency domain image is performed in order to convert the frequency domain representation back to a spatial domain representation, resulting in an image of the saliency region.
S3-6, respectively obtaining salient region images of the region to be processed and the region not to be processed according to the processing procedure of the step S3-5;
S3-7, setting a salient region image of the region to be processed as The saliency area image of the non-to-be-processed area is. Respectively use/>And/>Representation/>And/>In (3) significant region, use/>And/>Representation/>And/>In the region of the difference in (c) and (d),And/>Respectively express/>And/>The coordination index is calculated according to the following formula:
Wherein:
A coordination index is represented, and the coordination degree between the to-be-processed area and the non-to-be-processed area is represented;
representing the similarity of the salient regions of the region to be processed and the non-region to be processed, wherein the higher the similarity is, the larger the product value is;
the difference between the salient areas of the to-be-treated area and the non-to-be-treated area is represented, and the lower the difference is, the smaller the denominator is;
And/> The total area of the to-be-treated area and the non-to-be-treated area are respectively represented, and the influence of the area size on harmony is considered through the ratio of the areas;
And/> The standard deviation and the average value of the saliency areas of the to-be-processed area and the non-to-be-processed area are respectively represented, and the influence of the statistical characteristics of the saliency areas on coordination is considered.
The coordination index indicates the degree of coordination between the region to be treated and the region not to be treated. In particular, a larger value indicates a higher coordination between the region to be treated and the region not to be treated, i.e. a more similar, less different, and more uniform distribution of the regions of salience between them. Conversely, a smaller value indicates a lower coordination between the region to be treated and the region not to be treated, i.e. a larger difference in the salient regions between them, a non-uniform distribution.
Step S3 includes the following:
the fractal complexity difference index and the coordination degree index are used for carrying out dimensionless comprehensive processing calculation to obtain an evaluation qualification coefficient, and the evaluation qualification coefficient can be calculated through the following formula:
Wherein, To evaluate the qualification factor,/>And/>Respectively the minimum value and the maximum value of the coordination index,/>And/>The minimum and maximum values of the fractal complexity difference index are respectively.
The evaluation qualification coefficient is used for comprehensively considering the structural complexity difference and coordination between the to-be-processed area and the non-to-be-processed area, so as to evaluate whether the non-to-be-processed area is qualified for evaluating the to-be-processed area. The larger the evaluation qualification coefficient is, the higher the evaluation qualification of the non-to-be-processed area to the to-be-processed area is, namely the structure complexity difference between the non-to-be-processed area and the to-be-processed area is smaller, and the coordination is better; conversely, the smaller the evaluation qualification coefficient, the lower the evaluation qualification of the non-to-be-processed area to the to-be-processed area, namely, the greater the difference of the structural complexity between the non-to-be-processed area and the to-be-processed area, or the poorer the coordination. Therefore, the size of the evaluation qualification coefficient may reflect the level of the evaluation readiness of the non-processing region to the processing region.
And comparing the evaluation qualification coefficient with the evaluation qualification threshold value to judge whether the non-to-be-processed area qualifies for evaluating the to-be-processed area. Specifically:
if the evaluation qualification coefficient is greater than or equal to the evaluation qualification threshold, the evaluation qualification coefficient indicates that the evaluation readiness of the non-to-be-processed area to the to-be-processed area is enough and the evaluation qualification coefficient qualifies the to-be-processed area to evaluate. The method means that the difference of the structural complexity between the non-to-be-processed area and the to-be-processed area is small, the coordination is good, and effective evaluation analysis can be performed to generate signals;
Otherwise, if the evaluation qualification coefficient is smaller than the evaluation qualification threshold, the evaluation qualification coefficient indicates that the evaluation readiness of the non-to-be-processed area to the to-be-processed area is insufficient, and the evaluation qualification coefficient does not qualify the to-be-processed area to evaluate. This means that the difference in structural complexity between the non-treated area and the treated area is large, or the coordination is poor, and no signal is generated.
The invention adopts fractal dimension algorithm and local contrast matching algorithm to evaluate the difference and coordination of the structural complexity between the area to be processed and the area not to be processed. Firstly, calculating a fractal complexity difference index through a fractal dimension algorithm, and carrying out fractal dimension analysis on a region to be processed and a region not to be processed, so as to describe the complexity of the region to be processed. Secondly, a local contrast matching algorithm is utilized to obtain a coordination index, and local textures and details of the image are analyzed through steps of window scanning, curvature calculation and the like so as to reflect coordination of the local textures and details. And obtaining an evaluation qualification coefficient by integrating the fractal complexity difference index and the coordination index, wherein the evaluation qualification coefficient is used for judging whether the non-to-be-processed area has qualification for evaluating the to-be-processed area. The evaluation qualification coefficient reflects the evaluation readiness of the non-to-be-processed area to the to-be-processed area, and further influences the subsequent image processing and optimization work. Thereby facilitating the high-efficiency analysis of the self-evaluable degree under the condition of partial commodity image processing, promoting a more accurate image processing scheme and improving the accuracy and effect of image processing.
And calculating the style difference degree between the migrated image and the original image, and proposing an image processing optimization suggestion according to the difference measurement result, thereby having great significance for partial area modification of the commodity image. Firstly, through the calculation of the style difference degree, the style consistency between the area to be processed and the non-processed area can be objectively evaluated, namely, the similarity of the processed image and the original image in style. This helps to determine whether the non-processing region qualifies for self-evaluation of the region to be processed, i.e., to determine whether the non-processing region is suitable for evaluating the effect of the region to be processed. Secondly, an image processing optimization suggestion is provided according to the style difference measurement result, subsequent processing work can be guided, and the region to be processed can be adjusted and improved in a targeted manner. If the style difference is large, meaning that the processed image and the original image have obvious difference in style, the processing method of the area to be processed is recommended to be optimized so as to improve the processing effect and the consistency with the original image. On the contrary, if the degree of style difference is small, the processing effect is considered to be good, and no substantial adjustment is required. Therefore, by calculating the style difference degree and proposing the optimization suggestion, the modification effect of the partial area of the commodity image can be effectively evaluated and improved, and the quality and effect of the image processing are improved, so that the requirements of users are better met, and the attraction and competitiveness of commodity display are improved.
Step S4 includes the following:
S4-1, marking the region as a processed region after the image replacement of the region to be processed is completed, and respectively loading the processed region and the non-processed region into a computer memory after the signal is obtained;
S4-2, respectively carrying out forward propagation on the images of the processed area and the non-processed area by using a pre-trained convolutional neural network (such as a VGG network), and extracting the style characteristics and the content characteristics of the images.
For example, the following is a complete example demonstrating how images of processed and non-processed regions are propagated forward using a pretrained VGG network, extracting their style and content features:
importtensorflowastf
fromtensorflow.keras.applicationsimportVGG19
fromtensorflow.keras.applications.vgg19importpreprocess_input
fromtensorflow.keras.modelsimportModel
importnumpyasnp
importcv2
# load Pre-trained VGG19 model (excluding fully connected layers)
vgg=VGG19(weights='imagenet',include_top=False)
Selecting certain middle layer outputs of VGG19 model as style and content features
style_layers=['block1_conv1','block2_conv1','block3_conv1','block4_conv1','block5_conv1']
content_layers=['block4_conv2']
# Create a new model, retaining only the specified middle layer output
style_outputs=[vgg.get_layer(name).outputfornameinstyle_layers]
content_outputs=[vgg.get_layer(name).outputfornameincontent_layers]
feature_extractor=Model(inputs=vgg.input,outputs=style_outputs+content_outputs)
# Loading and preprocessing images of the processed and non-processed regions
Processed_img1= preprocess _input (cv2.imread ('processed_region. Jpg')) # processed region image
Processed_img1=tf.image.size (processsed_img1, (224, 224)) # resizing the image
Processed_img1=tf.expand_ dims (processsed_img1, axis=0) # expands dimensions to match model input format
Processed_img2= preprocess _input (cv2.imread ('non-processed_region. Jpg')) # non-region image to be processed
Processed_img2=tf.image.size (processsed_img2, (224, 224)) # resizing the image
Processed_img2=tf.expand_ dims (processsed_img2, axis=0) # expands dimensions to match model input format
Forward propagation using model, # extraction of style features and content features
style_features=feature_extractor(processed_img1)[:len(style_layers)]
content_features=feature_extractor(processed_img2)[len(style_layers):]
# Convert extracted features into NumPy arrays and remove batch dimensions
style_features=[style_layer[0]forstyle_layerinstyle_features]
content_features=[content_layer[0]forcontent_layerincontent_features]
Shape of# print extracted feature
fori,style_featureinenumerate(style_features):
print(f'Stylefeature{i+1}shape:{style_feature.shape}')
fori,content_featureinenumerate(content_features):
print(f'Contentfeature{i+1}shape:{content_feature.shape}')
The above code demonstrates how the pre-trained VGG19 model can be used to perform feature extraction on images of both the treated region and the non-treated region. First, a pre-trained VGG19 model is loaded, and some intermediate layers are selected as extraction layers for style and content features. Then, the images of the processed and non-processed regions are loaded and preprocessed, and their style and content features are extracted by forward propagation of the model. Finally, the extracted features are converted into NumPy arrays and their shapes are output for further processing and analysis.
S4-3, calculating Gram matrix of the extracted style characteristics. The Gram matrix may reflect the correlation between features and is an important indicator of style representation.
S4-4, defining style losses and content losses, wherein the style losses measure the difference of styles by comparing the differences between Gram matrixes, and the content losses measure the retention of the content by comparing the differences between feature graphs;
S4-5, optimizing the image of the processed area by minimizing the style loss and the content loss so that the image is similar to the style of the non-processed area. Optimization is performed using an optimization algorithm such as gradient descent, and the following is an example of optimization by gradient descent:
S4-5-1, firstly, taking an image of a processed area as an initial value of optimization, wherein the image is taken as an initial input of an optimization algorithm;
S4-5-2, defining style loss and content loss functions. The style loss function measures the style difference between the image of the processed region and the image of the non-processed region, and the content loss function measures the difference between the image of the processed region and its original content.
And S4-5-3, adding the style loss and the content loss in a weighted manner to obtain a total loss function. The total loss function is an optimization objective and needs to be minimized.
S4-5-4, calculating the gradient of the total loss function relative to the processed area image by using a back propagation algorithm. The gradient represents the direction in which the total loss function changes most rapidly at the current processed region image location.
S4-5-5, updating the image of the processed area along the gradient direction of the loss function by using a gradient descent algorithm. The updated step size is controlled by the learning rate parameter.
Steps S4-5-3 and S4-5-4 are repeated until a set number of optimization iterations is reached or the loss function converges to a set threshold.
After optimization, the obtained image is the image after migration, wherein the content of the processed area is matched with the style of the non-processed area.
S4-6, extracting style characteristics of the migrated image of the processed area and the original image of the area to be processed respectively in a mode of extracting the image through a convolutional neural network of S4-2;
s4-7, using the extracted style characteristic representation, calculating the style difference degree between the migrated image and the original image, namely calculating the Frobenius norm between the style characteristics of the migrated image and the original image to obtain the style difference degree.
The style difference degree is used for measuring the style difference degree between the migrated processed area image and the original image of the area to be processed. When the style difference degree is larger, the style difference between the two images is more obvious, and the style difference degree between the migrated image and the original image is larger; on the contrary, when the style difference degree is smaller, the style difference between the two is smaller, and the style similarity between the migrated image and the original image is higher. Therefore, the magnitude of the style difference may reflect the degree of style consistency and similarity between the migrated image and the original image.
The effect of the image after processing can be known by calculating the style difference between the migrated image and the original image. When the style difference degree is smaller, the style similarity between the migrated image and the original image is higher, and the effect of the processed image is good, and the style of the processed image is consistent with that of the original image; on the contrary, when the style difference degree is larger, the processing effect of the partial region is poor, and the processed image and the original image have obvious difference in style. Therefore, through the evaluation of the style difference degree, important enlightenment can be provided for the partial area processing, and further optimization and adjustment are guided so as to improve the quality and effect of the image processing.
Step S5 includes the following:
And comparing the style difference degree with a difference threshold value, and judging whether the effect of the partial area processing reaches the expected value. When the style difference degree is larger than or equal to the difference threshold value, the style difference between the processed image and the original image is larger, the expected consistency requirement is not met, and further optimization is needed to generate a prompt signal; otherwise, when the style difference degree is smaller than the difference threshold, the style difference between the processed image and the original image is smaller, the expected requirement is met, the processing effect is good, and a qualified signal is generated.
According to the invention, after the non-to-be-processed area is definitely qualified for evaluating the to-be-processed area, the convolution neural network and the style migration technology are adopted, and the quality and effect of image processing can be effectively improved by replacing the image of the to-be-processed area and performing style optimization. Firstly, extracting the content and style characteristics of an image by using a convolutional neural network, and then migrating the style of a non-to-be-processed area to the to-be-processed area by using a style migration technology, so that the image of the to-be-processed area is matched with the style of the non-to-be-processed area. Then, the processing effect can be objectively evaluated by calculating the style difference degree between the original image and the image after style migration. The method is beneficial to maintaining the consistency of the overall style of the image and improving the consistency and consistency of image processing. Meanwhile, through a style migration technology, style optimization of the image of the region to be processed can be realized, so that the image can meet the requirement of replacement and modification while the original style is not damaged. The original style is not destroyed, and the replacement and modification requirements are met, so that the image quality and visual effect are improved, and powerful support is provided for displaying and popularizing commodity images.
Examples
FIG. 2 shows a commodity display picture generation system of the present invention, which comprises a detection boundary module, a content analysis module, a region assessment module, a style migration module and a processing prompt module;
The boundary detection module inputs the picture to be processed, marks the boundary of the picture to be processed to obtain a region to be processed, and sends the marking result to the content analysis module;
the content analysis module evaluates whether the non-processing area has qualification of self-evaluating the area to be processed through the combination of the structural characteristics of the area and the significance difference analysis, and sends the result of the structural characteristics and the significance difference analysis to the area evaluation module;
the region evaluation module judges whether the non-processing region has qualification of self-evaluation on the region to be processed based on the structure and content difference evaluation result, and sends the evaluation result to the style migration module;
The style migration module utilizes an image style migration technology to migrate the style of the non-to-be-processed area to the processed content of the to-be-processed area, calculates the style difference degree between the migrated image and the original image, and sends the style difference degree to the processing prompt module;
And the processing prompt module provides an image processing quality signal according to the style difference measurement result.
The above formulas are all formulas with dimensionality removed and numerical calculation, the formulas are formulas with the latest real situation obtained by software simulation through collecting a large amount of data, and preset parameters and threshold selection in the formulas are set by those skilled in the art according to the actual situation.
The above embodiments may be implemented in whole or in part by software, hardware, firmware, or any other combination. When implemented in software, the above-described embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product comprises one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, the processes or functions described in accordance with embodiments of the present application are produced in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website site, computer, server, or data center to another website site, computer, server, or data center by wired (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains one or more sets of available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. The semiconductor medium may be a solid state disk.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the several embodiments provided in the present application, it should be understood that the disclosed system and apparatus may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other forms.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, or the like) to perform all or part of the steps described in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, or other various media capable of storing program codes.
The foregoing is merely illustrative of the present application, and the present application is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Finally: the foregoing description of the preferred embodiments of the invention is not intended to limit the invention to the precise form disclosed, and any such modifications, equivalents, and alternatives falling within the spirit and principles of the invention are intended to be included within the scope of the invention.

Claims (8)

1. The commodity display picture generation method is characterized by comprising the following steps of:
Step S1, inputting a picture to be processed, and marking the boundary of the picture to be processed to obtain a region to be processed and a region not to be processed;
step S2, evaluating whether the non-processing area has the qualification of self-evaluating the area to be processed through the structural feature of the area combined with the significance difference analysis;
Step S3, judging whether the non-processing area has qualification of self-evaluation on the area to be processed based on the structure and content difference evaluation result;
Step S4, migrating the style of the non-to-be-processed area to the processed content of the to-be-processed area by using an image style migration technology, and calculating the style difference degree between the migrated image and the original image;
step S5, according to the style difference measurement result, an image processing quality signal is provided;
step S2 includes the following:
based on the obtained to-be-processed area and the non-to-be-processed area, obtaining a fractal complexity difference index through a fractal dimension algorithm; obtaining a coordination index through a local contrast matching algorithm;
the fractal complexity difference index is obtained as follows:
S2-1, different scales are set Each window will cover the image and be used to calculate the local curvature, the window sliding from the upper left corner of the image in the horizontal and vertical directions to cover the entire image;
S2-2, combining pixels in each window into a curve by connecting adjacent pixel points to obtain a continuous curve;
once the pixel curve is obtained, calculating a curvature value at each point on the curve by using a three-point difference method;
Counting the curvature values to obtain local curvature characteristics of the whole window;
S2-2, estimating local fractal dimension by using a fractal geometric theory according to a calculation result of the local curvature, wherein a calculation formula is as follows:
Wherein, Representing local fractal dimension,/>Is the scale/>The minimum number of units covered by the lower curve;
s2-3, respectively calculating fractal dimensions of the area to be treated and the untreated area to describe the complexity of the fractal dimensions;
S2-4, comparing the to-be-processed area with the untreated area, calculating the difference value of the to-be-processed area and the untreated area, and marking the difference value as a fractal complexity difference index.
2. The merchandise display picture generation method according to claim 1, wherein:
step S1 includes the following:
s1-1, loading a picture to be processed into a computer as an input of processing;
S1-2, accurately marking the region to be processed by using an interactive marking tool, wherein the region to be processed obtained after marking is represented as a binary mask, the pixel value of the binary mask is 1, and the pixel value of the binary mask is 0, which represents the region to be processed;
S1-3, storing the pixel value corresponding to the area to be processed as backup data for subsequent processing.
3. The merchandise display picture generation method according to claim 2, wherein:
the acquisition process of the cooperative scheduling index is as follows:
s3-1, dividing an image into a plurality of overlapped image blocks;
S3-2, calculating the local contrast of each image block to reflect the texture and detail information in the image block, wherein the local contrast is obtained through the following calculation formula:
Wherein, Representing the upper left corner coordinates of an image block,/>Representing gray values of pixels in an image block,/>Representing the size of an image block,/>Representing pixel positions relative to the upper left corner of the image block;
s3-3, performing contrast matching on each image block, comparing the local contrast with the adjacent image blocks to obtain a contrast difference value, and marking the image blocks as saliency mutation areas if the contrast difference value exceeds a contrast threshold value;
s3-4, comparing each image block in the step S3-3 to determine a salient mutation region;
s3-5, analyzing and extracting the salient mutation region to obtain a salient region;
s3-6, respectively obtaining salient region images of the region to be processed and the region not to be processed according to the processing procedure of the step S3-5;
S3-7, setting a salient region image of the region to be processed as The salient region image of the non-treated region is/>Respectively use/>And/>Representation/>And/>In (3) significant region, use/>And/>Representation/>And/>In (a) difference region,/>AndRespectively express/>And/>The coordination index is calculated according to the following formula:
Wherein, Representing the coordination index.
4. A merchandise display picture generation method according to claim 3, wherein:
s3-5, analyzing and extracting the salient mutation region to obtain the salient region, wherein the process is as follows:
Firstly, performing two-dimensional Fourier transform on an image of a salient mutation region to obtain a spectrum representation
Then, the significance score is calculated by using the amplitude and phase information of the frequency components, and the calculation formula is as follows:;
Wherein, Representing the amplitude of the frequency component,/>Representing the phase of the frequency component,/>Is the circumference ratio;
and finally, carrying out inverse Fourier transform on the frequency domain image according to the saliency score to obtain a saliency region image.
5. The commodity display picture generating method according to claim 4, wherein:
step S3 includes the following:
carrying out dimensionless comprehensive processing calculation by utilizing the fractal complexity difference index and the coordination index to obtain an evaluation qualification coefficient;
Comparing the evaluation qualification coefficient with an evaluation qualification threshold value, and judging whether the non-to-be-processed area qualifies for evaluating the to-be-processed area, specifically:
if the evaluation qualification coefficient is greater than or equal to the evaluation possessing threshold value, generating possessing signals;
otherwise, if the evaluation qualification coefficient is smaller than the evaluation qualification threshold, generating a non-qualification signal.
6. The commodity display picture generating method according to claim 5, wherein:
Step S4 includes the following:
S4-1, marking the region as a processed region after the image replacement of the region to be processed is completed, and respectively loading the processed region and the non-processed region into a computer memory after the signal is obtained;
S4-2, respectively carrying out forward propagation on the images of the processed area and the non-processed area by using a pre-trained convolutional neural network, and extracting corresponding style characteristics and content characteristics;
s4-3, calculating a Gram matrix of the extracted style characteristics;
S4-4, defining style losses and content losses, wherein the style losses measure the difference of styles by comparing the differences between Gram matrixes, and the content losses measure the retention of the content by comparing the differences between feature graphs;
S4-5, optimizing the image of the processed area by minimizing style loss and content loss by using an optimization algorithm such as gradient descent;
S4-6, extracting style characteristics of the migrated image of the processed area and the original image of the area to be processed respectively in a mode of extracting the image through a convolutional neural network of S4-2;
s4-7, using the extracted style characteristic representation, calculating the style difference degree between the migrated image and the original image, namely calculating the Frobenius norm between the style characteristics of the migrated image and the original image to obtain the style difference degree.
7. The merchandise display picture generation method of claim 6, wherein:
Step S5 includes the following:
Comparing the style difference degree with a difference threshold value, and generating a prompt signal when the style difference degree is greater than or equal to the difference threshold value; and otherwise, when the style difference degree is smaller than the difference threshold value, generating a qualified signal.
8. A merchandise display picture generation system for implementing the merchandise display picture generation method of any one of claims 1-7, comprising a detection boundary module, a content analysis module, a region assessment module, a style migration module, and a processing prompt module;
The boundary detection module inputs the picture to be processed, marks the boundary of the picture to be processed to obtain a region to be processed, and sends the marking result to the content analysis module;
the content analysis module evaluates whether the non-processing area has qualification of self-evaluating the area to be processed through the combination of the structural characteristics of the area and the significance difference analysis, and sends the result of the structural characteristics and the significance difference analysis to the area evaluation module;
the region evaluation module judges whether the non-processing region has qualification of self-evaluation on the region to be processed based on the structure and content difference evaluation result, and sends the evaluation result to the style migration module;
The style migration module utilizes an image style migration technology to migrate the style of the non-to-be-processed area to the processed content of the to-be-processed area, calculates the style difference degree between the migrated image and the original image, and sends the style difference degree to the processing prompt module;
And the processing prompt module provides an image processing quality signal according to the style difference measurement result.
CN202410518744.6A 2024-04-28 2024-04-28 Commodity display picture generation method and system Pending CN118096505A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410518744.6A CN118096505A (en) 2024-04-28 2024-04-28 Commodity display picture generation method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410518744.6A CN118096505A (en) 2024-04-28 2024-04-28 Commodity display picture generation method and system

Publications (1)

Publication Number Publication Date
CN118096505A true CN118096505A (en) 2024-05-28

Family

ID=91146011

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202410518744.6A Pending CN118096505A (en) 2024-04-28 2024-04-28 Commodity display picture generation method and system

Country Status (1)

Country Link
CN (1) CN118096505A (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111242841A (en) * 2020-01-15 2020-06-05 杭州电子科技大学 Image background style migration method based on semantic segmentation and deep learning
US20220383580A1 (en) * 2021-05-28 2022-12-01 Intel Corporation Real-time temporally consistent object segmented style transfer in media and gaming
WO2023151299A1 (en) * 2022-02-11 2023-08-17 华为云计算技术有限公司 Data generation method and apparatus, device, and storage medium
CN117726502A (en) * 2023-12-20 2024-03-19 贵州大学 Image nerve style migration method

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111242841A (en) * 2020-01-15 2020-06-05 杭州电子科技大学 Image background style migration method based on semantic segmentation and deep learning
US20220383580A1 (en) * 2021-05-28 2022-12-01 Intel Corporation Real-time temporally consistent object segmented style transfer in media and gaming
WO2023151299A1 (en) * 2022-02-11 2023-08-17 华为云计算技术有限公司 Data generation method and apparatus, device, and storage medium
CN117726502A (en) * 2023-12-20 2024-03-19 贵州大学 Image nerve style migration method

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
高文冉: "局部可控的风格迁移神经网络模型研究", 硕士电子期刊, 15 December 2021 (2021-12-15) *

Similar Documents

Publication Publication Date Title
CN109815865B (en) Water level identification method and system based on virtual water gauge
Liu et al. GcsDecolor: gradient correlation similarity for efficient contrast preserving decolorization
CN111598998A (en) Three-dimensional virtual model reconstruction method and device, computer equipment and storage medium
CN111709914B (en) Non-reference image quality evaluation method based on HVS characteristics
CN109657567B (en) Weak supervision characteristic analysis method and system based on 3D fingerprint image
CN107784288A (en) A kind of iteration positioning formula method for detecting human face based on deep neural network
CN112288645B (en) Skull face restoration model construction method and restoration method and system
CN109165654B (en) Training method of target positioning model and target positioning method and device
CN111666813B (en) Subcutaneous sweat gland extraction method of three-dimensional convolutional neural network based on non-local information
CN117372604B (en) 3D face model generation method, device, equipment and readable storage medium
US11880913B2 (en) Generation of stylized drawing of three-dimensional shapes using neural networks
CN111652876B (en) Method for detecting three-dimensional basin bottom ultrasonic image
CN115953330B (en) Texture optimization method, device, equipment and storage medium for virtual scene image
CN105069767A (en) Image super-resolution reconstruction method based on representational learning and neighbor constraint embedding
CN118096505A (en) Commodity display picture generation method and system
CN113467881B (en) Method and device for automatically adjusting chart style, computer equipment and storage medium
CN111914751B (en) Image crowd density identification detection method and system
CN111062343B (en) Three-dimensional human ear recognition method based on average curvature and normal distribution transformation
CN113052798A (en) Screen aging detection model training method and screen aging detection method
CN117218422B (en) Power grid image recognition method and system based on machine learning
CN117392735B (en) Face data processing method, device, computer equipment and storage medium
CN112396648B (en) Target identification method and system capable of positioning mass center of target object
Chen et al. Structure guided image completion using texture synthesis and region segmentation
CN116050282B (en) User demand computing method for information and implicit feedback demand prediction method
CN117876757A (en) Image mole pattern research model construction method and terminal

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination