CN110335330B - Image simulation generation method and system, deep learning algorithm training method and electronic equipment - Google Patents
Image simulation generation method and system, deep learning algorithm training method and electronic equipment Download PDFInfo
- Publication number
- CN110335330B CN110335330B CN201910634805.4A CN201910634805A CN110335330B CN 110335330 B CN110335330 B CN 110335330B CN 201910634805 A CN201910634805 A CN 201910634805A CN 110335330 B CN110335330 B CN 110335330B
- Authority
- CN
- China
- Prior art keywords
- image
- simulation
- initial image
- initial
- processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/40—Filling a planar surface by adding surface attributes, e.g. colour or texture
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/60—Editing figures and text; Combining figures or text
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
Abstract
The invention provides an image simulation generation method, which is based on the preliminary analysis of an initial image, determines a simulation degree threshold value, determines to perform translation of preset pixel values in at least two directions on the initial image based on the simulation degree threshold value, and performs weighted average superposition on the translated images to output a required simulation image. The method can solve the technical problem that the image aberration and the like caused by a lens, an optical element, a photosensitive element and the like cannot be effectively simulated by the conventional image simulation technology. The image simulation generation system and the electronic equipment thereof have the same technical effect as the method. The deep learning algorithm training method provided by the invention can also solve the problem that the existing data amplification technology cannot well simulate the generated image into a real sampling image so as to obtain a better training effect.
Description
[ technical field ] A method for producing a semiconductor device
The invention relates to the field of image data simulation, in particular to an image simulation generation method and system, a deep learning algorithm training method and electronic equipment.
[ background of the invention ]
The development of deep learning technology enables computer image vision technology to be supported strongly, but a deep learning network structure often contains a large number of parameters, and a large amount of data needs to be provided for training the network in order to enable the network to have better performance (accuracy, generalization performance) and the like. In practical application, data acquisition, labeling and the like are difficult to meet the magnitude required by a training network, so a new technical scheme for rapidly acquiring or generating image data is urgently needed to be provided.
[ summary of the invention ]
In order to solve the technical problem of rapid acquisition of the existing image data, the invention provides an image simulation generation method and a system thereof, a deep learning algorithm training method and electronic equipment.
In order to solve the technical problems, the invention provides the following technical scheme: a method of image simulation generation, comprising: step S1, acquiring an initial image, and analyzing the size characteristic and the color space structure characteristic of the initial image; specifically, the original image can be generated or the image to be processed can be input based on the scale and the color space structure of the initial image; step S2, determining a simulation degree threshold value based on the size characteristic and the color space structure characteristic of the initial image; step S201, carrying out amplification or reduction processing on the initial image; the magnification of the initial image magnification or reduction processing can be set based on the simulation degree threshold, wherein the initial image is subjected to optical aberration simulation processing, and the optical aberration comprises spherical aberration, astigmatism, field curvature, distortion, position chromatic aberration and magnification chromatic aberration; step S3, the initial image and the translated image are weighted average superposed according to the predetermined pixel value of the initial image translated along the preset direction, so as to output the required simulation image; wherein the predetermined pixel value is determined based on a simulation degree threshold.
Preferably, the step S3 specifically includes the following steps: step S31, setting a first direction and a second direction, and sequentially translating the initial image along the first direction and the second direction by predetermined pixel values to obtain a first translated image and a second translated image, respectively; step S32, processing the first and second translation images to obtain a numerical value matching the position coordinates; and step S33, carrying out weighted average superposition on the numerical values matched with the same position coordinates in the first translation image and the second translation image so as to output the required simulation image. Preferably, after the step S3, the method further includes: and step S4, filling the missing area by using a reflection model simulation or image interpolation method.
Preferably, after the step S2, the method further includes the following steps: S3A, simulating the color temperature of the light source, matching the color temperature of the blackbody light radiation and the ratio of RGB values based on a simulation degree threshold value, and adjusting the color temperature of the initial image; and/or step S3B, simulating exposure, converting the color space structure of the initial image into YUV color space, wherein the YUV color space comprises a Y channel, the Y channel is a brightness channel, and carrying out exposure correction on the Y channel, and the exposure correction can be processed based on the simulation degree threshold; and/or step S3C, performing data amplification processing on the initial image; in addition, one or more of the steps S3A, S3B and S3C may be optionally combined with the step S3 in any order.
Preferably, in the step S3C, the data amplification process further includes: and performing any one or combination of a plurality of processes of turning, rotating, scaling, cropping, translating, noise processing, color channel processing and image shear transformation on the initial image.
In order to solve the above technical problems, the present invention provides another technical solution as follows: an image simulation generation system, comprising: the image preliminary analysis module is configured to acquire an initial image and analyze the size characteristic and the color space structure characteristic of the initial image; the simulation degree threshold acquisition module is configured to determine a simulation degree threshold based on the size characteristic and the color space structure characteristic of the initial image, wherein the initial image is subjected to optical aberration simulation processing, and the optical aberration comprises spherical aberration, astigmatism, field curvature, distortion, position chromatic aberration and magnification chromatic aberration; the image size adjusting module is used for carrying out amplification or reduction processing on the initial image; the simulation image generation module is configured to perform weighted average superposition on the initial image and the translated image according to a preset pixel value of the initial image translated along a preset direction so as to output a required simulation image; wherein the predetermined pixel value is determined based on a simulation degree threshold.
Preferably, the image simulation generation system further includes: the image size adjusting module is used for carrying out amplification or reduction processing on the initial image; the missing region filling module is used for filling the missing region by using a reflection model or an image numerical average value; the simulation light source color temperature module is used for matching by combining the ratio of the black body light radiation color temperature and the RGB value based on a simulation degree threshold value and adjusting the color temperature of the initial image; the analog exposure module is used for converting the color space structure of the initial image into a YUV color space; and/or a data amplification module for performing data amplification processing on the initial image.
In order to solve the above technical problems, the present invention provides another technical solution as follows: a deep learning algorithm training method, wherein the deep learning algorithm is obtained by training the simulated image obtained by the image simulation generation method and combining the initial image as a training set.
In order to solve the above technical problems, the present invention provides another technical solution as follows: an electronic device comprising a storage unit for storing a computer program and a processing unit for executing the image simulation generation method as described above by the computer program stored by the storage unit.
Compared with the prior art, the image simulation generation method and the system thereof, the deep learning algorithm training method and the electronic equipment provided by the invention have the following beneficial effects:
the invention aims to introduce optical errors into the field of image simulation generation through optical simulation, so that the enhancement and expansion of image data can be realized based on a small amount of basic image data, and a generated data set can be enhanced to fit real data. The invention can supplement the problem that the data amplification result of the traditional method can not cover all possibilities of real situations by an optical method under the condition of insufficient data quantity through optical-based simulation. There are good physical simulation results for the process of generating image data. By the method, the training set can be closer to the real situation, and the network performance of the final training can be improved. Specifically, in the present invention, based on a preliminary analysis of an initial image, a simulation degree threshold value is determined, the initial image is shifted by a predetermined pixel value to obtain a shifted image, wherein the predetermined pixel value is determined based on the simulation degree threshold value, and the initial image and the shifted image are subjected to weighted average superposition to output a desired simulated image.
The image simulation generation method provided by the invention can solve the technical problem that the image aberration and the like caused by a lens, an optical element, a photosensitive element and the like cannot be effectively simulated by the conventional image simulation technology. Meanwhile, the image simulation generation method provided by the invention can also solve the problem that the generated image cannot be well simulated into a real sampling image by the existing data amplification technology. Therefore, the image simulation generation method provided by the invention has wide applicability.
In the image simulation generation method provided by the invention, the steps of determining a simulation degree threshold value based on the preliminary analysis of an initial image, determining predetermined pixel values for performing translation in at least two directions on the initial image based on the simulation degree threshold value, and performing weighted average superposition on the translated image specifically comprise the following steps: setting a first direction and a second direction, and translating the initial image in the first direction and the second direction in sequence by preset pixel values to respectively obtain a first translation image and a second translation image; processing the first and second translation images to obtain a numerical value matched with the position coordinates; and carrying out weighted average superposition on numerical values matched with the coordinates of the same position in the first translation image and the second translation image so as to output a required simulation image. Based on the weighted average superposition of the numerical values matched with the same position coordinates, an image simulating astigmatism or defocus can be obtained.
In order to effectively process initial images with different sizes, the image simulation generation method further comprises the step of carrying out amplification or reduction processing on the initial images; wherein a magnification of the initial image enlarging or reducing process may be set based on the simulation degree threshold. Based on this, the precision of the magnification of the enlargement or reduction processing of the initial image can be made higher.
In order to avoid the unusable simulated image due to the missing area generated by the zooming processing or the translation processing on the initial image, the image simulation generation method further comprises filling the missing area by using a reflection model or an image numerical average value.
In the invention, the method can also comprise the simulated light source color temperature processing, the simulated exposure processing and the data amplification processing of the initial image, and the processing can be carried out based on the simulation degree threshold value, so that the accuracy of the simulation processing can be improved. Furthermore, in the image simulation generation method provided by the invention, the processing sequence of the corresponding steps of simulated astigmatism, simulated defocus, simulated light source color temperature processing, simulated exposure processing and data amplification processing can be replaced at will so as to meet the requirement of accurate processing of different initial images.
In the present invention, the data amplification process further comprises: the method comprises the following steps of carrying out any one or combination of a plurality of turning processing, rotating processing, zooming processing, cutting processing, translation processing, noise processing, color channel processing and image miscut transformation processing on an initial image, and based on the result, the method can meet various different image simulation generation methods and has wider applicability.
The invention also provides an image simulation generation system and electronic equipment thereof, which have the same beneficial effects as the image simulation generation method, and can supplement the problem that the data amplification result of the traditional method can not cover all possibilities of real situations through an optical method under the condition of insufficient data quantity through optical-based simulation. There are good physical simulation results for the process of generating image data. By the method, the training set can be closer to the real situation, and the network performance of the final training can be improved. Specifically, in the present invention, a simulation degree threshold is determined based on a preliminary analysis of an initial image, predetermined pixel values for performing translation of the initial image in at least two directions are determined based on the simulation degree threshold, and the translated images are subjected to weighted average superposition to output a desired simulated image.
The invention also provides a deep learning algorithm training method, which is obtained by training by combining the simulated images obtained by the image simulation generation method with the initial images as a training set. The deep learning algorithm training method can supplement the problem that the data amplification result of the traditional method cannot cover all possibilities of the real situation through an optical method under the condition of insufficient data quantity. Therefore, more comprehensive training data are provided for the training of the deep learning algorithm, and the training effect of the deep learning algorithm can be improved.
[ description of the drawings ]
Fig. 1 is a schematic flow chart illustrating steps of an image simulation generation method according to a first embodiment of the present invention.
Fig. 2 is a flowchart illustrating a detailed procedure of the content shown in step S3 in fig. 1.
Fig. 3 is a schematic flow chart of steps of one of the variant embodiments of the image simulation generation method shown in fig. 1.
Fig. 4 is a schematic flow chart of the second step of the variant embodiment of the image simulation generation method shown in fig. 1.
Fig. 5 is a schematic diagram before image translation for a method of simulating defocus as an optical simulation of an image.
Fig. 6 is a schematic diagram after image shift in a method of simulating defocus as an optical simulation of an image.
Figure 7 is a schematic representation before image translation in a method of simulating astigmatism as an optical simulation of the image.
Figure 8 is a schematic representation after image translation with simulated astigmatism as a method of image optical simulation.
Fig. 9 is a schematic flow chart of steps of one of the variant embodiments of the image simulation generation method shown in fig. 1.
Fig. 10 is a functional block diagram of an image simulation generation system provided in the second embodiment of the present invention.
Fig. 11 is a functional block diagram of the analog image generation block shown in fig. 10.
FIG. 12 is a functional block diagram of one of the embodiments of the image simulation generation system shown in FIG. 10.
FIG. 13 is a functional block diagram of a second embodiment of the image simulation generation system shown in FIG. 10.
Fig. 14 is a functional block diagram of an electronic device provided in a fourth embodiment of the present invention.
The attached drawings indicate the following:
20. an image simulation generation system; 21. an image preliminary analysis module; 22. a simulation degree threshold acquisition module; 23. a simulated image generation module; 231. a translation unit; 232. a position coordinate matching unit; 233. a weighted average superposition unit; 24. an image size adjustment module; 25. a missing region padding module; 26. the simulation light source color temperature module; 27. a simulated exposure module; 28. data amplification module
30. An electronic device; 31. a storage unit; 32. and a processing unit.
[ detailed description ] embodiments
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Referring to fig. 1, in a first embodiment of the present invention, an image simulation generating method S10 is provided, which includes the following steps:
step S1, acquiring an initial image, and analyzing the size characteristic and the color space structure characteristic of the initial image; specifically, the original image can be generated or the image to be processed can be input based on the scale and the color space structure of the initial image;
step S2, determining a simulation degree threshold value based on the size characteristic and the color space structure characteristic of the initial image; and
step S3, performing weighted average superposition on the initial image and the translated image according to the predetermined pixel value of the initial image translated along the preset direction so as to output a required simulation image; wherein the predetermined pixel value is determined based on a simulation degree threshold.
In the above step S1, the size characteristic refers to the actual size of the initial image, and based on the specific size characteristic, it can be determined whether the image needs to be enlarged or reduced for the subsequent optical aberration simulation processing on the initial image. The optical phase difference may include spherical aberration, astigmatism, field curvature, distortion, position chromatic aberration, magnification chromatic aberration, and the like.
The color space structural features may include a base color space and a color and brightness separation color space. The former is typically RGB (Red, Green, Blue), and also includes CMY (Cyan, Magenta, Yellow), CMYK (Cyan, Magenta, Yellow, black), CIE XYZ (Color System), and the like; the latter includes YCC/YUV, Lab, and a set of hue class color spaces.
Further, in step S2, the degree threshold of the optical simulation of the image is determined based on the size characteristic and the color space structure characteristic of the initial image, that is, the more accurate degree threshold of the simulation can be set and obtained based on the specific characteristics of the initial image.
In the present invention, the simulation degree threshold may include image simulation based on image optical simulation, and specifically, the method of image optical simulation may include, but is not limited to, any one or combination of simulated defocus, simulated astigmatism, simulated chromatic aberration, and the like.
The simulation degree threshold value can be directly reflected as a difference value between a simulated image obtained after the image optical simulation and the initial image, wherein the larger the required difference value is, the larger the corresponding degree threshold value is, and the smaller the required difference value is, the smaller the corresponding degree threshold value is.
As shown in fig. 2, in the step S3, the method includes the following steps:
step S31, setting a first direction and a second direction, and sequentially translating the initial image along the first direction and the second direction by predetermined pixel values to obtain a first translated image and a second translated image, respectively;
step S32, processing the first and second translation images to obtain a numerical value matching the position coordinates; and
step S33, performing weighted average superposition on the numerical values matched with the coordinates of the same position in the first and second translation images to output a desired simulated image.
Specifically, in some other embodiments, the preset direction may be adjusted correspondingly based on the actual characteristics of the picture and the requirement of the user, and specifically may be further two, four, six, or eight, and in some special embodiments, may also be three, five, seven, and the like, which are only used as examples and are not limited to the present invention.
Referring to fig. 3, in order to obtain more diversified analog images, in some variations of the present embodiment, the following steps may be further included between the steps S2 and S3:
step S201, performing an enlargement or reduction process on the initial image.
Specifically, the magnification of the initial image enlargement or reduction process may be set based on the simulation degree threshold.
Further, referring to fig. 4, in order to avoid missing regions in the weighted-average superimposed image due to multi-directional translation, when a region with a value of zero is detected in the weighted-average superimposed image, the method further includes the following steps:
in step S4, the missing area is filled in by using a Reflection Model (Reflection Model) simulation or an image difference method.
Wherein the reflection model comprises any one of a Cook-Torrance reflection model, a Blinn-Phong reflection model and a Phong reflection model.
The image numerical value average value is the average value of numerical values obtained by calculating the numerical value of each pixel point corresponding to the initial image.
For better explaining the above embodiments, the generation of the DM (data matrix) two-dimensional code analog image is taken as an example, that is, the DM two-dimensional code initial image is obtained.
Specifically, in the present embodiment, as shown in fig. 5 and fig. 6, the method of using simulated defocus as the image optical simulation specifically includes:
for the DM two-dimensional code initial image, the out-of-focus of the camera is a very common source of image error. Based on the above step S2, a simulation degree threshold corresponding to the DM two-dimensional code initial image is obtained, so as to obtain the degree of image defocus, specifically, the degree of astigmatism represented by n/m as a rational number, after the image is enlarged to m × m times of the initial image, 1 pixel and 2 pixels are translated to n pixels along multiple directions, so as to obtain the numerical value corresponding to (n +1) new images for weighted average superposition. For example, in some embodiments, after the initial image a shown in fig. 5 is enlarged by m × m times, 1 pixel is shifted based on the X positive direction, the X negative direction, the Y positive direction, and the Y negative direction (i.e., n is 1), four new shifted images a 'are obtained, and the shifted image a' obtained after n pixels are shifted in the X direction is shown in fig. 6.
Further carrying out weighted average superposition on the numerical values corresponding to the initial image and the four new translation images A'; further, since a missing region appears on the image after the translation, the data average value of the image is further used for padding.
In order to obtain a richer simulated out-of-focus image, the direction of translation may be further adjusted based on specific content, and the above example is only an example, and is not a limitation of the present invention.
In the present embodiment, as shown in fig. 7 and fig. 8, the method for simulating astigmatism as an optical simulation of an image specifically includes:
and (2) using n/m to represent the astigmatism degree of a rational number, magnifying the image to m × m times of the initial image, translating 1 pixel and 2 pixels along a preset direction until n pixels, and superposing the obtained (n +1) images in a weighted average manner. For example, in some other embodiments, after the initial image B shown in fig. 7 is enlarged by m × m, 2 pixels are shifted based on the positive Y direction (i.e., n is 2), and two new shifted images B 'are obtained, and the shifted image B' obtained after n pixels are shifted in the positive Y direction is shown in fig. 8.
And further performing weighted average superposition on the initial image and the data corresponding to the two new translation images B'.
In the above example, the defocus degree and the astigmatism degree are the simulation degree threshold values.
In this embodiment, the method for simulating chromatic aberration as an optical simulation of an image specifically includes:
and setting an optical axis X for the initial image, assuming that the length direction (namely the first direction) of the DM two-dimensional code initial image comprises n pixels, and the width direction (namely the second direction) comprises m pixels, translating along the direction opposite to the optical axis X, and simulating a chromatic aberration result after superposition. In addition, a color difference simulation parameter may be generated and a color difference simulation may be performed.
Because the focal lengths of the light with different wavelengths passing through the lens group have slight difference, the three channels of RGB are subjected to defocusing processing with different degrees to obtain a simulated defocused image.
Referring to fig. 9, in some other embodiments of the present invention, in order to meet the optical aberration simulation requirements of various initial images, after step S2, the method further includes the following steps:
S3A, simulating the color temperature of the light source, matching the color temperature of the blackbody light radiation and the ratio of RGB values based on a simulation degree threshold value, and adjusting the color temperature of the initial image; and/or
Step S3B, simulating exposure, converting the color space structure of the initial image into YUV (YCrCb, color coding method) color space, where the Y channel is a luminance channel, and performing exposure correction on the Y channel, where the exposure correction may be processed based on the simulation degree threshold; and/or
And step S3C, performing data amplification processing on the initial image, wherein the data amplification processing comprises any one or a combination of several of turning processing, rotating processing, zooming processing, cropping processing, translation processing, noise processing, color channel processing, image miscut transform processing and the like on the initial image.
As shown in fig. 9, the sequence between the above three steps is any one of step S3A, step S3B, and step S3C. This sequence of steps is exemplary only and not intended as a limitation of the present invention. Further, the step S3A, the step S3B and the step S3C may be optionally combined with one or more of the steps S3 in any order.
In step S3A, in the black body radiation, the color of light varies with temperature, and the black body exhibits a gradual change process of red-orange-yellow-white-blue-white, that is, the color temperature of the black body radiation can represent the gradual change process of the black body with temperature. The RGB values (Red, Green, Blue) are R values, G values, and B values of each pixel point corresponding to the initial image. And adjusting the corresponding color temperature based on the black body radiation and the R value, the G value and the B value of the corresponding pixel points.
In step S3B, the color space structure of the initial image is converted into a YUV color space, where the Y channel is a luminance channel, and exposure correction is performed on the Y channel, so that a new diagram can be obtained.
Specifically, in the above step S3C, the image flipping process refers to flipping an image in the horizontal or vertical direction; the image rotation processing refers to rotating the image by different angles such as 90 degrees, 180 degrees, 270 degrees and the like, and some technical schemes cut the rotated image to ensure that the image proportion is unchanged. The scaling processing refers to scaling the image in different scales, different interpolation methods may be adopted in different technical solutions, and the scaled image may be clipped or completed (the possible methods include reflection, translation, and the like). The random cutting processing refers to that after the image is cut randomly, the cut part is zoomed into the original size; the translation processing refers to the translation in random direction of the image, and part of technical schemes fill up the missing area after the translation; the random noise processing mode comprises various modes such as salt-pepper noise, Gaussian noise and the like, and the generalization capability of the deep learning network is enhanced by modifying random pixels in the image; the color channel processing refers to that the image is modified by color channels, and the generated result is the deviation of the whole color style; the image cross-cut transform processing is to generate a deformed image by matrix transformation of the image and to fill up a missing region in the image.
For example, as the DM two-dimensional code listed above is taken as an example, in combination with the above specific processing steps, since the color temperature of the light environment in the final application scene is about 6600K, the color temperature parameter is adjusted to 6600K. Further setting the exposure condition and the degree of overexposure and underexposure. It is also possible to decide whether to defocus or astigmatism and to generate random parameters within the appropriate range. The data pair is formed by performing appropriate random transformation on the generated result and the generated original image.
Referring to fig. 10, a second embodiment of the present invention provides an image simulation generating system 20, which includes:
an image preliminary analysis module 21 configured to acquire an initial image and analyze a size characteristic and a color space structure characteristic of the initial image;
a simulation degree threshold value obtaining module 22 configured to determine a simulation degree threshold value based on the size characteristic and the color space structure characteristic of the initial image; and
the analog image generation module 23 is configured to perform weighted average superposition on the initial image and the translated image according to a predetermined pixel value of the initial image translated along a preset direction so as to output a required analog image; wherein the predetermined pixel value is determined based on a simulation degree threshold.
As shown in fig. 11, in order to improve the accuracy of the simulated image generation to obtain a satisfactory simulated image, the simulated image generation module 23 further includes:
a translation unit 231, configured to set a first direction and a second direction, and translate the initial image by predetermined pixel values in the first direction and the second direction in sequence to obtain a first translation image and a second translation image, respectively;
a position coordinate matching unit 232, configured to process the first and second translation images to obtain a numerical value matched with a position coordinate; and
a weighted average superimposing unit 233, configured to perform weighted average superimposing on the numerical values matching the coordinates of the same position in the first and second translation images, so as to output a desired analog image.
As shown in fig. 12, in some embodiments of the present embodiment, the image simulation generating system 20 further includes:
an image size adjusting module 24, configured to perform an enlargement or reduction process on the initial image; and
and a missing region filling module 25, configured to fill the missing region by using a Reflection Model (Reflection Model) simulation or an image interpolation method.
Wherein the reflection model comprises any one of a Cook-Torrance reflection model, a Blinn-Phong reflection model and a Phong reflection model.
The image interpolation method is to use the gray value of the known adjacent pixel point to obtain the gray value of the unknown pixel point so as to generate an image with higher resolution from the original image, and the image interpolation method may include a nearest neighbor method, a bilinear interpolation method, and the like.
In other embodiments of this embodiment, as shown in fig. 13, in order to meet the optical aberration simulation requirements of various initial images in this embodiment, the image simulation generating system 20 further includes:
the simulated light source color temperature module 26 is used for matching the black body light radiation color temperature and the RGB value based on a simulation degree threshold value and adjusting the color temperature of the initial image; and/or
An analog exposure module 27, configured to convert a color space structure of the initial image into a YUV (YCrCb, color coding method) color space, where a Y channel is a luminance channel, and perform exposure correction on the Y channel, where the exposure correction may be processed based on the analog degree threshold; and/or
And a data amplification module 28, configured to perform data amplification processing on the initial image.
Specifically, the data amplification module 28 performs any one or a combination of several of flipping processing, rotation processing, scaling processing, cropping processing, translation processing, noise processing, color channel processing, image cross-cut transformation processing, and the like on the initial image.
In this embodiment, the related content related to the simulation generation of the specific image is consistent with the related description in the first embodiment, and is not described herein again.
A third embodiment of the present invention provides a deep learning algorithm training method, which is obtained by training a simulation image obtained by the image simulation generation method as described in the first embodiment and an initial image as a training set.
The specific contents of the image simulation generation method are the same as those described in the first embodiment, and are not described herein again.
Referring to fig. 14, a fourth embodiment of the present invention provides an electronic device 30, where the electronic device 30 includes a storage unit 31 and a processing unit 32, the storage unit 31 is used for storing a computer program, and the processing unit 32 is used for executing the specific steps of the data verification method for detecting abnormal values in the first embodiment by using the computer program stored in the storage unit 31.
In some specific embodiments of the present invention, the electronic device 30 may be hardware or software. When the electronic device 30 is hardware, it may be various electronic devices having a display screen and supporting video playing, including but not limited to a smart phone, a tablet computer, an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III, motion Picture Experts Group Audio Layer 3), an MP4 player (Moving Picture Experts Group Audio Layer IV, motion Picture Experts Group Audio Layer 4), a laptop computer, a desktop computer, and the like. When the electronic device 30 is software, it can be installed in the electronic devices listed above. It may be implemented as multiple pieces of software or software modules (e.g., multiple pieces of software or software modules to provide distributed services) or as a single piece of software or software module. And is not particularly limited herein.
The storage unit 31 includes a storage portion of a Read Only Memory (ROM), a Random Access Memory (RAM), a hard disk, and the like, and the processing unit 32 may perform various appropriate actions and processes according to a program stored in the Read Only Memory (ROM) or a program loaded into the Random Access Memory (RAM). In a Random Access Memory (RAM), various programs and data necessary for the operation of the electronic device 30 are also stored.
The electronic device 30 may further include an input portion (not shown) of a keyboard, a mouse, and the like; the electronic device 30 may further include an output portion (not shown) such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker and the like; and the electronic device 30 may further include a communication part (not shown) of a network interface card such as a LAN card, a modem, and the like. The communication section performs communication processing via a network such as the internet.
In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, the disclosed embodiments of the invention may include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication section.
When executed by the processing unit 32, the computer program performs the above-described functions defined in the method for training a neural network model with an anti-counterfeiting function of the present application. It should be noted that the computer readable medium described herein can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
In the present application, a computer readable storage medium may also be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures of the present application illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will be understood that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units involved in the embodiments of the present invention may be implemented by software or hardware. The described units may also be located in the processor.
As another aspect, a fifth embodiment of the present invention also provides a computer-readable medium, which may be contained in the apparatus described in the above embodiments; or may be present separately and not assembled into the device. The computer-readable medium carries one or more programs, which specifically include acquiring an initial image and analyzing a size characteristic and a color space structure characteristic of the initial image; determining a simulation degree threshold value based on the size characteristic and the color space structure characteristic of the initial image; performing weighted average superposition on the initial image and the translated image according to the preset pixel value of the initial image translated along the preset direction so as to output a required simulation image; wherein the predetermined pixel value is determined based on a simulation degree threshold.
Compared with the prior art, the image simulation generation method and the system thereof, the deep learning algorithm training method and the electronic equipment provided by the invention have the following beneficial effects:
the invention aims to introduce optical errors into the field of image simulation generation through optical simulation, so that the enhancement and expansion of image data can be realized based on a small amount of basic image data, and a generated data set can be enhanced to fit real data. The invention can supplement the problem that the data amplification result of the traditional method can not cover all possibilities of real situations by an optical method under the condition of insufficient data quantity through optical-based simulation. There are good physical simulation results for the process of generating image data. By the method, the training set can be closer to the real situation, and the network performance of the final training can be improved. Specifically, in the present invention, a simulation degree threshold is determined based on a preliminary analysis of an initial image, predetermined pixel values for performing translation of the initial image in at least two directions are determined based on the simulation degree threshold, and the translated images are subjected to weighted average superposition to output a desired simulated image.
The image simulation generation method provided by the invention can solve the technical problem that the existing image simulation technology can not effectively simulate the image aberration and the like caused by lenses, optical elements, photosensitive elements and the like. Meanwhile, the image simulation generation method provided by the invention can also solve the problem that the generated image cannot be well simulated into a real sampling image by the existing data amplification technology. Therefore, the image simulation generation method provided by the invention has wide applicability.
The invention also provides an image simulation generation system and electronic equipment thereof, which have the same beneficial effects as the image simulation generation method, and can supplement the problem that the data amplification result of the traditional method can not cover all possibilities of real situations through an optical method under the condition of insufficient data quantity through optical-based simulation. There are good physical simulation results for the process of generating image data. By the method, the training set can be closer to the real situation, and the network performance of the final training can be improved. Specifically, in the present invention, a simulation degree threshold is determined based on a preliminary analysis of an initial image, predetermined pixel values for performing translation of the initial image in at least two directions are determined based on the simulation degree threshold, and the translated images are subjected to weighted average superposition to output a desired simulated image.
The invention also provides a deep learning algorithm training method, which is obtained by training by combining the simulated images obtained by the image simulation generation method with the initial images as a training set. The deep learning algorithm training method can supplement the problem that the data amplification result of the traditional method cannot cover all possibilities of the real situation through an optical method under the condition of insufficient data quantity. Therefore, more comprehensive training data are provided for the training of the deep learning algorithm, and the training effect of the deep learning algorithm can be improved.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit of the present invention are intended to be included within the scope of the present invention.
Claims (9)
1. An image simulation generation method, characterized in that: the image simulation generation method comprises the following steps:
step S1, acquiring an initial image, and analyzing the size characteristic and the color space structure characteristic of the initial image; specifically, the scale and the color space structure of the initial image may be determined to generate an original image or input an image to be processed, and the initial image may be further subjected to optical aberration simulation processing, where the optical aberration includes spherical aberration, astigmatism, field curvature, distortion, positional chromatic aberration, and chromatic aberration of magnification;
step S2, determining a simulation degree threshold value based on the size characteristic and the color space structure characteristic of the initial image;
step S201, carrying out amplification or reduction processing on the initial image; wherein a multiple of the initial image enlarging or reducing process may be set based on the simulation degree threshold; and
step S3, performing weighted average superposition on the initial image and the translated image according to the predetermined pixel value of the initial image translated along the preset direction so as to output a required simulation image; wherein the predetermined pixel value is determined based on a simulation degree threshold.
2. A method for analog generation of an image as claimed in claim 1, characterized by: the step S3 specifically includes the following steps:
step S31, setting a first direction and a second direction, and sequentially translating the initial image along the first direction and the second direction by predetermined pixel values to obtain a first translated image and a second translated image, respectively;
step S32, processing the first and second translation images to obtain a numerical value matching the position coordinates; and
step S33, performing weighted average superposition on the numerical values matched with the coordinates of the same position in the first and second translation images to output a desired simulated image.
3. A method for analog generation of an image as claimed in claim 1, characterized by: after the step S3, the method further includes:
and step S4, filling the missing area by using a reflection model simulation or image interpolation method.
4. A method for analog generation of an image as claimed in claim 1, characterized by: after the above step S2, the method further includes the following steps:
S3A, simulating the color temperature of the light source, matching the color temperature of the blackbody light radiation and the ratio of RGB values based on a simulation degree threshold value, and adjusting the color temperature of the initial image; and/or
Step S3B, simulating exposure, converting the color space structure of the initial image into YUV color space, wherein the YUV color space comprises a Y channel, the Y channel is a brightness channel, and carrying out exposure correction on the Y channel, and the exposure correction can be processed based on the simulation degree threshold; and/or
Step S3C, performing data amplification processing on the initial image;
in addition, one or more of the steps S3A, S3B and S3C may be optionally combined with the step S3 in any order.
5. A method for analog generation of an image as claimed in claim 4, characterized by: in the above step S3C, the data amplification process further includes: and performing any one or combination of a plurality of processes of turning, rotating, scaling, cropping, translating, noise processing, color channel processing and image shear transformation on the initial image.
6. An image simulation generation system characterized by: it includes:
the image preliminary analysis module is configured to acquire an initial image and analyze the size characteristic and the color space structure characteristic of the initial image;
the simulation degree threshold acquisition module is configured to determine a simulation degree threshold based on the size characteristic and the color space structure characteristic of the initial image, wherein the initial image is subjected to optical aberration simulation processing, and the optical aberration comprises spherical aberration, astigmatism, field curvature, distortion, position chromatic aberration and magnification chromatic aberration;
the image size adjusting module is used for carrying out amplification or reduction processing on the initial image; and
the simulation image generation module is configured to perform weighted average superposition on the initial image and the translated image according to a preset pixel value of the initial image translated along a preset direction so as to output a required simulation image; wherein the predetermined pixel value is determined based on a simulation degree threshold.
7. The image simulation generation system of claim 6, wherein:
the image size adjusting module is used for carrying out amplification or reduction processing on the initial image;
the missing region filling module is used for filling the missing region by using a reflection model simulation or image interpolation method;
the simulation light source color temperature module is used for matching by combining the ratio of the black body light radiation color temperature and the RGB value based on a simulation degree threshold value and adjusting the color temperature of the initial image;
the analog exposure module is used for converting the color space structure of the initial image into a YUV color space; and/or
And the data amplification module is used for carrying out data amplification processing on the initial image.
8. A deep learning algorithm training method, wherein the deep learning algorithm is obtained by training a simulated image obtained by the image simulation generation method according to any one of claims 1 to 5 in combination with an initial image as a training set.
9. An electronic device, characterized in that: the electronic device comprises a storage unit for storing a computer program and a processing unit for executing the image simulation generation method according to any one of claims 1 to 5 by means of the computer program stored by the storage unit.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910634805.4A CN110335330B (en) | 2019-07-12 | 2019-07-12 | Image simulation generation method and system, deep learning algorithm training method and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910634805.4A CN110335330B (en) | 2019-07-12 | 2019-07-12 | Image simulation generation method and system, deep learning algorithm training method and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110335330A CN110335330A (en) | 2019-10-15 |
| CN110335330B true CN110335330B (en) | 2021-04-20 |
Family
ID=68146719
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910634805.4A Active CN110335330B (en) | 2019-07-12 | 2019-07-12 | Image simulation generation method and system, deep learning algorithm training method and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110335330B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111243058B (en) * | 2019-12-31 | 2024-03-22 | 富联裕展科技(河南)有限公司 | Object simulation image generation method and computer readable storage medium |
| CN111259968A (en) * | 2020-01-17 | 2020-06-09 | 腾讯科技（深圳）有限公司 | Illegal image recognition method, device, equipment and computer readable storage medium |
| CN111652199B (en) * | 2020-08-04 | 2020-11-24 | 湖南莱博赛医用机器人有限公司 | Image processing method, device, equipment and medium |
| CN111931783A (en) * | 2020-08-18 | 2020-11-13 | 创新奇智(重庆)科技有限公司 | Training sample generation method, machine-readable code identification method and device |
| CN113553985A (en) * | 2021-08-02 | 2021-10-26 | 中再云图技术有限公司 | High-altitude smoke detection and identification method based on artificial intelligence, storage device and server |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101540832B (en) * | 2009-04-24 | 2011-02-09 | 段江 | Methods for matching dynamic range of image signals |
| CN102867318B (en) * | 2012-09-25 | 2015-11-11 | 哈尔滨工业大学 | The discrete degraded image building method of view picture fold return motion of subimage weighted stacking |
| CN103310420B (en) * | 2013-06-19 | 2016-03-30 | 武汉大学 | Based on coloured image cavity restorative procedure and the system of texture and geometric similarity |
| CN105631439B (en) * | 2016-02-18 | 2019-11-08 | 北京旷视科技有限公司 | Face image processing process and device |
| US10403026B2 (en) * | 2016-11-04 | 2019-09-03 | Disney Enterprises, Inc. | Noise reduction on G-buffers for Monte Carlo filtering |
-
2019
- 2019-07-12 CN CN201910634805.4A patent/CN110335330B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN110335330A (en) | 2019-10-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110335330B (en) | Image simulation generation method and system, deep learning algorithm training method and electronic equipment | |
| CN112989904B (en) | Method for generating style image, method, device, equipment and medium for training model | |
| CN109508681B (en) | Method and device for generating human body key point detection model | |
| CN110136066B (en) | Video-oriented super-resolution method, device, equipment and storage medium | |
| GB2536904A (en) | Image filtering based on image gradients | |
| CN111353948A (en) | Image noise reduction method, device and equipment | |
| CN111489322B (en) | Method and device for adding sky filter to static picture | |
| CN112241933A (en) | Face image processing method and device, storage medium and electronic equipment | |
| CN112017222A (en) | Video panorama stitching and three-dimensional fusion method and device | |
| CN102202162A (en) | Image processing apparatus, image processing method and program | |
| CN113658085B (en) | Image processing method and device | |
| CN109102484B (en) | Method and apparatus for processing image | |
| CN113962859A (en) | Panorama generation method, device, equipment and medium | |
| US9997134B2 (en) | Chromatic Calibration of an HDR display using 3D octree forests | |
| WO2023010751A1 (en) | Information compensation method and apparatus for highlighted area of image, device, and storage medium | |
| KR20120015980A (en) | Method, system and computer program product for object color correction | |
| CN113706400A (en) | Image correction method, image correction device, microscope image correction method, and electronic apparatus | |
| CN116403142A (en) | Video processing method, device, electronic equipment and medium | |
| CN111311722B (en) | Information processing method and device, electronic equipment and storage medium | |
| US20230368340A1 (en) | Gating of Contextual Attention and Convolutional Features | |
| CN115049559A (en) | Model training method, human face image processing method, human face model processing device, electronic equipment and readable storage medium | |
| CN114663570A (en) | Map generation method and device, electronic device and readable storage medium | |
| CN115049558A (en) | Model training method, human face image processing device, electronic equipment and readable storage medium | |
| CN114627293A (en) | Image matting method based on multi-task learning | |
| CN116485979B (en) | Mapping relation calculation method, color calibration method and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |