CN114004968A - Image processing method, image processing device, electronic equipment and storage medium - Google Patents

Image processing method, image processing device, electronic equipment and storage medium Download PDF

Info

Publication number
CN114004968A
CN114004968A CN202010740717.5A CN202010740717A CN114004968A CN 114004968 A CN114004968 A CN 114004968A CN 202010740717 A CN202010740717 A CN 202010740717A CN 114004968 A CN114004968 A CN 114004968A
Authority
CN
China
Prior art keywords
image processing
model
data
training
data type
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010740717.5A
Other languages
Chinese (zh)
Inventor
陈怡桦
蔡东佐
孙国钦
林子甄
李宛真
郭锦斌
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Futaihua Industry Shenzhen Co Ltd
Hon Hai Precision Industry Co Ltd
Original Assignee
Futaihua Industry Shenzhen Co Ltd
Hon Hai Precision Industry Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Futaihua Industry Shenzhen Co Ltd, Hon Hai Precision Industry Co Ltd filed Critical Futaihua Industry Shenzhen Co Ltd
Priority to CN202010740717.5A priority Critical patent/CN114004968A/en
Priority to US17/386,326 priority patent/US20220036192A1/en
Publication of CN114004968A publication Critical patent/CN114004968A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/217—Validation; Performance evaluation; Active pattern learning techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
    • G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
    • G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
    • G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • General Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Neurology (AREA)
  • Multimedia (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Biodiversity & Conservation Biology (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides an image processing method, an image processing device, an electronic device and a storage medium. The method comprises the following steps: responding to the received image processing instruction, and acquiring training data; training a preset deep learning model by using the training data by using a preset deep learning architecture to obtain an initial model; converting the data type of the initial model; adding a correction layer into the converted initial model; training the weight of the correction layer to optimize the initial model to obtain an image processing model; acquiring an image to be processed from the image processing instruction; and inputting the image to be processed into the image processing model, and outputting an image processing result. The invention can process the image based on the optimized image processing model, and simultaneously ensures the data processing speed and accuracy.

Description

Image processing method, image processing device, electronic equipment and storage medium
Technical Field
The present invention relates to the field of deep learning technologies, and in particular, to an image processing method and apparatus, an electronic device, and a storage medium.
Background
An original deep learning framework, namely, a Convolutional structure for Fast Feature Embedding (Fast Feature Embedding), only supports a float (single-precision floating point type) data type and a double-precision floating point type) data type, and most training, testing and application processes are finished based on the float data type at present.
However, for some large networks, the time consumption and the memory consumption are very serious, especially in embedded devices, many networks cannot be used directly.
In current solutions, a balance between data processing speed and accuracy is still not achieved.
Disclosure of Invention
In view of the foregoing, it is desirable to provide an image processing method, an image processing apparatus, an electronic device, and a storage medium, which can perform image processing based on an optimized image processing model while ensuring data processing speed and accuracy.
An image processing method, the image processing method comprising:
responding to the received image processing instruction, and acquiring training data;
training a preset deep learning model by using the training data by using a preset deep learning architecture to obtain an initial model;
converting the data type of the initial model;
adding a correction layer into the converted initial model;
training the weight of the correction layer to optimize the initial model to obtain an image processing model;
acquiring an image to be processed from the image processing instruction;
and inputting the image to be processed into the image processing model, and outputting an image processing result.
According to a preferred embodiment of the present invention, the converting the data type of the initial model includes:
acquiring demand data from the image processing instruction;
determining a target data type according to the demand data;
converting the initial model to the target data type.
According to a preferred embodiment of the present invention, the determining the target data type according to the demand data includes:
determining a required data processing speed and a required accuracy according to the demand data;
in response to the required data processing speed being greater than or equal to a preset speed, determining that the target data type is int 8; or
In response to the requested accuracy being greater than or equal to a preset accuracy, determining the target data type to be semi-accurate.
According to the preferred embodiment of the present invention, the data processing speed of the model with the data type int8 is higher than that of the model with the data type half precision, and the accuracy of the model with the data type int8 is lower than that of the model with the data type half precision.
According to a preferred embodiment of the present invention, the training the weights of the correction layer to optimize the initial model, and obtaining the image processing model includes:
fixing the weight of each layer in the initial model to train the weight of the correction layer;
and stopping training in response to the fact that the accuracy of the model is larger than or equal to a preset threshold value, and obtaining the image processing model.
According to a preferred embodiment of the present invention, the training the weights of the correction layer to optimize the initial model, and obtaining the image processing model further includes:
and training the weight of the correction layer by taking the full precision as a data type to obtain the image processing model.
According to a preferred embodiment of the present invention, the image processing method further comprises:
constructing a verification data set;
and verifying the image processing model by using the data in the verification data set.
An image processing apparatus, the image processing apparatus comprising:
an acquisition unit configured to acquire training data in response to a received image processing instruction;
the training unit is used for training a preset deep learning model by using the training data by using a preset deep learning architecture to obtain an initial model;
a conversion unit for converting the data type of the initial model;
the adding unit is used for adding a correction layer into the converted initial model;
the optimization unit is used for training the weight of the correction layer to optimize the initial model to obtain an image processing model;
the acquisition unit is further used for acquiring an image to be processed from the image processing instruction;
and the processing unit is used for inputting the image to be processed into the image processing model and outputting an image processing result.
According to a preferred embodiment of the present invention, the converting unit converting the data type of the initial model includes:
acquiring demand data from the image processing instruction;
determining a target data type according to the demand data;
converting the initial model to the target data type.
According to a preferred embodiment of the present invention, the determining, by the conversion unit, the target data type according to the demand data includes:
determining a required data processing speed and a required accuracy according to the demand data;
in response to the required data processing speed being greater than or equal to a preset speed, determining that the target data type is int 8; or
In response to the requested accuracy being greater than or equal to a preset accuracy, determining the target data type to be semi-accurate.
According to the preferred embodiment of the present invention, the data processing speed of the model with the data type int8 is higher than that of the model with the data type half precision, and the accuracy of the model with the data type int8 is lower than that of the model with the data type half precision.
According to a preferred embodiment of the present invention, the optimization unit is specifically configured to:
fixing the weight of each layer in the initial model to train the weight of the correction layer;
and stopping training in response to the fact that the accuracy of the model is larger than or equal to a preset threshold value, and obtaining the image processing model.
According to a preferred embodiment of the present invention, the optimization unit is further specifically configured to:
and training the weight of the correction layer by taking the full precision as a data type to obtain the image processing model.
According to a preferred embodiment of the present invention, the image processing apparatus further comprises:
a construction unit for constructing a verification data set;
and the verification unit is used for verifying the image processing model by using the data in the verification data set.
An electronic device, the electronic device comprising:
a memory storing at least one instruction; and
a processor executing instructions stored in the memory to implement the image processing method.
A computer-readable storage medium having stored therein at least one instruction, the at least one instruction being executable by a processor in an electronic device to implement the image processing method.
According to the technical scheme, the image processing method and the device can process the image based on the optimized image processing model, and simultaneously ensure the data processing speed and accuracy.
Drawings
FIG. 1 is a flow chart of a preferred embodiment of the image processing method of the present invention.
FIG. 2 is a functional block diagram of an image processing apparatus according to a preferred embodiment of the present invention.
FIG. 3 is a schematic structural diagram of an electronic device implementing an image processing method according to a preferred embodiment of the invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in detail with reference to the accompanying drawings and specific embodiments.
FIG. 1 is a flow chart of an image processing method according to a preferred embodiment of the present invention. The order of the steps in the flow chart may be changed and some steps may be omitted according to different needs.
The image processing method is applied to one or more electronic devices, which are devices capable of automatically performing numerical calculation and/or information processing according to preset or stored instructions, and the hardware thereof includes, but is not limited to, a microprocessor, an Application Specific Integrated Circuit (ASIC), a Programmable Gate Array (FPGA), a Digital Signal Processor (DSP), an embedded device, and the like.
The electronic device may be any electronic product capable of performing human-computer interaction with a user, for example, a Personal computer, a tablet computer, a smart phone, a Personal Digital Assistant (PDA), a game machine, an interactive Internet Protocol Television (IPTV), an intelligent wearable device, and the like.
The electronic device may also include a network device and/or a user device. The network device includes, but is not limited to, a single network server, a server group consisting of a plurality of network servers, or a Cloud Computing (Cloud Computing) based Cloud consisting of a large number of hosts or network servers.
The Network where the electronic device is located includes, but is not limited to, the internet, a wide area Network, a metropolitan area Network, a local area Network, a Virtual Private Network (VPN), and the like.
S10, in response to the received image processing instruction, training data is acquired.
Wherein the image processing instruction may be triggered by the relevant staff, which is not limited by the present invention.
The training data may include image data, imagery data, and the like.
And S11, training a preset deep learning model by the training data by using a preset deep learning architecture to obtain an initial model.
Wherein the preset deep learning architecture may include, but is not limited to: TensorFlow, Caffe (Convolutional structure for Fast Feature Embedding) Architecture.
The Caffe architecture is a deep learning framework with expressiveness, speed and thinking modularization, supports various types of deep learning architectures, and is oriented to image classification and image segmentation.
Moreover, the original Caffe architecture only supports float (single precision floating point type) data types and double (double precision floating point type) data types.
In this embodiment, the preset deep learning model may include any deep learning model supported by the Caffe architecture.
For example: the preset deep learning model can be CNN (Convolutional Neural Networks), RCNN (Regions with CNN features), LSTM (Long Short-Term Memory network), fully-connected Neural network, and the like.
And S12, converting the data type of the initial model.
The data type of the initial model is full-precision, and the data processing speed is low, so that the data type of the initial model is converted in order to improve the data processing speed of the model.
In this embodiment, the converting the data type of the initial model includes:
acquiring demand data from the image processing instruction;
determining a target data type according to the demand data;
converting the initial model to the target data type.
Specifically, the determining the target data type according to the demand data includes:
determining a required data processing speed and a required accuracy from the demand data, further:
(1) and in response to the required data processing speed being greater than or equal to a preset speed, determining the target data type to be int 8.
It can be understood that the data processing speed of the model with the data type int8 is higher than that of the model with the data type half-precision, and therefore, when the requirement on the data processing speed is high, that is, the required data processing speed is greater than or equal to the preset speed, the target data type can be preferentially determined as int 8.
The preset speed may be configured according to a speed that can be actually achieved by the model, which is not limited in the present invention.
(2) Or, in response to the requested accuracy being greater than or equal to a preset accuracy, determining the target data type to be semi-accurate.
It will be appreciated that the accuracy of a model with data type int8 is lower than a model with data type half precision, and therefore, when the accuracy requirement is high, i.e. the required accuracy is greater than or equal to a preset accuracy, the target data type may be preferentially determined to be half precision.
The preset accuracy may be configured according to an accuracy that can be actually achieved by the model, which is not limited in the present invention.
In the above embodiment, after the data types are converted into int8 or half precision, since the memory occupation of each data is reduced, the memory consumption of the whole network is reduced, the bandwidth consumption is reduced at the same time, and the performance is improved, both the two conversion modes can improve the data processing speed to a certain extent.
Of course, in the case of other cases than the above (1) and (2), either one of the two conversion modes may be selected, or a default selection may be made according to a preset configuration, and the present invention is not limited.
It should be noted that, since the conversion between data types is well known in the art, it is not described herein in detail.
And S13, adding a correction layer into the converted initial model.
Wherein, the correction layer can be any one layer of the newly added initial model.
For example: the correction layer can be a newly added convolution layer, a newly added full connection layer and the like.
And S14, training the weight of the correction layer to optimize the initial model to obtain an image processing model.
It can be understood that after the initial model is subjected to the data type conversion, the accuracy of the initial model will be reduced, and therefore, in order to improve the data processing speed and ensure the accuracy of the model, the initial model needs to be optimized.
Specifically, the training of the weight of the correction layer to optimize the initial model, and obtaining the image processing model includes:
fixing the weight of each layer in the initial model to train the weight of the correction layer;
and stopping training in response to the fact that the accuracy of the model is larger than or equal to a preset threshold value, and obtaining the image processing model.
The preset threshold value can be configured according to actual requirements so as to ensure that the precision of the model meets task requirements.
In this embodiment, the training the weight of the correction layer to optimize the initial model, and obtaining the image processing model further includes:
and training the weight of the correction layer by taking the full precision as a data type to obtain the image processing model.
And performing multiple operations on the correction layer by taking full precision as a data type, so that the accuracy of the model is continuously improved, the accuracy requirement is finally met, and the optimized data processing speed and accuracy meet the requirement of the image processing model.
In this embodiment, the image processing method further includes:
constructing a verification data set;
and verifying the image processing model by using the data in the verification data set.
The verification data set may be obtained from historical data, or may be provided by a designated organization or staff, and the invention is not limited as long as the reliability of the data is ensured.
Through the implementation mode, the image processing model can be further verified to ensure the correctness and the usability of the image processing model.
And S15, acquiring the image to be processed from the image processing instruction.
The image to be processed may be uploaded by a relevant person, or may be acquired by using a web crawler technology, which is not limited in the present invention.
And S16, inputting the image to be processed into the image processing model, and outputting an image processing result.
For example: the image processing result may be an image classification result, or an image segmentation result, or the like.
It should be noted that, image processing models obtained according to different training are different, and corresponding image processing results are also different, so as to satisfy various types of image processing tasks.
According to the technical scheme, the method can respond to the received image processing instruction, obtain training data, train a preset deep learning model by the training data by utilizing a preset deep learning framework to obtain an initial model, further convert the data type of the initial model to improve the data processing speed of the model, add a correction layer into the converted initial model, train the weight of the correction layer to optimize the initial model to obtain the image processing model to further improve the accuracy of the model, obtain the image to be processed from the image processing instruction, input the image to be processed into the image processing model, output an image processing result, realize image processing based on the optimized image processing model and simultaneously ensure the data processing speed and accuracy.
FIG. 2 is a functional block diagram of an image processing apparatus according to a preferred embodiment of the present invention. The image processing apparatus 11 includes an acquisition unit 110, a training unit 111, a conversion unit 112, an adding unit 113, an optimization unit 114, a processing unit 115, a construction unit 116, and a verification unit 117. The module/unit referred to in the present invention refers to a series of computer program segments that can be executed by the processor 13 and that can perform a fixed function, and that are stored in the memory 12. In the present embodiment, the functions of the modules/units will be described in detail in the following embodiments.
In response to the received image processing instruction, the acquisition unit 110 acquires training data.
Wherein the image processing instruction may be triggered by the relevant staff, which is not limited by the present invention.
The training data may include image data, imagery data, and the like.
The training unit 111 trains a preset deep learning model with the training data by using a preset deep learning architecture to obtain an initial model.
Wherein the preset deep learning architecture may include, but is not limited to: TensorFlow, Caffe (Convolutional structure for Fast Feature Embedding) Architecture.
The Caffe architecture is a deep learning framework with expressiveness, speed and thinking modularization, supports various types of deep learning architectures, and is oriented to image classification and image segmentation.
Moreover, the original Caffe architecture only supports float (single precision floating point type) data types and double (double precision floating point type) data types.
In this embodiment, the preset deep learning model may include any deep learning model supported by the Caffe architecture.
For example: the preset deep learning model can be CNN (Convolutional Neural Networks), RCNN (Regions with CNN features), LSTM (Long Short-Term Memory network), fully-connected Neural network, and the like.
The conversion unit 112 converts the data type of the initial model.
The data type of the initial model is full-precision, and the data processing speed is low, so that the data type of the initial model is converted in order to improve the data processing speed of the model.
In this embodiment, the converting unit 112 converts the data type of the initial model, including:
acquiring demand data from the image processing instruction;
determining a target data type according to the demand data;
converting the initial model to the target data type.
Specifically, the determining, by the conversion unit 112, the target data type according to the demand data includes:
determining a required data processing speed and a required accuracy from the demand data, further:
(1) and in response to the required data processing speed being greater than or equal to a preset speed, determining the target data type to be int 8.
It can be understood that the data processing speed of the model with the data type int8 is higher than that of the model with the data type half-precision, and therefore, when the requirement on the data processing speed is high, that is, the required data processing speed is greater than or equal to the preset speed, the target data type can be preferentially determined as int 8.
The preset speed may be configured according to a speed that can be actually achieved by the model, which is not limited in the present invention.
(2) Or, in response to the requested accuracy being greater than or equal to a preset accuracy, determining the target data type to be semi-accurate.
It will be appreciated that the accuracy of a model with data type int8 is lower than a model with data type half precision, and therefore, when the accuracy requirement is high, i.e. the required accuracy is greater than or equal to a preset accuracy, the target data type may be preferentially determined to be half precision.
The preset accuracy may be configured according to an accuracy that can be actually achieved by the model, which is not limited in the present invention.
In the above embodiment, after the data types are converted into int8 or half precision, since the memory occupation of each data is reduced, the memory consumption of the whole network is reduced, the bandwidth consumption is reduced at the same time, and the performance is improved, both the two conversion modes can improve the data processing speed to a certain extent.
Of course, in the case of other cases than the above (1) and (2), either one of the two conversion modes may be selected, or a default selection may be made according to a preset configuration, and the present invention is not limited.
It should be noted that, since the conversion between data types is well known in the art, it is not described herein in detail.
The adding unit 113 adds the correction layer to the converted initial model.
Wherein, the correction layer can be any one layer of the newly added initial model.
For example: the correction layer can be a newly added convolution layer, a newly added full connection layer and the like.
The optimization unit 114 trains the weights of the modified layer to optimize the initial model, resulting in an image processing model.
It can be understood that after the initial model is subjected to the data type conversion, the accuracy of the initial model will be reduced, and therefore, in order to improve the data processing speed and ensure the accuracy of the model, the initial model needs to be optimized.
Specifically, the optimizing unit 114 trains the weights of the correction layers to optimize the initial model, and obtaining the image processing model includes:
fixing the weight of each layer in the initial model to train the weight of the correction layer;
and stopping training in response to the fact that the accuracy of the model is larger than or equal to a preset threshold value, and obtaining the image processing model.
The preset threshold value can be configured according to actual requirements so as to ensure that the precision of the model meets task requirements.
In this embodiment, the optimizing unit 114 trains the weights of the correction layer to optimize the initial model, and obtaining the image processing model further includes:
the optimization unit 114 trains the weight of the correction layer with full precision as a data type to obtain the image processing model.
And performing multiple operations on the correction layer by taking full precision as a data type, so that the accuracy of the model is continuously improved, the accuracy requirement is finally met, and the optimized data processing speed and accuracy meet the requirement of the image processing model.
In the present embodiment, the construction unit 116 constructs the verification data set;
the verification unit 117 verifies the image processing model with data in the verification data set.
The verification data set may be obtained from historical data, or may be provided by a designated organization or staff, and the invention is not limited as long as the reliability of the data is ensured.
Through the implementation mode, the image processing model can be further verified to ensure the correctness and the usability of the image processing model.
The acquisition unit 110 acquires an image to be processed from the image processing instruction.
The image to be processed may be uploaded by a relevant person, or may be acquired by using a web crawler technology, which is not limited in the present invention.
The processing unit 115 inputs the image to be processed into the image processing model, and outputs an image processing result.
For example: the image processing result may be an image classification result, or an image segmentation result, or the like.
It should be noted that, image processing models obtained according to different training are different, and corresponding image processing results are also different, so as to satisfy various types of image processing tasks.
According to the technical scheme, the method can respond to the received image processing instruction, obtain training data, train a preset deep learning model by the training data by utilizing a preset deep learning framework to obtain an initial model, further convert the data type of the initial model to improve the data processing speed of the model, add a correction layer into the converted initial model, train the weight of the correction layer to optimize the initial model to obtain the image processing model to further improve the accuracy of the model, obtain the image to be processed from the image processing instruction, input the image to be processed into the image processing model, output an image processing result, realize image processing based on the optimized image processing model and simultaneously ensure the data processing speed and accuracy.
Fig. 3 is a schematic structural diagram of an electronic device implementing the image processing method according to the preferred embodiment of the invention.
The electronic device 1 may comprise a memory 12, a processor 13 and a bus, and may further comprise a computer program, such as an image processing program, stored in the memory 12 and executable on the processor 13.
It will be understood by those skilled in the art that the schematic diagram is merely an example of the electronic device 1, and does not constitute a limitation to the electronic device 1, the electronic device 1 may have a bus-type structure or a star-type structure, the electronic device 1 may further include more or less hardware or software than those shown in the figures, or different component arrangements, for example, the electronic device 1 may further include an input and output device, a network access device, and the like.
It should be noted that the electronic device 1 is only an example, and other existing or future electronic products, such as those that can be adapted to the present invention, should also be included in the scope of the present invention, and are included herein by reference.
The memory 12 includes at least one type of readable storage medium, which includes flash memory, removable hard disks, multimedia cards, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, magnetic disks, optical disks, etc. The memory 12 may in some embodiments be an internal storage unit of the electronic device 1, for example a removable hard disk of the electronic device 1. The memory 12 may also be an external storage device of the electronic device 1 in other embodiments, such as a plug-in mobile hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like provided on the electronic device 1. Further, the memory 12 may also include both an internal storage unit and an external storage device of the electronic device 1. The memory 12 may be used not only to store application software installed in the electronic apparatus 1 and various types of data such as codes of image processing programs, etc., but also to temporarily store data that has been output or is to be output.
The processor 13 may be composed of an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or may be composed of a plurality of integrated circuits packaged with the same or different functions, including one or more Central Processing Units (CPUs), microprocessors, digital Processing chips, graphics processors, and combinations of various control chips. The processor 13 is a Control Unit (Control Unit) of the electronic device 1, connects various components of the electronic device 1 by various interfaces and lines, and executes various functions and processes data of the electronic device 1 by running or executing programs or modules (e.g., executing image processing programs and the like) stored in the memory 12 and calling data stored in the memory 12.
The processor 13 executes an operating system of the electronic device 1 and various installed application programs. The processor 13 executes the application program to implement the steps in the respective image processing method embodiments described above, such as steps S10, S11, S12, S13, S14, S15, S16 shown in fig. 1.
Alternatively, the processor 13, when executing the computer program, implements the functions of the modules/units in the above device embodiments, for example:
responding to the received image processing instruction, and acquiring training data;
training a preset deep learning model by using the training data by using a preset deep learning architecture to obtain an initial model;
converting the data type of the initial model;
adding a correction layer into the converted initial model;
training the weight of the correction layer to optimize the initial model to obtain an image processing model;
acquiring an image to be processed from the image processing instruction;
and inputting the image to be processed into the image processing model, and outputting an image processing result.
Illustratively, the computer program may be divided into one or more modules/units, which are stored in the memory 12 and executed by the processor 13 to accomplish the present invention. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution process of the computer program in the electronic device 1. For example, the computer program may be divided into an acquisition unit 110, a training unit 111, a transformation unit 112, an addition unit 113, an optimization unit 114, a processing unit 115, a construction unit 116 and a verification unit 117.
The integrated unit implemented in the form of a software functional module may be stored in a computer-readable storage medium. The software functional module is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a computer device, or a network device) or a processor (processor) to execute parts of the image processing method according to the embodiments of the present invention.
The integrated modules/units of the electronic device 1 may be stored in a computer-readable storage medium if they are implemented in the form of software functional units and sold or used as separate products. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may be implemented by a computer program, which may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method embodiments may be implemented.
Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying said computer program code, recording medium, U-disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM).
The bus may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one arrow is shown in FIG. 3, but this does not indicate only one bus or one type of bus. The bus is arranged to enable connection communication between the memory 12 and at least one processor 13 or the like.
Although not shown, the electronic device 1 may further include a power supply (such as a battery) for supplying power to each component, and preferably, the power supply may be logically connected to the at least one processor 13 through a power management device, so as to implement functions of charge management, discharge management, power consumption management, and the like through the power management device. The power supply may also include any component of one or more dc or ac power sources, recharging devices, power failure detection circuitry, power converters or inverters, power status indicators, and the like. The electronic device 1 may further include various sensors, a bluetooth module, a Wi-Fi module, and the like, which are not described herein again.
Further, the electronic device 1 may further include a network interface, and optionally, the network interface may include a wired interface and/or a wireless interface (such as a WI-FI interface, a bluetooth interface, etc.), which are generally used for establishing a communication connection between the electronic device 1 and other electronic devices.
Optionally, the electronic device 1 may further comprise a user interface, which may be a Display (Display), an input unit (such as a Keyboard), and optionally a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, or the like. The display, which may also be referred to as a display screen or display unit, is suitable for displaying information processed in the electronic device 1 and for displaying a visualized user interface, among other things.
It is to be understood that the described embodiments are for purposes of illustration only and that the scope of the appended claims is not limited to such structures.
Fig. 3 only shows the electronic device 1 with components 12-13, and it will be understood by a person skilled in the art that the structure shown in fig. 3 does not constitute a limitation of the electronic device 1, and may comprise fewer or more components than shown, or a combination of certain components, or a different arrangement of components.
With reference to fig. 1, the memory 12 of the electronic device 1 stores a plurality of instructions to implement an image processing method, and the processor 13 executes the plurality of instructions to implement:
responding to the received image processing instruction, and acquiring training data;
training a preset deep learning model by using the training data by using a preset deep learning architecture to obtain an initial model;
converting the data type of the initial model;
adding a correction layer into the converted initial model;
training the weight of the correction layer to optimize the initial model to obtain an image processing model;
acquiring an image to be processed from the image processing instruction;
and inputting the image to be processed into the image processing model, and outputting an image processing result.
Specifically, the processor 13 may refer to the description of the relevant steps in the embodiment corresponding to fig. 1 for a specific implementation method of the instruction, which is not described herein again.
In the embodiments provided in the present invention, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is only one logical functional division, and other divisions may be realized in practice.
The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
In addition, functional modules in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional module.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof.
The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference signs in the claims shall not be construed as limiting the claim concerned.
Furthermore, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or means recited in the system claims may also be implemented by one unit or means in software or hardware. The terms second, etc. are used to denote names, but not any particular order.
Finally, it should be noted that the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting, and although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions may be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims (10)

1. An image processing method, characterized in that the image processing method comprises:
responding to the received image processing instruction, and acquiring training data;
training a preset deep learning model by using the training data by using a preset deep learning architecture to obtain an initial model;
converting the data type of the initial model;
adding a correction layer into the converted initial model;
training the weight of the correction layer to optimize the initial model to obtain an image processing model;
acquiring an image to be processed from the image processing instruction;
and inputting the image to be processed into the image processing model, and outputting an image processing result.
2. The image processing method of claim 1, wherein the converting the data type of the initial model comprises:
acquiring demand data from the image processing instruction;
determining a target data type according to the demand data;
converting the initial model to the target data type.
3. The image processing method of claim 2, wherein said determining a target data type from said demand data comprises:
determining a required data processing speed and a required accuracy according to the demand data;
in response to the required data processing speed being greater than or equal to a preset speed, determining that the target data type is int 8; or
In response to the requested accuracy being greater than or equal to a preset accuracy, determining the target data type to be semi-accurate.
4. The image processing method according to claim 3, characterized in that:
the data processing speed of the model of data type int8 is higher than that of the model of data type half precision, and the accuracy of the model of data type int8 is lower than that of the model of data type half precision.
5. The image processing method of claim 1, wherein the training of the weights of the modified layer to optimize the initial model, resulting in an image processing model comprises:
fixing the weight of each layer in the initial model to train the weight of the correction layer;
and stopping training in response to the fact that the accuracy of the model is larger than or equal to a preset threshold value, and obtaining the image processing model.
6. The image processing method of claim 1, wherein the training of the weights of the modified layer to optimize the initial model, resulting in an image processing model further comprises:
and training the weight of the correction layer by taking the full precision as a data type to obtain the image processing model.
7. The image processing method according to claim 1, further comprising:
constructing a verification data set;
and verifying the image processing model by using the data in the verification data set.
8. An image processing apparatus characterized by comprising:
an acquisition unit configured to acquire training data in response to a received image processing instruction;
the training unit is used for training a preset deep learning model by using the training data by using a preset deep learning architecture to obtain an initial model;
a conversion unit for converting the data type of the initial model;
the adding unit is used for adding a correction layer into the converted initial model;
the optimization unit is used for training the weight of the correction layer to optimize the initial model to obtain an image processing model;
the acquisition unit is further used for acquiring an image to be processed from the image processing instruction;
and the processing unit is used for inputting the image to be processed into the image processing model and outputting an image processing result.
9. An electronic device, characterized in that the electronic device comprises:
a memory storing at least one instruction; and
a processor executing instructions stored in the memory to implement the image processing method of any one of claims 1 to 7.
10. A computer-readable storage medium characterized by: the computer-readable storage medium has stored therein at least one instruction that is executed by a processor in an electronic device to implement the image processing method of any one of claims 1 to 7.
CN202010740717.5A 2020-07-28 2020-07-28 Image processing method, image processing device, electronic equipment and storage medium Pending CN114004968A (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202010740717.5A CN114004968A (en) 2020-07-28 2020-07-28 Image processing method, image processing device, electronic equipment and storage medium
US17/386,326 US20220036192A1 (en) 2020-07-28 2021-07-27 Image processing method, electronic device and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010740717.5A CN114004968A (en) 2020-07-28 2020-07-28 Image processing method, image processing device, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN114004968A true CN114004968A (en) 2022-02-01

Family

ID=79920610

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010740717.5A Pending CN114004968A (en) 2020-07-28 2020-07-28 Image processing method, image processing device, electronic equipment and storage medium

Country Status (2)

Country Link
US (1) US20220036192A1 (en)
CN (1) CN114004968A (en)

Also Published As

Publication number Publication date
US20220036192A1 (en) 2022-02-03

Similar Documents

Publication Publication Date Title
CN109740732B (en) Neural network processor, convolutional neural network data multiplexing method and related equipment
CN111738212B (en) Traffic signal lamp identification method, device, equipment and medium based on artificial intelligence
CN111639153A (en) Query method and device based on legal knowledge graph, electronic equipment and medium
CN111476225B (en) In-vehicle human face identification method, device, equipment and medium based on artificial intelligence
CN111931729B (en) Pedestrian detection method, device, equipment and medium based on artificial intelligence
CN111950621A (en) Target data detection method, device, equipment and medium based on artificial intelligence
CN114511038A (en) False news detection method and device, electronic equipment and readable storage medium
CN113868528A (en) Information recommendation method and device, electronic equipment and readable storage medium
CN111985545A (en) Target data detection method, device, equipment and medium based on artificial intelligence
CN115600644A (en) Multitasking method and device, electronic equipment and storage medium
CN115205225A (en) Training method, device and equipment of medical image recognition model and storage medium
CN109583579B (en) Computing device and related product
CN114913371A (en) Multitask learning model training method and device, electronic equipment and storage medium
CN113204698B (en) News subject term generation method, device, equipment and medium
CN114022841A (en) Personnel monitoring and identifying method and device, electronic equipment and readable storage medium
CN111950707B (en) Behavior prediction method, device, equipment and medium based on behavior co-occurrence network
CN111985194A (en) Data storage method and device, electronic equipment and storage medium
CN112069782A (en) Document template generation method and device, electronic equipment and storage medium
CN114004968A (en) Image processing method, image processing device, electronic equipment and storage medium
CN116401602A (en) Event detection method, device, equipment and computer readable medium
CN115346095A (en) Visual question answering method, device, equipment and storage medium
CN112132037B (en) Pavement detection method, device, equipment and medium based on artificial intelligence
CN115239958A (en) Wheel hub damage detection method based on artificial intelligence and related equipment
TWI807204B (en) Image processing method, device, electronic equipment and storage medium
CN112233194B (en) Medical picture optimization method, device, equipment and computer readable storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination