CN116778488A - Image processing method, device, terminal equipment, server and storage medium - Google Patents
Image processing method, device, terminal equipment, server and storage medium Download PDFInfo
- Publication number
- CN116778488A CN116778488A CN202311048069.7A CN202311048069A CN116778488A CN 116778488 A CN116778488 A CN 116778488A CN 202311048069 A CN202311048069 A CN 202311048069A CN 116778488 A CN116778488 A CN 116778488A
- Authority
- CN
- China
- Prior art keywords
- annotation
- labeling
- image
- information
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 26
- 238000002372 labelling Methods 0.000 claims abstract description 239
- 238000012545 processing Methods 0.000 claims abstract description 20
- 238000000034 method Methods 0.000 claims description 32
- 238000004590 computer program Methods 0.000 claims description 6
- 238000004422 calculation algorithm Methods 0.000 description 26
- 238000012549 training Methods 0.000 description 20
- 238000010586 diagram Methods 0.000 description 8
- 238000007726 management method Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 230000002093 peripheral effect Effects 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 238000012550 audit Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000008713 feedback mechanism Effects 0.000 description 1
- 238000002955 isolation Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Abstract
The invention provides an image processing method, an image processing device, terminal equipment, a server and a storage medium, and relates to the technical field of images. The image processing method comprises the following steps: reporting the annotation data information of the annotated images to a server under the condition that the number of the annotated images is larger than a preset threshold value, wherein the server is used for generating a first annotation model based on the annotation data information, and the first annotation model is a model for predicting the annotation information of the images; sending the image to be annotated to the server; receiving a labeling result sent by the server, wherein the labeling result is a labeling information prediction result obtained by inputting the image to be labeled into the first labeling model by the server for labeling information prediction; and labeling the image to be labeled based on the labeling result. Therefore, the labeling efficiency of the image can be effectively improved.
Description
Technical Field
The present invention relates to the field of image technologies, and in particular, to an image processing method, an image processing device, a terminal device, a server, and a storage medium.
Background
At present, in the labeling process of images, labeling personnel usually label the images to be labeled manually one by one. However, when there are many images to be marked, the manual marking method is time-consuming and labor-consuming, and has a problem of low marking efficiency.
Therefore, the image labeling mode in the related art has the problem of low labeling efficiency.
Disclosure of Invention
The embodiment of the invention provides an image processing method, an image processing device, terminal equipment, a server and a storage medium, which are used for solving the problem of low labeling efficiency of an image labeling mode in the related technology.
In order to solve the technical problems, the invention is realized as follows:
in a first aspect, an embodiment of the present invention provides an image processing method, which is used for a terminal device, and the method includes:
reporting the annotation data information of the annotated images to a server under the condition that the number of the annotated images is larger than a preset threshold value, wherein the server is used for generating a first annotation model based on the annotation data information, and the first annotation model is a model for predicting the annotation information of the images;
sending the image to be annotated to the server;
Receiving a labeling result sent by the server, wherein the labeling result is a labeling information prediction result obtained by inputting the image to be labeled into the first labeling model by the server for labeling information prediction;
and labeling the image to be labeled based on the labeling result.
In a second aspect, an embodiment of the present invention provides an image processing method, for a server, including:
receiving marking data information sent by terminal equipment, wherein the marking data information is image marking data of marked images sent by the terminal equipment under the condition that the number of the marked images is larger than a preset threshold value;
generating a first annotation model based on the annotation data information, wherein the first annotation model is a model for carrying out annotation information prediction on the image;
receiving an image to be marked sent by the terminal equipment;
inputting the image to be marked into the first marking model for marking information prediction, and obtaining marking results of the image to be marked;
and sending the labeling result to the terminal equipment, wherein the labeling result is used for labeling the image to be labeled.
In a third aspect, an embodiment of the present invention provides an image processing apparatus for a terminal device, including:
The system comprises a reporting module, a server and a prediction module, wherein the reporting module is used for reporting the annotation data information of the annotated image to the server under the condition that the number of the annotated image is larger than a preset threshold value, and the server is used for generating a first annotation model based on the annotation data information, wherein the first annotation model is a model for carrying out annotation information prediction on the image;
the sending module is used for sending the image to be marked to the server;
the receiving module is used for receiving the labeling result sent by the server, wherein the labeling result is a labeling information prediction result obtained by the server by inputting the image to be labeled into the first labeling model for labeling information prediction;
and the labeling module is used for labeling the image to be labeled based on the labeling result.
In a fourth aspect, an embodiment of the present invention provides an image processing apparatus for a server, the apparatus including:
the first receiving module is used for receiving marking data information sent by the terminal equipment, wherein the marking data information is image marking data of marked images sent by the terminal equipment under the condition that the number of the marked images is larger than a preset threshold value;
The generation module is used for generating a first annotation model based on the annotation data information, wherein the first annotation model is a model for carrying out annotation information prediction on the image;
the second receiving module is used for receiving the image to be marked sent by the terminal equipment;
the prediction module is used for inputting the image to be marked into the first marking model to perform marking information prediction and obtaining a marking result of the image to be marked;
the sending module is used for sending the labeling result to the terminal equipment, and the labeling result is used for labeling the image to be labeled.
In a fifth aspect, an embodiment of the present invention provides a terminal device, including a transceiver and a processor,
the transceiver is used for reporting the marked data information of the marked images to the server under the condition that the number of the marked images is larger than a preset threshold value, and the server is used for generating a first marked model based on the marked data information, wherein the first marked model is a model for carrying out marked information prediction on the images;
the transceiver is used for sending the image to be marked to the server;
the transceiver is used for receiving the labeling result sent by the server, wherein the labeling result is a labeling information prediction result obtained by the server by inputting the image to be labeled into the first labeling model for labeling information prediction;
And the processor is used for marking the image to be marked based on the marking result.
In a sixth aspect, embodiments of the present invention provide a server, comprising a transceiver and a processor,
the transceiver is used for receiving marking data information sent by the terminal equipment, wherein the marking data information is image marking data of marked images sent by the terminal equipment under the condition that the number of the marked images is larger than a preset threshold value;
the processor is used for generating a first annotation model based on the annotation data information, wherein the first annotation model is a model for carrying out annotation information prediction on the image;
the transceiver is used for receiving the image to be marked sent by the terminal equipment;
the processor is used for inputting the image to be marked into the first marking model for marking information prediction and obtaining marking results of the image to be marked;
the transceiver is used for sending the labeling result to the terminal equipment, and the labeling result is used for labeling the image to be labeled.
In a seventh aspect, an embodiment of the present invention provides a terminal device, including: a processor, a memory, and a program stored on the memory and executable on the processor, which when executed by the processor, implements the steps of the image processing method described in the first aspect.
In an eighth aspect, an embodiment of the present invention provides a server, including: a processor, a memory, and a program stored on the memory and executable on the processor, which when executed by the processor, implements the steps of the image processing method described in the second aspect.
In a ninth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, which when executed by a processor, implements the steps of the image processing method described in the first aspect above; or the computer program when executed by a processor implements the steps of the image processing method of the second aspect described above.
In the embodiment of the invention, the first annotation model is generated by training based on the preset number of annotation image data, so that in the subsequent image annotation process, the corresponding annotation result can be predicted based on the first annotation model, and the user can conveniently annotate the image to be annotated based on the annotation result, thereby achieving the purpose of improving the annotation efficiency of the image.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments of the present invention will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort to a person of ordinary skill in the art.
Fig. 1 is a flowchart of an image processing method according to an embodiment of the present invention;
FIG. 2 is a flowchart of another image processing method according to an embodiment of the present invention;
FIG. 3 is a block diagram of an image annotation system according to an embodiment of the present invention;
fig. 4 is a schematic structural view of an image processing apparatus according to an embodiment of the present invention;
fig. 5 is a schematic structural view of another image processing apparatus according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a terminal device according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of a server according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Referring to fig. 1, fig. 1 is a flowchart of an image processing method according to an embodiment of the present invention. The image processing method provided by the embodiment of the invention can be used for terminal equipment, and as shown in fig. 1, the method comprises the following steps:
And step 101, reporting the labeling data information of the labeled images to a server under the condition that the number of the labeled images is larger than a preset threshold value.
The noted image is understood to be an image that has been manually noted.
The server can generate a first annotation model based on the annotation data information, wherein the first annotation model is a model for carrying out annotation information prediction on the image.
The number of the marked images is larger than the preset threshold, and the number of the marked images can be understood to be used for model training, and a marking model with accurate marking information prediction can be obtained.
For example, if the minimum number of samples for training the labeling model is 100, the preset threshold may be set to 110 or 120, so that a corresponding number of training samples are obtained based on the number of labeled images reaching the preset threshold, and the initial labeling model is trained based on the training samples until the first labeling model is generated.
When the preset threshold is set to 100, that is, when the number of images marked by the marking personnel reaches 100, marking data information of 100 images which have been marked can be collected and reported to the server, so that the server can perform marking model training based on the reported marking data information and generate a first marking model.
The marking data information comprises central point coordinate information, first coordinate information and second coordinate information of the marking frame, wherein the first coordinate information and the second coordinate information are coordinate information of two diagonal positions on a diagonal line of the marking frame;
in the case that the labeling frame is a rectangular frame, the first coordinate information may be coordinate information of an upper left corner of the labeling frame, and the second coordinate information may be coordinate information of a lower right corner of the labeling frame; alternatively, the first coordinate information may be the coordinate information of the lower left corner of the label frame, and the second coordinate information may be the coordinate information of the upper right corner of the label frame.
The server may use the labeling data as a training sample to train the initial labeling model, thereby obtaining a first labeling model.
And 102, sending the image to be annotated to the server.
And step 103, receiving the labeling result sent by the server.
The labeling result is a labeling information prediction result obtained by inputting the image to be labeled into the first labeling model by the server for labeling information prediction.
The labeling result may include coordinate parameters, width and height dimensions, etc. of the labeling frame.
And 104, labeling the image to be labeled based on the labeling result.
The image to be marked can be marked based on the marking result sent by the server, so that the quick marking of the image to be marked is realized, and the marking efficiency of the image to be marked is improved.
For example, the labeling result can be directly displayed on the image to be labeled, so that labeling personnel can finish labeling only by simple fine adjustment, and the labeling efficiency of the image is improved.
In the embodiment of the invention, the first annotation model is generated by training based on the preset number of annotation image data, so that in the subsequent image annotation process, the corresponding annotation result can be predicted based on the first annotation model, and the user can conveniently annotate the image to be annotated based on the annotation result, thereby achieving the purpose of improving the annotation efficiency of the image.
Optionally, the labeling the image to be labeled based on the labeling result includes:
labeling the image to be labeled according to the labeling result to obtain a first labeling image;
and storing the first annotation model in a local storage space of the terminal equipment under the condition that the annotation information of the first annotation image meets the preset condition.
The fact that the labeling information of the first labeling image meets the preset condition can be understood that the parameter difference value between the labeling information of the first labeling image and the target labeling information of the image to be labeled is smaller than a preset value, and the preset value is used for indicating the accuracy of a labeling result obtained by prediction of the first labeling model; and the smaller the parameter difference value between the labeling information of the first labeling image and the target labeling information of the image to be labeled is, the higher the accuracy of the labeling result predicted based on the first labeling model is.
In this embodiment, when the labeling information of the first labeling image meets the preset condition, that is, when the accuracy of the labeling result predicted by the first labeling model reaches the preset accuracy, the first labeling model may be downloaded and stored in the local storage space of the terminal device, so that the labeling personnel may label the image to be labeled locally of the terminal device, that is, off-line labeling may be achieved, and the influence of the network environment on image labeling is reduced.
In addition, under the condition that the accuracy of the marking result predicted by the first marking model reaches the preset accuracy, the marking result predicted by the first marking model can be directly marked on the image to be marked, automatic marking of the image to be marked is achieved, and the marking efficiency of the image is further improved.
In the embodiment of the invention, when the number of marked images is larger than a preset threshold value, marking data information of the marked images is reported to a server, wherein the server is used for generating a first marking model based on the marking data information, and the first marking model is a model for marking information prediction of the images; sending the image to be annotated to the server; receiving a labeling result sent by the server, wherein the labeling result is a labeling information prediction result obtained by inputting the image to be labeled into the first labeling model by the server for labeling information prediction; and labeling the image to be labeled based on the labeling result. Therefore, the labeling efficiency of the image can be effectively improved.
Referring to fig. 2, fig. 2 is a flowchart of another image processing method according to an embodiment of the present invention. The image processing method provided by the embodiment of the invention can be used for a server, and as shown in fig. 2, the method comprises the following steps:
step 201, receiving label data information sent by a terminal device.
The labeling data information may be image labeling data of the labeled image, which is sent by the terminal device when the number of labeled images is greater than a preset threshold.
And 202, generating a first annotation model based on the annotation data information.
The first labeling model is a model for predicting labeling information of the image.
And 203, receiving the image to be marked sent by the terminal equipment.
And 204, inputting the image to be annotated into the first annotation model for annotation information prediction, and obtaining an annotation result of the image to be annotated.
And 205, transmitting the labeling result to the terminal equipment, wherein the labeling result is used for labeling the image to be labeled.
In the embodiment of the invention, the first annotation model is generated and obtained by receiving the annotation data information sent by the terminal equipment so that the initial annotation model can be trained based on the annotation data information on the basis of the server; in the subsequent labeling process, the user can predict the labeling information of the image to be labeled based on the first labeling model and obtain the corresponding labeling result, and then the user can label the image to be labeled based on the labeling result, so that the purpose of improving the labeling efficiency of the image is achieved.
Optionally, the labeling data information includes central point coordinate information, first coordinate information and second coordinate information of a labeling frame, where the first coordinate information and the second coordinate information are coordinate information of two diagonal positions on a diagonal line of the labeling frame;
the generating a first annotation model based on the annotation data information includes:
normalizing the central point coordinate information, the first coordinate information and the second coordinate information to obtain a first target value;
generating the annotation data information of the prediction annotation frame based on the annotation data information of the target annotation frame and the first target value;
calculating a first loss value based on the marking data information of the prediction marking frame and the marking data information of the marked image;
updating parameters of a second annotation model based on the first loss value, obtaining an updated second annotation model, and determining the converged second annotation model as a first annotation model under the condition that the updated second annotation model is converged, wherein the second annotation model is used for carrying out annotation information prediction on an image.
The preset annotation frame can be understood as an annotation frame predicted by the current iteration update model.
The noted data information of the noted image may be understood as training sample data information corresponding to an iteration update.
For example, if the second labeling model is a labeling model obtained after i-1 times of iterative updating is performed on the initial labeling model, the predicted labeling frame is a labeling frame obtained by inputting an image to be labeled into the second labeling model; when the initial annotation model is subjected to the ith iteration update, a loss value can be calculated based on the annotation data information of the prediction annotation frame and the real annotation frame corresponding to the ith training sample, parameters of the second annotation model are updated based on the calculated loss value, and the updated second annotation model is obtained, so that the iteration update of the annotation model is realized until the annotation model is completely converged, and the converged second annotation model is determined as the first annotation model.
In the embodiment, the first labeling model is obtained through training based on the labeling data information of the labeled image, so that the labeling result obtained through prediction based on the first labeling model can be labeled on the image to be labeled in the subsequent image labeling process, and the purpose of improving the labeling efficiency of the image is achieved.
In the embodiment of the invention, the labeling data information sent by the terminal equipment is received, and the labeling data information is the image labeling data of the labeled images sent by the terminal equipment under the condition that the number of the labeled images is larger than a preset threshold value; generating a first annotation model based on the annotation data information, wherein the first annotation model is a model for carrying out annotation information prediction on the image; receiving an image to be marked sent by the terminal equipment; inputting the image to be marked into the first marking model for marking information prediction, and obtaining marking results of the image to be marked; and sending the labeling result to the terminal equipment, wherein the labeling result is used for labeling the image to be labeled. Therefore, the labeling efficiency of the image can be effectively improved.
Referring to fig. 3, fig. 3 is a schematic diagram of an image labeling system according to an embodiment of the present invention. The image annotation system is provided with various roles: a system administrator, a labeling person, a quality inspector and a project administrator. The system administrator has the highest authority of the system and can create/delete users. The project manager is responsible for creating the project and uploading annotation tags, annotation data (images), selection of annotators, quality inspectors, etc. The labeling personnel can pick up the labeling task and label, and the quality inspector can audit the labeling result and determine that the labeling task passes or is rejected. The system can be a multi-tenant mode with institutions as basic isolation, users among different institutions are completely isolated in marking data, and the marking data are not affected by each other.
As shown in fig. 3, the image labeling system may adopt a micro-service architecture with separated front and rear ends, and based on the image labeling system shown in fig. 3, the following steps may be implemented, so as to implement automatic labeling of the image to be labeled:
step one, developing front and back ends of an image marking system, wherein the front end can adopt React (JavaScript library for constructing a user interface), and rapidly constructing a background management system by using an open source ant design pro. The backend interface may be developed using Golang.
And secondly, constructing algorithm training and prediction services, wherein the real-time requirement of algorithm prediction is considered, and the embodiment of the invention selects the YOLO V3 algorithm, so that compared with other image detection algorithms (R-CNN, SPP-net, fast R-CNN), the YOLO V3 prediction speed is Faster.
The specific training steps comprise: acquiring image information to be trained, namely acquiring marking data information of marked images, carrying out data enhancement (such as enhancing data in a coding mode) on marking data of the images, reading coordinates of the upper left corner and the lower right corner of a marking frame, calculating information of a central point of the marking frame, and carrying out normalization processing on yolov3 according to the central point and width and height information of a rectangular frame, wherein four normalized values are x, y, w and h (x and y represent central point coordinates of the rectangular frame, and w and h represent width and height information of the rectangular frame). The original real frame has a value between 0 and 1, and needs to be multiplied by the width and height of the prior frame to be converted into the form of a characteristic layer, and then can be compared with the prior frame. And then calculating the center point, the width and the height of the real annotation frame on the feature layer, and calculating which network of the feature layer the real annotation frame belongs to.
The prior width can be understood as the target labeling frame in the previous embodiment.
The specific prediction steps comprise: and (5) performing cyclic calculation on the loss functions loss on the three effective feature layers. And then obtaining the width and height of the feature layer, scaling the prior frame size to the size of the feature layer, and scaling by utilizing the original picture size and the proportion of the feature layer. The feature layer obtains a prediction result, namely the adjustment parameters of the prior frame. And decoding the prediction result, calculating the IOU with the real frame, and neglecting the prediction result with large overlapping degree. Since this is a box that belongs to a prediction that is relatively accurate, it is not appropriate as a negative sample. And comparing the parameters of the predicted frame with the parameters of the real frame on the feature layer to find loss, continuously taking out the real frame, and carrying out the same operation.
And step three, the algorithm service provides an external web api interface. The algorithm service provides loading, deployment and management of the model through tf-service, and provides webservice interfaces to the outside. tf-service is a high performance library for machine learning model deployment that can directly bring the trained model on line and provide services.
One important feature of tf-serving is, among others: supporting hot updates and model version auto-management.
And fourthly, building a training service cluster. The whole model training and resource management system can be built based on OpenPAI, and the system is based on a pure Kubernetes architecture. The specific installation flow is as follows: prepare 5 independent machines: one was a dev box machine, a master machine and three worker machines. Wherein the dev box machine controls the master machine and the worker machine through ssh during installation, maintenance and uninstallation. The master machine is used to run Kubernetes components and core OpenPAI services. The worker machine is used to perform training tasks and may specify one or more.
And fifthly, copying the model and automatically predicting. The machine where the algorithm service is located communicates with the HDFS server through an SSH protocol, an algorithm model file is copied to the algorithm server, the model is loaded through tf-service, and the front-end page indirectly calls the algorithm prediction service through a back-end service interface to complete image annotation prediction.
And step six, in the process of labeling pictures, labeling personnel can carry out secondary labeling on the result predicted by the algorithm, and when the whole labeling task is finished, the server side automatically realizes statistics on the accuracy of the result predicted by the algorithm. Thereby helping the algorithm personnel to further optimize the algorithm training parameters.
And step seven, after the whole labeling task is finished, depending on the fact that the server in the step five already stores the model file trained by the algorithm, the model file stored at the server end can be downloaded locally by sending an http request through the front-end page of the labeling terminal, and the archiving of the model file of the algorithm is realized.
The embodiment of the invention provides an image annotation system which can be automatically trained, deployed and predicted. When the number of the marked images of the marking personnel reaches a set threshold, the OpenPAI training task is triggered, the training service cluster is scheduled to train the image data, and the trained algorithm model file is copied to a server where tf-service is located through an SSH protocol, so that automatic deployment of the algorithm is realized. And simultaneously restarting the tf-serving service to finish loading the algorithm model file, realizing the external prediction service capability, and realizing the automatic prediction of the algorithm when a annotator annotates the new picture again, wherein the specific implementation mode can refer to the specific implementation step five.
In addition, the embodiment of the invention also provides an algorithm prediction feedback mechanism. Reference may be made specifically to the above-mentioned implementation step six. The automatic prediction result of the algorithm is marked secondarily to form a closed loop, and the accuracy test is carried out on the algorithm during marking, so that the subsequent further optimization of algorithm parameters is facilitated.
Compared with the existing labeling system, the image labeling system provided by the embodiment of the invention can also directly export the model file trained by the algorithm, and the problem that the format of the label data set exported by the existing system is inconsistent with the format required by the algorithm training is avoided.
Referring to fig. 4, fig. 4 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present invention. As shown in fig. 4, the image processing apparatus 400 includes:
the reporting module 401 is configured to report, when the number of marked images is greater than a preset threshold, marking data information of the marked images to a server, where the server is configured to generate a first marking model based on the marking data information, and the first marking model is a model for performing marking information prediction on the images;
a sending module 402, configured to send an image to be annotated to the server;
the receiving module 403 is configured to receive a labeling result sent by the server, where the labeling result is a labeling information prediction result obtained by inputting the image to be labeled into the first labeling model by the server for labeling information prediction;
and the labeling module 404 is configured to label the image to be labeled based on the labeling result.
Optionally, the labeling module 404 is specifically configured to:
labeling the image to be labeled according to the labeling result to obtain a first labeling image;
and storing the first annotation model in a local storage space of the terminal equipment under the condition that the annotation information of the first annotation image meets the preset condition.
Optionally, the labeling data information includes central point coordinate information, first coordinate information and second coordinate information of a labeling frame, where the first coordinate information and the second coordinate information are coordinate information of two diagonal positions on a diagonal line of the labeling frame.
The image processing apparatus 400 can implement the processes of the method embodiment of fig. 1 in the embodiment of the present invention, and achieve the same beneficial effects, and in order to avoid repetition, a detailed description is omitted here.
Referring to fig. 5, fig. 5 is a schematic structural diagram of another image processing apparatus according to an embodiment of the present invention. As shown in fig. 5, the image processing apparatus 500 includes:
a first receiving module 501, configured to receive annotation data information sent by a terminal device, where the annotation data information is image annotation data of an annotated image sent by the terminal device when the number of annotated images is greater than a preset threshold;
The generating module 502 is configured to generate a first labeling model based on the labeling data information, where the first labeling model is a model for predicting labeling information of an image;
a second receiving module 503, configured to receive an image to be annotated sent by the terminal device;
the prediction module 504 is configured to input the image to be annotated into the first annotation model to perform annotation information prediction, and obtain an annotation result of the image to be annotated;
and the sending module 505 is configured to send the labeling result to the terminal device, where the labeling result is used to label the image to be labeled.
Optionally, the labeling data information includes central point coordinate information, first coordinate information and second coordinate information of a labeling frame, where the first coordinate information and the second coordinate information are coordinate information of two diagonal positions on a diagonal line of the labeling frame;
the generating module 502 is specifically configured to:
normalizing the central point coordinate information, the first coordinate information and the second coordinate information to obtain a first target value;
generating the annotation data information of the prediction annotation frame based on the annotation data information of the target annotation frame and the first target value;
Calculating a first loss value based on the marking data information of the prediction marking frame and the marking data information of the marked image;
updating parameters of a second annotation model based on the first loss value, obtaining an updated second annotation model, and determining the converged second annotation model as a first annotation model under the condition that the updated second annotation model is converged, wherein the second annotation model is used for carrying out annotation information prediction on an image.
The image processing apparatus 500 can implement the processes of the method embodiment of fig. 2 in the embodiment of the present invention, and achieve the same beneficial effects, and in order to avoid repetition, a detailed description is omitted here.
The embodiment of the invention also provides a terminal device, which comprises: the image processing device comprises a processor, a memory and a program stored in the memory and capable of running on the processor, wherein the program realizes the processes of the image processing method embodiment when being executed by the processor, and can achieve the same technical effects, and the repetition is avoided, and the description is omitted here.
Specifically, referring to fig. 6, an embodiment of the present invention provides a schematic structural diagram of a terminal device. As shown in fig. 6, the embodiment of the present invention further provides a terminal device, which includes a bus 601, a transceiver 602, an antenna 603, a bus interface 604, a processor 605 and a memory 606.
The transceiver 602 is configured to report, when the number of marked images is greater than a preset threshold, marking data information of the marked images to a server, where the server is configured to generate a first marking model based on the marking data information, and the first marking model is a model for predicting marking information of the images;
the transceiver 602 is configured to send an image to be annotated to the server;
the transceiver 602 is configured to receive a labeling result sent by the server, where the labeling result is a labeling information prediction result obtained by inputting, by the server, the image to be labeled into the first labeling model for labeling information prediction;
the processor 605 is configured to annotate the image to be annotated based on the annotation result.
Optionally, the processor 605 is configured to:
labeling the image to be labeled according to the labeling result to obtain a first labeling image;
and storing the first annotation model in a local storage space of the terminal equipment under the condition that the annotation information of the first annotation image meets the preset condition.
Optionally, the labeling data information includes central point coordinate information, first coordinate information and second coordinate information of a labeling frame, where the first coordinate information and the second coordinate information are coordinate information of two diagonal positions on a diagonal line of the labeling frame.
In fig. 6, a bus architecture (represented by bus 601), the bus 601 may include any number of interconnected buses and bridges, with the bus 601 linking together various circuits, including one or more processors, represented by processor 605, and memory, represented by memory 606. The bus 601 may also link together various other circuits such as peripheral devices, voltage regulators, power management circuits, etc., which are well known in the art and, therefore, will not be described further herein. Bus interface 604 provides an interface between bus 601 and transceiver 602. The transceiver 602 may be one element or may be multiple elements, such as multiple receivers and transmitters, providing a means for communicating with various other apparatus over a transmission medium. The data processed by the processor 605 is transmitted over a wireless medium via an antenna 603, and further, the antenna 603 also receives data and transmits the data to the processor 605.
The processor 605 is responsible for managing the bus 601 and general processing, and may also provide various functions including timing, peripheral interfaces, voltage regulation, power management, and other control functions. And memory 606 may be used to store data used by processor 605 in performing operations.
Alternatively, the processor 605 may be CPU, ASIC, FPGA or a CPLD.
The embodiment of the invention also provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor, implements the processes of the above-mentioned image processing method embodiment, and can achieve the same technical effects, so that repetition is avoided, and no further description is given here. Wherein the computer readable storage medium is selected from Read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), magnetic disk or optical disk.
The embodiment of the invention also provides a server, which comprises: the image processing device comprises a processor, a memory and a program stored in the memory and capable of running on the processor, wherein the program realizes the processes of the image processing method embodiment when being executed by the processor, and can achieve the same technical effects, and the repetition is avoided, and the description is omitted here.
Specifically, referring to fig. 7, an embodiment of the present invention provides a schematic structural diagram of a server. As shown in fig. 7, an embodiment of the present invention further provides a server comprising a bus 701, a transceiver 702, an antenna 703, a bus interface 704, a processor 705 and a memory 706.
The transceiver 702 is configured to receive annotation data information sent by a terminal device, where the annotation data information is image annotation data of an annotated image sent by the terminal device when the number of annotated images is greater than a preset threshold;
the processor 705 is configured to generate a first labeling model based on the labeling data information, where the first labeling model is a model for predicting labeling information of an image;
the transceiver 702 is configured to receive an image to be annotated sent by the terminal device;
the processor 705 is configured to input the image to be annotated into the first annotation model for annotation information prediction, and obtain an annotation result of the image to be annotated;
the transceiver 702 is configured to send the labeling result to the terminal device, where the labeling result is used to label the image to be labeled.
Optionally, the labeling data information includes central point coordinate information, first coordinate information and second coordinate information of a labeling frame, where the first coordinate information and the second coordinate information are coordinate information of two diagonal positions on a diagonal line of the labeling frame;
the processor 705 is configured to:
Normalizing the central point coordinate information, the first coordinate information and the second coordinate information to obtain a first target value;
generating the annotation data information of the prediction annotation frame based on the annotation data information of the target annotation frame and the first target value;
calculating a first loss value based on the marking data information of the prediction marking frame and the marking data information of the marked image;
updating parameters of a second annotation model based on the first loss value, obtaining an updated second annotation model, and determining the converged second annotation model as a first annotation model under the condition that the updated second annotation model is converged, wherein the second annotation model is used for carrying out annotation information prediction on an image.
In FIG. 7, a bus architecture (represented by bus 701), the bus 701 may include any number of interconnected buses and bridges, with the bus 701 linking together various circuits, including one or more processors, represented by the processor 705, and memory, represented by the memory 706. The bus 701 may also link together various other circuits such as peripheral devices, voltage regulators, power management circuits, etc., which are well known in the art and, therefore, will not be described further herein. Bus interface 704 provides an interface between bus 701 and transceiver 702. The transceiver 702 may be one element or a plurality of elements, such as a plurality of receivers and transmitters, providing a means for communicating with various other apparatus over a transmission medium. The data processed by the processor 705 is transmitted over a wireless medium via the antenna 703, and further, the antenna 703 receives and transmits data to the processor 705.
The processor 705 is responsible for managing the bus 701 and general processing and may also provide various functions including timing, peripheral interfaces, voltage regulation, power management, and other control functions. And memory 706 may be used to store data used by processor 705 in performing operations.
Alternatively, the processor 705 may be a CPU, ASIC, FPGA or CPLD.
The embodiment of the invention also provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor, implements the processes of the above-mentioned embodiments of the repeated transmission control method, and can achieve the same technical effects, so that repetition is avoided, and no further description is given here. Wherein the computer readable storage medium is such as ROM, RAM, magnetic or optical disk.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising instructions for causing a terminal (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method according to the embodiments of the present invention.
The embodiments of the present invention have been described above with reference to the accompanying drawings, but the present invention is not limited to the above-described embodiments, which are merely illustrative and not restrictive, and many forms may be made by those having ordinary skill in the art without departing from the spirit of the present invention and the scope of the claims, which are to be protected by the present invention.
Claims (15)
1. An image processing method for a terminal device, the method comprising:
reporting the annotation data information of the annotated images to a server under the condition that the number of the annotated images is larger than a preset threshold value, wherein the server is used for generating a first annotation model based on the annotation data information, and the first annotation model is a model for predicting the annotation information of the images;
sending the image to be annotated to the server;
receiving a labeling result sent by the server, wherein the labeling result is a labeling information prediction result obtained by inputting the image to be labeled into the first labeling model by the server for labeling information prediction;
and labeling the image to be labeled based on the labeling result.
2. The method according to claim 1, wherein labeling the image to be labeled based on the labeling result comprises:
labeling the image to be labeled according to the labeling result to obtain a first labeling image;
and storing the first annotation model in a local storage space of the terminal equipment under the condition that the annotation information of the first annotation image meets the preset condition.
3. The method according to claim 1 or 2, wherein the labeling data information includes center point coordinate information of a labeling frame, first coordinate information, and second coordinate information, the first coordinate information and the second coordinate information being coordinate information of two diagonal positions on a diagonal of the labeling frame.
4. An image processing method for a server, the method comprising:
receiving marking data information sent by terminal equipment, wherein the marking data information is image marking data of marked images sent by the terminal equipment under the condition that the number of the marked images is larger than a preset threshold value;
generating a first annotation model based on the annotation data information, wherein the first annotation model is a model for carrying out annotation information prediction on the image;
receiving an image to be marked sent by the terminal equipment;
inputting the image to be marked into the first marking model for marking information prediction, and obtaining marking results of the image to be marked;
and sending the labeling result to the terminal equipment, wherein the labeling result is used for labeling the image to be labeled.
5. The method of claim 4, wherein the annotation data information comprises central point coordinate information, first coordinate information and second coordinate information of an annotation frame, the first coordinate information and the second coordinate information being coordinate information of two diagonal positions on a diagonal of the annotation frame;
The generating a first annotation model based on the annotation data information includes:
normalizing the central point coordinate information, the first coordinate information and the second coordinate information to obtain a first target value;
generating the annotation data information of the prediction annotation frame based on the annotation data information of the target annotation frame and the first target value;
calculating a first loss value based on the marking data information of the prediction marking frame and the marking data information of the marked image;
updating parameters of a second annotation model based on the first loss value, obtaining an updated second annotation model, and determining the converged second annotation model as a first annotation model under the condition that the updated second annotation model is converged, wherein the second annotation model is used for carrying out annotation information prediction on an image.
6. An image processing apparatus for a terminal device, the apparatus comprising:
the system comprises a reporting module, a server and a prediction module, wherein the reporting module is used for reporting the annotation data information of the annotated image to the server under the condition that the number of the annotated image is larger than a preset threshold value, and the server is used for generating a first annotation model based on the annotation data information, wherein the first annotation model is a model for carrying out annotation information prediction on the image;
The sending module is used for sending the image to be marked to the server;
the receiving module is used for receiving the labeling result sent by the server, wherein the labeling result is a labeling information prediction result obtained by the server by inputting the image to be labeled into the first labeling model for labeling information prediction;
and the labeling module is used for labeling the image to be labeled based on the labeling result.
7. The device according to claim 6, wherein the labeling module is specifically configured to:
labeling the image to be labeled according to the labeling result to obtain a first labeling image;
and storing the first annotation model in a local storage space of the terminal equipment under the condition that the annotation information of the first annotation image meets the preset condition.
8. The apparatus according to claim 6 or 7, wherein the labeling data information includes center point coordinate information of a labeling frame, first coordinate information, and second coordinate information, the first coordinate information and the second coordinate information being coordinate information of two diagonal positions on a diagonal line of the labeling frame.
9. An image processing apparatus for a server, the apparatus comprising:
The first receiving module is used for receiving marking data information sent by the terminal equipment, wherein the marking data information is image marking data of marked images sent by the terminal equipment under the condition that the number of the marked images is larger than a preset threshold value;
the generation module is used for generating a first annotation model based on the annotation data information, wherein the first annotation model is a model for carrying out annotation information prediction on the image;
the second receiving module is used for receiving the image to be marked sent by the terminal equipment;
the prediction module is used for inputting the image to be marked into the first marking model to perform marking information prediction and obtaining a marking result of the image to be marked;
the sending module is used for sending the labeling result to the terminal equipment, and the labeling result is used for labeling the image to be labeled.
10. The apparatus of claim 9, wherein the annotation data information comprises central point coordinate information, first coordinate information and second coordinate information of an annotation frame, the first coordinate information and the second coordinate information being coordinate information of two diagonal positions on a diagonal of the annotation frame;
The generating module is specifically configured to:
normalizing the central point coordinate information, the first coordinate information and the second coordinate information to obtain a first target value;
generating the annotation data information of the prediction annotation frame based on the annotation data information of the target annotation frame and the first target value;
calculating a first loss value based on the marking data information of the prediction marking frame and the marking data information of the marked image;
updating parameters of a second annotation model based on the first loss value, obtaining an updated second annotation model, and determining the converged second annotation model as a first annotation model under the condition that the updated second annotation model is converged, wherein the second annotation model is used for carrying out annotation information prediction on an image.
11. A terminal device comprising a transceiver and a processor,
the transceiver is used for reporting the marked data information of the marked images to the server under the condition that the number of the marked images is larger than a preset threshold value, and the server is used for generating a first marked model based on the marked data information, wherein the first marked model is a model for carrying out marked information prediction on the images;
The transceiver is used for sending the image to be marked to the server;
the transceiver is used for receiving the labeling result sent by the server, wherein the labeling result is a labeling information prediction result obtained by the server by inputting the image to be labeled into the first labeling model for labeling information prediction;
and the processor is used for marking the image to be marked based on the marking result.
12. A server is characterized by comprising a transceiver and a processor,
the transceiver is used for receiving marking data information sent by the terminal equipment, wherein the marking data information is image marking data of marked images sent by the terminal equipment under the condition that the number of the marked images is larger than a preset threshold value;
the processor is used for generating a first annotation model based on the annotation data information, wherein the first annotation model is a model for carrying out annotation information prediction on the image;
the transceiver is used for receiving the image to be marked sent by the terminal equipment;
the processor is used for inputting the image to be marked into the first marking model for marking information prediction and obtaining marking results of the image to be marked;
The transceiver is used for sending the labeling result to the terminal equipment, and the labeling result is used for labeling the image to be labeled.
13. A terminal device, comprising: a processor, a memory and a program stored on the memory and executable on the processor, which when executed by the processor, implements the steps of the image processing method according to any one of claims 1 to 3.
14. A server, comprising: a processor, a memory and a program stored on the memory and executable on the processor, which when executed by the processor, implements the steps of the image processing method according to claim 4 or 5.
15. A computer-readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, implements the steps of the image processing method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311048069.7A CN116778488A (en) | 2023-08-21 | 2023-08-21 | Image processing method, device, terminal equipment, server and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311048069.7A CN116778488A (en) | 2023-08-21 | 2023-08-21 | Image processing method, device, terminal equipment, server and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116778488A true CN116778488A (en) | 2023-09-19 |
Family
ID=87993402
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311048069.7A Pending CN116778488A (en) | 2023-08-21 | 2023-08-21 | Image processing method, device, terminal equipment, server and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116778488A (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110110811A (en) * | 2019-05-17 | 2019-08-09 | 北京字节跳动网络技术有限公司 | Method and apparatus for training pattern, the method and apparatus for predictive information |
| KR20200042629A (en) * | 2018-10-16 | 2020-04-24 | 주식회사 키센스 | Method for generating annotation and image based on touch of mobile device to learn artificial intelligence and apparatus therefor |
| CN116433997A (en) * | 2021-12-31 | 2023-07-14 | 深圳开立生物医疗科技股份有限公司 | Image labeling method, device and medium |
-
2023
- 2023-08-21 CN CN202311048069.7A patent/CN116778488A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20200042629A (en) * | 2018-10-16 | 2020-04-24 | 주식회사 키센스 | Method for generating annotation and image based on touch of mobile device to learn artificial intelligence and apparatus therefor |
| CN110110811A (en) * | 2019-05-17 | 2019-08-09 | 北京字节跳动网络技术有限公司 | Method and apparatus for training pattern, the method and apparatus for predictive information |
| CN116433997A (en) * | 2021-12-31 | 2023-07-14 | 深圳开立生物医疗科技股份有限公司 | Image labeling method, device and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109726763B (en) | Information asset identification method, device, equipment and medium | |
| CN109190763A (en) | Inquiry response equipment | |
| US20210012187A1 (en) | Adaptation of Deep Learning Models to Resource Constrained Edge Devices | |
| CN114945817A (en) | Task processing method, device and equipment based on defect detection and storage medium | |
| CN113138927B (en) | Software function test method and device | |
| CN110991871A (en) | Risk monitoring method, device, equipment and computer readable storage medium | |
| US8990058B2 (en) | Generating and evaluating expert networks | |
| CN113688986A (en) | Longitudinal federal prediction optimization method, device, medium, and computer program product | |
| CN114492601A (en) | Resource classification model training method and device, electronic equipment and storage medium | |
| CN111784401A (en) | Order taking rate prediction method, device, equipment and readable storage medium | |
| CN113709708B (en) | Data transmission method, device, equipment and storage medium | |
| CN113792892A (en) | Federal learning modeling optimization method, apparatus, readable storage medium, and program product | |
| CN113095508A (en) | Regression model construction optimization method, device, medium, and computer program product | |
| CN113869774A (en) | Method and device for predicting quality of building object, electronic device and storage medium | |
| CN109522282A (en) | Picture management method, device, computer installation and storage medium | |
| CN112860736A (en) | Big data query optimization method and device and readable storage medium | |
| CN112541556A (en) | Model construction optimization method, device, medium, and computer program product | |
| CN116778488A (en) | Image processing method, device, terminal equipment, server and storage medium | |
| CN117370162A (en) | Test tool management method, device, equipment and storage medium | |
| CN112749746A (en) | Method, system and device for iteratively updating defect sample | |
| US11893383B2 (en) | Configuration properties management for software | |
| US11775264B2 (en) | Efficient deployment of machine learning and deep learning model's pipeline for serving service level agreement | |
| CN108256753B (en) | Emergency material allocation method and device | |
| CN115437627A (en) | Method, device and equipment for developing learning object and storage medium | |
| CN101267337A (en) | Network management system and method of managing the same |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |