CN110969072A - Model optimization method and device and image analysis system - Google Patents

Model optimization method and device and image analysis system Download PDF

Info

Publication number
CN110969072A
CN110969072A CN201910556196.5A CN201910556196A CN110969072A CN 110969072 A CN110969072 A CN 110969072A CN 201910556196 A CN201910556196 A CN 201910556196A CN 110969072 A CN110969072 A CN 110969072A
Authority
CN
China
Prior art keywords
detection model
image
analysis result
user
model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910556196.5A
Other languages
Chinese (zh)
Other versions
CN110969072B (en
Inventor
谭晶晶
户军
许毅
张记伟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Hikvision System Technology Co Ltd
Original Assignee
Hangzhou Hikvision System Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Hikvision System Technology Co Ltd filed Critical Hangzhou Hikvision System Technology Co Ltd
Publication of CN110969072A publication Critical patent/CN110969072A/en
Application granted granted Critical
Publication of CN110969072B publication Critical patent/CN110969072B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Multimedia (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a model optimization method, model optimization equipment and an image analysis system, and belongs to the field of image monitoring. The method comprises the following steps: acquiring alternative images from a plurality of images acquired by first image acquisition equipment, selecting training samples from the alternative images, and training a detection model of the first user equipment based on the training samples, wherein the detection model of the first user equipment is used for detecting the images acquired by the first image acquisition equipment; and when the trained detection model is superior to the detection model, updating the detection model of the first user terminal equipment by adopting the trained detection model. The problem that the detection model obtained through training cannot be matched with the image analysis scene corresponding to the user side equipment is solved, and the reliability of the optimized detection model is improved.

Description

Model optimization method and device and image analysis system
The present application claims priority from chinese patent application No. 201811161087.5 entitled "model optimization method, apparatus, and image analysis system" filed on 30/9/2018, the entire contents of which are incorporated herein by reference.
Technical Field
The invention relates to the field of video monitoring, in particular to a model optimization method, model optimization equipment and an image analysis system.
Background
The image analysis system is widely applied to various fields such as security, transportation, scientific research and entertainment, and the image analysis system based on big data may include a Central Management Server (CMS), a plurality of user equipments, each of which manages a plurality of image capturing equipments in an image analysis scene. The image acquisition equipment can send the acquired image to the corresponding user side equipment, and the user side equipment automatically analyzes the key information for the image monitoring personnel to process.
In the related art, the CMS establishes a detection model according to a pre-established image library, and sends the detection model to each customer premise equipment, and each customer premise equipment analyzes an image acquired by an image acquisition device managed by the customer premise equipment based on the detection model. The image library may be continuously updated based on images provided by the plurality of client devices. When a certain user terminal device puts forward a model optimization requirement, the CMS may obtain a plurality of training samples from the image library, then train the detection model based on the plurality of training samples, and update the detection model in the user terminal device using the trained detection model as an optimized model.
However, the detection model obtained by training in the related art cannot be adapted to the image analysis scene corresponding to the user end device, and therefore, the reliability of the optimized model is low.
Disclosure of Invention
The embodiment of the invention provides a model optimization method, a device and an image analysis system, which can solve the problem of low reliability of an optimized model in the related art. The technical scheme is as follows:
according to a first aspect of the present invention, there is provided a model optimization method, the method comprising:
acquiring an alternative image from a plurality of images acquired by first image acquisition equipment, wherein the first image acquisition equipment is image acquisition equipment managed by first user side equipment;
selecting a training sample from the alternative image, and adding the training sample to a training sample library corresponding to the first user equipment;
training a detection model of the first user equipment based on samples in the training sample library to obtain a trained detection model, wherein the detection model of the first user equipment is used for detecting images acquired by the first image acquisition equipment;
and when the trained detection model is superior to the detection model, updating the detection model of the first user equipment by adopting the trained detection model.
Optionally, the acquiring the alternative image from the plurality of images acquired by the first image acquisition device includes:
inputting the plurality of images into the detection model respectively;
receiving a first analysis result of each image output by the detection model;
receiving an auditing instruction input by a user at the first user terminal equipment, wherein the auditing instruction is used for indicating whether the first analysis result is accurate or not;
when the auditing instruction indicates that the first analysis result of any image is accurate, determining any image as a candidate image, and saving the first analysis result.
Optionally, the acquiring the alternative image from the plurality of images acquired by the first image acquisition device further includes:
when the auditing instruction indicates that the first analysis result is inaccurate, receiving a first analysis result updating instruction input by a user at the first user terminal equipment, wherein the first analysis result updating instruction is used for indicating an updated first analysis result,
determining an image containing an updated first analysis result as a candidate image, and storing the updated first analysis result;
or when the auditing instruction indicates that the first analysis result of any image is inaccurate, determining any image as a non-alternative image.
Optionally, before the updating the detection model of the first user equipment by using the trained detection model, the method further includes:
selecting a test image from the alternative images;
judging whether the trained detection model is superior to the detection model or not based on the test image;
the updating the detection model of the first user equipment by using the trained detection model includes:
and when the trained detection model is superior to the detection model, updating the detection model of the first user equipment by adopting the trained detection model.
Optionally, the determining whether the trained detection model is better than the detection model based on the test image includes:
inputting the test image into the trained detection model to obtain a second analysis result;
detecting whether the accuracy of the trained detection model meets a specified accuracy condition or not based on a second analysis result of the test image and a first analysis result which is stored in advance;
and when the accuracy of the trained detection model meets the specified accuracy condition, judging that the trained detection model is superior to the detection model.
Optionally, there are a plurality of test images, and the detecting whether the accuracy of the trained detection model meets a specified accuracy condition based on the second analysis result of the test image and the first analysis result stored in advance includes:
counting a first number of test images of which the first analysis results are the same as the corresponding second analysis results;
when the ratio of the first number to the total number of the test images is larger than a specified proportion threshold value, determining that the accuracy of the trained detection model meets a specified accuracy condition;
and when the ratio of the first number to the total number of the test images is not greater than a specified proportion threshold value, determining that the accuracy of the trained detection model does not meet a specified accuracy condition.
Optionally, the selecting a training sample in the candidate image includes:
selecting images with a specified proportion from the alternative images as training samples;
the selecting a test image from the candidate images comprises:
and taking the images except the training sample in the alternative images as test images.
Optionally, before the training the detection model of the first user equipment based on the samples in the training sample library, the method further includes:
backing up a detection model of the first user terminal equipment;
and when the backup detection model meets the deletion condition, deleting the backup detection model.
Optionally, the method is performed by the first user equipment, and the training sample library is stored locally in the first user equipment.
According to a second aspect of the present invention, there is provided a model optimization apparatus, characterized in that the apparatus comprises:
the system comprises an acquisition module, a processing module and a display module, wherein the acquisition module is used for acquiring alternative images from a plurality of images acquired by first image acquisition equipment, and the first image acquisition equipment is image acquisition equipment managed by first user equipment;
the adding module is used for selecting a training sample from the alternative image and adding the training sample to a training sample library corresponding to the first user terminal device;
a training module, configured to train a detection model of the first user equipment based on a sample in the training sample library to obtain a trained detection model, where the detection model of the first user equipment is used to detect an image acquired by the first image acquisition device;
and the updating module is used for updating the detection model of the first user terminal equipment by adopting the trained detection model when the trained detection model is superior to the detection model.
Optionally, the obtaining module is configured to:
inputting the plurality of images into the detection model respectively;
receiving a first analysis result of each image output by the detection model;
receiving an auditing instruction input by a user at the first user terminal equipment, wherein the auditing instruction is used for indicating whether the first analysis result is accurate or not;
when the auditing instruction indicates that the first analysis result of any image is accurate, determining any image as a candidate image, and saving the first analysis result.
Optionally, the obtaining module is further configured to:
when the auditing instruction indicates that the first analysis result is inaccurate, receiving a first analysis result updating instruction input by a user at the first user terminal equipment, wherein the first analysis result updating instruction is used for indicating an updated first analysis result,
determining an image containing an updated first analysis result as a candidate image, and storing the updated first analysis result;
or when the auditing instruction indicates that the first analysis result of any image is inaccurate, determining any image as a non-alternative image.
Optionally, the apparatus further comprises:
the selection module is used for selecting a test image from the alternative images;
the judging module is used for judging whether the trained detection model is superior to the detection model or not based on the test image;
and the updating module is used for updating the detection model of the first user terminal equipment by adopting the trained detection model when the trained detection model is superior to the detection model.
Optionally, the determining module includes:
the input submodule is used for inputting the test image into the trained detection model to obtain a second analysis result;
the detection submodule is used for detecting whether the accuracy of the trained detection model meets the specified accuracy condition or not based on the second analysis result of the test image and the first analysis result which is stored in advance;
and the judging submodule is used for judging that the trained detection model is superior to the detection model when the accuracy of the trained detection model meets the specified accuracy condition.
Optionally, there are a plurality of test images, and the detection sub-module is configured to:
counting a first number of test images of which the first analysis results are the same as the corresponding second analysis results;
when the ratio of the first number to the total number of the test images is larger than a specified proportion threshold value, determining that the accuracy of the trained detection model meets a specified accuracy condition;
and when the ratio of the first number to the total number of the test images is not greater than a specified proportion threshold value, determining that the accuracy of the trained detection model does not meet a specified accuracy condition.
Optionally, the apparatus further comprises:
and the backup module is used for backing up the detection model of the first user terminal equipment.
Optionally, the apparatus further comprises:
and the deleting module is used for deleting the backup detection model when the backup detection model meets the deleting condition.
Optionally, the model optimization apparatus is the first user equipment, and the training sample library is stored locally in the first user equipment.
According to a third aspect of the present invention, there is provided an image analysis system comprising the model optimization apparatus of the second aspect;
the image analysis system further comprises at least one user end device, and each user end device manages at least one image acquisition device.
According to a fourth aspect of the invention, there is provided a computer device, comprising a processor and a memory,
wherein, the memory is used for storing computer programs;
the processor is configured to execute the program stored in the memory, and implement the model optimization method according to the first aspect.
According to a fifth aspect of the present invention, there is provided a storage medium having stored therein a computer program which, when executed by a processor, implements the model optimization method of the first aspect.
The technical scheme provided by the embodiment of the invention has the following beneficial effects:
the detection model of the first user end equipment is trained by selecting the training sample from the alternative images to obtain the trained detection model, and the alternative images are images acquired from a plurality of images acquired by the first image acquisition equipment, and the first image acquisition equipment acquires images for a certain image analysis scene where the first user end equipment is located, so that the trained detection model is the detection model for the first user end equipment and can be adapted to the certain image analysis scene where the first user end equipment is located, the problem that the detection model obtained by training in the related technology cannot be adapted to the image analysis scene corresponding to the user end equipment is solved, and the reliability of the optimized detection model is improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a schematic diagram of an image analysis system provided by an embodiment of the present invention;
FIG. 2 is a flow chart of a model optimization method provided by an embodiment of the invention;
FIG. 3 is a flow chart of another model optimization method provided by embodiments of the present invention;
fig. 4 is a flowchart of a method for training a detection model of a first user equipment based on samples in a training sample library to obtain a trained detection model according to an embodiment of the present invention;
FIG. 5 is a flowchart of a method for determining whether a trained test model is better than a test model according to an embodiment of the present invention;
fig. 6 is a flowchart of a method for determining whether a backup detection model meets a deletion condition according to an embodiment of the present invention;
FIG. 7 is a flowchart of a method for optimizing a model according to another embodiment of the present invention;
FIG. 8 is a block diagram of a model optimization apparatus according to an embodiment of the present invention;
FIG. 9 is a block diagram of another model optimization apparatus provided in an embodiment of the present invention;
FIG. 10 is a block diagram of a determination module provided by an embodiment of the invention;
FIG. 11 is a block diagram of another model optimization apparatus provided in an embodiment of the present invention;
FIG. 12 is a block diagram of another model optimization apparatus provided in an embodiment of the present invention;
FIG. 13 is a block diagram of another model optimization apparatus provided in an embodiment of the present invention;
fig. 14 is a block diagram of a client device according to an embodiment of the present invention.
With the above figures, certain embodiments of the invention have been illustrated and described in more detail below. The drawings and the description are not intended to limit the scope of the inventive concept in any way, but rather to illustrate it by those skilled in the art with reference to specific embodiments.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the following will describe embodiments of the present invention in further detail with reference to the accompanying drawings, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 illustrates an image analysis system 10 according to the model optimization method provided by the embodiment of the present invention, where the image analysis system 10 may include at least one client device 12 and at least one image acquisition device 13, and the image analysis system 10 may be a big data-based image analysis system. Wherein each customer premises device 12 manages at least one image acquisition device 13. Fig. 1 illustrates an example of the image analysis system 10 including two client devices 12 and two image capturing devices 13 managed by each client device 12, but the present invention is not limited thereto.
The image pickup apparatus 13 may be an image pickup apparatus or a photographing apparatus or the like capable of image pickup, and may be, for example, a two-sided camera or a wide-angle camera or the like. Optionally, the image capturing device may be a network Camera (Internet Protocol Camera, IPC for short) such as a gunlock or a dome Camera, and the network Camera may transmit the captured video to a client device connected to the network through the network. In the embodiment of the present invention, the image capturing device 13 may be installed in a scene where an image analysis system performs image analysis, for example, in an image analysis system applied to an urban management environment, an object to be subjected to image analysis by the image analysis system may be an event such as a vendor or a store in a city, and the purpose of analysis is to determine an illegal event, so the image capturing device may be installed in an area where urban management is patrolled. The customer premises device 12 may comprise a terminal operated by a user, and in an alternative embodiment, the customer premises device 12 may further comprise a server connected to the terminal operated by the user.
Optionally, the image analysis system 10 may further include a central management server 14, where the central management server 14 may be a server, a server cluster composed of a plurality of servers, or a cloud computing service center, and the central management server 14 may provide and optimize a service required by the user end device, for example, in the image analysis system provided in the embodiment of the present invention, the central management server 14 may establish an initial detection model for the user end device according to a scene to be subjected to image analysis by the image analysis system 10 and a required analysis result, and optionally, the central management server 14 may further optimize the initial detection model. In other alternative embodiments, the central management server may also be regarded as a laboratory device for providing services, and the embodiments of the present invention are not limited herein.
In the image analysis system 10, the user end device 12 is installed with a designated client, and the central management server 14 can provide services for the user end device through the designated client.
Optionally, the image analysis system 10 may further include a supervision platform 15, the supervision platform 15 may include a terminal for the supervisor to operate, and in an alternative embodiment, the supervision platform 15 may further include a server connected to the terminal for the supervisor. The monitoring platform 15 is used for monitoring the scene of image analysis according to the analysis result obtained by the monitoring platform 15, and compared with the user end device 12, the monitoring platform 15 is a user device with higher authority, and a monitoring person can further process the analysis result according to the monitoring platform 15.
Currently, image analysis systems can be applied to different application environments (i.e., uses), and for each application environment, there are a variety of image analysis scenarios. Generally, in an image analysis system applied to an application environment, a client device may be configured for each image analysis scene, and the client device may manage one or more image capturing devices to perform image analysis on the image analysis scene. Of course, multiple client devices may be configured for each image analysis scenario. The embodiment of the present invention is described by taking an example that each image analysis scene may correspond to one user end device. When each image analysis scene corresponds to a client device, the actions performed by each client device refer to the following embodiments.
The image analysis system is applied to an urban management environment, and can divide the image analysis scene in an urban division manner. For example, the china includes multiple cities such as Chongqing city, Hangzhou city, Tianjin city, etc., the city management environment of the traffic image analysis system may include an image analysis scene of Chongqing city, an image analysis scene of Hangzhou city, an image analysis scene of Tianjin city, etc.
In other alternative embodiments, the image analysis system may be applied to a traffic environment, which may be a traffic image analysis system, and the image analysis scene may be divided in a manner of an internal partition of a city, for example, a number of partitions of the city of western security including an image analysis scene of an anser tower area, a tombstone area, and a lotus lake area, and the traffic environment of the traffic image analysis system may include an image analysis scene of an anser tower area, an image analysis scene of a tombstone area, an image analysis scene of a lotus lake area, and the like. Of course, the image analysis system described in the embodiment of the present invention may also be applied in other application environments, and the embodiment of the present invention is not listed here.
At present, in an image analysis system corresponding to each application environment, a feature parameter to be analyzed needs to be set, and a detection model is established based on the feature parameter, so that after an image is input into the detection model, the detection model outputs an analysis result matched with the feature parameter.
For example, the image analysis system is an urban management image analysis system for analyzing an illegal event in a city, and the object to be subjected to image analysis by the image analysis system is an event such as a vendor or a store in the city. When a detection model is established, feature parameters for representing multiple violation event types need to be set, and then multiple images of the multiple violation event types are used as a training sample set to train the detection model, so that an analysis result of the detection model includes: the type of violation event in the image, and/or the location where the violation event occurred.
For another example, the image analysis system is a traffic image analysis system for analyzing a violation vehicle, and the object to be subjected to image analysis by the image analysis system is a vehicle traveling on a road. When the detection model is established, characteristic parameters for representing violation vehicles of various violation types need to be set, then a plurality of images of the violation vehicles of various violation types are used as a training sample set to train the detection model, and then the analysis result of the detection model comprises the following steps: the type of violation of the violation vehicle in the image, and/or the location of the violation vehicle.
However, in each existing image analysis system, each detection model provided by the CMS is for all image analysis scenarios in an application environment, that is, for the application environment, no matter how many image analysis scenarios the application environment includes, the training sample used by the CMS to optimize the detection model is selected from a fixed image library, and the finally optimized detection model is applied to each analysis scenario in the application environment, and cannot be applied to the image analysis scenario corresponding to the ue.
Fig. 2 is a flow chart of a model optimization method according to an embodiment of the present invention, which is used in the image analysis system 10 shown in fig. 1, and the model optimization method may include the following steps:
step 201, acquiring a candidate image from a plurality of images acquired by a first image acquisition device.
The first image acquisition device is an image acquisition device managed by the first user side device. For example, the first client device is one of the at least one client device in the image analysis system 10.
Step 202, selecting a training sample from the candidate image, and adding the training sample to a training sample library corresponding to the first user equipment.
And 203, training the detection model of the first user equipment based on the samples in the training sample library to obtain the trained detection model.
The detection model of the first user terminal device is used for detecting the image acquired by the first image acquisition device.
And 204, when the trained detection model is superior to the detection model, updating the detection model of the first user equipment by using the trained detection model.
In summary, in the model optimization method provided in the embodiment of the present invention, the training sample is selected from the candidate images to train the detection model of the first user equipment, so as to obtain the trained detection model, and since the candidate images are images obtained from a plurality of images acquired by the first image acquisition device, and the first image acquisition device performs image acquisition on a certain image analysis scene where the first user equipment is located, the trained detection model can be adapted to the certain image analysis scene, a problem that the detection model trained in the related art cannot be adapted to the image analysis scene corresponding to the user equipment is solved, and reliability of the optimized detection model is improved.
The model optimization method described above is further described below in an alternative implementation. If the model optimization method provided by the embodiment of the invention is adopted to optimize the detection model in the image analysis system of the national city management in China, taking the example that the urban management environment of the country includes three image analysis scenes, namely, an image analysis scene of Chongqing city, an image analysis scene of Hangzhou city and an image analysis scene of Tianjin city, assuming that the three image analysis scenes correspond to three user end devices one by one, namely, a first user end device, a second user end device and a third user end device, wherein the first user end device can correspondingly manage a first image acquisition device in the image analysis scene of Chongqing city, the second user end device can correspondingly manage a second image acquisition device in the image analysis scene of Hangzhou city, and the third user end device can correspondingly manage a third image acquisition device in the image analysis scene of Tianjin city. Taking a Chongqing city image analysis scene as an example, the multiple images acquired by the first image acquisition device are all multiple images in the Chongqing city image analysis scene, the candidate images are acquired from the multiple images, and then the training sample is selected to train the detection model corresponding to the Chongqing city image analysis scene, the trained detection model can be used for the Chongqing city image analysis scene, if the first image acquisition device continues to input the acquired images into the detection model, the analysis result output by the detection model is more suitable for the Chongqing city image analysis scene, so that the detection model is more pertinent and higher in reliability, and the detection model for the Hangzhou city image analysis scene and the detection model for the Tianjin city image analysis scene can be obtained by analogy in sequence.
It should be noted that, the model optimization method may be executed by any one of the at least one customer premise equipment, or executed by the central management server, and the detection model may be an initial detection model sent by the central management server.
In a first optional implementation manner, if the model optimization method is executed by a first user equipment, where the first user equipment is any user equipment in at least one user equipment of a model optimization system, refer to fig. 3, where fig. 3 shows a flowchart of another model optimization method provided in an embodiment of the present invention, where the model optimization method may include multiple rounds of model optimization processes that are sequentially executed, and in the embodiment of the present invention, a round of model optimization process is taken as an example for description, in an actual implementation of the embodiment of the present invention, each round of model optimization process may refer to the round of model optimization process, which is not described again in the embodiment of the present invention, and the model optimization method includes:
step 301, the first user equipment inputs the plurality of images acquired by the first image acquisition device into the detection model of the first user equipment respectively.
The first image capturing device may be an image capturing device managed by the first user equipment. The first image capturing device may capture images in a scene where the first image capturing device is located at a fixed frequency and transmit the images to the first user end device in real time, or transmit the number of images to the first user end device when the number of images in the scene where the first image capturing device is located at the fixed frequency reaches a fixed threshold, which is not limited in the embodiment of the present invention. The detection model of the first user terminal device is used for detecting the image acquired by the first image acquisition device.
The first image acquisition device and the first user terminal device can be connected through a network, the first image acquisition device can be IPC, and the IPC can transmit the acquired video to the first user terminal device through the network. In practical implementation, the first user equipment may extract a single-frame image in the video, and input a plurality of single-frame images into the detection model respectively. Optionally, the first client device may extract a single frame image in the video stream according to a predetermined frequency (for example, capturing one frame image every 5 minutes), and the embodiment of the present invention is not limited herein.
When the one-round model optimization process provided by the embodiment of the invention is the first-round model optimization process, the detection model is the initial detection model sent by the central management server. The initial detection model may be established by the central management server and provided to the first client device.
Optionally, in order to ensure that the memory usage amount of the initial detection model is small, and the installation is convenient, the central management server may generate the initial detection model by using a small amount of training samples, where the training samples may be images acquired by any image acquisition device or images acquired from the internet, or images acquired by other methods. Optionally, in order to ensure the effectiveness of the established initial detection model, a part of image with obvious features may be manually selected from images acquired by any image acquisition device, and then a detection algorithm is adopted to generate the initial detection model. Since in machine vision learning, the detection algorithm typically refers to a target detection algorithm (e.g., a target detection algorithm that classifies based on sliding windows), accordingly, the detection model typically refers to a target detection model. Therefore, the initial detection model may be established based on a target detection algorithm, which may be multiple, for example, a Single-Shot multi-box Detector (SSD) target detection algorithm or a one-eye target detection algorithm (You Only Look one), and the embodiments of the present invention are not limited herein.
It should be noted that, because the accuracy of the analysis result output by the detection model by the image analysis system is generally required to be higher, the detection model in the embodiment of the present invention may be established based on a supervised learning method.
In addition, when the one-round model optimization process provided by the embodiment of the present invention is the n-th round model optimization process, and n is an integer greater than 1, the detection model is obtained by updating the initial detection model at least once.
Step 302, the first user equipment receives a first analysis result of each image output by the detection model.
After the first user equipment inputs the multiple images acquired by the first image acquisition equipment into the detection model respectively, the first user equipment receives a first analysis result of each image output by the detection model respectively, so that the analysis result is acquired.
Step 303, the first user equipment receives an audit instruction input by the user at the first user equipment, where the audit instruction is used to indicate whether the first analysis result is accurate.
The first user equipment can be a terminal operated by a user, a tablet computer, a smart phone or other equipment with a display screen, and the first user equipment can present a first analysis result of each image for the user to check. For example, a first analysis result output by the detection model is presented through a User Interface (UI).
The first analysis result is an analysis result obtained by analyzing each image by using the detection model in the optimization process of the model. For example, the first analysis result may include a type of a target object of the image and a region of the target object, and optionally, the type of the target object may be marked in the form of text near the region of the target object.
Of course, for each image, in addition to presenting the first analysis result output by the detection model, information such as time and area for acquiring the image may also be presented on each image, and the embodiment of the present invention is not limited herein.
In other optional implementation manners, when the first user equipment presents the first analysis result output by the detection model, prompt information may be sent to prompt the user to check the first analysis result, and the prompt information may be sent in a form of text, sound, and/or light, which is not limited herein.
After the first user equipment presents the first analysis result output by the detection model through the UI, the user of the first user equipment may audit the first analysis result by combining with own experience, and operate the first user equipment to trigger the first user equipment to generate an audit instruction for the first analysis result.
Optionally, the UI of the first user-side device may sequentially present a plurality of images, so that the user may sequentially audit the first analysis result of the plurality of images, and the first user-side device may sequentially receive a plurality of audit instructions. Certainly, the UI of the first user equipment may also present multiple images at one time for the user to audit the first analysis result of the multiple images, and after the first user equipment receives the audit instruction of the multiple images, present multiple new images on the UI for the user to audit.
Optionally, the first user equipment may receive the audit instruction in a variety of ways, for example, a corresponding button is set in the UI, and the user triggers the button to send the audit instruction for the first analysis result to the first user equipment. The button may include an accurate button indicating that the first analysis result is accurate and/or an inaccurate button indicating that the first analysis result is inaccurate. Alternatively, in the UI, a corresponding button may be provided for each image.
And step 304, when the auditing instruction indicates that the first analysis result of any image is accurate, the first user terminal equipment determines any image as a candidate image and stores the first analysis result.
After the first user-side device receives an audit instruction indicating that the first analysis result of any image is accurate, the first user-side device may determine the any image as a candidate image, and store the first analysis result. The saved first analysis result is an analysis result determined by the user, and the analysis result can be used for training the detection model of the first user equipment and verifying the reliability of the trained detection model. For example, when a certain candidate image is used as a training sample, the content recorded in the first analysis result thereof may be used as an annotation result, i.e. a label, of the training sample.
Step 305, when the audit instruction indicates that the first analysis result is inaccurate, the first user equipment receives a first analysis result update instruction input by a user at the first user equipment, where the first analysis result update instruction is used to indicate an updated first analysis result.
When the user determines that the first analysis result is inaccurate through auditing, the user may continue to trigger the first analysis result update instruction after triggering the audit instruction indicating that the first analysis result is inaccurate, and correspondingly, the first user equipment receives the audit instruction and the first analysis result update instruction in sequence. The manner in which the user triggers the first analysis result update instruction may be various, and two manners are described as examples below:
in a first manner, a user may directly update the first analysis result, and a UI presented by the first user-side device may be provided with a function for the user to select an area in the image, and then the updating manner may include: and re-checking the area in the image, and/or updating the text description of the area, wherein when the user re-checks the area in the image and/or revises the text description of the area, the first user end equipment can receive a first analysis result updating instruction.
In a second manner, a button marked with "update first analysis result" is set in the UI presented by the first user end device, and when the user triggers the button, the first user end device presents an update interface for the user to input a new analysis result on the update interface, and correspondingly, the first user end device receives a first analysis result update instruction input by the user on the first user end device.
In another optional embodiment, when the user determines that the first analysis result is inaccurate through the audit, the generation of the audit instruction and the first analysis result update instruction may be triggered through the same operation, that is, the audit instruction and the first analysis result update instruction are the same instruction. For other alternative implementations, the embodiments of the present invention are not limited herein.
Step 306, the first user equipment determines the image containing the updated first analysis result as the alternative image, and stores the updated first analysis result.
Optionally, after the first user-side device receives the first analysis result update instruction, an option for prompting the user whether to complete updating of the first analysis result may be presented in the UI, and after the user triggers the option, the first user-side device may determine an image containing the updated first analysis result as the alternative image, and store the updated first analysis result.
And 307, when the auditing instruction indicates that the first analysis result of any image is inaccurate, the first user end equipment determines any image as a non-alternative image.
In another optional embodiment, when the audit instruction indicates that the first analysis result of any image is inaccurate, the user may not update the first analysis result, but determine any image as a non-candidate image, which is not used in subsequent operations. That is, the steps 305 and 307 may be alternatively performed.
The above steps 304 to 307 describe how to determine the candidate images, and it can be seen from the above steps that the candidate images are all images that are checked by the user of the first user side, and since the process of determining the candidate images is participated by the user of the first user side, the user is more familiar with the image analysis scene due to long-term contact with the image in the image analysis scene corresponding to the first user side device, so that the result of checking is more accurate, and therefore, the accuracy of the candidate images is guaranteed.
Further, after the step 304 and the step 306, if the image analysis system includes a supervision platform, the first user equipment may send the alternative image to the supervision platform (refer to fig. 1) for a supervisor to further process the alternative image, for example, for an urban management image analysis system, the supervision platform may be a reporting case platform, and after the alternative image is sent to the reporting case platform, the supervisor may process the type of the violation event and the reporting case platform generates a case event.
For example, assume that the detection model is applied to an image analysis system of a city management environment. The image containing the first analysis result is presented in the UI, the first analysis result includes an area where the violation event is located and a type of the violation event, and a user (e.g., city management) of the first user-side device can review the first analysis result.
When the user considers that the first analysis result is accurate, a button of 'finishing auditing' in the UI can be clicked, the first user terminal equipment receives an auditing instruction corresponding to the button, the auditing instruction can indicate that the first analysis result presented in the UI is accurate, and the first user terminal equipment can determine the image as a candidate image and store the first analysis result; when the user considers that at least one of the area where the violation event in the first analysis result is located and the type of the violation event is inaccurate, the first analysis result may be updated, and the updating process may include: when the area where the violation event is located is not accurate, the user can re-check the area where the violation event is located in the image, when the type of the violation event is not accurate, the user can re-select the violation type in the UI, when the user re-checks the area where the violation event is located or re-selects the type of the violation event, the auditing instruction and the first analysis result updating instruction in the above steps are simultaneously triggered, when the user considers that the updated first analysis result is accurate, the user can click an "auditing completion" button of the UI, the first user end device determines the image containing the updated first analysis result as a candidate image, and stores the updated first analysis result, when the area and the type where the violation event is located are not accurate, the two updating processes can be referred to, and details are not repeated herein.
Optionally, when the user thinks that the first analysis result is accurate, or after the user updates the first analysis result, a "report case" button below the UI may be triggered, and the alternative image is reported to the monitoring platform.
Besides, in the UI, the number (e.g., Camera 01) of the image capturing device that captures the image, time information (e.g., alarm time 10:00) for capturing the image, handling opinions (e.g., confirming violation, non-violation and incomplete filing), remark information, and the like may be presented in the image, and the embodiment of the present invention is not limited herein.
And 308, the first user end equipment selects a training sample from the alternative image and adds the training sample to a training sample library corresponding to the first user end equipment.
When the first user-side device selects the test image from the candidate images, images with a specified proportion may be selected as training samples, and of course, a specified number of images may also be selected as training samples from the candidate images, which is not limited in the embodiment of the present invention.
The user of the first user end equipment verifies the first analysis result output by the detection model according to the related experience of the user, and updates the first analysis result which is considered to be inaccurate by the user, so that the accuracy of the alternative image is ensured, and the accuracy of the training sample is further ensured.
For example, for an urban management image analysis system, the urban management image analysis system divides an image analysis scene in a city division manner, and since urban management of each city has different criteria for violation: for the same behavior, the city management of some cities may consider the behavior not to be illegal, and the city management of some cities may consider the behavior to be illegal; for the same violation, some urban city managers may consider the violation to belong to the outside-of-store operation, and some urban city managers may consider the violation to belong to the outside-of-store operation. If a uniform detection model is established for the image analysis system of the city management by adopting the CMS in the related technology, the detection model cannot effectively analyze scenes for each city image, but in the model optimization method provided by the embodiment of the invention, the city management of the fixed city is taken as a user, and the images acquired by the city are checked based on the rule-breaking judgment standard of the user, so that the trained detection model is more consistent with the cognition and habit of the city management of the city, and the pertinence of the trained detection model is improved. For example, the city manager in Chongqing city may review the images collected in the image analysis scene in Chongqing city based on the criterion of violation in Chongqing city, the city manager in Hangzhou city may review the images collected in the image analysis scene in Hangzhou city based on the criterion of violation in Hangzhou city, and the city manager in Tianjin city may review the images collected in the image analysis scene in Tianjin city based on the criterion of violation in Tianjin city, so that each detection model may be trained for each image analysis scene respectively.
Optionally, the training sample library may be a local sample library of the first user equipment, so that image resource leakage is avoided, and image confidentiality is achieved. Especially, when the image acquired by the first image acquisition device is a security image or a security image such as an image of a security place, the first user terminal device can perform model training based on a local training sample library, and other external devices are not required to participate, so that the safety of the image can be effectively ensured. And the local sample library can be updated in real time along with the images acquired by the first image acquisition equipment managed by the first user terminal equipment, so that the timeliness is high, and the updated detection model can be ensured to be more adaptive to the specific image analysis scene of the continuously changed first user terminal equipment.
Step 309, the first user equipment backs up the detection model of the first user equipment.
In the model optimization method provided in the embodiment of the present invention, even if the subsequently trained detection model is optimized compared to the previous detection model, the situation that the use requirement of the user of the first user equipment in the actual operation cannot be met may occur. Therefore, to ensure that the detection model can be rolled back in this case, the first ue may backup the detection model of the first ue (i.e. the detection model for which the training process has not been currently performed).
Further, the first user equipment may also back up each trained detection model to provide a reference for a subsequent model optimization process.
Because the backup detection model is in an idle state, the backup detection model can be encrypted in order to avoid that the backup detection model leaks out to cause the safety of the image analysis system to be damaged.
And 310, training the detection model of the first user equipment by the first user equipment based on the samples in the training sample library to obtain the trained detection model.
As shown in fig. 4, step 310 may include the steps of:
step 3101, when the number of samples in the training sample library is greater than the threshold value of the number of designated samples, the first user equipment performs at least one training process on the detection model until the detection model meets the training stop condition.
The first user equipment may perform at least one training process on the detection model of the first user equipment actively by a user, or the first user equipment may automatically perform at least one training process on the detection model when a specified condition is satisfied.
Optionally, the specified condition may be that the number of samples in the training sample library is greater than a specified sample number threshold, and then, when the number of samples in the training sample library is greater than the specified sample number threshold, the first user equipment automatically performs at least one training process on the detection model until the detection model meets the training stop condition.
By setting the specified conditions, the efficiency of the model optimization method can be improved, and the system performance of the first user equipment is prevented from being consumed in a transition manner.
In order not to affect the normal use of the first user equipment by the user, when the first user equipment meets the specified condition, the first user equipment may detect whether the first user equipment is in an idle state at the moment, and when the first user equipment is in the idle state, the first user equipment automatically performs at least one training process on the detection model.
Optionally, the training stop condition may include: the training times reach a specified training time threshold and/or the training error converges within a specified range. There are various ways to determine whether the training error converges within the specified range, for example, calculating the training error through a loss function, and then minimizing the training error through a gradient descent algorithm. Of course, the training stopping condition may also include other conditions, and the embodiment of the present invention is not limited herein.
Step 3102, the first user equipment obtains the trained detection model and empties the training sample library.
Because the memory space of the first user equipment is limited, the first user equipment can empty the training sample library after acquiring the trained detection model. When a new training sample is added to the training sample library, the number of samples in the training sample library is re-counted to re-perform step 3101 described above.
Step 311, the first user equipment selects a test image from the candidate images.
The test images may be used to determine whether the test model trained by the training samples is superior to the test model. The first user end equipment can select a training sample from the alternative images, and can also select a test image from the alternative images, wherein the image as the training sample and the image as the test image can be the same image, but if the image as the training sample is reused as the test image, the test image cannot accurately test the reliability of the detection model because the detection model has adaptability to the image.
Therefore, in order to ensure the accuracy of the judgment result, the test image is different from the training sample. The first user-side device may treat the images other than the training sample in the alternative images as the test images.
Step 312, the first user equipment determines whether the trained detection model is better than the detection model based on the test image.
Optionally, as shown in fig. 5, the step of the first user equipment determining whether the trained detection model is better than the detection model (i.e. the detection model that does not perform the training process) may include:
and 3121, the first user terminal device inputs the test image into the trained detection model to obtain a second analysis result.
Similar to the first analysis result, the second analysis result is an analysis result obtained by analyzing each test by using the trained detection model in the model optimization process. For example, the second analysis result may include the type of the target object of the image and the region of the target object.
And 3122, the first user end device detects whether the accuracy of the trained detection model meets the specified accuracy condition based on the second analysis result of the test image and the first analysis result stored in advance.
Referring to step 304, the pre-saved first analysis result is an analysis result that is audited by the user, and the second analysis result based on the test image is an analysis result output by the trained detection model, and by comparing the two sets of analysis results, it can be detected whether the accuracy of the trained detection model meets the specified accuracy condition.
Optionally, step 3122 may include:
and step X1, the first user terminal device counts a first number of the test images of which the first analysis results are the same as the corresponding second analysis results.
Optionally, determining whether the first analysis result is the same as the corresponding second analysis result may include: and judging whether the similarity between the first analysis result and the corresponding second analysis result is greater than a preset threshold, and when the similarity is greater than the preset threshold, determining that the first analysis result is the same as the corresponding second analysis result.
The corresponding first analysis result and second analysis result refer to the analysis results for the same image, for example, for the same candidate image, the first analysis result of the candidate image is stored in step 304, and the second analysis result is obtained by inputting the candidate image into the trained detection model in step 3121, and the first analysis result corresponds to the second analysis result.
And step X2, when the ratio of the first number to the total number of the test images is greater than a specified proportion threshold value, the first user equipment determines that the accuracy of the trained detection model meets a specified accuracy condition.
The specified proportion threshold value can be preset by a user and can be adjusted at a later stage so as to meet the personalized requirements of the user.
And step X3, when the ratio of the first number to the total number of the test images is not greater than the specified proportion threshold, the first user equipment determines that the accuracy of the trained detection model does not meet the specified accuracy condition.
And step X4, when the accuracy of the trained detection model meets the specified accuracy condition, the first user equipment judges that the trained detection model is superior to the detection model.
It should be noted that, in the scenario described in the step X1 to the step X4, when there are more than one test image, when there are 1 test image, the process of detecting whether the accuracy of the trained detection model meets the specified accuracy condition may include: when the first analysis result of the test image is the same as the second analysis result, the first user terminal equipment determines that the accuracy of the trained detection model meets the specified accuracy condition; and when the first analysis result of the test image is different from the second analysis result, the first user end equipment determines that the accuracy of the trained detection model does not accord with the specified accuracy condition.
And 313, when the trained detection model is better than the detection model, the first user end equipment adopts the trained detection model to update the detection model of the first user end equipment.
It should be noted that, in the process of performing the model optimization method, various intermediate files may be generated, for example, an xml-format markup file (generated when the user updates the first analysis result) may be generated, and after the first user-side device updates the detection model of the first user-side device by using the trained detection model, the intermediate files may be deleted, so as to save the memory usage.
And step 314, when the backup detection model meets the deletion condition, the first user equipment deletes the backup detection model.
In order to ensure that the memory occupation of the image analysis system is as small as possible, in the embodiment of the present invention, when the detection model that is backed up meets the deletion condition, the detection side model that is backed up may be deleted, and optionally, the step of determining whether the detection model meets the deletion condition occurs in a new round of model optimization process, as shown in fig. 6, the step may include the following steps:
step 3141, the first user equipment obtains a plurality of new images collected by the first image collecting equipment.
Step 3142, the first user equipment inputs a plurality of new images into the trained detection model respectively.
The related process of step 3142 may refer to step 301, which is not described herein again in this embodiment of the present invention.
Step 3143, the first user equipment receives the third analysis result of each updated image output by the detection model.
The related process of step 3143 may refer to step 302, which is not described herein again in this embodiment of the present invention.
Similar to the first analysis result and the second analysis result, the third analysis result is an analysis result obtained by analyzing each test by using the detection model in the first user terminal in the model optimization process. For example, the third analysis result may include the type of the target object of the image and the region of the target object.
Step 3144, the first user equipment receives an audit instruction for the updated image, which is input by the user at the first user equipment, where the audit instruction is used to indicate whether the third analysis result is accurate.
The related process of step 3144 may refer to step 303, which is not described herein again in this embodiment of the present invention.
Step 3145, based on the auditing instructions corresponding to the new images, the first user equipment counts the proportion of the accurate third analysis results of the new images in all the third analysis results.
For example, if the number of audit instructions indicating that the third analysis result is accurate is 80, and the third analysis result has 100, the ratio is 8/10. Of course, the ratio of the inaccurate third analysis results of the multiple new images in all the third analysis results may also be counted, and the embodiment of the present invention is not limited herein.
Step 3146, when the ratio is greater than the specified ratio threshold, the first user equipment determines that the backup detection model meets the deletion condition, and deletes the backup detection model.
For example, if the specified ratio threshold is 1/2, it may be determined that the backup detection model meets the deletion condition and the backup detection model is deleted when the ratio of the accurate third analysis results of the plurality of new images to all the third analysis results is 8/10. Of course, in other possible embodiments, the detection model of the backup may not be deleted, but the detection model may be archived.
If the ratio is not greater than the specified ratio threshold, the first user-side device may determine that the backup detection model does not meet the deletion condition, and at this time, the backup detection model may be reinstalled in the first user-side device to replace the detection model of the first user-side device at this time.
It can be seen that, in the model optimization method provided in the embodiment of the present invention, a plurality of methods (refer to step 309 and step 314 above) for rolling back the version of the detection model are provided, so that the flexibility of the use of the image analysis system is ensured, and various use requirements of the user can be met.
It should be noted that, when a plurality of client devices are configured in each image analysis scene, the plurality of client devices are in the same image analysis scene, and therefore can share the same detection model, the model optimization method provided in steps 301 to 314 can be performed in one of the client devices, and then the detection model finally retained in the client device is substituted for the detection models in other client devices, so that the plurality of client devices can all analyze the image in the image analysis scene in a targeted manner.
In the related art, a CMS establishes a detection model according to a pre-established image library, sends the detection model to each customer premise equipment, and when a certain customer premise equipment puts forward a model optimization requirement, the CMS can acquire a plurality of training samples in the image library, then trains the detection model based on the training samples, and updates the detection model in the customer premise equipment by using the trained detection model as an optimized model.
Since each user terminal device performs image analysis by using the detection model established by the CMS, when the CMS establishes the detection model, in order to cater to the usage scenario of each user terminal device, a large number of images are required to be used as training samples, so as to increase the universality of the trained detection model.
Before the detection model is trained by using the training samples, the training samples need to be labeled according to the analysis result of the detection model, for example, the type of the target event in the image and the area of the target event are labeled. When a large number of images are used as training samples, a large amount of manpower is consumed in the process of labeling the images by the staff of the CMS.
In the model optimization method provided by the embodiment of the present invention, the user of the first user equipment performs the auditing on the first analysis result of the image, and since the auditing process of the first analysis result by the user is a step that the user usually performs to ensure the validity of the analysis result output by the detection model, the model optimization method provided by the embodiment of the present invention combines the auditing process in the model optimization process, which is equivalent to issuing the work of image labeling to each user, so as to implement the marginalization of model optimization. Especially in a big data scene, the labeling process of the CMS side can be effectively simplified, and the labor cost is obviously reduced.
Moreover, in the related art, for some specific image analysis systems, for example, for an image analysis system of an urban management system, since images for training are often only from some specified image analysis scenes, for example, images for training are mainly from image analysis scenes of Chongqing city and image analysis scenes of Hangzhou city, when images in a large number of image analysis scenes of Chongqing city and image analysis scenes of Hangzhou city are used as training samples, and a detection model trained based on the training samples is applied to image analysis scenes of various cities throughout the country, a situation of overfitting occurs for a certain image analysis scene, and reliability of the detection model is low.
The model optimization method provided by the embodiment of the invention selects the training sample in the alternative image to train the detection model of the first user terminal device to obtain the trained detection model, since the alternative image is an image acquired in a plurality of images acquired by the first image acquisition device, the first image acquisition device acquires images aiming at a certain image analysis scene where the first user terminal device is positioned, therefore, the detection model aiming at the first user terminal equipment can be trained, the trained detection model can be adapted to the certain image analysis scene, and the problems in the related art are solved, the training samples are obtained by the CMS in the image library in a random selection mode, so that the problem that the detection model obtained through final training cannot be adapted to the image analysis scene corresponding to the user side equipment is solved, and the reliability of the optimized detection model is improved.
In addition, in the related art, if the detection model established by the CMS is installed in a new client device, the new client device means that the images collected by the image collecting component managed by the client device are not added into the image library of the CMS. Because the new ue corresponds to the new image analysis scenario, if the detection model established by the CMS is used to analyze the image in the new image analysis scenario, the accuracy of the analysis result output by the detection model is low, and at this time, the CMS needs to add the image in the image analysis scenario in the image library and select the training sample based on the image to retrain the detection model. If the number of the new customer premise equipment is large, the frequency of the CMS optimization detection model is high; if the image in the image analysis scene cannot be sent to the CMS due to the secret, the accuracy of the analysis result output by the detection model for the image analysis scene is low.
In the model optimization method provided by the embodiment of the invention, the model optimization is performed on the user side equipment, so that the problem of high frequency of a CMS optimization detection model is avoided, and the problem of low accuracy of an analysis result caused by the fact that images under the image analysis scene cannot be added to a CMS image library is also avoided.
In a second optional implementation manner, the model optimization method is executed by a central management server, and assuming that the central management server performs model optimization on a first user equipment based on a requirement of the first user equipment, where the first user equipment is any user equipment in the image analysis system 10 shown in fig. 1, please refer to fig. 7, which shows a flowchart of another model optimization method provided in an embodiment of the present invention, where the method includes:
step 701, the central management server obtains an alternative image from a plurality of images acquired by the first image acquisition device.
The first image acquisition device is an image acquisition device managed by the first user side device.
The first image acquisition device may acquire an image analysis scene in which the first user-side device is located, and therefore, a plurality of images acquired by the first image acquisition device are all images in the image analysis scene, and the alternative images acquired from the plurality of images are all images in the image analysis scene. The central management server can establish a plurality of image libraries, each image library corresponds to an image analysis scene where the user end equipment is located, and is used for storing a plurality of images in the image analysis scene where the corresponding user end equipment is located. The central management server acquires the alternative images from the image library of the first user terminal equipment to form a training sample library.
Optionally, there may be multiple ways for the central management server to obtain the alternative image, and the embodiment of the present invention is described by taking the following two cases as examples:
first, as described in steps 301 to 307, a user of the first user equipment checks a first analysis result of the image through the first user equipment, and the first user equipment sends the candidate image to the central management server, taking the checked image as a candidate image.
And secondly, manually labeling the images in the image library provided by the first user terminal equipment to take the labeled images as alternative images.
In an optional embodiment, if the image analysis system includes a monitoring platform, since the staff of the monitoring platform can check the analysis result of the image output by the detection model, the staff can send the checked image as a candidate image to the central management server.
It should be noted that, if the image analysis scene where the first user equipment is located is a secret-related scene, and the image acquired by the first image acquisition equipment is a security image or a secret-related image such as an image of a secret-related place, the process of acquiring the alternative image by the central management server needs authorization by the first user equipment, so as to avoid secret leakage. In the first case, the candidate image is sent by the first user equipment, and in the second case, the image library is provided by the first user equipment. This ensures that the alternative image is an image that the first client device allows the central management server to obtain.
Step 702, the central management server selects a training sample from the alternative images, and adds the training sample to a training sample library corresponding to the first user equipment.
Because the alternative images are established based on the images acquired by the first image acquisition equipment, the trained detection models are all suitable for the model analysis scene where the first image acquisition equipment is located, so that the trained detection models are more targeted, the image features of the scene can be more accurately extracted, the feature parameters related to the analysis result can be more conveniently and accurately extracted, the trained detection models can be accurately matched with the scene corresponding to the first user terminal equipment, and the reliability of the optimized models is improved.
Step 703, the central management server trains the detection model of the first user equipment based on the samples in the training sample library to obtain the trained detection model.
The detection model of the first user terminal device is used for detecting the image acquired by the first image acquisition device.
In step 703, the central management server trains the detection model of the first user equipment based on the samples in the training sample library, and reference may be made to the related process in step 310 for a process of obtaining the trained detection model, which is not described in detail herein.
And 704, when the trained detection model is superior to the detection model, the central management server updates the detection model of the first user equipment by using the trained detection model.
Before the central management server updates the detection model of the first user equipment by using the trained detection model, the central management server can also select a test image from the alternative images to judge whether the detection model trained by the training sample is superior to the detection model, and when the trained detection model is superior to the detection model, the detection model of the first user equipment is updated by using the trained detection model.
Optionally, in step 704, the central management server selects a test image from the candidate images to determine whether the detection model trained by the training sample is better than the detection model, which may refer to the relevant processes in steps 311 to 313 above, and the embodiments of the present invention are not described herein again.
Optionally, to ensure that the detection model can be rolled back to the detection model obtained by the previous round of optimization, the central tube server may backup the detection model (including the initial detection model and the model obtained by updating the initial detection model at least once) sent to the first client device. In order to ensure that the memory occupation of the image analysis system is as small as possible, the backup detection side model can be deleted when the backup detection model meets the deletion condition. The process of determining whether the detection model meets the deletion condition may refer to step 314 above. Of course, in other alternative embodiments, the first user end device may also back up the detection model sent by the central management server, and then delete the backed-up detection side model when the backed-up detection model meets the deletion condition, which is not limited herein in the embodiment of the present invention.
In summary, in the model optimization method provided in the embodiment of the present invention, the training sample is selected from the candidate images to train the detection model of the first user equipment, so as to obtain the trained detection model, and since the candidate images are images obtained from a plurality of images acquired by the first image acquisition device, and the first image acquisition device performs image acquisition on a certain image analysis scene where the first user equipment is located, the trained detection model can be adapted to the certain image analysis scene, a problem that the detection model trained in the related art cannot be adapted to the image analysis scene corresponding to the user equipment is solved, and reliability of the optimized detection model is improved.
It should be noted that, when the second optional implementation manner is applied, because a plurality of images acquired by the first image acquisition device may be confidential, generally, the first user equipment cannot provide the images to the central management server, and when the first optional implementation manner is used for performing model optimization, because the first user equipment may perform model optimization locally, the first user equipment may directly acquire the candidate images from the plurality of images, and select the training samples from the candidate images to train the detection model of the first user equipment, thereby further improving the pertinence of the trained detection model to the image analysis scene where the first user equipment is located.
Further, in the two alternative implementations, the users who label the training samples may be the same or different. In the two implementation manners, the users for labeling are all users of the first user equipment; or, in the first optional implementation manner, the user performing the labeling is a user of the first user-side device, and in the second optional implementation manner, the user performing the labeling is a staff of the CMS. When the user of the first user equipment directly audits the first analysis result, the user contacts the image in the image analysis scene corresponding to the first user equipment for a long time, and the image analysis scene is more familiar with the image analysis scene, so that the audit result is more accurate, namely, the result of labeling the training sample is higher in accuracy, and the accuracy of the trained detection model is further guaranteed. And because the user can audit the first analysis result, the participation of the user is improved, the user experience is enhanced, and the process of auditing the first analysis result by the user is easier and easier along with the continuous optimization of the detection model, so that the user experience is further improved.
It should be noted that the model optimization method may also be executed by the first ue and the central management server, for example, the steps 301, 302, 308 to 314 may be executed by the central management server, and the steps 303 to 307 are executed by the first ue. That is, the central management server is used for performing the training and updating process of the detection model of the first user equipment; the first user end equipment is used for determining the alternative images, and the related information in the first user end equipment and the central management server are interacted through a wired or wireless network between the first user end equipment and the central management server. The first user terminal device and the central management server may also cooperate with each other to execute the model optimization method, as long as it is ensured that the detection model for the first image acquisition device is obtained by training the alternative images acquired from the images acquired by the first image acquisition device. The embodiment of the present invention is not limited thereto.
It should be noted that, for different image analysis systems, the image capturing device and the user end device provided therein may be the same device. For example, the image acquisition device arranged in the image analysis scene of the tombstone area in the city of west ann may also be used as the image acquisition device in the image analysis scene in the city of west ann for acquiring the violation events in the city of west ann in the city of city urban management image analysis system. Accordingly, the same user end device may be located in different image analysis scenarios, and therefore, different detection models corresponding to the different image analysis scenarios one to one may be installed therein, and a user may select to use different detection models in the first user end device for the different image analysis scenarios. The foregoing embodiment takes an optimization method of one detection model of the first user equipment as an example, and the foregoing optimization method may be referred to as an optimization method of other detection models of the first user equipment.
An embodiment of the present invention provides a model optimization apparatus 80, which is applied to an image analysis system, where the image analysis system includes at least one user end device, and each user end device manages at least one image acquisition device, as shown in fig. 8, the apparatus 80 includes:
an obtaining module 801, configured to obtain an alternative image from multiple images collected by a first image collection device, where the first image collection device is an image collection device managed by a first user equipment;
an adding module 802, configured to select a training sample from the candidate image, and add the training sample to a training sample library corresponding to the first user equipment;
a training module 803, configured to train a detection model of the first user equipment based on a sample in the training sample library to obtain a trained detection model, where the detection model of the first user equipment is used to detect an image acquired by the first image acquisition device;
an updating module 804, configured to update the detection model of the first ue with the trained detection model when the trained detection model is better than the detection model.
In summary, in the model optimization device provided in the embodiment of the present invention, the training sample is selected from the candidate images to train the detection model of the first user equipment, so as to obtain the trained detection model, and since the candidate images are images obtained from a plurality of images collected by the first image collecting device, and the first image collecting device performs image collection on a certain image analysis scene where the first user equipment is located, the trained detection model can be adapted to the certain image analysis scene, a problem that the detection model obtained by training in the related art cannot be adapted to the image analysis scene corresponding to the user equipment is solved, and reliability of the optimized detection model is improved.
Optionally, the obtaining module 801 is configured to:
inputting the plurality of images into the detection model respectively;
receiving a first analysis result of each image output by the detection model;
receiving an auditing instruction input by a user at the first user terminal equipment, wherein the auditing instruction is used for indicating whether the first analysis result is accurate or not;
when the auditing instruction indicates that the first analysis result of any image is accurate, determining any image as a candidate image, and saving the first analysis result.
Optionally, the obtaining module 801 is further configured to:
when the auditing instruction indicates that the first analysis result is inaccurate, receiving a first analysis result updating instruction input by a user at the first user terminal equipment, wherein the first analysis result updating instruction is used for indicating an updated first analysis result,
determining an image containing an updated first analysis result as a candidate image, and storing the updated first analysis result;
or when the auditing instruction indicates that the first analysis result of any image is inaccurate, determining any image as a non-alternative image.
Optionally, as shown in fig. 9, the apparatus 80 further includes:
a selection module 805 configured to select a test image from the candidate images;
a determining module 806, configured to determine whether the trained detection model is better than the detection model based on the test image;
an updating module 804, configured to update the detection model of the first ue with the trained detection model when the trained detection model is better than the detection model.
Optionally, as shown in fig. 10, the determining module 806 includes:
an input sub-module 8061, configured to input the test image into the trained detection model to obtain a second analysis result;
a detection submodule 8062, configured to detect, based on the second analysis result of the test image and a first analysis result that is pre-stored, whether the accuracy of the trained detection model meets a specified accuracy condition;
a judging submodule 8063, configured to judge that the trained detection model is superior to the detection model when the accuracy of the trained detection model meets the specified accuracy condition.
Optionally, if there are multiple test images, the detection sub-module 8062 is configured to:
counting a first number of test images of which the first analysis results are the same as the corresponding second analysis results;
when the ratio of the first number to the total number of the test images is larger than a specified proportion threshold value, determining that the accuracy of the trained detection model meets a specified accuracy condition;
and when the ratio of the first number to the total number of the test images is not greater than a specified proportion threshold value, determining that the accuracy of the trained detection model does not meet a specified accuracy condition.
Optionally, the adding module 802 is configured to: selecting images with a specified proportion from the alternative images as training samples;
a selecting module 805 for: and taking the images except the training sample in the alternative images as test images.
Optionally, the training module 803 is configured to:
when the number of samples in the training sample library is larger than a specified sample number threshold value, executing at least one training process on the detection model until the detection model meets a training stopping condition;
and acquiring the trained detection model, and emptying the training sample library.
Optionally, as shown in fig. 11, the apparatus 80 further includes:
a backup module 807 for backing up the detection model of the first user equipment;
optionally, as shown in fig. 12, the apparatus 80 further includes:
a deleting module 808, configured to delete the backup detection model when the backup detection model meets the deleting condition.
Optionally, the deleting module 808 is configured to:
acquiring a plurality of new images acquired by first image acquisition equipment;
inputting the new images into the trained detection model respectively;
receiving a third analysis result of each new image output by the trained detection model;
receiving an auditing instruction of a third analysis result of the new image, which is input by a user at first user equipment, wherein the auditing instruction is used for indicating whether the third analysis result is accurate or not;
counting the proportion of accurate third analysis results of the plurality of new images in all the third analysis results based on the auditing instructions corresponding to the plurality of new images;
and when the proportion is larger than a specified proportion threshold value, determining that the backup detection model meets a deletion condition, and deleting the backup detection model.
Optionally, the image analysis system further includes a supervision platform, as shown in fig. 13, the apparatus 80 further includes:
a sending module 809, configured to send the alternative image and the saved first analysis result to the monitoring platform.
Optionally, the image analysis system is an urban management image analysis system, and the analysis result output by the detection model includes: a type of violation event in the image, and/or a location at which the violation event occurred.
Optionally, the image analysis system further includes: and the detection model is an initial detection model sent by the central management server, or a model obtained by updating the initial detection model at least once by the first user equipment.
In summary, in the model optimization device provided in the embodiment of the present invention, the training sample is selected from the candidate images to train the detection model of the first user equipment, so as to obtain the trained detection model, and since the candidate images are images obtained from a plurality of images collected by the first image collecting device, and the first image collecting device performs image collection on a certain image analysis scene where the first user equipment is located, the trained detection model can be adapted to the certain image analysis scene, a problem that the detection model obtained by training in the related art cannot be adapted to the image analysis scene corresponding to the user equipment is solved, and reliability of the optimized detection model is improved.
An embodiment of the present invention provides a storage medium, which may be a non-volatile computer-readable storage medium, where a computer program is stored in the storage medium, and when the computer program is executed by a processor, the computer program implements any one of the model optimization methods provided in the foregoing embodiments.
Embodiments of the present invention further provide a computer program product containing instructions, which when run on a computer, cause the computer to execute the model optimization method provided by the above method embodiments.
An embodiment of the present invention provides an image analysis system, including: the image analysis system comprises a model optimisation device as described in any of figures 8, 9 and 11 to 13. Other structures and architectures of the image analysis system can refer to the image analysis system shown in fig. 1.
Fig. 14 is a block diagram illustrating a structure of a user end device 900 according to an exemplary embodiment of the present invention. The client device 900 may be: smart phones, tablet computers, MP3 players (Moving Picture Experts group Audio Layer III, motion video Experts compression standard Audio Layer 3), MP4 players (Moving Picture Experts compression standard Audio Layer IV, motion video Experts compression standard Audio Layer 4), notebook computers, or desktop computers. The user end device 900 may also be referred to by other names such as user equipment, portable terminal, laptop terminal, desktop terminal, etc.
In general, the client device 900 includes: a processor 901 and a memory 902.
Processor 901 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and so forth. The processor 901 may be implemented in at least one hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 901 may also include a main processor and a coprocessor, where the main processor is a processor for processing data in an awake state, and is also called a Central Processing Unit (CPU); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 901 may be further integrated with a GPU (Graphics Processing Unit) for Processing computing operations related to machine learning, such as training and analyzing detection models. In some embodiments, the processor 901 may include an AI (Artificial Intelligence) processor, which may have the same function as the GPU, i.e., processing computational operations related to machine learning, such as training and analysis of detection models, and the like.
Memory 902 may include one or more computer-readable storage media, which may be non-transitory. The memory 902 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in memory 902 is used to store at least one instruction for execution by processor 901 to implement the model optimization methods provided by the method embodiments herein.
In some embodiments, the client device 900 may further include: a peripheral interface 903 and at least one peripheral. The processor 901, memory 902, and peripheral interface 903 may be connected by buses or signal lines. Various peripheral devices may be connected to the peripheral interface 903 via a bus, signal line, or circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 904, a touch display screen 905, a camera 906, an audio circuit 907, a positioning component 908, and a power supply 909.
The peripheral interface 903 may be used to connect at least one peripheral related to I/O (Input/Output) to the processor 901 and the memory 902. In some embodiments, the processor 901, memory 902, and peripheral interface 903 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 901, the memory 902 and the peripheral interface 903 may be implemented on a separate chip or circuit board, which is not limited by this embodiment.
The Radio Frequency circuit 904 is used for receiving and transmitting RF (Radio Frequency) signals, also called electromagnetic signals. The radio frequency circuitry 904 communicates with communication networks and other communication devices via electromagnetic signals. The radio frequency circuit 904 converts an electrical signal into an electromagnetic signal to transmit, or converts a received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 904 comprises: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a subscriber identity module card, and so forth. The radio frequency circuit 904 may communicate with other terminals via at least one wireless communication protocol. The wireless communication protocols include, but are not limited to: metropolitan area networks, various generation mobile communication networks (2G, 3G, 4G, and 5G), Wireless local area networks, and/or WiFi (Wireless Fidelity) networks. In some embodiments, the radio frequency circuit 904 may also include NFC (Near Field Communication) related circuits, which are not limited in this application.
The display screen 905 is used to display a UI (User Interface). The UI may include graphics, text, icons, video, and any combination thereof. When the display screen 905 is a touch display screen, the display screen 905 also has the ability to capture touch signals on or over the surface of the display screen 905. The touch signal may be input to the processor 901 as a control signal for processing. At this point, the display 905 may also be used to provide virtual buttons and/or a virtual keyboard, also referred to as soft buttons and/or a soft keyboard. In some embodiments, the display 905 may be one, and is configured as a front panel of the user end device 900; in other embodiments, the number of the display screens 905 may be at least two, and the two display screens are respectively disposed on different surfaces of the user end device 900 or in a folding design; in still other embodiments, the display 905 may be a flexible display, disposed on a curved surface or on a folded surface of the user end device 900. Even more, the display screen 905 may be arranged in a non-rectangular irregular figure, i.e. a shaped screen. The Display 905 may be an LCD (Liquid Crystal Display) or an OLED (Organic Light-emitting diode) Display.
The camera assembly 906 is used to capture images or video. Optionally, camera assembly 906 includes a front camera and a rear camera. Generally, a front camera is disposed at a front panel of the terminal, and a rear camera is disposed at a rear surface of the terminal. In some embodiments, the number of the rear cameras is at least two, and each rear camera is any one of a main camera, a depth-of-field camera, a wide-angle camera and a telephoto camera, so that the main camera and the depth-of-field camera are fused to realize a background blurring function, and the main camera and the wide-angle camera are fused to realize panoramic shooting and VR (Virtual Reality) shooting functions or other fusion shooting functions. In some embodiments, camera assembly 906 may also include a flash. The flash lamp can be a monochrome temperature flash lamp or a bicolor temperature flash lamp. The double-color-temperature flash lamp is a combination of a warm-light flash lamp and a cold-light flash lamp, and can be used for light compensation at different color temperatures.
Audio circuit 907 may include a microphone and a speaker. The microphone is used for collecting sound waves of a user and the environment, converting the sound waves into electric signals, and inputting the electric signals to the processor 901 for processing, or inputting the electric signals to the radio frequency circuit 904 for realizing voice communication. For the purpose of stereo sound collection or noise reduction, a plurality of microphones may be provided at different positions of the user end device 900. The microphone may also be an array microphone or an omni-directional pick-up microphone. The speaker is used to convert electrical signals from the processor 901 or the radio frequency circuit 904 into sound waves. The loudspeaker can be a traditional film loudspeaker or a piezoelectric ceramic loudspeaker. When the speaker is a piezoelectric ceramic speaker, the speaker can be used for purposes such as converting an electric signal into a sound wave audible to a human being, or converting an electric signal into a sound wave inaudible to a human being to measure a distance. In some embodiments, audio circuit 907 may also include a headphone jack.
The positioning component 908 is used to locate the current geographical Location of the client device 900 to implement navigation or LBS (Location Based Service). The positioning component 908 may be a positioning component based on the GPS (global positioning System) of the united states, the beidou System of china, the graves System of russia, or the galileo System of the european union.
The power source 909 is used to supply power to each component in the client device 900. The power source 909 may be alternating current, direct current, disposable or rechargeable. When power source 909 comprises a rechargeable battery, the rechargeable battery may support wired or wireless charging. The rechargeable battery may also be used to support fast charge technology.
In some embodiments, the client device 900 further comprises one or more sensors 910. The one or more sensors 910 include, but are not limited to: acceleration sensor 911, gyro sensor 912, pressure sensor 913, fingerprint sensor 914, optical sensor 915, and proximity sensor 916.
The acceleration sensor 911 can detect the magnitude of acceleration in three coordinate axes of the coordinate system established by the user end device 900. For example, the acceleration sensor 911 may be used to detect the components of the gravitational acceleration in three coordinate axes. The processor 901 can control the touch display 905 to display the user interface in a landscape view or a portrait view according to the gravitational acceleration signal collected by the acceleration sensor 911. The acceleration sensor 911 may also be used for acquisition of motion data of a game or a user.
The gyroscope sensor 912 may detect a body direction and a rotation angle of the user end device 900, and the gyroscope sensor 912 and the acceleration sensor 911 cooperate to acquire a 3D motion of the user to the user end device 900. The processor 901 can implement the following functions according to the data collected by the gyro sensor 912: motion sensing (such as changing the UI according to a user's tilting operation), image stabilization at the time of photographing, game control, and inertial navigation.
The pressure sensor 913 may be disposed on the side frame of the user end device 900 and/or under the touch screen 905. When the pressure sensor 913 is disposed on the side frame of the user end device 900, the user's holding signal to the user end device 900 may be detected, and the processor 901 performs left-right hand recognition or shortcut operation according to the holding signal collected by the pressure sensor 913. When the pressure sensor 913 is disposed at a lower layer of the touch display 905, the processor 901 controls the operability control on the UI interface according to the pressure operation of the user on the touch display 905. The operability control comprises at least one of a button control, a scroll bar control, an icon control and a menu control.
The fingerprint sensor 914 is used for collecting a fingerprint of the user, and the processor 901 identifies the user according to the fingerprint collected by the fingerprint sensor 914, or the fingerprint sensor 914 identifies the user according to the collected fingerprint. Upon recognizing that the user's identity is a trusted identity, processor 901 authorizes the user to perform relevant sensitive operations including unlocking the screen, viewing encrypted information, downloading software, paying, and changing settings, etc. The fingerprint sensor 914 may be arranged on the front, back or side of the user end device 900. When a physical button or vendor Logo is provided on the client device 900, the fingerprint sensor 914 may be integrated with the physical button or vendor Logo.
The optical sensor 915 is used to collect ambient light intensity. In one embodiment, the processor 901 may control the display brightness of the touch display 905 based on the ambient light intensity collected by the optical sensor 915. Specifically, when the ambient light intensity is high, the display brightness of the touch display screen 905 is increased; when the ambient light intensity is low, the display brightness of the touch display screen 905 is turned down. In another embodiment, the processor 901 can also dynamically adjust the shooting parameters of the camera assembly 906 according to the ambient light intensity collected by the optical sensor 915.
The proximity sensor 916, also called distance sensor, is usually located on the front panel of the customer premises equipment 900. The proximity sensor 916 is used to collect the distance between the user and the front side of the customer premises device 900. In one embodiment, when the proximity sensor 916 detects that the distance between the user and the front face of the user end device 900 gradually decreases, the processor 901 controls the touch display 905 to switch from the bright screen state to the dark screen state; when the proximity sensor 916 detects that the distance between the user and the front of the user end device 900 gradually becomes larger, the processor 901 controls the touch display 905 to switch from the breath screen state to the bright screen state.
Those skilled in the art will appreciate that the architecture shown in fig. 14 is not limiting of client device 900 and may include more or fewer components than shown, or some components may be combined, or a different arrangement of components may be used.
It should be noted that the client device may also be a server.
The term "and/or" in the present invention is only an association relationship describing an associated object, and means that there may be three relationships, for example, a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship.
In the present invention, the terms "first" and "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. The term "plurality" means two or more unless expressly limited otherwise.
Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.
It will be understood that the invention is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the invention is limited only by the appended claims.

Claims (19)

1. A method of model optimization, the method comprising:
acquiring an alternative image from a plurality of images acquired by first image acquisition equipment, wherein the first image acquisition equipment is image acquisition equipment managed by first user side equipment;
selecting a training sample from the alternative image, and adding the training sample to a training sample library corresponding to the first user equipment;
training a detection model of the first user equipment based on samples in the training sample library to obtain a trained detection model, wherein the detection model of the first user equipment is used for detecting images acquired by the first image acquisition equipment;
and when the trained detection model is superior to the detection model, updating the detection model of the first user equipment by adopting the trained detection model.
2. The model optimization method according to claim 1, wherein the acquiring the alternative image from the plurality of images acquired by the first image acquisition device comprises:
inputting the plurality of images into the detection model respectively;
receiving a first analysis result of each image output by the detection model;
receiving an auditing instruction input by a user at the first user terminal equipment, wherein the auditing instruction is used for indicating whether the first analysis result is accurate or not;
when the auditing instruction indicates that the first analysis result of any image is accurate, determining any image as a candidate image, and saving the first analysis result.
3. The model optimization method according to claim 2, wherein the acquiring the alternative image from the plurality of images acquired by the first image acquisition device further comprises:
when the auditing instruction indicates that the first analysis result is inaccurate, receiving a first analysis result updating instruction input by a user at the first user terminal equipment, wherein the first analysis result updating instruction is used for indicating an updated first analysis result,
determining an image containing an updated first analysis result as a candidate image, and storing the updated first analysis result;
or when the auditing instruction indicates that the first analysis result of any image is inaccurate, determining any image as a non-alternative image.
4. The model optimization method according to claim 2 or 3, wherein before said updating the detection model of the first user equipment with the trained detection model, the method further comprises:
selecting a test image from the alternative images;
judging whether the trained detection model is superior to the detection model or not based on the test image;
the updating the detection model of the first user equipment by using the trained detection model includes:
and when the trained detection model is superior to the detection model, updating the detection model of the first user equipment by adopting the trained detection model.
5. The model optimization method of claim 4, wherein the determining whether the trained inspection model is better than the inspection model based on the test image comprises:
inputting the test image into the trained detection model to obtain a second analysis result;
detecting whether the accuracy of the trained detection model meets a specified accuracy condition or not based on a second analysis result of the test image and a first analysis result which is stored in advance;
and when the accuracy of the trained detection model meets the specified accuracy condition, judging that the trained detection model is superior to the detection model.
6. The model optimization method according to claim 5, wherein there are a plurality of test images, and the detecting whether the accuracy of the trained detection model meets a specified accuracy condition based on the second analysis result of the test images and the pre-stored first analysis result comprises:
counting a first number of test images of which the first analysis results are the same as the corresponding second analysis results;
when the ratio of the first number to the total number of the test images is larger than a specified proportion threshold value, determining that the accuracy of the trained detection model meets a specified accuracy condition;
and when the ratio of the first number to the total number of the test images is not greater than a specified proportion threshold value, determining that the accuracy of the trained detection model does not meet a specified accuracy condition.
7. The model optimization method of claim 1, wherein prior to said training the detection model of the first client device based on the samples in the training sample library, the method further comprises:
backing up a detection model of the first user terminal equipment;
and when the backup detection model meets the deletion condition, deleting the backup detection model.
8. The model optimization method according to any one of claims 1 to 7, wherein the method is performed by the first user equipment, and wherein the training sample library is stored locally at the first user equipment.
9. An apparatus for model optimization, the apparatus comprising:
the system comprises an acquisition module, a processing module and a display module, wherein the acquisition module is used for acquiring alternative images from a plurality of images acquired by first image acquisition equipment, and the first image acquisition equipment is image acquisition equipment managed by first user equipment;
the adding module is used for selecting a training sample from the alternative image and adding the training sample to a training sample library corresponding to the first user terminal device;
a training module, configured to train a detection model of the first user equipment based on a sample in the training sample library to obtain a trained detection model, where the detection model of the first user equipment is used to detect an image acquired by the first image acquisition device;
and the updating module is used for updating the detection model of the first user terminal equipment by adopting the trained detection model when the trained detection model is superior to the detection model.
10. The model optimization device of claim 9, wherein the obtaining module is configured to:
inputting the plurality of images into the detection model respectively;
receiving a first analysis result of each image output by the detection model;
receiving an auditing instruction input by a user at the first user terminal equipment, wherein the auditing instruction is used for indicating whether the first analysis result is accurate or not;
when the auditing instruction indicates that the first analysis result of any image is accurate, determining any image as a candidate image, and saving the first analysis result.
11. The model optimization device of claim 10, wherein the obtaining module is further configured to:
when the auditing instruction indicates that the first analysis result is inaccurate, receiving a first analysis result updating instruction input by a user at the first user terminal equipment, wherein the first analysis result updating instruction is used for indicating an updated first analysis result,
determining an image containing an updated first analysis result as a candidate image, and storing the updated first analysis result;
or when the auditing instruction indicates that the first analysis result of any image is inaccurate, determining any image as a non-alternative image.
12. The model optimization device according to claim 10 or 11, characterized in that the device further comprises:
the selection module is used for selecting a test image from the alternative images;
the judging module is used for judging whether the trained detection model is superior to the detection model or not based on the test image;
and the updating module is used for updating the detection model of the first user terminal equipment by adopting the trained detection model when the trained detection model is superior to the detection model.
13. The model optimization device of claim 12, wherein the determining module comprises:
the input submodule is used for inputting the test image into the trained detection model to obtain a second analysis result;
the detection submodule is used for detecting whether the accuracy of the trained detection model meets the specified accuracy condition or not based on the second analysis result of the test image and the first analysis result which is stored in advance;
and the judging submodule is used for judging that the trained detection model is superior to the detection model when the accuracy of the trained detection model meets the specified accuracy condition.
14. The model optimization device of claim 13, wherein there are a plurality of said test images, and said detection sub-module is configured to:
counting a first number of test images of which the first analysis results are the same as the corresponding second analysis results;
when the ratio of the first number to the total number of the test images is larger than a specified proportion threshold value, determining that the accuracy of the trained detection model meets a specified accuracy condition;
and when the ratio of the first number to the total number of the test images is not greater than a specified proportion threshold value, determining that the accuracy of the trained detection model does not meet a specified accuracy condition.
15. The model optimization device of claim 9, further comprising:
the backup module is used for backing up the detection model of the first user terminal equipment;
and the deleting module is used for deleting the backup detection model when the backup detection model meets the deleting condition.
16. The model optimization device of any one of claims 9 to 15, wherein the model optimization device is the first user equipment and the training sample library is stored locally at the first user equipment.
17. An image analysis system, characterized in that it comprises a model optimization device according to any one of claims 9 to 16;
the image analysis system further comprises at least one user end device, and each user end device manages at least one image acquisition device.
18. A computer device comprising a processor and a memory,
wherein, the memory is used for storing computer programs;
the processor is configured to execute the program stored in the memory to implement the model optimization method according to any one of claims 1 to 8.
19. A storage medium, characterized in that the storage medium has stored therein a computer program which, when executed by a processor, implements the model optimization method of any one of claims 1 to 8.
CN201910556196.5A 2018-09-30 2019-06-25 Model optimization method, device and image analysis system Active CN110969072B (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811161087 2018-09-30
CN2018111610875 2018-09-30

Publications (2)

Publication Number Publication Date
CN110969072A true CN110969072A (en) 2020-04-07
CN110969072B CN110969072B (en) 2023-05-02

Family

ID=70029506

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910556196.5A Active CN110969072B (en) 2018-09-30 2019-06-25 Model optimization method, device and image analysis system

Country Status (1)

Country Link
CN (1) CN110969072B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113569947A (en) * 2021-07-27 2021-10-29 合肥阳光智维科技有限公司 Arc detection method and system
CN113627403A (en) * 2021-10-12 2021-11-09 深圳市安软慧视科技有限公司 Method, system and related equipment for selecting and pushing picture

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2015078183A1 (en) * 2013-11-29 2015-06-04 华为技术有限公司 Image identity recognition method and related device, and identity recognition system
US20160328613A1 (en) * 2015-05-05 2016-11-10 Xerox Corporation Online domain adaptation for multi-object tracking
CN106709917A (en) * 2017-01-03 2017-05-24 青岛海信医疗设备股份有限公司 Neural network model training method, device and system
CN107622281A (en) * 2017-09-20 2018-01-23 广东欧珀移动通信有限公司 Image classification method, device, storage medium and mobile terminal
CN108009525A (en) * 2017-12-25 2018-05-08 北京航空航天大学 A kind of specific objective recognition methods over the ground of the unmanned plane based on convolutional neural networks

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2015078183A1 (en) * 2013-11-29 2015-06-04 华为技术有限公司 Image identity recognition method and related device, and identity recognition system
US20160328613A1 (en) * 2015-05-05 2016-11-10 Xerox Corporation Online domain adaptation for multi-object tracking
CN106709917A (en) * 2017-01-03 2017-05-24 青岛海信医疗设备股份有限公司 Neural network model training method, device and system
CN107622281A (en) * 2017-09-20 2018-01-23 广东欧珀移动通信有限公司 Image classification method, device, storage medium and mobile terminal
CN108009525A (en) * 2017-12-25 2018-05-08 北京航空航天大学 A kind of specific objective recognition methods over the ground of the unmanned plane based on convolutional neural networks

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
周俊;王明军;邵乔林;: "农田图像绿色植物自适应分割方法" *
张帆;刘星;张宇;: "基于实时样本采集的个性化手写汉字输入系统设计" *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113569947A (en) * 2021-07-27 2021-10-29 合肥阳光智维科技有限公司 Arc detection method and system
CN113627403A (en) * 2021-10-12 2021-11-09 深圳市安软慧视科技有限公司 Method, system and related equipment for selecting and pushing picture

Also Published As

Publication number Publication date
CN110969072B (en) 2023-05-02

Similar Documents

Publication Publication Date Title
CN108924737B (en) Positioning method, device, equipment and computer readable storage medium
CN110278464B (en) Method and device for displaying list
CN111338910B (en) Log data processing method, log data display method, log data processing device, log data display device, log data processing equipment and log data storage medium
CN110569220B (en) Game resource file display method and device, terminal and storage medium
CN111104980B (en) Method, device, equipment and storage medium for determining classification result
CN110457571B (en) Method, device and equipment for acquiring interest point information and storage medium
CN111127509A (en) Target tracking method, device and computer readable storage medium
CN110535890B (en) File uploading method and device
CN111416996B (en) Multimedia file detection method, multimedia file playing device, multimedia file equipment and storage medium
CN111818050A (en) Target access behavior detection method, system, device, equipment and storage medium
CN110969072B (en) Model optimization method, device and image analysis system
CN107944024B (en) Method and device for determining audio file
CN111586279A (en) Method, device and equipment for determining shooting state and storage medium
CN112231666A (en) Illegal account processing method, device, terminal, server and storage medium
CN112529871A (en) Method and device for evaluating image and computer storage medium
CN110990728A (en) Method, device and equipment for managing point of interest information and storage medium
CN110768843A (en) Network problem analysis method, device, terminal and storage medium
CN112449308A (en) Mobile phone signal-based activity early warning method and device for active personnel
CN113099378B (en) Positioning method, device, equipment and storage medium
CN113205069B (en) False license plate detection method and device and computer storage medium
CN110336881B (en) Method and device for executing service processing request
CN114900559A (en) Management system, terminal, management method, and storage medium
CN110519319B (en) Method and device for splitting partitions
CN107948171B (en) User account management method and device
CN111918084A (en) Wheat loading method and device, server and terminal

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant