WO2024114440A1 - Procédé d'apprentissage en ligne, procédé et appareil d'identification d'image d'inspection de sécurité, dispositif et support - Google Patents

Procédé d'apprentissage en ligne, procédé et appareil d'identification d'image d'inspection de sécurité, dispositif et support Download PDF

Info

Publication number
WO2024114440A1
WO2024114440A1 PCT/CN2023/132906 CN2023132906W WO2024114440A1 WO 2024114440 A1 WO2024114440 A1 WO 2024114440A1 CN 2023132906 W CN2023132906 W CN 2023132906W WO 2024114440 A1 WO2024114440 A1 WO 2024114440A1
Authority
WO
WIPO (PCT)
Prior art keywords
security inspection
inspected
recognition
inspection image
online
Prior art date
Application number
PCT/CN2023/132906
Other languages
English (en)
Chinese (zh)
Inventor
陈志强
李元景
张丽
唐虎
孙运达
戴智晟
魏国华
傅罡
Original Assignee
同方威视技术股份有限公司
清华大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 同方威视技术股份有限公司, 清华大学 filed Critical 同方威视技术股份有限公司
Publication of WO2024114440A1 publication Critical patent/WO2024114440A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/10—Image acquisition
    • G06V10/12—Details of acquisition arrangements; Constructional details thereof
    • G06V10/14—Optical characteristics of the device performing the acquisition or on the illumination arrangements
    • G06V10/143—Sensing or illuminating at different wavelengths
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/70—Labelling scene content, e.g. deriving syntactic or semantic representations

Definitions

  • the present disclosure relates to the field of security inspection technology, and more specifically, to an online training method, an online training system, a security inspection image recognition method, a device, a equipment, a medium and a program product of a security inspection image recognition model.
  • Security inspection machines are usually set up in places where security inspection is required to scan the objects being inspected, and determine whether there are security issues based on the scanned images.
  • an artificial intelligence-based security inspection image recognition model can be used to automatically identify the scanned images.
  • the above-mentioned security inspection image recognition model is obtained through offline training. For example, security inspection X-ray images containing contraband at the actual security inspection site are collected, and the intelligent recognition technology manufacturer performs internal annotation and training to generate a recognition algorithm model, which is then manually updated to the on-site equipment.
  • the inventors found that the offline training method requires a series of manual operations, which is long, time-consuming and labor-intensive.
  • some images of contraband at the scene may not be collected and sent back to the technology manufacturer due to information security restrictions, resulting in the algorithm model generated after the loss of these images not being fully matched with the actual application scene.
  • the present disclosure provides an online training method, an online training system, a security inspection image recognition method, an apparatus, a device, a medium and a program product of a security inspection image recognition model which are different from an offline training method.
  • One aspect of the disclosed embodiment provides an online training method for a security inspection image recognition model, comprising: obtaining M images of an object to be inspected with specific recognition results from N recognition terminals, wherein each recognition terminal is used to determine the recognition result of the image of the object to be inspected from the corresponding security inspection machine, and M and N are respectively greater than or equal to 1; obtaining the M images of the object to be inspected The data annotation results of each image of the object to be inspected are obtained; and each image of the object to be inspected and its data annotation results are used as training samples for online training to obtain a first security inspection image recognition model.
  • the method further includes: sending the first security inspection image recognition model to the N recognition terminals, wherein the N recognition terminals are configured to locally deploy the first security inspection image recognition model.
  • the method before sending the first security inspection image recognition model to the N recognition terminals, the method also includes: obtaining an evaluation index of the first security inspection image recognition model; when the evaluation index meets a preset condition, sending a model update instruction to the N recognition terminals, wherein the N recognition terminals are configured to obtain the first security inspection image recognition model in response to the model update instruction.
  • the step of acquiring M images of the object to be inspected having specific recognition results from N recognition terminals includes: receiving the M images of the object to be inspected uploaded by the N recognition terminals.
  • obtaining the data annotation result of each of the M images of the object to be inspected includes: distributing the M images of the object to be inspected according to task type, and the task type is determined according to the recognition result of each image of the object to be inspected; and obtaining the data annotation result of each image of the object to be inspected after distribution.
  • the data annotation result of each image of the object to be inspected includes the object type of at least one object to be inspected in the image of the object to be inspected, and/or the position information of at least one object to be inspected.
  • the online training of each image of the object to be inspected and the data annotation results thereof as training samples to obtain the first security inspection image recognition model includes: online training of the second security inspection image recognition model pre-deployed at each recognition terminal to obtain the first security inspection image recognition model; and/or online training of a pre-trained third security inspection image recognition model to obtain the first security inspection image recognition model; and/or online training of an untrained fourth security inspection image recognition model to obtain the first security inspection image recognition model.
  • the M images of the object to be inspected belong to at least one
  • the method comprises: performing online training on at least one image of the object to be inspected of the same task type and its data annotation results as training samples to obtain a first security inspection image recognition model corresponding to the task type.
  • the online training of each image of the object to be inspected and the data annotation results thereof as training samples to obtain the first security inspection image recognition model includes: when the number of the training samples is greater than or equal to a preset threshold, automatically performing online training to obtain the first security inspection image recognition model.
  • Another aspect of an embodiment of the present disclosure provides an online training method for a security inspection image recognition model, which is used for an identification terminal, including: obtaining S images of objects to be inspected from a corresponding security inspection machine, where S is greater than or equal to 1; determining the recognition result of each of the S images of objects to be inspected; and sending at least one image of the object to be inspected with a specific recognition result to an online terminal, wherein the online terminal is used to execute the online training method as described above to obtain a first security inspection image recognition model.
  • the identification terminal is communicatively connected with the security inspection machine, and obtaining S images of the object to be inspected from the corresponding security inspection machine includes: after the security inspection machine scans the inspected object to obtain each image of the object to be inspected, obtaining the image of the object to be inspected in real time.
  • the method further includes: acquiring the first security inspection image recognition model from the online end; and automatically deploying the first security inspection image recognition model.
  • the method before acquiring the first security inspection image recognition model from the online end, the method further includes: receiving a model update instruction sent by the online end.
  • Another aspect of an embodiment of the present disclosure provides a security inspection image recognition method for use at an identification end, comprising: obtaining Q images of objects to be inspected from a corresponding security inspection machine, where Q is greater than or equal to 1; and using a first security inspection image recognition model to identify the Q images of objects to be inspected to obtain a recognition result, wherein the first security inspection image recognition model is obtained according to the online training method as described above.
  • Another aspect of the disclosed embodiment provides an online training system for a security inspection image recognition model, comprising N recognition terminals and an online terminal, N being greater than or equal to 1, wherein: each of the N recognition terminals comprises a recognition device, the recognition device being used to determine a recognition result of an image of an object to be inspected from a corresponding security inspection machine, and to convert a specific recognition result into a recognition result.
  • At least one image of the object to be inspected is sent to the online end;
  • the online end includes a data management platform and an online training platform, wherein: the data management platform is used to receive the at least one image of the object to be inspected sent by the recognition device, and send the at least one image of the object to be inspected and the data annotation result to the online training platform; the online training platform is used to perform online training using the at least one image of the object to be inspected and the data annotation result as training samples to obtain a first security inspection image recognition model.
  • the online training platform is also used to send the first security inspection image recognition model to the recognition device; the recognition device is used to receive and automatically deploy the first security inspection image recognition model.
  • an online training device for a security inspection image recognition model comprising: a first image module, used to obtain M images of an object to be inspected with specific recognition results from N recognition terminals, wherein each recognition terminal is used to determine the recognition result of the image of the object to be inspected from a corresponding security inspection machine, and M and N are respectively greater than or equal to 1; a data annotation module, used to obtain the data annotation result of each image of the object to be inspected in the M images of the object to be inspected; and an online training module, used to perform online training using each image of the object to be inspected and its data annotation result as a training sample to obtain a first security inspection image recognition model.
  • an online training device for a security inspection image recognition model which is used for an identification end and includes: a second image module, which is used to obtain S images of an object to be inspected from a corresponding security inspection machine, where S is greater than or equal to 1; a first identification module, which is used to determine the identification result of each image of the object to be inspected in the S images of the object to be inspected; and a third image module, which is used to send at least one image of the object to be inspected with a specific identification result to an online end, wherein the online end is used to execute the online training method as described above to obtain a first security inspection image recognition model.
  • a security inspection image recognition device for an identification end, comprising: a fourth image module, used to obtain Q images of objects to be inspected from a corresponding security inspection machine, where Q is greater than or equal to 1; a second identification module, used to identify the Q images of objects to be inspected using a first security inspection image recognition model to obtain an identification result, wherein the first security inspection image recognition model is obtained according to the method described above.
  • Another aspect of the present disclosure provides an electronic device, including: one or more a processor; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the method as described above.
  • Another aspect of the embodiments of the present disclosure further provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the method as described above.
  • Another aspect of the embodiments of the present disclosure further provides a computer program product, including a computer program, which implements the above method when executed by a processor.
  • the online training process improves the conversion efficiency from image data to recognition models, and the entire data flow is highly automated, which greatly reduces the manual participation compared to the old offline solution. Due to the use of the online training solution, some sites that are restricted by information security do not have offline processing data and therefore meet information security requirements. Therefore, the recognition model obtained by using the site image as a training sample is more suitable for the actual site, which improves the applicability of the recognition model.
  • FIG1 schematically shows a flow chart of an online training method for an online terminal according to an embodiment of the present disclosure
  • FIG2 schematically shows a flow chart of model updating according to an embodiment of the present disclosure
  • FIG3 schematically shows a flow chart of obtaining data annotation results according to an embodiment of the present disclosure
  • FIG4 schematically shows a flow chart of an online training method for an identification terminal according to an embodiment of the present disclosure
  • FIG5 schematically shows a flow chart of model updating according to another embodiment of the present disclosure
  • FIG6 schematically shows a flow chart of a security inspection image recognition method according to an embodiment of the present disclosure
  • FIG7 schematically shows an architecture diagram of an online training system according to an embodiment of the present disclosure
  • FIG. 8 schematically shows a data closure of an online training system according to an embodiment of the present disclosure. Circulation diagram
  • FIG9 schematically shows a structural block diagram of an online training device for an online terminal according to an embodiment of the present disclosure
  • FIG10 schematically shows a structural block diagram of an online training device for an identification terminal according to an embodiment of the present disclosure
  • FIG11 schematically shows a structural block diagram of a security inspection image recognition device for an identification terminal according to an embodiment of the present disclosure.
  • FIG12 schematically shows a block diagram of an electronic device suitable for implementing an online training method or a recognition method according to an embodiment of the present disclosure.
  • the collection, storage, use, processing, transmission, provision, disclosure and application of user security inspection images involved are in compliance with the provisions of relevant laws and regulations, necessary confidentiality measures are taken, and do not violate public order and good morals.
  • the embodiments of the present disclosure provide an online training method, an online training system, a security inspection image recognition method, a device, equipment, a medium and a program product for a security inspection image recognition model.
  • the online training process improves the conversion efficiency from image data to an algorithm model, and the entire data flow is highly automated. Compared with the old offline solution, the manual participation part is greatly reduced. Due to the use of an online training solution, some sites restricted by information security do not have offline processing data, so they meet information security requirements. Therefore, the algorithm model obtained as a training sample is more suitable for the actual site, which improves the applicability of the algorithm model.
  • FIG1 schematically shows a flow chart of an online training method for an online terminal according to an embodiment of the present disclosure.
  • the online training method of the security inspection image recognition model of this embodiment includes operations S110 to S130. In some embodiments, it may also include operation S140.
  • M images of the object to be inspected with specific recognition results are obtained from N recognition terminals, wherein each recognition terminal is used to determine the recognition result of the image of the object to be inspected from the corresponding security inspection machine, and M and N are respectively greater than or equal to 1.
  • the security inspection machine may include an X-ray security inspection machine, which scans the inspected object through X-rays to obtain an image.
  • the recognition end may include a terminal device or
  • the server and the terminal device may include various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.
  • the terminal device may be installed with a manual image judgment application and/or an automatic image judgment application.
  • the manual image judgment application may display the image of the object to be inspected to the judge to obtain the judgment result input by the judge, that is, as the determined recognition result.
  • the automatic image judgment application may call the security inspection image recognition model for automatic recognition to obtain the recognition result.
  • the recognition result may include the type of the object to be inspected in each image of the object to be inspected, such as clothing, food, electronic products or contraband, etc.
  • a specific recognition result may include a result of identifying that contraband is contained. For example, for a certain recognition terminal, if it identifies that 5 images out of 100 images of the object to be inspected contain contraband within a certain time period, then the 5 images are regarded as images with specific recognition results.
  • the online terminal can receive M images of the object to be inspected uploaded by N recognition terminals.
  • a data transmission interface is configured between each recognition terminal and the online terminal, and the recognition terminal can send M images of the object to be inspected online by calling the interface.
  • each recognition terminal may upload the image of the object to be detected having a specific recognition result to a shared database or a cloud server, and the online terminal may obtain M images of the object to be detected from the shared database or the cloud server.
  • the transmission of the image of the object to be inspected between the recognition end and the online end is performed online, which can reduce manual operations and save time, and avoid information security issues caused by offline.
  • the data annotation result may be automatically annotated by the recognition end during recognition, in which case it is obtained from the recognition end. It may also be annotated manually after obtaining M images of the object to be inspected, or it may be automatically annotated using automatic annotation software.
  • the data annotation result of each image of the object to be inspected includes the object type of at least one object to be inspected in the image of the object to be inspected, and/or the position information of at least one object to be inspected.
  • the object type may include the specific name of the object to be inspected, such as lighters, mobile phones, Knives, guns, explosives, drugs, etc., and may also include types of contraband, such as weapon types, inflammable and explosive types, or biological types, etc.
  • the location information may include an object annotation box on the image, which may be used to select the location of the object to be inspected.
  • each image of the object to be inspected and its data annotation results are used as training samples for online training to obtain a first security inspection image recognition model.
  • the first security inspection image recognition model is constructed based on the convolutional neural network algorithm and trained online.
  • Each training sample can include an image of the object to be inspected and the type and location labels of the contraband therein.
  • the online training process is to input multiple training samples into the initialized first security inspection image recognition model, and update the parameters in the model through continuous iteration and back propagation until the objective function of the model meets the requirements or the iteration ends.
  • each image of the object to be inspected and its data annotation results are used as training samples for online training to obtain a first security inspection image recognition model, including: when the number of training samples is greater than or equal to a preset threshold, automatically performing online training to obtain the first security inspection image recognition model.
  • the offline training method is to collect all the pre-collected training samples together and learn them at the same time.
  • the training time is long, and the model cannot be adjusted in real time to adapt to the actual security inspection scene that may change (such as the appearance of new contraband).
  • it since it involves many manual links, it will also take a long time and the final generated model will not be able to adapt.
  • the online training method can run automatically when the number of training samples is greater than or equal to a preset threshold, which can adapt well to the actual security inspection scene and can adapt well to the scene where new types of contraband or hiding methods may appear.
  • training can also be manually started according to user needs, such as canceling automatic training, waiting for manual training to be started regardless of whether the number of training samples is greater than or equal to a preset threshold or less than a preset threshold.
  • the method of automatically starting training when the number is greater than or equal to a preset threshold is retained, and training can also be manually started when the number of training samples is less than the preset threshold.
  • the conversion efficiency from the on-site image data of the security inspection machine to the recognition model can be improved, the degree of automation of the entire data flow process is improved, and the applicability of the recognition model to the actual security inspection site can also be improved.
  • each image of the object to be inspected and its data annotation results are used as training samples for online training to obtain the first security inspection image recognition model, including: online training of the second security inspection image recognition model pre-deployed at each recognition terminal to obtain the first security inspection image recognition model. Or online training of the pre-trained third security inspection image recognition model to obtain the first security inspection image recognition model. Or online training of the untrained fourth security inspection image recognition model to obtain the first security inspection image recognition model.
  • a variety of training measures can be provided online.
  • the third security inspection image recognition model may be pre-trained but not deployed. Continuing to train the third security inspection image recognition model in a transfer learning manner can obtain better recognition effects, and the training cost will also be reduced.
  • the fourth security inspection image recognition model can be initialized, and the model obtained by training it can be better applied to actual security inspection scenarios.
  • the first security inspection image recognition model after the first security inspection image recognition model is obtained by online training, it can be automatically deployed to N recognition terminals.
  • the first security inspection image recognition model is sent to the N recognition terminals, wherein the N recognition terminals are configured to deploy the first security inspection image recognition model locally.
  • the recognition end can send images of the actual security inspection site to the online end, and the online end can use the above images for online training to obtain a first security inspection image recognition model, which can then be deployed to the recognition end.
  • the online data closed loop the on-site image data is used to complete the training and generation of a new model, and then the new model is fed back to the on-site intelligent recognition device, thereby realizing the online training and upgrading of the security inspection image recognition model.
  • the online end may directly push the first security inspection image recognition model to the recognition end, or may send a model update instruction to the recognition end as shown in FIG. 2 , as follows.
  • FIG. 2 schematically shows a flow chart of model updating according to an embodiment of the present disclosure.
  • operations S210 to S220 may be included.
  • the samples in the test set can be input into the model, and the evaluation indicators such as accuracy, precision, recall or F1 value of the model can be obtained according to the recognition results.
  • the indicator evaluation report of the new model allows the user to choose whether to update the model and also execute the step of "comparing the indicators of the new model and the old model". If the new model is better than the old model, it will be automatically updated.
  • a model update instruction is sent to the N recognition terminals, wherein the N recognition terminals are configured to obtain a first security inspection image recognition model in response to the model update instruction.
  • the recognition terminals after sending a model update instruction to N recognition terminals, the recognition terminals can decide when to update, and can flexibly update in consideration of the actual local conditions of the recognition terminals.
  • FIG3 schematically shows a flow chart of obtaining data annotation results according to an embodiment of the present disclosure.
  • obtaining the data annotation result of each of the M images of the object to be inspected in operation S130 includes operations S310 to S320 .
  • M images of objects to be inspected are distributed according to task types, and the task type is determined according to the recognition result of each image of the object to be inspected.
  • the task type may correspond to the type of contraband, such as weapon type, inflammable and explosive type, or biological type, etc. For example, if the recognition result of an image of an object to be inspected is that there is a knife, it is classified as a weapon type, and if the recognition result of an image of an object to be inspected is that there is explosives, it is classified as an inflammable and explosive type.
  • the distribution may include sending the images to different annotators according to the task type, and manually annotating the images of the type. It may also include calling an automatic annotation application to set annotation tasks according to different task types. It may also include sending the images to different servers for automatic annotation when the amount of data is large.
  • M images of objects to be inspected belong to at least one task type, and each image of the object to be inspected and its data annotation results are used as training samples for online training to obtain a first security inspection image recognition model, including: using at least one image of the object to be inspected of the same task type and its data annotation results as training samples for online training to obtain a first security inspection image recognition model corresponding to the task type.
  • first security inspection image recognition models There may be multiple first security inspection image recognition models, for example, a model trained with samples of weapon type is used to identify weapons, and a model trained with samples of flammable and explosive type is used to identify The obtained model is used to identify inflammable and explosive items such as fireworks and explosives.
  • there is only one first security inspection image recognition model which can be obtained by training samples of multiple task types to identify different types of objects.
  • online training is performed for each task type to obtain a corresponding first security inspection image recognition model, which can improve the accuracy of image recognition of the object to be inspected.
  • FIG4 schematically shows a flow chart of an online training method for an identification terminal according to an embodiment of the present disclosure.
  • the online training method of the security inspection image recognition model of this embodiment includes operations S410 to S430 , which can be applied to any recognition end among N recognition ends.
  • S images of the object to be inspected are obtained from the corresponding security inspection machine, where S is greater than or equal to 1.
  • the identification terminal is in communication connection with the security inspection machine, and obtaining S images of the object to be inspected from the corresponding security inspection machine includes: after the security inspection machine scans the inspected object to obtain each image of the object to be inspected, obtaining the image of the object to be inspected in real time.
  • the security inspection machine uses a conveyor belt to transport the inspected object (such as luggage) into the X-ray inspection channel.
  • the luggage enters the X-ray inspection channel, triggering the X-ray source to emit an X-ray beam.
  • the X-ray beam passes through the inspected object on the conveyor belt, is absorbed by the inspected object, and finally bombards the detector installed in the channel.
  • the detector converts the X-ray into a signal and sends it to the recognition end in real time. After these signals are processed, an image can be formed and displayed on the display screen, which means that the image of the object to be inspected is acquired in real time.
  • At least one image of the object to be inspected having a specific recognition result is sent to an online end, wherein the online end is used to execute one or more embodiments corresponding to FIGS. 1 to 3 to obtain a first security inspection image recognition model.
  • the recognition end can directly send the image of the object to be inspected to the online end, and the image transmission is performed online, which improves the data flow efficiency, reduces the time cost, and ensures data security.
  • FIG. 5 schematically shows a flow chart of model updating according to another embodiment of the present disclosure.
  • the model updating of this embodiment includes operations S510 to S520 .
  • a first security inspection image recognition model is obtained from an online end.
  • the recognition end may monitor the model update status of the online end, may periodically send a query instruction to the online end to inquire whether the model has been updated, and may also receive the first security inspection image recognition model sent by the online end.
  • the method before obtaining the first security inspection image recognition model from the online terminal, the method further includes: receiving a model update instruction sent by the online terminal. After receiving the model update instruction, the recognition terminal can automatically obtain the first security inspection image recognition model from the online terminal.
  • a first security inspection image recognition model is automatically deployed.
  • the recognition end may originally be manually judged, in which case the first security inspection image recognition model can be automatically installed.
  • the recognition end may obtain the recognition result by using the pre-deployed second security inspection image recognition model, in which case the model upgrade may be completed automatically.
  • the model may be deployed by manual upgrade.
  • the first security inspection image recognition obtained by using the on-site image data is fed back online to the recognition end through an online data closed-loop method, thereby avoiding the defect that the offline training method still requires external manufacturers to arrange manual on-site upgrades.
  • FIG6 schematically shows a flow chart of a security inspection image recognition method according to an embodiment of the present disclosure.
  • the security inspection image recognition method of this embodiment includes operations S610 to S620 , which can be applied to any recognition terminal among N recognition terminals.
  • Q images of the object to be inspected are obtained from the corresponding security inspection machine, where Q is greater than or equal to 1.
  • the Q images of the object to be inspected may also be obtained in real time through data transmission after the object to be inspected enters the security inspection machine for scanning.
  • the first security inspection image recognition model is used to identify Q images of the object to be inspected. Perform recognition to obtain a recognition result, wherein the first security inspection image recognition model is obtained according to one or more embodiments corresponding to Figures 1 to 5.
  • the model can be run to recognize Q images of objects to be inspected, thereby obtaining more accurate recognition results with better applicability to the current actual security inspection site.
  • FIG. 7 schematically shows an architecture diagram of an online training system according to an embodiment of the present disclosure.
  • the online training system of this embodiment includes N identification terminals and online terminals, specifically including N security inspection machines (such as 711-security inspection machine 1, 712-security inspection machine 2 ... 713-security inspection machine N), N identification devices (such as 721-identification device 1, 722-identification device 2 ... 723-identification device N), a data management platform 731 and an online training platform 732.
  • the N identification terminals may include N identification devices and N security inspection machines
  • the online terminal includes a data management platform and 731 online training platform 732.
  • the data management platform 731 and the online training platform 732 may also be combined into one platform.
  • the above platform includes the operating environment of the computer hardware and/or software required to implement the corresponding functions.
  • one identification device can correspond to multiple security inspection machines, and there is no need for one-to-one correspondence, and the data management platform can still receive images transmitted by one or more identification devices.
  • each recognition terminal includes a recognition device, which is used to determine the recognition result of the image of the object to be inspected from the corresponding security inspection machine, and send at least one image of the object to be inspected with a specific recognition result to the online terminal.
  • the online end includes a data management platform 731 and an online training platform 732, wherein: the data management platform 731 is used to receive at least one image of the object to be inspected sent by N recognition devices, and send at least one image of the object to be inspected and the data annotation result to the online training platform 732.
  • the online training platform 732 is used to perform online training using at least one image of the object to be inspected and the data annotation result as training samples to obtain a first security inspection image recognition model.
  • the online training platform is further used to send the first security inspection image recognition model to N recognition devices.
  • the N recognition devices are used to receive and automatically deploy the first security inspection image recognition model.
  • the identification end of this embodiment can execute the above-mentioned FIG. 1 to FIG. 3 for the identification end
  • the online end may execute one or more steps of the online training method for the online end described in FIG. 4 to FIG. 5 .
  • Figure 7 are data flow diagrams. Taking 711-security inspection machine 1 and 721-identification device 1, data management platform 731 and online training platform 732 as examples, Figure 8 is used to expand the description of the data flow during the operation of the online training system.
  • FIG8 schematically shows a data closed-loop flow diagram of an online training system according to an embodiment of the present disclosure.
  • the data closed-loop flow process during the operation of the online training system of this embodiment includes operations S810 to S880 .
  • Operation S810 711-security inspection machine 1 performs X-ray scanning on the inspected object and transmits the signal to 721-identification device 1.
  • Operation S820, 721-the identification device 1 performs real-time contraband identification on the X-ray image passing through the security inspection machine.
  • 721-recognition device 1 uploads the X-ray image that intelligently identifies contraband to the data management platform 731.
  • 721-recognition device 1 can also obtain the manual judgment conclusion of the X-ray image from an external system, and then upload the X-ray image that is manually judged to contain contraband to the data management platform 731.
  • the data management platform 731 distributes the uploaded X-ray images to the image annotation terminals according to the task type, and the X-ray images are annotated manually or automatically.
  • the data management platform 731 uploads the annotated X-ray image and data to the online training platform 732 .
  • Operation S860 when a certain number of annotated X-ray images are accumulated, the online training platform 732 uses these images to automatically perform training and learning, generates a new algorithm model, and simultaneously provides an indicator evaluation report of the new model.
  • Operation S870 The user selects whether to update the existing algorithm model of 721-recognition device 1 according to the evaluation report. If the update is performed, a model update instruction is sent to 721-recognition device 1.
  • Operation S880 if the update instruction is executed, then 721- the recognition device 1 will obtain the specified algorithm model to complete the automatic upgrade.
  • an online training scheme including intelligent recognition equipment,
  • the data management platform and online training platform are several components that realize online training and upgrading of intelligent recognition algorithm models by real-time collection of security X-ray images containing prohibited items at the actual security inspection site.
  • the present disclosure also provides an online training device for the online security inspection image recognition model.
  • the device will be described in detail below in conjunction with FIG.
  • FIG9 schematically shows a structural block diagram of an online training device for an online terminal according to an embodiment of the present disclosure.
  • the online training device 900 of this embodiment includes a first image module 910 , a data annotation module 920 and a data annotation module 930 .
  • the first image module 910 can perform operation S110 to obtain M images of the object to be inspected with specific recognition results from N recognition terminals, wherein each recognition terminal is used to determine the recognition result of the image of the object to be inspected from the corresponding security inspection machine, and M and N are respectively greater than or equal to 1.
  • the first image module 910 is used to receive M images of the object to be inspected uploaded by N recognition terminals.
  • the data annotation module 920 may perform operation S120 to obtain a data annotation result for each of the M images of the object to be inspected.
  • the data labeling module 920 may perform operations S310 to S320, which will not be described in detail herein.
  • the online training module 930 may perform operation S130 to perform online training using each image of the object to be inspected and its data annotation result as a training sample to obtain a first security inspection image recognition model.
  • the online training module 930 is used to perform online training on the second security inspection image recognition model pre-deployed on each recognition terminal to obtain the first security inspection image recognition model. And/or perform online training on the pre-trained third security inspection image recognition model to obtain the first security inspection image recognition model. And/or perform online training on the untrained fourth security inspection image recognition model to obtain the first security inspection image recognition model.
  • the online training module 930 is used to perform online training using at least one image of an object to be inspected of the same task type and its data annotation results as training samples to obtain Obtain the first security inspection image recognition model corresponding to the task type.
  • online training is automatically performed to obtain a first security inspection image recognition model.
  • the online training device 900 may further include a model sending module, which may perform operation S140 to send the first security inspection image recognition model to N recognition terminals.
  • the module may further perform operations S210 to S220, which are not described in detail herein.
  • the present disclosure also provides an online training device for the security inspection image recognition model of the recognition terminal.
  • the device will be described in detail below in conjunction with FIG.
  • FIG10 schematically shows a structural block diagram of an online training device for an identification terminal according to an embodiment of the present disclosure.
  • the online training device 1000 of this embodiment includes a second image module 1010 , a first recognition module 1020 , and a third image module 1030 .
  • the second image module 1010 may perform operation S410 to obtain S images of the object to be inspected from the corresponding security inspection machine, where S is greater than or equal to 1.
  • the second image module 1010 is used to obtain each image of the object to be inspected in real time after the security inspection machine scans the object to be inspected to obtain the image of the object to be inspected.
  • the first recognition module 1020 may perform operation S420 to determine a recognition result of each of the S images of the object to be detected.
  • the third image module 1030 can perform operation S430 to send at least one image of the object to be inspected having a specific recognition result to an online end, wherein the online end is used to execute the online training method shown in Figures 1 to 3 to obtain a first security inspection image recognition model.
  • the online training device 1000 may further include a model upgrade module, which may perform operations S510 to S520, which are not described in detail herein.
  • the model upgrade module is used to receive a model update instruction sent by the online end before acquiring the first security inspection image recognition model from the online end.
  • the present disclosure further provides a security inspection image recognition device for an identification terminal.
  • the device will be described in detail below in conjunction with FIG.
  • FIG. 11 schematically shows a structural block diagram of a security inspection image recognition device for an identification terminal according to an embodiment of the present disclosure.
  • the security inspection image recognition device 1100 of this embodiment includes a fourth image module 1110 and a second recognition module 1120 .
  • the fourth image module 1110 may perform operation S610 to obtain Q images of the object to be inspected from the corresponding security inspection machine, where Q is greater than or equal to 1.
  • the second recognition module 1120 can perform operation S620 to recognize Q images of objects to be inspected by using the first security inspection image recognition model to obtain a recognition result, wherein the first security inspection image recognition model is obtained according to the online training method described in Figures 1 to 5.
  • any multiple modules in the online training device 900, the online training device 1000 or the security inspection image recognition device 1100 may be combined into one module for implementation, or any one of the modules may be split into multiple modules. Or both, at least part of the functions of one or more of these modules may be combined with at least part of the functions of other modules and implemented in one module.
  • At least one of the online training device 900, the online training device 1000, or the security inspection image recognition device 1100 may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or may be implemented by hardware or firmware in any other reasonable manner of integrating or packaging the circuit, or in any one of the three implementation methods of software, hardware, and firmware, or in a suitable combination of any of them.
  • FPGA field programmable gate array
  • PLA programmable logic array
  • ASIC application specific integrated circuit
  • at least one of the online training device 900, the online training device 1000, or the security inspection image recognition device 1100 may be at least partially implemented as a computer program module, and when the computer program module is executed, the corresponding function may be executed.
  • FIG12 schematically shows a block diagram of an electronic device suitable for implementing an online training method or a recognition method according to an embodiment of the present disclosure.
  • the electronic device 1200 includes a processor 1201, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1202 or a program loaded from a storage portion 1208 into a random access memory (RAM) 1203.
  • the processor 1201 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and/or a related chipset and/or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc.
  • the processor 1201 may also include an onboard memory for caching purposes.
  • the processor 1201 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.
  • RAM 1203 various programs and data required for the operation of electronic device 1200 are stored.
  • Processor 1201, ROM 1202 and RAM 1203 are connected to each other through bus 1204.
  • Processor 1201 performs various operations of the method flow according to the embodiment of the present disclosure by executing the programs in ROM 1202 and/or RAM 1203. It should be noted that the program can also be stored in one or more memories other than ROM 1202 and RAM 1203.
  • Processor 1201 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the programs stored in one or more memories.
  • the electronic device 1200 may further include an input/output (I/O) interface 1205, which is also connected to the bus 1204.
  • the electronic device 1200 may further include one or more of the following components connected to the I/O interface 1205: an input portion 1206 including a keyboard, a mouse, etc.
  • An output portion 1207 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.
  • a storage portion 1208 including a hard disk, etc.
  • a communication portion 1209 including a network interface card such as a LAN card, a modem, etc.
  • the communication portion 1209 performs communication processing via a network such as the Internet.
  • a drive 1210 is also connected to the I/O interface 1205 as needed.
  • a removable medium 1211 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1210 as needed, so that the computer program read therefrom is installed into the storage portion 1208 as needed.
  • the present disclosure also provides a computer-readable storage medium, which may be included in the device/apparatus/system described in the above embodiments. It may also exist independently without being assembled into the device/apparatus/system.
  • the above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the implementation Now, the method according to the embodiment of the present disclosure is described.
  • a computer-readable storage medium may be a non-volatile computer-readable storage medium, such as but not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
  • a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.
  • a computer-readable storage medium may include the ROM 1202 and/or RAM 1203 described above and/or one or more memories other than ROM 1202 and RAM 1203.
  • the embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart.
  • the program code is used to enable the computer system to implement the method provided by the embodiment of the present disclosure.
  • the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc.
  • the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 1209, and/or installed from the removable medium 1211.
  • the program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.
  • the computer program can be downloaded and installed from the network through the communication part 1209, and/or installed from the removable medium 1211.
  • the computer program is executed by the processor 1201, the above functions defined in the system of the embodiment of the present disclosure are performed.
  • the system, device, means, module, unit, etc. described above can be implemented by a computer program module.
  • any combination of one or more programming languages can be used.
  • the program code for executing the computer program provided by the embodiment of the present disclosure can be written together.
  • these computing programs can be implemented using high-level process and/or object-oriented programming languages, and/or assembly/machine languages.
  • Programming languages include, but are not limited to, Java, C++, python, "C" language or similar programming languages.
  • the program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server.
  • the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).
  • LAN local area network
  • WAN wide area network
  • an Internet service provider to connect through the Internet
  • each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function.
  • the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved.
  • each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • Image Analysis (AREA)

Abstract

L'invention concerne un procédé d'apprentissage en ligne pour un modèle d'identification d'image d'inspection de sécurité, se rapportant au domaine technique de l'inspection de sécurité. Un procédé d'apprentissage en ligne pour une extrémité en ligne consiste à : obtenir M images d'objet à détecter ayant des résultats d'identification spécifiques à partir de N extrémités d'identification ; obtenir un résultat d'annotation de données de chacune des M images d'objet à détecter ; et effectuer un apprentissage en ligne en utilisant chaque image d'objet à détecter et le résultat d'annotation de données comme échantillons d'apprentissage, afin d'obtenir un premier modèle d'identification d'image d'inspection de sécurité. Un procédé d'apprentissage en ligne pour une extrémité d'identification consiste à : obtenir S images d'objet à détecter à partir d'une machine d'inspection de sécurité correspondante ; déterminer un résultat d'identification pour chacune des S images d'objet à détecter ; et envoyer au moins une image d'objet à détecter ayant un résultat d'identification spécifique à une extrémité en ligne. L'efficacité de conversion des données d'image sur site d'une machine d'inspection de sécurité en un modèle d'inspection peut être améliorée, le degré d'automatisation de l'ensemble du processus de transfert de données est amélioré et l'applicabilité du modèle d'identification au site d'inspection de sécurité réel peut être améliorée.
PCT/CN2023/132906 2022-12-01 2023-11-21 Procédé d'apprentissage en ligne, procédé et appareil d'identification d'image d'inspection de sécurité, dispositif et support WO2024114440A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211545328.2A CN118135347A (zh) 2022-12-01 2022-12-01 在线训练方法、安检图像识别方法、装置、设备和介质
CN202211545328.2 2022-12-01

Publications (1)

Publication Number Publication Date
WO2024114440A1 true WO2024114440A1 (fr) 2024-06-06

Family

ID=91236550

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/132906 WO2024114440A1 (fr) 2022-12-01 2023-11-21 Procédé d'apprentissage en ligne, procédé et appareil d'identification d'image d'inspection de sécurité, dispositif et support

Country Status (2)

Country Link
CN (1) CN118135347A (fr)
WO (1) WO2024114440A1 (fr)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019096181A1 (fr) * 2017-11-14 2019-05-23 深圳码隆科技有限公司 Procédé, appareil et système de détection pour inspection de sécurité et dispositif électronique
CN112712121A (zh) * 2020-12-30 2021-04-27 浙江智慧视频安防创新中心有限公司 一种基于深度神经网络的图像识别模型训练方法、装置及存储介质
CN112884085A (zh) * 2021-04-02 2021-06-01 中国科学院自动化研究所 基于x光图像的违禁物品检测识别方法、系统及设备
CN113688887A (zh) * 2021-08-13 2021-11-23 百度在线网络技术（北京）有限公司 图像识别模型的训练与图像识别方法、装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019096181A1 (fr) * 2017-11-14 2019-05-23 深圳码隆科技有限公司 Procédé, appareil et système de détection pour inspection de sécurité et dispositif électronique
CN112712121A (zh) * 2020-12-30 2021-04-27 浙江智慧视频安防创新中心有限公司 一种基于深度神经网络的图像识别模型训练方法、装置及存储介质
CN112884085A (zh) * 2021-04-02 2021-06-01 中国科学院自动化研究所 基于x光图像的违禁物品检测识别方法、系统及设备
CN113688887A (zh) * 2021-08-13 2021-11-23 百度在线网络技术（北京）有限公司 图像识别模型的训练与图像识别方法、装置

Also Published As

Publication number Publication date
CN118135347A (zh) 2024-06-04

Similar Documents

Publication Publication Date Title
US20200249936A1 (en) Method and system for a platform for api based user supplied algorithm deployment
US20200387836A1 (en) Machine learning model surety
JP5540160B2 (ja) プログラム解析・検証サービス提供システム、その制御方法、制御プログラム、コンピュータを機能させるための制御プログラム、プログラム解析・検証装置、プログラム解析・検証ツール管理装置
US20190244139A1 (en) Using meta-learning for automatic gradient-based hyperparameter optimization for machine learning and deep learning models
CN108304201A (zh) 对象更新方法、装置及设备
US11250951B2 (en) Feature engineering method, apparatus, and system
WO2024027624A1 (fr) Procédé de mise en correspondance de ressources intelligentes à base de métavers, et système de métavers
CN105432075A (zh) 用于移动质量管理检查的方法及设备
CN110391918A (zh) 利用机器学习的通讯频道推荐系统、方法和计算机可读介质
US11615269B2 (en) Processing method, system, program, and storage medium for generating learning data, and learning data generation method and system
US20230298333A1 (en) Information processing system
CN109857630A (zh) 代码检测方法、系统及设备
US20230004486A1 (en) Software testing using machine learning
US20220366244A1 (en) Modeling Human Behavior in Work Environment Using Neural Networks
CN107481018A (zh) 基于互联网技术的物品属性鉴定方法、装置及系统
CN109522214A (zh) 测试方法、平台、客户端及系统、计算设备及可读介质
CN115766861A (zh) 用于车软件升级的自动驾驶数据闭环系统、方法和介质
WO2024114440A1 (fr) Procédé d'apprentissage en ligne, procédé et appareil d'identification d'image d'inspection de sécurité, dispositif et support
Miranda et al. AKFruitData: A dual software application for Azure Kinect cameras to acquire and extract informative data in yield tests performed in fruit orchard environments
US20210158214A1 (en) Method of performing a process using artificial intelligence
US11599801B2 (en) Method for solving problem, computing system and program product
CN117235527A (zh) 端到端容器化的大数据模型构建方法、装置、设备及介质
CN113590484B (zh) 算法模型服务测试方法、系统、设备及存储介质
US20220044150A1 (en) Systems, methods, and apparatus to classify personalized data
CN113609018A (zh) 测试方法、训练方法、装置、设备、介质和程序产品