WO2024114440A1 - Online training method, security inspection image identification method and apparatus, device, and medium - Google Patents

Abstract

Provided is an online training method for a security inspection image identification model, relating to the technical field of security inspection. An online training method for an online end comprises: obtaining M object images to be detected having specific identification results from N identification ends; obtaining a data annotation result of each of the M object images to be detected; and performing online training by using each object image to be detected and the data annotation result thereof as training samples, to obtain a first security inspection image identification model. An online training method for an identification end comprises: obtaining S object images to be detected from a corresponding security inspection machine; determining an identification result of each of the S object images to be detected; and sending at least one object image to be detected having a specific identification result to an online end. The conversion efficiency from on-site image data of a security inspection machine to an inspection model can be improved, the automation degree of the whole data transfer process is improved, and the applicability of the identification model to the actual security inspection site can be improved.

Description

Online training method, security inspection image recognition method, device, equipment and medium

This application claims priority to Chinese Patent Application No. 202211545328.2 filed on December 1, 2022, the contents of which are incorporated herein by reference.

Technical Field

The present disclosure relates to the field of security inspection technology, and more specifically, to an online training method, an online training system, a security inspection image recognition method, a device, a equipment, a medium and a program product of a security inspection image recognition model.

Background technique

Security inspection machines are usually set up in places where security inspection is required to scan the objects being inspected, and determine whether there are security issues based on the scanned images. In the related technology, an artificial intelligence-based security inspection image recognition model can be used to automatically identify the scanned images. The above-mentioned security inspection image recognition model is obtained through offline training. For example, security inspection X-ray images containing contraband at the actual security inspection site are collected, and the intelligent recognition technology manufacturer performs internal annotation and training to generate a recognition algorithm model, which is then manually updated to the on-site equipment.

In the process of implementing the disclosed invention, the inventors found that the offline training method requires a series of manual operations, which is long, time-consuming and labor-intensive. In addition, some images of contraband at the scene may not be collected and sent back to the technology manufacturer due to information security restrictions, resulting in the algorithm model generated after the loss of these images not being fully matched with the actual application scene.

Summary of the invention

In view of the above problems, the present disclosure provides an online training method, an online training system, a security inspection image recognition method, an apparatus, a device, a medium and a program product of a security inspection image recognition model which are different from an offline training method.

One aspect of the disclosed embodiment provides an online training method for a security inspection image recognition model, comprising: obtaining M images of an object to be inspected with specific recognition results from N recognition terminals, wherein each recognition terminal is used to determine the recognition result of the image of the object to be inspected from the corresponding security inspection machine, and M and N are respectively greater than or equal to 1; obtaining the M images of the object to be inspected The data annotation results of each image of the object to be inspected are obtained; and each image of the object to be inspected and its data annotation results are used as training samples for online training to obtain a first security inspection image recognition model.

According to an embodiment of the present disclosure, after obtaining the first security inspection image recognition model, the method further includes: sending the first security inspection image recognition model to the N recognition terminals, wherein the N recognition terminals are configured to locally deploy the first security inspection image recognition model.

According to an embodiment of the present disclosure, before sending the first security inspection image recognition model to the N recognition terminals, the method also includes: obtaining an evaluation index of the first security inspection image recognition model; when the evaluation index meets a preset condition, sending a model update instruction to the N recognition terminals, wherein the N recognition terminals are configured to obtain the first security inspection image recognition model in response to the model update instruction.

According to an embodiment of the present disclosure, the step of acquiring M images of the object to be inspected having specific recognition results from N recognition terminals includes: receiving the M images of the object to be inspected uploaded by the N recognition terminals.

According to an embodiment of the present disclosure, obtaining the data annotation result of each of the M images of the object to be inspected includes: distributing the M images of the object to be inspected according to task type, and the task type is determined according to the recognition result of each image of the object to be inspected; and obtaining the data annotation result of each image of the object to be inspected after distribution.

According to an embodiment of the present disclosure, the data annotation result of each image of the object to be inspected includes the object type of at least one object to be inspected in the image of the object to be inspected, and/or the position information of at least one object to be inspected.

According to an embodiment of the present disclosure, the online training of each image of the object to be inspected and the data annotation results thereof as training samples to obtain the first security inspection image recognition model includes: online training of the second security inspection image recognition model pre-deployed at each recognition terminal to obtain the first security inspection image recognition model; and/or online training of a pre-trained third security inspection image recognition model to obtain the first security inspection image recognition model; and/or online training of an untrained fourth security inspection image recognition model to obtain the first security inspection image recognition model.

According to an embodiment of the present disclosure, the M images of the object to be inspected belong to at least one The method comprises: performing online training on at least one image of the object to be inspected of the same task type and its data annotation results as training samples to obtain a first security inspection image recognition model corresponding to the task type.

According to an embodiment of the present disclosure, the online training of each image of the object to be inspected and the data annotation results thereof as training samples to obtain the first security inspection image recognition model includes: when the number of the training samples is greater than or equal to a preset threshold, automatically performing online training to obtain the first security inspection image recognition model.

Another aspect of an embodiment of the present disclosure provides an online training method for a security inspection image recognition model, which is used for an identification terminal, including: obtaining S images of objects to be inspected from a corresponding security inspection machine, where S is greater than or equal to 1; determining the recognition result of each of the S images of objects to be inspected; and sending at least one image of the object to be inspected with a specific recognition result to an online terminal, wherein the online terminal is used to execute the online training method as described above to obtain a first security inspection image recognition model.

According to an embodiment of the present disclosure, the identification terminal is communicatively connected with the security inspection machine, and obtaining S images of the object to be inspected from the corresponding security inspection machine includes: after the security inspection machine scans the inspected object to obtain each image of the object to be inspected, obtaining the image of the object to be inspected in real time.

According to an embodiment of the present disclosure, the method further includes: acquiring the first security inspection image recognition model from the online end; and automatically deploying the first security inspection image recognition model.

According to an embodiment of the present disclosure, before acquiring the first security inspection image recognition model from the online end, the method further includes: receiving a model update instruction sent by the online end.

Another aspect of an embodiment of the present disclosure provides a security inspection image recognition method for use at an identification end, comprising: obtaining Q images of objects to be inspected from a corresponding security inspection machine, where Q is greater than or equal to 1; and using a first security inspection image recognition model to identify the Q images of objects to be inspected to obtain a recognition result, wherein the first security inspection image recognition model is obtained according to the online training method as described above.

Another aspect of the disclosed embodiment provides an online training system for a security inspection image recognition model, comprising N recognition terminals and an online terminal, N being greater than or equal to 1, wherein: each of the N recognition terminals comprises a recognition device, the recognition device being used to determine a recognition result of an image of an object to be inspected from a corresponding security inspection machine, and to convert a specific recognition result into a recognition result. At least one image of the object to be inspected is sent to the online end; the online end includes a data management platform and an online training platform, wherein: the data management platform is used to receive the at least one image of the object to be inspected sent by the recognition device, and send the at least one image of the object to be inspected and the data annotation result to the online training platform; the online training platform is used to perform online training using the at least one image of the object to be inspected and the data annotation result as training samples to obtain a first security inspection image recognition model.

According to an embodiment of the present disclosure, wherein: the online training platform is also used to send the first security inspection image recognition model to the recognition device; the recognition device is used to receive and automatically deploy the first security inspection image recognition model.

Another aspect of an embodiment of the present disclosure provides an online training device for a security inspection image recognition model, comprising: a first image module, used to obtain M images of an object to be inspected with specific recognition results from N recognition terminals, wherein each recognition terminal is used to determine the recognition result of the image of the object to be inspected from a corresponding security inspection machine, and M and N are respectively greater than or equal to 1; a data annotation module, used to obtain the data annotation result of each image of the object to be inspected in the M images of the object to be inspected; and an online training module, used to perform online training using each image of the object to be inspected and its data annotation result as a training sample to obtain a first security inspection image recognition model.

Another aspect of an embodiment of the present disclosure provides an online training device for a security inspection image recognition model, which is used for an identification end and includes: a second image module, which is used to obtain S images of an object to be inspected from a corresponding security inspection machine, where S is greater than or equal to 1; a first identification module, which is used to determine the identification result of each image of the object to be inspected in the S images of the object to be inspected; and a third image module, which is used to send at least one image of the object to be inspected with a specific identification result to an online end, wherein the online end is used to execute the online training method as described above to obtain a first security inspection image recognition model.

Another aspect of an embodiment of the present disclosure provides a security inspection image recognition device for an identification end, comprising: a fourth image module, used to obtain Q images of objects to be inspected from a corresponding security inspection machine, where Q is greater than or equal to 1; a second identification module, used to identify the Q images of objects to be inspected using a first security inspection image recognition model to obtain an identification result, wherein the first security inspection image recognition model is obtained according to the method described above.

Another aspect of the present disclosure provides an electronic device, including: one or more a processor; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the method as described above.

Another aspect of the embodiments of the present disclosure further provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the method as described above.

Another aspect of the embodiments of the present disclosure further provides a computer program product, including a computer program, which implements the above method when executed by a processor.

One or more of the above embodiments have at least the following beneficial effects: the online training process improves the conversion efficiency from image data to recognition models, and the entire data flow is highly automated, which greatly reduces the manual participation compared to the old offline solution. Due to the use of the online training solution, some sites that are restricted by information security do not have offline processing data and therefore meet information security requirements. Therefore, the recognition model obtained by using the site image as a training sample is more suitable for the actual site, which improves the applicability of the recognition model.

BRIEF DESCRIPTION OF THE DRAWINGS

The above contents and other purposes, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

FIG1 schematically shows a flow chart of an online training method for an online terminal according to an embodiment of the present disclosure;

FIG2 schematically shows a flow chart of model updating according to an embodiment of the present disclosure;

FIG3 schematically shows a flow chart of obtaining data annotation results according to an embodiment of the present disclosure;

FIG4 schematically shows a flow chart of an online training method for an identification terminal according to an embodiment of the present disclosure;

FIG5 schematically shows a flow chart of model updating according to another embodiment of the present disclosure;

FIG6 schematically shows a flow chart of a security inspection image recognition method according to an embodiment of the present disclosure;

FIG7 schematically shows an architecture diagram of an online training system according to an embodiment of the present disclosure;

FIG. 8 schematically shows a data closure of an online training system according to an embodiment of the present disclosure. Circulation diagram;

FIG9 schematically shows a structural block diagram of an online training device for an online terminal according to an embodiment of the present disclosure;

FIG10 schematically shows a structural block diagram of an online training device for an identification terminal according to an embodiment of the present disclosure;

FIG11 schematically shows a structural block diagram of a security inspection image recognition device for an identification terminal according to an embodiment of the present disclosure; and

FIG12 schematically shows a block diagram of an electronic device suitable for implementing an online training method or a recognition method according to an embodiment of the present disclosure.

Detailed ways

Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "include", "comprising", etc. used herein indicate the existence of the features, steps, operations and/or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

When using expressions such as "at least one of A, B, and C", they should generally be interpreted according to the meaning of the expression commonly understood by technical personnel in this field (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and/or A, B, C, etc.).

In the technical solution disclosed in the present invention, the collection, storage, use, processing, transmission, provision, disclosure and application of user security inspection images involved are in compliance with the provisions of relevant laws and regulations, necessary confidentiality measures are taken, and do not violate public order and good morals.

In the related technologies, offline training methods will collect a certain amount of images of contraband (such as weapons, drugs, explosives, smuggled goods, smuggled organisms or other items prohibited by security inspection sites) at the security inspection site, and then process them internally before manually updating the on-site equipment. This requires a series of manual operations, which is costly and inefficient. In airports, customs or security inspection sites with information security requirements, the internal network may be isolated from the external network. For example, the scanned images of security inspection machines are not allowed to leave the intranet and be copied to external technology manufacturers. Manufacturers can only collect the required images from other channels. On the one hand, it takes a long time for manufacturers to collect a sufficient number of images of contraband. On the other hand, there are certain differences between the images collected by manufacturers and the actual security inspection site, resulting in the final generated algorithm model not being able to fully match the application site, resulting in limited operation and poor matching.

The embodiments of the present disclosure provide an online training method, an online training system, a security inspection image recognition method, a device, equipment, a medium and a program product for a security inspection image recognition model. The online training process improves the conversion efficiency from image data to an algorithm model, and the entire data flow is highly automated. Compared with the old offline solution, the manual participation part is greatly reduced. Due to the use of an online training solution, some sites restricted by information security do not have offline processing data, so they meet information security requirements. Therefore, the algorithm model obtained as a training sample is more suitable for the actual site, which improves the applicability of the algorithm model.

The following will be further described in detail with reference to FIGS. 1 to 12 .

FIG1 schematically shows a flow chart of an online training method for an online terminal according to an embodiment of the present disclosure.

As shown in Fig. 1, the online training method of the security inspection image recognition model of this embodiment includes operations S110 to S130. In some embodiments, it may also include operation S140.

In operation S110, M images of the object to be inspected with specific recognition results are obtained from N recognition terminals, wherein each recognition terminal is used to determine the recognition result of the image of the object to be inspected from the corresponding security inspection machine, and M and N are respectively greater than or equal to 1.

For example, the security inspection machine may include an X-ray security inspection machine, which scans the inspected object through X-rays to obtain an image. The recognition end may include a terminal device or The server and the terminal device may include various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

The terminal device may be installed with a manual image judgment application and/or an automatic image judgment application. For example, the manual image judgment application may display the image of the object to be inspected to the judge to obtain the judgment result input by the judge, that is, as the determined recognition result. For another example, the automatic image judgment application may call the security inspection image recognition model for automatic recognition to obtain the recognition result.

Exemplarily, the recognition result may include the type of the object to be inspected in each image of the object to be inspected, such as clothing, food, electronic products or contraband, etc. A specific recognition result may include a result of identifying that contraband is contained. For example, for a certain recognition terminal, if it identifies that 5 images out of 100 images of the object to be inspected contain contraband within a certain time period, then the 5 images are regarded as images with specific recognition results.

In some embodiments, the online terminal can receive M images of the object to be inspected uploaded by N recognition terminals. For example, a data transmission interface is configured between each recognition terminal and the online terminal, and the recognition terminal can send M images of the object to be inspected online by calling the interface.

In other embodiments, for example, each recognition terminal may upload the image of the object to be detected having a specific recognition result to a shared database or a cloud server, and the online terminal may obtain M images of the object to be detected from the shared database or the cloud server.

According to the embodiments of the present disclosure, unlike the offline training method which requires manual copying, the transmission of the image of the object to be inspected between the recognition end and the online end is performed online, which can reduce manual operations and save time, and avoid information security issues caused by offline.

In operation S120, a data annotation result of each of the M images of the object to be inspected is obtained.

For example, the data annotation result may be automatically annotated by the recognition end during recognition, in which case it is obtained from the recognition end. It may also be annotated manually after obtaining M images of the object to be inspected, or it may be automatically annotated using automatic annotation software.

In some embodiments, the data annotation result of each image of the object to be inspected includes the object type of at least one object to be inspected in the image of the object to be inspected, and/or the position information of at least one object to be inspected.

The object type may include the specific name of the object to be inspected, such as lighters, mobile phones, Knives, guns, explosives, drugs, etc., and may also include types of contraband, such as weapon types, inflammable and explosive types, or biological types, etc. The location information may include an object annotation box on the image, which may be used to select the location of the object to be inspected.

In operation S130, each image of the object to be inspected and its data annotation results are used as training samples for online training to obtain a first security inspection image recognition model.

For example, the first security inspection image recognition model is constructed based on the convolutional neural network algorithm and trained online. Each training sample can include an image of the object to be inspected and the type and location labels of the contraband therein. The online training process is to input multiple training samples into the initialized first security inspection image recognition model, and update the parameters in the model through continuous iteration and back propagation until the objective function of the model meets the requirements or the iteration ends.

In some embodiments, each image of the object to be inspected and its data annotation results are used as training samples for online training to obtain a first security inspection image recognition model, including: when the number of training samples is greater than or equal to a preset threshold, automatically performing online training to obtain the first security inspection image recognition model.

The offline training method is to collect all the pre-collected training samples together and learn them at the same time. When the amount of data is large, the training time is long, and the model cannot be adjusted in real time to adapt to the actual security inspection scene that may change (such as the appearance of new contraband). Similarly, since it involves many manual links, it will also take a long time and the final generated model will not be able to adapt.

The online training method can run automatically when the number of training samples is greater than or equal to a preset threshold, which can adapt well to the actual security inspection scene and can adapt well to the scene where new types of contraband or hiding methods may appear.

In some embodiments, training can also be manually started according to user needs, such as canceling automatic training, waiting for manual training to be started regardless of whether the number of training samples is greater than or equal to a preset threshold or less than a preset threshold. For another example, the method of automatically starting training when the number is greater than or equal to a preset threshold is retained, and training can also be manually started when the number of training samples is less than the preset threshold.

According to the embodiments of the present disclosure, the conversion efficiency from the on-site image data of the security inspection machine to the recognition model can be improved, the degree of automation of the entire data flow process is improved, and the applicability of the recognition model to the actual security inspection site can also be improved.

In some embodiments, each image of the object to be inspected and its data annotation results are used as training samples for online training to obtain the first security inspection image recognition model, including: online training of the second security inspection image recognition model pre-deployed at each recognition terminal to obtain the first security inspection image recognition model. Or online training of the pre-trained third security inspection image recognition model to obtain the first security inspection image recognition model. Or online training of the untrained fourth security inspection image recognition model to obtain the first security inspection image recognition model.

According to the embodiments of the present disclosure, a variety of training measures can be provided online. By training the second security inspection image recognition model online, the accuracy and applicability can be further improved on the basis of the original recognition effect of the recognition end, and the training cost can be reduced. The third security inspection image recognition model may be pre-trained but not deployed. Continuing to train the third security inspection image recognition model in a transfer learning manner can obtain better recognition effects, and the training cost will also be reduced. The fourth security inspection image recognition model can be initialized, and the model obtained by training it can be better applied to actual security inspection scenarios.

In some embodiments, after the first security inspection image recognition model is obtained by online training, it can be automatically deployed to N recognition terminals. In operation S140, the first security inspection image recognition model is sent to the N recognition terminals, wherein the N recognition terminals are configured to deploy the first security inspection image recognition model locally.

According to the embodiments of the present disclosure, the recognition end can send images of the actual security inspection site to the online end, and the online end can use the above images for online training to obtain a first security inspection image recognition model, which can then be deployed to the recognition end. In other words, through the online data closed loop, the on-site image data is used to complete the training and generation of a new model, and then the new model is fed back to the on-site intelligent recognition device, thereby realizing the online training and upgrading of the security inspection image recognition model.

In some embodiments, the online end may directly push the first security inspection image recognition model to the recognition end, or may send a model update instruction to the recognition end as shown in FIG. 2 , as follows.

FIG. 2 schematically shows a flow chart of model updating according to an embodiment of the present disclosure.

Before sending the first security inspection image recognition model to N recognition terminals, as shown in FIG. 2 , operations S210 to S220 may be included.

In operation S210, an evaluation index of a first security inspection image recognition model is obtained.

For example, the samples in the test set can be input into the model, and the evaluation indicators such as accuracy, precision, recall or F1 value of the model can be obtained according to the recognition results. The indicator evaluation report of the new model allows the user to choose whether to update the model and also execute the step of "comparing the indicators of the new model and the old model". If the new model is better than the old model, it will be automatically updated.

In operation S220, when the evaluation index meets the preset conditions, a model update instruction is sent to the N recognition terminals, wherein the N recognition terminals are configured to obtain a first security inspection image recognition model in response to the model update instruction.

According to an embodiment of the present disclosure, after sending a model update instruction to N recognition terminals, the recognition terminals can decide when to update, and can flexibly update in consideration of the actual local conditions of the recognition terminals.

FIG3 schematically shows a flow chart of obtaining data annotation results according to an embodiment of the present disclosure.

As shown in FIG. 3 , obtaining the data annotation result of each of the M images of the object to be inspected in operation S130 includes operations S310 to S320 .

In operation S310 , M images of objects to be inspected are distributed according to task types, and the task type is determined according to the recognition result of each image of the object to be inspected.

The task type may correspond to the type of contraband, such as weapon type, inflammable and explosive type, or biological type, etc. For example, if the recognition result of an image of an object to be inspected is that there is a knife, it is classified as a weapon type, and if the recognition result of an image of an object to be inspected is that there is explosives, it is classified as an inflammable and explosive type.

In operation S320, the data annotation result of each image of the object to be inspected after distribution is obtained.

For example, the distribution may include sending the images to different annotators according to the task type, and manually annotating the images of the type. It may also include calling an automatic annotation application to set annotation tasks according to different task types. It may also include sending the images to different servers for automatic annotation when the amount of data is large.

In some embodiments, M images of objects to be inspected belong to at least one task type, and each image of the object to be inspected and its data annotation results are used as training samples for online training to obtain a first security inspection image recognition model, including: using at least one image of the object to be inspected of the same task type and its data annotation results as training samples for online training to obtain a first security inspection image recognition model corresponding to the task type.

There may be multiple first security inspection image recognition models, for example, a model trained with samples of weapon type is used to identify weapons, and a model trained with samples of flammable and explosive type is used to identify The obtained model is used to identify inflammable and explosive items such as fireworks and explosives. In some embodiments, there is only one first security inspection image recognition model, which can be obtained by training samples of multiple task types to identify different types of objects.

According to the embodiments of the present disclosure, online training is performed for each task type to obtain a corresponding first security inspection image recognition model, which can improve the accuracy of image recognition of the object to be inspected.

FIG4 schematically shows a flow chart of an online training method for an identification terminal according to an embodiment of the present disclosure.

As shown in FIG. 4 , the online training method of the security inspection image recognition model of this embodiment includes operations S410 to S430 , which can be applied to any recognition end among N recognition ends.

In operation S410, S images of the object to be inspected are obtained from the corresponding security inspection machine, where S is greater than or equal to 1.

In some embodiments, the identification terminal is in communication connection with the security inspection machine, and obtaining S images of the object to be inspected from the corresponding security inspection machine includes: after the security inspection machine scans the inspected object to obtain each image of the object to be inspected, obtaining the image of the object to be inspected in real time.

For example, the security inspection machine uses a conveyor belt to transport the inspected object (such as luggage) into the X-ray inspection channel. The luggage enters the X-ray inspection channel, triggering the X-ray source to emit an X-ray beam. The X-ray beam passes through the inspected object on the conveyor belt, is absorbed by the inspected object, and finally bombards the detector installed in the channel. The detector converts the X-ray into a signal and sends it to the recognition end in real time. After these signals are processed, an image can be formed and displayed on the display screen, which means that the image of the object to be inspected is acquired in real time.

In operation S420, a recognition result of each of the S images of the object to be inspected is determined.

During manual image judgment, security personnel use the display screen to identify prohibited items based on the outline of the item and the color of the item image, and make a judgment conclusion, which is the recognition result. During automatic image judgment, if the security inspection image recognition model is used to process the image, the display screen can be omitted, and the model automatically outputs the recognition result. Therefore, based on the recognition result, it is possible to effectively exclude whether the carry-on bag carried by the person contains prohibited items.

In operation S430, at least one image of the object to be inspected having a specific recognition result is sent to an online end, wherein the online end is used to execute one or more embodiments corresponding to FIGS. 1 to 3 to obtain a first security inspection image recognition model.

According to the embodiments of the present disclosure, the recognition end can directly send the image of the object to be inspected to the online end, and the image transmission is performed online, which improves the data flow efficiency, reduces the time cost, and ensures data security.

FIG. 5 schematically shows a flow chart of model updating according to another embodiment of the present disclosure.

As shown in FIG. 5 , the model updating of this embodiment includes operations S510 to S520 .

In operation S510, a first security inspection image recognition model is obtained from an online end.

Exemplarily, the recognition end may monitor the model update status of the online end, may periodically send a query instruction to the online end to inquire whether the model has been updated, and may also receive the first security inspection image recognition model sent by the online end.

In some embodiments, before obtaining the first security inspection image recognition model from the online terminal, the method further includes: receiving a model update instruction sent by the online terminal. After receiving the model update instruction, the recognition terminal can automatically obtain the first security inspection image recognition model from the online terminal.

In operation S520, a first security inspection image recognition model is automatically deployed.

In some embodiments, the recognition end may originally be manually judged, in which case the first security inspection image recognition model can be automatically installed.

In other embodiments, the recognition end may obtain the recognition result by using the pre-deployed second security inspection image recognition model, in which case the model upgrade may be completed automatically. In other embodiments, different from operation S520, the model may be deployed by manual upgrade.

According to the embodiments of the present disclosure, the first security inspection image recognition obtained by using the on-site image data is fed back online to the recognition end through an online data closed-loop method, thereby avoiding the defect that the offline training method still requires external manufacturers to arrange manual on-site upgrades.

FIG6 schematically shows a flow chart of a security inspection image recognition method according to an embodiment of the present disclosure.

As shown in FIG. 6 , the security inspection image recognition method of this embodiment includes operations S610 to S620 , which can be applied to any recognition terminal among N recognition terminals.

In operation S610, Q images of the object to be inspected are obtained from the corresponding security inspection machine, where Q is greater than or equal to 1.

Exemplarily, the Q images of the object to be inspected may also be obtained in real time through data transmission after the object to be inspected enters the security inspection machine for scanning.

In operation S620, the first security inspection image recognition model is used to identify Q images of the object to be inspected. Perform recognition to obtain a recognition result, wherein the first security inspection image recognition model is obtained according to one or more embodiments corresponding to Figures 1 to 5.

After the first security inspection image recognition model is trained and upgraded online as described above, the model can be run to recognize Q images of objects to be inspected, thereby obtaining more accurate recognition results with better applicability to the current actual security inspection site.

FIG. 7 schematically shows an architecture diagram of an online training system according to an embodiment of the present disclosure.

As shown in FIG7 , the online training system of this embodiment includes N identification terminals and online terminals, specifically including N security inspection machines (such as 711-security inspection machine 1, 712-security inspection machine 2 ... 713-security inspection machine N), N identification devices (such as 721-identification device 1, 722-identification device 2 ... 723-identification device N), a data management platform 731 and an online training platform 732. Among them, the N identification terminals may include N identification devices and N security inspection machines, and the online terminal includes a data management platform and 731 online training platform 732. In some embodiments, the data management platform 731 and the online training platform 732 may also be combined into one platform. The above platform includes the operating environment of the computer hardware and/or software required to implement the corresponding functions. It should be noted that the architecture diagram shown in FIG7 is only exemplary. Under the concept of implementing the online training model of the embodiment of the present disclosure, the architecture of the online training system can be deformed. For example, in some embodiments, one identification device can correspond to multiple security inspection machines, and there is no need for one-to-one correspondence, and the data management platform can still receive images transmitted by one or more identification devices.

7 , each recognition terminal includes a recognition device, which is used to determine the recognition result of the image of the object to be inspected from the corresponding security inspection machine, and send at least one image of the object to be inspected with a specific recognition result to the online terminal.

The online end includes a data management platform 731 and an online training platform 732, wherein: the data management platform 731 is used to receive at least one image of the object to be inspected sent by N recognition devices, and send at least one image of the object to be inspected and the data annotation result to the online training platform 732. The online training platform 732 is used to perform online training using at least one image of the object to be inspected and the data annotation result as training samples to obtain a first security inspection image recognition model.

In some other embodiments, the online training platform is further used to send the first security inspection image recognition model to N recognition devices. The N recognition devices are used to receive and automatically deploy the first security inspection image recognition model.

The identification end of this embodiment can execute the above-mentioned FIG. 1 to FIG. 3 for the identification end One or more steps of the online training method, the online end may execute one or more steps of the online training method for the online end described in FIG. 4 to FIG. 5 .

In particular, the arrows in Figure 7 are data flow diagrams. Taking 711-security inspection machine 1 and 721-identification device 1, data management platform 731 and online training platform 732 as examples, Figure 8 is used to expand the description of the data flow during the operation of the online training system.

FIG8 schematically shows a data closed-loop flow diagram of an online training system according to an embodiment of the present disclosure.

As shown in FIG. 8 , the data closed-loop flow process during the operation of the online training system of this embodiment includes operations S810 to S880 .

Operation S810, 711-security inspection machine 1 performs X-ray scanning on the inspected object and transmits the signal to 721-identification device 1.

Operation S820, 721-the identification device 1 performs real-time contraband identification on the X-ray image passing through the security inspection machine.

Operation S830, 721-recognition device 1 uploads the X-ray image that intelligently identifies contraband to the data management platform 731. 721-recognition device 1 can also obtain the manual judgment conclusion of the X-ray image from an external system, and then upload the X-ray image that is manually judged to contain contraband to the data management platform 731.

In operation S840, the data management platform 731 distributes the uploaded X-ray images to the image annotation terminals according to the task type, and the X-ray images are annotated manually or automatically.

In operation S850 , the data management platform 731 uploads the annotated X-ray image and data to the online training platform 732 .

Operation S860, when a certain number of annotated X-ray images are accumulated, the online training platform 732 uses these images to automatically perform training and learning, generates a new algorithm model, and simultaneously provides an indicator evaluation report of the new model.

Operation S870: The user selects whether to update the existing algorithm model of 721-recognition device 1 according to the evaluation report. If the update is performed, a model update instruction is sent to 721-recognition device 1.

Operation S880, if the update instruction is executed, then 721- the recognition device 1 will obtain the specified algorithm model to complete the automatic upgrade.

According to the embodiment of the present disclosure, an online training scheme is adopted, including intelligent recognition equipment, The data management platform and online training platform are several components that realize online training and upgrading of intelligent recognition algorithm models by real-time collection of security X-ray images containing prohibited items at the actual security inspection site.

Based on the above-mentioned online training method for the online security inspection image recognition model, the present disclosure also provides an online training device for the online security inspection image recognition model. The device will be described in detail below in conjunction with FIG.

FIG9 schematically shows a structural block diagram of an online training device for an online terminal according to an embodiment of the present disclosure.

As shown in FIG. 9 , the online training device 900 of this embodiment includes a first image module 910 , a data annotation module 920 and a data annotation module 930 .

The first image module 910 can perform operation S110 to obtain M images of the object to be inspected with specific recognition results from N recognition terminals, wherein each recognition terminal is used to determine the recognition result of the image of the object to be inspected from the corresponding security inspection machine, and M and N are respectively greater than or equal to 1.

In some embodiments, the first image module 910 is used to receive M images of the object to be inspected uploaded by N recognition terminals.

The data annotation module 920 may perform operation S120 to obtain a data annotation result for each of the M images of the object to be inspected.

In some embodiments, the data labeling module 920 may perform operations S310 to S320, which will not be described in detail herein.

The online training module 930 may perform operation S130 to perform online training using each image of the object to be inspected and its data annotation result as a training sample to obtain a first security inspection image recognition model.

In some embodiments, the online training module 930 is used to perform online training on the second security inspection image recognition model pre-deployed on each recognition terminal to obtain the first security inspection image recognition model. And/or perform online training on the pre-trained third security inspection image recognition model to obtain the first security inspection image recognition model. And/or perform online training on the untrained fourth security inspection image recognition model to obtain the first security inspection image recognition model.

In some embodiments, the online training module 930 is used to perform online training using at least one image of an object to be inspected of the same task type and its data annotation results as training samples to obtain Obtain the first security inspection image recognition model corresponding to the task type.

In some embodiments, when the number of training samples is greater than or equal to a preset threshold, online training is automatically performed to obtain a first security inspection image recognition model.

In some embodiments, the online training device 900 may further include a model sending module, which may perform operation S140 to send the first security inspection image recognition model to N recognition terminals. In some embodiments, the module may further perform operations S210 to S220, which are not described in detail herein.

Based on the above-mentioned online training method for the security inspection image recognition model of the recognition terminal, the present disclosure also provides an online training device for the security inspection image recognition model of the recognition terminal. The device will be described in detail below in conjunction with FIG.

FIG10 schematically shows a structural block diagram of an online training device for an identification terminal according to an embodiment of the present disclosure.

As shown in FIG. 10 , the online training device 1000 of this embodiment includes a second image module 1010 , a first recognition module 1020 , and a third image module 1030 .

The second image module 1010 may perform operation S410 to obtain S images of the object to be inspected from the corresponding security inspection machine, where S is greater than or equal to 1.

In some embodiments, the second image module 1010 is used to obtain each image of the object to be inspected in real time after the security inspection machine scans the object to be inspected to obtain the image of the object to be inspected.

The first recognition module 1020 may perform operation S420 to determine a recognition result of each of the S images of the object to be detected.

The third image module 1030 can perform operation S430 to send at least one image of the object to be inspected having a specific recognition result to an online end, wherein the online end is used to execute the online training method shown in Figures 1 to 3 to obtain a first security inspection image recognition model.

In some embodiments, the online training device 1000 may further include a model upgrade module, which may perform operations S510 to S520, which are not described in detail herein. In some embodiments, the model upgrade module is used to receive a model update instruction sent by the online end before acquiring the first security inspection image recognition model from the online end.

Based on the above security inspection image recognition method for an identification terminal, the present disclosure further provides a security inspection image recognition device for an identification terminal. The device will be described in detail below in conjunction with FIG.

FIG. 11 schematically shows a structural block diagram of a security inspection image recognition device for an identification terminal according to an embodiment of the present disclosure.

As shown in FIG. 11 , the security inspection image recognition device 1100 of this embodiment includes a fourth image module 1110 and a second recognition module 1120 .

The fourth image module 1110 may perform operation S610 to obtain Q images of the object to be inspected from the corresponding security inspection machine, where Q is greater than or equal to 1.

The second recognition module 1120 can perform operation S620 to recognize Q images of objects to be inspected by using the first security inspection image recognition model to obtain a recognition result, wherein the first security inspection image recognition model is obtained according to the online training method described in Figures 1 to 5.

It should be noted that the implementation methods, technical problems solved, functions realized, and technical effects achieved of each module/unit/sub-unit, etc. in each device part embodiment are the same or similar to the implementation methods, technical problems solved, functions realized, and technical effects achieved of each corresponding step in the method part embodiment, and will not be repeated here.

According to an embodiment of the present disclosure, any multiple modules in the online training device 900, the online training device 1000 or the security inspection image recognition device 1100 may be combined into one module for implementation, or any one of the modules may be split into multiple modules. Or both, at least part of the functions of one or more of these modules may be combined with at least part of the functions of other modules and implemented in one module.

According to an embodiment of the present disclosure, at least one of the online training device 900, the online training device 1000, or the security inspection image recognition device 1100 may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or may be implemented by hardware or firmware in any other reasonable manner of integrating or packaging the circuit, or in any one of the three implementation methods of software, hardware, and firmware, or in a suitable combination of any of them. Alternatively, at least one of the online training device 900, the online training device 1000, or the security inspection image recognition device 1100 may be at least partially implemented as a computer program module, and when the computer program module is executed, the corresponding function may be executed.

FIG12 schematically shows a block diagram of an electronic device suitable for implementing an online training method or a recognition method according to an embodiment of the present disclosure.

As shown in FIG12 , the electronic device 1200 according to an embodiment of the present disclosure includes a processor 1201, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1202 or a program loaded from a storage portion 1208 into a random access memory (RAM) 1203. The processor 1201 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and/or a related chipset and/or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 1201 may also include an onboard memory for caching purposes. The processor 1201 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

In RAM 1203, various programs and data required for the operation of electronic device 1200 are stored. Processor 1201, ROM 1202 and RAM 1203 are connected to each other through bus 1204. Processor 1201 performs various operations of the method flow according to the embodiment of the present disclosure by executing the programs in ROM 1202 and/or RAM 1203. It should be noted that the program can also be stored in one or more memories other than ROM 1202 and RAM 1203. Processor 1201 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the programs stored in one or more memories.

According to an embodiment of the present disclosure, the electronic device 1200 may further include an input/output (I/O) interface 1205, which is also connected to the bus 1204. The electronic device 1200 may further include one or more of the following components connected to the I/O interface 1205: an input portion 1206 including a keyboard, a mouse, etc. An output portion 1207 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc. A storage portion 1208 including a hard disk, etc. And a communication portion 1209 including a network interface card such as a LAN card, a modem, etc. The communication portion 1209 performs communication processing via a network such as the Internet. A drive 1210 is also connected to the I/O interface 1205 as needed. A removable medium 1211, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1210 as needed, so that the computer program read therefrom is installed into the storage portion 1208 as needed.

The present disclosure also provides a computer-readable storage medium, which may be included in the device/apparatus/system described in the above embodiments. It may also exist independently without being assembled into the device/apparatus/system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the implementation Now, the method according to the embodiment of the present disclosure is described.

According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, such as but not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include the ROM 1202 and/or RAM 1203 described above and/or one or more memories other than ROM 1202 and RAM 1203.

The embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the method provided by the embodiment of the present disclosure.

The above functions defined in the system/device of the embodiment of the present disclosure are performed when the computer program is executed by the processor 1201. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 1209, and/or installed from the removable medium 1211. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1209, and/or installed from the removable medium 1211. When the computer program is executed by the processor 1201, the above functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

According to the embodiments of the present disclosure, any combination of one or more programming languages can be used. The program code for executing the computer program provided by the embodiment of the present disclosure can be written together. Specifically, these computing programs can be implemented using high-level process and/or object-oriented programming languages, and/or assembly/machine languages. Programming languages include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

It will be appreciated by those skilled in the art that the features described in the various embodiments and/or claims of the present disclosure may be combined and/or combined in a variety of ways, even if such combinations and/or combinations are not explicitly described in the present disclosure. In particular, the features described in the various embodiments and/or claims of the present disclosure may be combined and/or combined in a variety of ways without departing from the spirit and teachings of the present disclosure. All of these combinations and/or combinations fall within the scope of the present disclosure.

The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although each embodiment is described above separately, it does not mean that the measures in each embodiment cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make various substitutions and modifications, which All should fall within the scope of this disclosure.

Claims (22)

1. An online training method for a security inspection image recognition model, comprising:

   Obtaining M images of the object to be inspected with specific recognition results from N recognition terminals, wherein each recognition terminal is used to determine the recognition result of the image of the object to be inspected from the corresponding security inspection machine, and M and N are respectively greater than or equal to 1;

   Obtaining a data annotation result for each of the M images of the object to be inspected;

   Each image of the object to be inspected and the data annotation results thereof are used as training samples for online training to obtain a first security inspection image recognition model.
2. The method according to claim 1, wherein after obtaining the first security inspection image recognition model, the method further comprises:

   The first security inspection image recognition model is sent to the N recognition terminals, wherein the N recognition terminals are configured to locally deploy the first security inspection image recognition model.
3. The method according to claim 2, wherein, before sending the first security inspection image recognition model to the N recognition terminals, the method further comprises:

   Obtaining evaluation indicators of the first security inspection image recognition model;

   When the evaluation index meets the preset condition, a model update instruction is sent to the N recognition terminals, wherein the N recognition terminals are configured to obtain the first security inspection image recognition model in response to the model update instruction.
4. The method according to claim 1, wherein obtaining M images of the object to be inspected having specific recognition results from N recognition terminals comprises:

   Receive the M images of the objects to be inspected uploaded by the N recognition terminals.
5. The method according to claim 4, wherein obtaining the data annotation result of each of the M images of the object to be inspected comprises:

   Distributing the M images of the objects to be inspected according to task types, wherein the task types are determined according to the recognition results of each of the images of the objects to be inspected;

   Obtain the data annotation results of each image of the object to be inspected after distribution.
6. The method according to claim 5, wherein each image of the object to be inspected The data annotation result includes the object type of at least one object to be inspected in the image of the object to be inspected, and/or the position information of at least one object to be inspected.
7. The method according to claim 5, wherein the step of performing online training using each image of the object to be inspected and the data annotation results thereof as training samples to obtain the first security inspection image recognition model comprises:

   Performing online training on the second security inspection image recognition model pre-deployed at each recognition terminal to obtain the first security inspection image recognition model; and/or

   Performing online training on the pre-trained third security inspection image recognition model to obtain the first security inspection image recognition model; and/or

   The untrained fourth security inspection image recognition model is trained online to obtain the first security inspection image recognition model.
8. The method according to claim 7, wherein the M images of the object to be inspected belong to at least one task type, and the step of performing online training using each image of the object to be inspected and its data annotation results as training samples to obtain a first security inspection image recognition model comprises:

   At least one image of the object to be inspected of the same task type and its data annotation results are used as training samples for online training to obtain a first security inspection image recognition model corresponding to the task type.
9. According to the method according to any one of claims 1 to 8, wherein the step of performing online training using each image of the object to be inspected and the data annotation result thereof as a training sample to obtain the first security inspection image recognition model comprises:

   When the number of the training samples is greater than or equal to a preset threshold, online training is automatically performed to obtain the first security inspection image recognition model.
10. An online training method for a security inspection image recognition model, used for a recognition terminal, includes:

    Obtain S images of the object to be inspected from the corresponding security inspection machine, where S is greater than or equal to 1;

    Determining a recognition result of each of the S images of the object to be inspected;

    At least one image of the object to be inspected having a specific recognition result is sent to an online terminal, wherein the online terminal is used to execute the method according to any one of claims 1 to 9 to obtain a first security inspection image recognition model.
11. The method according to claim 10, wherein the identification terminal is in communication with the security inspection machine, and the acquiring S images of the object to be inspected from the corresponding security inspection machine comprises:

    After the security inspection machine scans the inspected object to obtain each image of the object to be inspected, the image of the object to be inspected is acquired in real time.
12. The method according to claim 10, wherein the method further comprises:

    Acquire the first security inspection image recognition model from the online terminal;

    The first security inspection image recognition model is automatically deployed.
13. The method according to claim 12, wherein, before acquiring the first security inspection image recognition model from the online end, the method further comprises:

    Receive the model update instruction sent by the online end.
14. A security inspection image recognition method, used for an identification terminal, comprising:

    Obtain Q images of the object to be inspected from the corresponding security inspection machine, where Q is greater than or equal to 1;

    The Q images of the objects to be inspected are identified by using a first security inspection image recognition model to obtain a recognition result, wherein the first security inspection image recognition model is obtained according to the method according to any one of claims 1 to 13.
15. An online training system for a security inspection image recognition model includes N recognition terminals and online terminals, where N is greater than or equal to 1, wherein:

    Each of the N recognition terminals includes a recognition device, which is used to determine the recognition result of the image of the object to be inspected from the corresponding security inspection machine, and send at least one image of the object to be inspected with a specific recognition result to the online terminal;

    The online end includes a data management platform and an online training platform, wherein:

    The data management platform is used to receive the at least one image of the object to be inspected sent by the recognition device, and send the at least one image of the object to be inspected and the data annotation result to the online training platform;

    The online training platform is used to perform online training using the at least one image of the object to be inspected and the data annotation result as training samples to obtain a first security inspection image recognition model.
16. The online training system according to claim 15, wherein:

    The online training platform is also used to send the first security inspection image recognition model to the recognition device;

    The recognition device is used to receive and automatically deploy the first security inspection image recognition model.
17. An online training device for a security inspection image recognition model, comprising:

    The first image module is used to obtain M images with specific recognition results from N recognition terminals. Each recognition terminal is used to determine the recognition result of the image of the object to be inspected from the corresponding security inspection machine, and M and N are respectively greater than or equal to 1;

    A data annotation module, used to obtain a data annotation result of each of the M images of the object to be inspected;

    The online training module is used to perform online training using each image of the object to be inspected and its data annotation results as training samples to obtain a first security inspection image recognition model.
18. An online training device for a security inspection image recognition model, used for a recognition terminal, comprising:

    The second image module is used to obtain S images of the object to be inspected from the corresponding security inspection machine, where S is greater than or equal to 1;

    A first recognition module, used to determine the recognition result of each of the S images of the object to be detected;

    The third image module is used to send at least one image of the object to be inspected with a specific recognition result to an online end, wherein the online end is used to execute the method according to any one of claims 1 to 9 to obtain a first security inspection image recognition model.
19. A security inspection image recognition device, used for an identification terminal, comprising:

    A fourth image module, used to obtain Q images of the object to be inspected from the corresponding security inspection machine, where Q is greater than or equal to 1;

    The second recognition module is used to use a first security inspection image recognition model to recognize the Q images of the objects to be inspected to obtain a recognition result, wherein the first security inspection image recognition model is obtained according to the method described in any one of claims 1 to 13.
20. An electronic device, comprising:

    one or more processors;

    a storage device for storing one or more programs,

    When the one or more programs are executed by the one or more processors, the one or more processors are enabled to execute the method according to any one of claims 1 to 14.
21. A computer-readable storage medium stores executable instructions, which, when executed by a processor, cause the processor to execute the method according to any one of claims 1 to 14.
22. A computer program product comprising a computer program, the computer program being When executed by a processor, the method according to any one of claims 1 to 14 is implemented.