CN111739009A - Image detection method and device, electronic equipment and storage medium - Google Patents
Image detection method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN111739009A CN111739009A CN202010583010.8A CN202010583010A CN111739009A CN 111739009 A CN111739009 A CN 111739009A CN 202010583010 A CN202010583010 A CN 202010583010A CN 111739009 A CN111739009 A CN 111739009A
- Authority
- CN
- China
- Prior art keywords
- image
- detection
- detection result
- feature
- detected
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
- G06V10/446—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering using Haar-like filters, e.g. using integral image techniques
Abstract
The embodiment of the invention provides an image detection method, an image detection device, electronic equipment and a storage medium, wherein the method comprises the following steps: extracting various characteristics of an image to be detected; and inputting each feature into a corresponding image detection model to obtain a detection result corresponding to the feature, and determining the detection result of the image to be detected based on a plurality of detection results corresponding to the various features. In the embodiment of the invention, the detection result of the image to be detected can be determined through various characteristics, so that the influence of each characteristic on the image to be detected can be comprehensively considered when the detection result of the image to be detected is determined, and the image detection performance can be further improved.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to an image detection method and apparatus, an electronic device, and a storage medium.
Background
With the wide application of image detection algorithms, intelligent devices using image detection algorithms are also widely used. When the intelligent device is used for detecting the image, the image to be detected can be input into the intelligent device, then the intelligent device can recognize a characteristic of the image to be detected by adopting a built-in image characteristic library, and then the detection result of the image to be detected is determined based on the recognized characteristic of the image to be detected.
However, since different images or different regions of the same image have different types of features, and the different types of features have different expressive power when representing the image, if a single feature is used to represent the image, the detection results of detecting the image will be uneven, which is not beneficial to improving the performance of image detection.
Disclosure of Invention
An object of embodiments of the present invention is to provide an image detection method, an image detection apparatus, an electronic device, and a storage medium, so as to implement detection of an image based on multiple features and improve performance of image detection. The specific technical scheme is as follows:
in a first aspect, an embodiment of the present invention provides an image detection method, where the method includes:
extracting various characteristics of an image to be detected;
inputting each feature into a corresponding image detection model to obtain a detection result corresponding to the feature;
and determining the detection result of the image to be detected based on a plurality of detection results corresponding to the various characteristics.
Optionally, inputting each feature into a corresponding image detection model to obtain a detection result corresponding to the feature, where the method includes:
inputting the characteristics into a corresponding image detection model aiming at each characteristic, and monitoring whether the image detection model corresponding to the characteristics outputs a detection result within a corresponding time length threshold value or not;
if so, acquiring a detection result output within a preset time threshold corresponding to the characteristic;
determining the detection result of the image to be detected based on a plurality of detection results corresponding to the plurality of characteristics, including:
and determining the detection result of the image to be detected based on the detection results output within the preset time length thresholds.
Optionally, after each feature is input into the corresponding image detection model to obtain a detection result corresponding to the feature, the image detection method further includes:
counting the detection time length of each feature, wherein the detection time length is the time length from inputting the feature to the corresponding image detection model to obtaining the detection result corresponding to the feature;
for each feature, when the detection duration corresponding to the feature is less than or equal to the time threshold corresponding to the feature, taking the detection result corresponding to the feature as an effective detection result;
when the detection duration corresponding to the characteristic is greater than the time threshold corresponding to the characteristic, taking the detection result corresponding to the characteristic as an invalid detection result;
determining the detection result of the image to be detected based on a plurality of detection results corresponding to the plurality of characteristics, including:
and determining the detection result of the image to be detected based on a plurality of effective detection results in the plurality of detection results.
Optionally, after the feature is input into the corresponding image detection model and a detection result corresponding to the feature is obtained, the image detection method further includes:
inputting the multiple characteristics into a preset universal image detection model to obtain comprehensive detection results corresponding to the multiple characteristics, wherein the preset universal image detection model is obtained by training a training sample containing the multiple characteristics;
determining the detection result of the image to be detected based on a plurality of detection results corresponding to the plurality of characteristics, including:
and determining the detection result of the image to be detected based on the plurality of detection results and the comprehensive detection result.
Optionally, the detection result and the comprehensive detection result are the similarity between the image to be detected and any image or the probability of belonging to any classification;
determining the detection result of the image to be detected based on a plurality of detection results and a comprehensive detection result, comprising:
selecting an image with the highest similarity with the image to be detected as a target image corresponding to the image to be detected from the plurality of detection results and the comprehensive detection result;
alternatively, the first and second electrodes may be,
and selecting the classification corresponding to the highest probability from the plurality of detection results and the comprehensive detection result as the target classification corresponding to the image to be detected.
Optionally, the detection result and the comprehensive detection result are the similarity between the image to be detected and any image or the probability of belonging to any classification;
determining the detection result of the image to be detected based on a plurality of detection results and a comprehensive detection result, comprising:
weighting each detection result and the comprehensive detection result respectively based on preset weight values;
in the weighted detection results and the weighted comprehensive detection results, the image with the highest similarity with the image to be detected is a target image corresponding to the image to be detected; or selecting the classification corresponding to the highest probability as the target classification corresponding to the image to be detected.
Optionally, before inputting the multiple features into a preset general image detection model to obtain a comprehensive detection result corresponding to the multiple features, the image detection method further includes:
counting the detection time length of each feature to obtain a plurality of detection time lengths, wherein the detection time length is the time length from inputting the feature to the corresponding image detection model to obtaining the detection result corresponding to the feature;
comparing the detection durations to determine the maximum detection duration in the detection durations;
and when the maximum detection time length is greater than the total preset time length threshold value, inputting the multiple characteristics into a preset general image detection model to obtain a comprehensive detection result corresponding to the multiple characteristics, wherein the total preset time length threshold value is the total time length threshold value of the multiple characteristics.
In a second aspect, an embodiment of the present invention further provides an image detection apparatus, including:
the characteristic extraction module is used for extracting various characteristics of the image to be detected;
the first input module is used for inputting each feature into the corresponding image detection model to obtain a detection result corresponding to the feature;
and the detection result acquisition module is used for determining the detection result of the image to be detected based on a plurality of detection results corresponding to the various characteristics.
Optionally, the first input module is specifically configured to:
inputting the characteristics into a corresponding image detection model aiming at each characteristic, and monitoring whether the image detection model corresponding to the characteristics outputs a detection result within a corresponding time length threshold value or not; if so, acquiring a detection result output within a preset time threshold corresponding to the characteristic;
the detection result acquisition module is specifically configured to:
and determining the detection result of the image to be detected based on the detection results output within the preset time length thresholds.
Optionally, the image detection apparatus further includes:
the statistical module is used for counting the detection time length of each feature, wherein the detection time length is the time length from inputting the feature to the corresponding image detection model to obtaining the detection result corresponding to the feature;
the effective detection result acquisition module is used for taking the detection result corresponding to each feature as an effective detection result when the detection duration corresponding to the feature is less than or equal to the time threshold corresponding to the feature;
an invalid detection result obtaining module, configured to take a detection result corresponding to the feature as an invalid detection result when a detection duration corresponding to the feature is greater than a time threshold corresponding to the feature;
the detection result acquisition module is specifically configured to:
and determining the detection result of the image to be detected based on a plurality of effective detection results in the plurality of detection results.
Optionally, the image detection apparatus further includes:
the second input module is used for inputting the multiple characteristics into a preset universal image detection model to obtain comprehensive detection results corresponding to the multiple characteristics, wherein the preset universal image detection model is obtained by training a training sample containing the multiple characteristics;
the detection result acquisition module is specifically configured to:
and determining the detection result of the image to be detected based on the plurality of detection results and the comprehensive detection result.
Optionally, the detection result and the comprehensive detection result are the similarity between the image to be detected and any image or the probability of belonging to any classification;
the detection result acquisition module is specifically configured to:
selecting an image with the highest similarity with the image to be detected as a target image corresponding to the image to be detected from the plurality of detection results and the comprehensive detection result;
alternatively, the first and second electrodes may be,
and selecting the classification corresponding to the highest probability from the plurality of detection results and the comprehensive detection result as the target classification corresponding to the image to be detected.
Optionally, the detection result and the comprehensive detection result are the similarity between the image to be detected and any image or the probability of belonging to any classification;
the detection result acquisition module is specifically configured to:
weighting each detection result and the comprehensive detection result respectively based on preset weight values;
selecting an image with the highest similarity with the image to be detected as a target image corresponding to the image to be detected from the plurality of weighted detection results and the weighted comprehensive detection results; or selecting the classification corresponding to the highest probability as the target classification corresponding to the image to be detected.
Optionally, the image detection apparatus further includes:
the statistical module is used for counting the detection time length of each feature to obtain a plurality of detection time lengths, wherein the detection time length is the time length from the time when the feature is input into the corresponding image detection model to the time when the detection result corresponding to the feature is obtained;
the maximum detection duration determining module is used for comparing the detection durations and determining the maximum detection duration in the detection durations; and when the maximum detection time length is greater than a preset time length total threshold value, triggering a second input module to execute a step of inputting the multiple characteristics into a preset general image detection model to obtain a comprehensive detection result corresponding to the multiple characteristics, wherein the preset time length total threshold value is a total time length threshold value of the multiple characteristics.
In a third aspect, an embodiment of the present invention further provides an electronic device, including a processor and a machine-readable storage medium, where the machine-readable storage medium stores machine-executable instructions capable of being executed by the processor, and the processor is caused by the machine-executable instructions to: the steps of the image detection method provided by the first aspect are implemented.
In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the image detection method provided in the first aspect are implemented.
In a fifth aspect, embodiments of the present invention further provide a computer program product containing instructions, which when run on a computer, cause the computer to perform the steps of the image detection method provided in the first aspect.
In a sixth aspect, an embodiment of the present invention further provides a computer program, which when run on a computer, causes the computer to execute the steps of the image detection method provided in the first aspect.
According to the image detection method, the image detection device, the electronic equipment and the storage medium, provided by the embodiment of the invention, after the image to be detected is obtained, various characteristics of the image to be detected can be extracted; then, each feature is input into the corresponding image detection model to obtain the detection result corresponding to the feature, and when multiple detection results corresponding to multiple features are obtained, the detection result of the image to be detected can be determined based on the multiple detection results. Therefore, the detection result of the image to be detected can be determined through various characteristics, so that the influence of each characteristic on the image to be detected can be comprehensively considered when the detection result of the image to be detected is determined, and the image detection performance can be improved. Of course, it is not necessary for any product or method of practicing the invention to achieve all of the above-described advantages at the same time.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1a is a flowchart of a first implementation of an image detection method according to an embodiment of the present invention;
FIG. 1b is a schematic diagram of a feature extraction database and an image matching module in an electronic device applying the image detection method shown in FIG. 1 a;
FIG. 2 is a flowchart illustrating a second embodiment of an image detection method according to an embodiment of the present invention;
FIG. 3 is a flowchart illustrating a third exemplary embodiment of an image detection method according to an embodiment of the present invention;
FIG. 4 is a flowchart illustrating a fourth implementation manner of an image detection method according to an embodiment of the present invention;
FIG. 5 is a schematic structural diagram of an image detection apparatus according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In order to solve the problems in the prior art, embodiments of the present invention provide an image detection method, an image detection apparatus, an electronic device, and a storage medium, so as to implement detection of an image based on multiple features and improve the performance of image detection.
In the following, an image detection method according to an embodiment of the present invention is first described, as shown in fig. 1a, which is a flowchart of a first implementation manner of an image detection method according to an embodiment of the present invention, where the method may include:
s110, extracting various characteristics of an image to be detected;
s120, inputting each feature into a corresponding image detection model to obtain a detection result corresponding to the feature;
s130, determining the detection result of the image to be detected based on a plurality of detection results corresponding to the plurality of characteristics.
In some examples, when detecting an image to be detected, various features of the image to be detected may be extracted first.
When various features of the image to be detected are extracted, a feature extraction algorithm capable of extracting the features can be adopted for the features to be extracted, and the features are extracted from the image to be detected.
For example, assuming that the MD5 feature of the image to be detected is to be extracted, a feature extraction algorithm or computer code capable of extracting MD5 value may be used to extract the MD5 feature in the image to be detected, and it is understood that the feature extraction algorithm or computer code used herein to extract the MD5 feature may be the prior art and is not limited herein.
For another example, assuming that the haar feature of the image to be detected is to be extracted, a feature extraction algorithm or computer code capable of extracting the haar feature may be used to extract the haar feature of the image to be detected.
In some examples, as shown in fig. 1b, an electronic device to which an image detection method according to an embodiment of the present invention is applied may be provided with a feature extraction database in advance.
A plurality of feature extraction algorithms may be preset in the feature extraction database, for example, an MD5 feature extraction algorithm, a haar feature extraction algorithm, and other feature extraction algorithms may be set. And then, for example, the MD5 feature of the image to be detected extracted based on the MD5 feature extraction algorithm is input to an image detection model (for example, an MD5 feature matching sub-module) corresponding to the MD5 feature, and the haar feature of the image to be detected extracted based on the haar feature extraction algorithm is input to an image detection model (for example, a haar feature matching sub-module) corresponding to the haar feature, and so on, that is, the corresponding feature is extracted based on the corresponding feature extraction algorithm, and then the feature is input to the image detection model corresponding to the feature, so as to obtain an image detection result corresponding to the feature of the image to be detected.
After obtaining the multiple features of the image to be detected, in order to comprehensively consider the multiple features when detecting the image to be detected, each feature can be input into the corresponding image detection model to obtain the detection result corresponding to the feature. Thus, a plurality of detection results corresponding to the plurality of characteristics can be obtained.
In some examples, each feature herein may correspond to one image detection model, and may also correspond to a plurality of image detection models. When each feature corresponds to a plurality of image detection models, the detection result output by each image detection model corresponding to the feature can be obtained, and then the detection results output by all the image detection models corresponding to the feature can be used as the detection result corresponding to the feature.
In still other examples, the image detection model corresponding to each feature may be obtained by training a preset image detection model using an image sample including the feature. It is understood that the process of training the preset image detection model may be a process of training by using the prior art, and is not described herein again.
In still other examples, as shown in fig. 1b, an image matching module may also be preset on an electronic device to which the image detection method according to the embodiment of the present invention is applied. The image matching module is used for detecting a plurality of features extracted through the feature extraction database.
The image matching module may be preset with a plurality of feature matching sub-modules, for example, an MD5 feature matching sub-module, a haar feature matching sub-module, and other feature matching sub-modules. Different image detection models are respectively arranged in different feature matching sub-modules in the image matching module.
After the various features of the image to be detected are obtained through the different feature extraction algorithms, the features can be input into the corresponding image matching sub-module for each feature, and the detection result corresponding to the features can be obtained.
For example, after the MD5 feature of the image to be detected is obtained through the MD5 feature extraction algorithm, the MD5 feature may be input into the MD5 feature matching sub-module, so that the detection result corresponding to the MD5 feature may be obtained. After the haar feature of the image to be detected is obtained through the haar feature extraction algorithm, the haar feature can be input into the haar feature matching submodule, and therefore a detection result corresponding to the haar feature can be obtained.
After the detection result corresponding to each of the various features is obtained, the detection result of the image to be detected can be determined based on the plurality of detection results corresponding to the plurality of features.
In some examples, the detection result output by the image detection model may be a probability that the image to be detected is a certain image or images. Or the probability that each region in the image to be detected is a certain or a certain image.
According to the image detection method provided by the embodiment of the invention, after the image to be detected is obtained, various characteristics of the image to be detected can be extracted; then, each feature is input into the corresponding image detection model to obtain the detection result corresponding to the feature, and when multiple detection results corresponding to multiple features are obtained, the detection result of the image to be detected can be determined based on the multiple detection results. Therefore, the detection result of the image to be detected can be determined through various characteristics, so that the influence of each characteristic on the image to be detected can be comprehensively considered when the detection result of the image to be detected is determined, and the image detection performance can be improved.
On the basis of the image detection method shown in fig. 1a, an embodiment of the present invention further provides a possible implementation manner, as shown in fig. 2, which is a flowchart of a second implementation manner of the image detection method according to the embodiment of the present invention, and the method may include:
s210, extracting various characteristics of the image to be detected;
s220, inputting each feature into a corresponding image detection model to obtain a detection result corresponding to the feature;
s230, counting the detection time length of each feature, wherein the detection time length is the time length from inputting the feature to the corresponding image detection model to obtaining the detection result corresponding to the feature;
s240, regarding each feature, when the detection duration corresponding to the feature is less than or equal to the time threshold corresponding to the feature, taking the detection result corresponding to the feature as an effective detection result; when the detection duration corresponding to the characteristic is greater than the time threshold corresponding to the characteristic, taking the detection result corresponding to the characteristic as an invalid detection result;
and S250, determining the detection result of the image to be detected based on a plurality of effective detection results in the plurality of detection results.
In some examples, in order to avoid too long detection time when detecting the image to be detected based on one feature, the detection efficiency is reduced. After a feature is input into a corresponding image detection model, counting the detection time length between the time when the feature is input into the corresponding image detection model and the time when the detection result corresponding to the feature is obtained, taking the detection result corresponding to the feature as an effective detection result when the detection time length corresponding to the feature is less than or equal to the time threshold corresponding to the feature, and taking the detection result corresponding to the feature as an ineffective detection result when the detection time length corresponding to the feature is greater than the time threshold corresponding to the feature; and then determining the detection result of the image to be detected based on a plurality of effective detection results in the plurality of detection results.
In some examples, when counting the detection time length of each feature, when inputting the feature to the corresponding image detection model, the time of input may be recorded, when outputting the detection result corresponding to the feature, the time of output may be recorded, and then based on the time of input and the time of output corresponding to the feature, the detection time length of the feature may be counted.
As can be understood, when the detection time of an image detection model for a feature is long, the image detection model can be considered to be difficult to detect the feature, and therefore the reliability of the obtained detection result will be low.
By setting a time threshold for each feature, when the detection duration corresponding to the feature is greater than the time threshold corresponding to the feature, it may be determined that the reliability of the detection result corresponding to the feature is lower than a reliability threshold. Therefore, the detection result corresponding to the feature may be determined as an invalid detection result, and conversely, the detection result may be determined as a valid detection result.
And finally, determining the detection result of the image to be detected based on a plurality of effective detection results in the plurality of detection results, so that the reliability of the finally determined detection result of the image to be detected is higher. Thereby, the performance of image detection can be further improved.
In some examples, the detection result output by the image detection model may be a probability that the image to be detected is a certain image or images.
In contrast, when determining the detection result of the to-be-detected image based on the plurality of valid detection results, the probabilities in the plurality of valid detection results may be multiplied by the respective weights, so as to obtain a plurality of weighted valid detection results. And then selecting the effective detection result with the highest probability from the effective detection results after the weighting processing as the detection result of the image to be detected.
In still other examples, the detection result output by the image detection model may also be the probability that each region in the image to be detected is a certain or certain image.
In this regard, when determining the detection result of the image to be detected based on the plurality of valid detection results among the plurality of detection results, the valid detection result having the highest probability may be selected as the detection result of the region among the plurality of valid detection results corresponding to the region for each region. Therefore, the detection result of each region in the image to be detected can be obtained, and the detection of the image to be detected is more detailed.
In still other examples, after obtaining the detection result of each region, the probability in the detection result of each region may be multiplied by the weight of each region in the image to be detected to obtain the detection result of each region after weighting processing, and finally, the detection result with the highest probability from the detection results of each region after weighting processing may be selected as the detection result of the image to be detected.
In some examples, when an image detection method according to an embodiment of the present invention is used to classify an image to be detected, that is, the image detection method is used to detect to which category the image to be detected belongs. The detection results corresponding to the features may be probabilities of the category to which the image to be detected belongs. Then, when determining the detection result of the image to be detected based on a plurality of valid detection results in the plurality of detection results, the category corresponding to the highest probability can be selected as the category of the image to be detected in the valid probabilities.
In still other examples, when an image detection method according to an embodiment of the present invention is used to detect whether an image to be detected includes an object, a plurality of detection results corresponding to the plurality of features may be probabilities that the image to be detected includes the object. When the detection result of the image to be detected is determined based on a plurality of effective detection results in the plurality of detection results, the target corresponding to the highest probability can be selected as the target contained in the image to be detected in the effective probabilities.
On the basis of the image detection method shown in fig. 1a, an embodiment of the present invention further provides a possible implementation manner, as shown in fig. 3, which is a flowchart of a third implementation manner of the image detection method according to the embodiment of the present invention, and the method may include:
s310, extracting various characteristics of the image to be detected;
s320, inputting each feature into a corresponding image detection model, and monitoring whether the image detection model corresponding to the feature outputs a detection result within a corresponding time length threshold value; if yes, go to step S330;
s330, obtaining a detection result output within a preset time length threshold corresponding to the characteristic;
s340, determining the detection result of the image to be detected based on the detection results output within the preset time length thresholds.
In some examples, if waiting for each image detection model to output the detection result corresponding to each feature may cause the overall detection duration of the image detection process to be too long, in order to avoid the overall detection duration of the image detection process to be too long, a duration threshold may be set for each image detection model, and when each feature is input into the corresponding image detection model, it may be monitored whether the image detection model corresponding to the feature outputs the detection result within the corresponding duration threshold.
In some examples, when monitoring whether the image detection model corresponding to each feature outputs the detection result within the corresponding time length threshold, when the feature is input into the corresponding image detection model, the input time may be recorded, then the input time may be added to the corresponding time length threshold to obtain the output cutoff time corresponding to the feature, and then whether the image detection model corresponding to the feature outputs the detection result before the corresponding output cutoff time may be monitored.
In still other examples, when monitoring whether the image detection model corresponding to each feature outputs the detection result within the corresponding time length threshold, the method may start to count from zero when the feature is input into the corresponding image detection model, and then monitor whether the counted time length reaches the time length threshold corresponding to the feature, or when the feature is input into the corresponding image detection model, count down according to the time length threshold corresponding to the feature, and then monitor whether the image detection model corresponding to the feature outputs the detection result when the count down is finished.
In this way, when a plurality of image detection models output detection results within the respective corresponding preset time length thresholds, the detection results output within the preset time length thresholds can be obtained; and then determining the detection result of the image to be detected based on the detection results output within the preset time length thresholds.
By the embodiment of the invention, when the time length of the image detection model for detecting the detection result corresponding to the characteristic is too long, the detection result can be directly abandoned, and the detection result of the image to be detected is determined only according to the detection result output in each preset time length. Therefore, the overtime management of the whole detection process can be realized, and the overlong total detection time length of the detection process is avoided, so that the detection efficiency of the detection process is improved, and the performance of image detection is also improved.
On the basis of the image detection method shown in fig. 1a, an embodiment of the present invention further provides a possible implementation manner, as shown in fig. 4, which is a flowchart of a fourth implementation manner of the image detection method according to the embodiment of the present invention, and the method may include:
s410, extracting various characteristics of the image to be detected;
s420, inputting each feature into a corresponding image detection model to obtain a detection result corresponding to the feature;
s430, inputting the multiple characteristics into a preset universal image detection model to obtain comprehensive detection results corresponding to the multiple characteristics, wherein the preset universal image detection model is obtained by training a training sample containing the multiple characteristics;
s440, determining the detection result of the image to be detected based on the plurality of detection results and the comprehensive detection result.
In some examples, after the detection result corresponding to each feature is obtained, in order to adjust the final detection result of the image to be detected, multiple features may be input into a preset general image detection model, so as to obtain a comprehensive detection result corresponding to the multiple features.
In some examples, the integrated detection result may include a plurality of probabilities, each of which may represent a probability that the image to be detected is a certain or a certain image, or may represent a probability that each region in the image to be detected is a certain or a certain image.
After the comprehensive detection result is obtained, the plurality of detection results and the comprehensive detection result can be integrated, so that the detection result of the image to be detected can be determined.
In some examples, the detection result and the comprehensive detection result are the similarity between the image to be detected and any image or the probability of belonging to any classification;
in this regard, in some examples, determining a detection result of an image to be detected based on a plurality of detection results and a comprehensive detection result includes:
in the plurality of detection results and the comprehensive detection result, the image with the highest similarity with the image to be detected is a target image corresponding to the image to be detected; or selecting the classification corresponding to the highest probability from the plurality of detection results and the comprehensive detection result as the target classification corresponding to the image to be detected.
And weighting the detection result and the comprehensive detection result based on a preset weight, and selecting the target image or the target classification corresponding to the highest probability from the plurality of weighted detection results and the plurality of weighted comprehensive detection results as the target image or the target classification corresponding to the image to be detected.
In still other examples, determining a detection result of the image to be detected based on the plurality of detection results and the integrated detection result includes:
weighting each detection result and the comprehensive detection result respectively based on preset weight values;
selecting an image with the highest similarity with the image to be detected as a target image corresponding to the image to be detected from the plurality of weighted detection results and the weighted comprehensive detection results; or selecting the classification corresponding to the highest probability as the target classification corresponding to the image to be detected.
Therefore, the detection result of the image to be detected can be adjusted in advance through the comprehensive detection result, so that the flexibility of the image detection method applying the embodiment of the invention can be improved.
In some examples, in addition to setting a duration threshold for each image detection model, embodiments of the present invention also provide a possible implementation manner, for example, setting a total duration threshold for all image detection models, that is, a preset total duration threshold.
In some examples, in order to avoid adjusting the detection results of all the images to be detected, in the embodiment of the present invention, after the detection results corresponding to various features are obtained, the detection time duration from the time when each feature is input to the corresponding image detection model to the time when the detection result corresponding to the feature is obtained may be counted to obtain a plurality of detection time durations; then comparing the plurality of detection durations, and determining the maximum detection duration from the plurality of detection durations; when the maximum detection time is greater than the total threshold of the preset time, step S440 is executed again to determine the detection result of the image to be detected based on the plurality of detection results and the comprehensive detection result.
In this way, the detection results corresponding to various characteristics output when the detection of the whole is too long can be adjusted. When the overall detection duration is too long, it can be considered that at least one image detection model has difficulty in detecting one feature in all the image detection models, the detection process is relatively slow, and then detection results with lower accuracy exist in detection results corresponding to various features output by each image detection model, so that the detection results corresponding to the various features can be adjusted by adopting the comprehensive detection result, and then the detection result of the image to be detected is determined based on the adjusted detection result, so that the finally determined detection result of the image to be detected is more accurate.
Corresponding to the above method embodiment, an embodiment of the present invention further provides an image detection apparatus, as shown in fig. 5, which is a schematic structural diagram of an image detection apparatus according to an embodiment of the present invention, and the apparatus may include:
a feature extraction module 510, configured to extract multiple features of an image to be detected;
a first input module 520, configured to input each feature into a corresponding image detection model to obtain a detection result corresponding to the feature;
a detection result obtaining module 530, configured to determine a detection result of the image to be detected based on a plurality of detection results corresponding to the plurality of features.
The image detection device provided by the embodiment of the invention can extract various characteristics of the image to be detected after the image to be detected is obtained; then, each feature is input into the corresponding image detection model to obtain the detection result corresponding to the feature, and when multiple detection results corresponding to multiple features are obtained, the detection result of the image to be detected can be determined based on the multiple detection results. Therefore, the detection result of the image to be detected can be determined through various characteristics, so that the influence of each characteristic on the image to be detected can be comprehensively considered when the detection result of the image to be detected is determined, and the image detection performance can be improved.
In some examples, the first input module 520 is specifically configured to:
inputting the characteristics into a corresponding image detection model aiming at each characteristic, and monitoring whether the image detection model corresponding to the characteristics outputs a detection result within a corresponding time length threshold value or not; if so, acquiring a detection result output within a preset time threshold corresponding to the characteristic;
the detection result obtaining module 530 is specifically configured to:
and determining the detection result of the image to be detected based on the detection results output within the preset time length thresholds.
In some examples, the image detection apparatus further includes:
the statistical module is used for counting the detection time length of each feature, wherein the detection time length is the time length from inputting the feature to the corresponding image detection model to obtaining the detection result corresponding to the feature;
the effective detection result acquisition module is used for taking the detection result corresponding to each feature as an effective detection result when the detection duration corresponding to the feature is less than or equal to the time threshold corresponding to the feature;
an invalid detection result obtaining module, configured to take a detection result corresponding to the feature as an invalid detection result when a detection duration corresponding to the feature is greater than a time threshold corresponding to the feature;
the detection result obtaining module 530 is specifically configured to:
and determining the detection result of the image to be detected based on a plurality of effective detection results in the plurality of detection results.
In some examples, the image detection apparatus further includes:
the second input module is used for inputting the multiple characteristics into a preset universal image detection model to obtain comprehensive detection results corresponding to the multiple characteristics, wherein the preset universal image detection model is obtained by training a training sample containing the multiple characteristics;
the detection result obtaining module 530 is specifically configured to:
and determining the detection result of the image to be detected based on the plurality of detection results and the comprehensive detection result.
In some examples, the detection result and the comprehensive detection result are the similarity between the image to be detected and any image or the probability of belonging to any classification; the detection result obtaining module 530 is specifically configured to:
selecting an image with the highest similarity with the image to be detected as a target image corresponding to the image to be detected from the plurality of detection results and the comprehensive detection result;
alternatively, the first and second electrodes may be,
and selecting the classification corresponding to the highest probability from the plurality of detection results and the comprehensive detection result as the target classification corresponding to the image to be detected.
In some examples, the detection result obtaining module 530 is specifically configured to:
weighting each detection result and the comprehensive detection result respectively based on preset weight values;
selecting an image with the highest similarity with the image to be detected as a target image corresponding to the image to be detected from the plurality of weighted detection results and the weighted comprehensive detection results; or selecting the classification corresponding to the highest probability as the target classification corresponding to the image to be detected.
In some examples, the image detection apparatus further includes:
the statistical module is used for counting the detection time length of each feature to obtain a plurality of detection time lengths, wherein the detection time length is the time length from the time when the feature is input into the corresponding image detection model to the time when the detection result corresponding to the feature is obtained;
the maximum detection duration determining module is used for comparing the detection durations and determining the maximum detection duration in the detection durations; and when the maximum detection time length is larger than the total threshold of the preset time length, triggering a second input module to execute the step of inputting the multiple characteristics into a preset general image detection model to obtain the comprehensive detection result corresponding to the multiple characteristics.
An electronic device is further provided, as shown in fig. 6, and is a schematic structural diagram of an electronic device to which an image detection method according to an embodiment of the present invention is applied, where the electronic device may include a processor 601 and a machine-readable storage medium 602, where the machine-readable storage medium 602 stores machine-executable instructions that can be executed by the processor 601, and the processor 601 is caused by the machine-executable instructions to: the steps of implementing the image detection method shown in any of the above embodiments may be implemented, for example, as follows:
extracting various characteristics of an image to be detected;
inputting each feature into a corresponding image detection model to obtain a detection result corresponding to the feature;
and determining the detection result of the image to be detected based on a plurality of detection results corresponding to the various characteristics.
The machine-readable storage medium 602 may include a Random Access Memory (RAM) and may also include a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor 601 may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.
According to the electronic equipment provided by the embodiment of the invention, after the image to be detected is obtained, various characteristics of the image to be detected can be extracted; then, each feature is input into the corresponding image detection model to obtain the detection result corresponding to the feature, and when multiple detection results corresponding to multiple features are obtained, the detection result of the image to be detected can be determined based on the multiple detection results. Therefore, the detection result of the image to be detected can be determined through various characteristics, so that the influence of each characteristic on the image to be detected can be comprehensively considered when the detection result of the image to be detected is determined, and the image detection performance can be improved.
An embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the image detection method shown in any of the above embodiments are implemented, for example, the following steps may be implemented:
extracting various characteristics of an image to be detected;
inputting each feature into a corresponding image detection model to obtain a detection result corresponding to the feature;
and determining the detection result of the image to be detected based on a plurality of detection results corresponding to the various characteristics.
The computer-readable storage medium provided by the embodiment of the invention can be used for extracting various characteristics of an image to be detected after the image to be detected is obtained; then, each feature is input into the corresponding image detection model to obtain the detection result corresponding to the feature, and when multiple detection results corresponding to multiple features are obtained, the detection result of the image to be detected can be determined based on the multiple detection results. Therefore, the detection result of the image to be detected can be determined through various characteristics, so that the influence of each characteristic on the image to be detected can be comprehensively considered when the detection result of the image to be detected is determined, and the image detection performance can be improved.
Embodiments of the present invention further provide a computer program product including instructions, which when run on a computer, cause the computer to perform the steps of an image detection method shown in any of the above embodiments, for example, the following steps may be performed:
extracting various characteristics of an image to be detected;
inputting each feature into a corresponding image detection model to obtain a detection result corresponding to the feature;
and determining the detection result of the image to be detected based on a plurality of detection results corresponding to the various characteristics.
The computer program product containing the instructions provided by the embodiment of the invention can extract various characteristics of the image to be detected after the image to be detected is obtained; then, each feature is input into the corresponding image detection model to obtain the detection result corresponding to the feature, and when multiple detection results corresponding to multiple features are obtained, the detection result of the image to be detected can be determined based on the multiple detection results. Therefore, the detection result of the image to be detected can be determined through various characteristics, so that the influence of each characteristic on the image to be detected can be comprehensively considered when the detection result of the image to be detected is determined, and the image detection performance can be improved.
Embodiments of the present invention further provide a computer program, which when running on a computer, causes the computer to execute the steps of an image detection method shown in any of the above embodiments, for example, the following steps may be executed:
extracting various characteristics of an image to be detected;
inputting each feature into a corresponding image detection model to obtain a detection result corresponding to the feature;
and determining the detection result of the image to be detected based on a plurality of detection results corresponding to the various characteristics.
According to the computer program provided by the embodiment of the invention, after the image to be detected is obtained, various characteristics of the image to be detected can be extracted; then, each feature is input into the corresponding image detection model to obtain the detection result corresponding to the feature, and when multiple detection results corresponding to multiple features are obtained, the detection result of the image to be detected can be determined based on the multiple detection results. Therefore, the detection result of the image to be detected can be determined through various characteristics, so that the influence of each characteristic on the image to be detected can be comprehensively considered when the detection result of the image to be detected is determined, and the image detection performance can be improved.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.
Claims (16)
1. An image detection method, characterized in that the method comprises:
extracting various characteristics of an image to be detected;
inputting each feature into a corresponding image detection model to obtain a detection result corresponding to the feature;
and determining the detection result of the image to be detected based on a plurality of detection results corresponding to the plurality of characteristics.
2. The method of claim 1, wherein inputting each feature into a corresponding image detection model to obtain a detection result corresponding to the feature comprises:
inputting the characteristics into a corresponding image detection model aiming at each characteristic, and monitoring whether the image detection model corresponding to the characteristics outputs a detection result within a corresponding time length threshold value or not;
if so, acquiring a detection result output within a preset time threshold corresponding to the characteristic;
the determining the detection result of the image to be detected based on the plurality of detection results corresponding to the plurality of characteristics comprises:
and determining the detection result of the image to be detected based on the detection results output within the preset time length thresholds.
3. The method according to claim 1, wherein after inputting each feature into the corresponding image detection model and obtaining the detection result corresponding to the feature, the method further comprises:
counting the detection time length of each feature, wherein the detection time length is the time length from inputting the feature to the corresponding image detection model to obtaining the detection result corresponding to the feature;
for each feature, when the detection duration corresponding to the feature is less than or equal to the time threshold corresponding to the feature, taking the detection result corresponding to the feature as an effective detection result;
when the detection duration corresponding to the characteristic is greater than the time threshold corresponding to the characteristic, taking the detection result corresponding to the characteristic as an invalid detection result;
the determining the detection result of the image to be detected based on the plurality of detection results corresponding to the plurality of characteristics comprises:
and determining the detection result of the image to be detected based on a plurality of effective detection results in the plurality of detection results.
4. The method according to claim 1, wherein after inputting the feature into the corresponding image detection model and obtaining the detection result corresponding to the feature, the method further comprises:
inputting the multiple features into a preset general image detection model to obtain a comprehensive detection result corresponding to the multiple features, wherein the preset general image detection model is obtained by training a training sample containing the multiple features;
the determining the detection result of the image to be detected based on the plurality of detection results corresponding to the plurality of characteristics comprises:
and determining the detection result of the image to be detected based on the plurality of detection results and the comprehensive detection result.
5. The method according to claim 4, wherein the detection result and the comprehensive detection result are the similarity between the image to be detected and any image or the probability of belonging to any classification;
determining the detection result of the image to be detected based on the plurality of detection results and the comprehensive detection result, comprising:
selecting an image with the highest similarity with the image to be detected from the plurality of detection results and the comprehensive detection result as a target image corresponding to the image to be detected;
alternatively, the first and second electrodes may be,
and selecting the classification corresponding to the highest probability from the plurality of detection results and the comprehensive detection result as the target classification corresponding to the image to be detected.
6. The method according to claim 4, wherein the detection result and the comprehensive detection result are the similarity between the image to be detected and any image or the probability of belonging to any classification;
determining the detection result of the image to be detected based on the plurality of detection results and the comprehensive detection result, comprising:
weighting each detection result and the comprehensive detection result respectively based on preset weight values;
selecting an image with the highest similarity with the image to be detected from the plurality of weighted detection results and the weighted comprehensive detection results as a target image corresponding to the image to be detected; or selecting the classification corresponding to the highest probability as the target classification corresponding to the image to be detected.
7. The method according to claim 4, wherein before the inputting the plurality of features into a preset general image detection model to obtain the comprehensive detection result corresponding to the plurality of features, the method further comprises:
counting the detection time length of each feature to obtain a plurality of detection time lengths, wherein the detection time length is the time length from inputting the feature to the corresponding image detection model to obtaining the detection result corresponding to the feature;
comparing the detection durations to determine the maximum detection duration in the detection durations;
and when the maximum detection time length is greater than a preset time length total threshold value, executing the step of inputting the multiple characteristics into a preset general image detection model to obtain a comprehensive detection result corresponding to the multiple characteristics, wherein the preset time length total threshold value is the total time length threshold value of the multiple characteristics.
8. An image detection apparatus, characterized in that the apparatus comprises:
the characteristic extraction module is used for extracting various characteristics of the image to be detected;
the first input module is used for inputting each feature into the corresponding image detection model to obtain a detection result corresponding to the feature;
and the detection result acquisition module is used for determining the detection result of the image to be detected based on a plurality of detection results corresponding to the plurality of characteristics.
9. The apparatus of claim 8, wherein the first input module is specifically configured to:
inputting the characteristics into a corresponding image detection model aiming at each characteristic, and monitoring whether the image detection model corresponding to the characteristics outputs a detection result within a corresponding time length threshold value or not; if so, acquiring a detection result output within a preset time threshold corresponding to the characteristic;
the detection result obtaining module is specifically configured to:
and determining the detection result of the image to be detected based on the detection results output within the preset time length thresholds.
10. The apparatus of claim 8, further comprising:
the statistical module is used for counting the detection time length of each feature, wherein the detection time length is the time length from inputting the feature into the corresponding image detection model to obtaining the detection result corresponding to the feature;
the effective detection result acquisition module is used for taking the detection result corresponding to each feature as an effective detection result when the detection duration corresponding to the feature is less than or equal to the time threshold corresponding to the feature;
an invalid detection result obtaining module, configured to take a detection result corresponding to the feature as an invalid detection result when a detection duration corresponding to the feature is greater than a time threshold corresponding to the feature;
the detection result obtaining module is specifically configured to:
and determining the detection result of the image to be detected based on a plurality of effective detection results in the plurality of detection results.
11. The apparatus of claim 8, further comprising:
the second input module is used for inputting the multiple features into a preset universal image detection model to obtain a comprehensive detection result corresponding to the multiple features, wherein the preset universal image detection model is obtained by training a training sample containing the multiple features;
the detection result obtaining module is specifically configured to:
and determining the detection result of the image to be detected based on the plurality of detection results and the comprehensive detection result.
12. The apparatus according to claim 11, wherein the detection result and the comprehensive detection result are the similarity between the image to be detected and any image or the probability of belonging to any classification;
the detection result obtaining module is specifically configured to:
selecting an image with the highest similarity with the image to be detected from the plurality of detection results and the comprehensive detection result as a target image corresponding to the image to be detected;
alternatively, the first and second electrodes may be,
and selecting the classification corresponding to the highest probability from the plurality of detection results and the comprehensive detection result as the target classification corresponding to the image to be detected.
13. The apparatus according to claim 11, wherein the detection result and the comprehensive detection result are the similarity between the image to be detected and any image or the probability of belonging to any classification;
the detection result obtaining module is specifically configured to:
weighting each detection result and the comprehensive detection result respectively based on preset weight values;
selecting an image with the highest similarity with the image to be detected from the plurality of weighted detection results and the weighted comprehensive detection results as a target image corresponding to the image to be detected; or selecting the classification corresponding to the highest probability as the target classification corresponding to the image to be detected.
14. The apparatus of claim 11, further comprising:
the statistical module is used for counting the detection time length of each feature to obtain a plurality of detection time lengths, wherein the detection time length is the time length from the time when the feature is input into the corresponding image detection model to the time when the detection result corresponding to the feature is obtained;
the maximum detection duration determining module is used for comparing the detection durations and determining the maximum detection duration in the detection durations; and when the maximum detection time length is greater than a preset time length total threshold value, triggering the second input module to execute the step of inputting the multiple features into a preset general image detection model to obtain a comprehensive detection result corresponding to the multiple features, wherein the preset time length total threshold value is the total time length threshold value of the multiple features.
15. An electronic device comprising a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor, the processor being caused by the machine-executable instructions to: carrying out the process steps of any one of claims 1 to 7.
16. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the method steps of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010583010.8A CN111739009A (en) | 2020-06-23 | 2020-06-23 | Image detection method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010583010.8A CN111739009A (en) | 2020-06-23 | 2020-06-23 | Image detection method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111739009A true CN111739009A (en) | 2020-10-02 |
Family
ID=72650743
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010583010.8A Pending CN111739009A (en) | 2020-06-23 | 2020-06-23 | Image detection method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111739009A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113723226A (en) * | 2021-08-13 | 2021-11-30 | 浙江大华技术股份有限公司 | Mobile stall detection method and device, electronic equipment and storage medium |
-
2020
- 2020-06-23 CN CN202010583010.8A patent/CN111739009A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113723226A (en) * | 2021-08-13 | 2021-11-30 | 浙江大华技术股份有限公司 | Mobile stall detection method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110163300B (en) | Image classification method and device, electronic equipment and storage medium | |
| CN111767783A (en) | Behavior detection method, behavior detection device, model training method, model training device, electronic equipment and storage medium | |
| CN109426785B (en) | Human body target identity recognition method and device | |
| CN109672936B (en) | Method and device for determining video evaluation set and electronic equipment | |
| CN110728258B (en) | Step detection method and system based on front-back frame connected domain matching | |
| CN112883782B (en) | Method, device, equipment and storage medium for identifying putting behaviors | |
| CN110928862A (en) | Data cleaning method, data cleaning apparatus, and computer storage medium | |
| CN111666907B (en) | Method, device and server for identifying object information in video | |
| CN111368143A (en) | Video similarity retrieval method and device, electronic equipment and storage medium | |
| CN109410198B (en) | Time sequence action detection method, device and equipment | |
| CN111739009A (en) | Image detection method and device, electronic equipment and storage medium | |
| CN113221601A (en) | Character recognition method, device and computer readable storage medium | |
| CN111064719A (en) | Method and device for detecting abnormal downloading behavior of file | |
| CN112330063B (en) | Equipment fault prediction method, equipment fault prediction device and computer readable storage medium | |
| CN116740586A (en) | Hail identification method, hail identification device, electronic equipment and computer readable storage medium | |
| CN110866831A (en) | Asset activity level determination method and device and server | |
| CN111738173B (en) | Video clip detection method and device, electronic equipment and storage medium | |
| CN112784691B (en) | Target detection model training method, target detection method and device | |
| CN114020905A (en) | Text classification external distribution sample detection method, device, medium and equipment | |
| CN111079466A (en) | Vehicle identification method and device, electronic equipment and storage medium | |
| CN111507238B (en) | Face data screening method and device and electronic equipment | |
| CN111667021B (en) | Front-end performance problem detection method based on artificial intelligence | |
| CN112199998B (en) | Face recognition method, device, equipment and medium | |
| CN110113228B (en) | Network connection detection method and device | |
| CN110633311B (en) | Data processing method, device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |