CN115713484A - Industrial defect sample detection method and system based on image representation - Google Patents
Industrial defect sample detection method and system based on image representation Download PDFInfo
- Publication number
- CN115713484A CN115713484A CN202211279714.1A CN202211279714A CN115713484A CN 115713484 A CN115713484 A CN 115713484A CN 202211279714 A CN202211279714 A CN 202211279714A CN 115713484 A CN115713484 A CN 115713484A
- Authority
- CN
- China
- Prior art keywords
- sample
- image representation
- positive
- detected
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Image Analysis (AREA)
Abstract
The invention provides an image representation-based industrial defect sample detection method and system, wherein the method comprises the following steps: acquiring an image representation model training data set, and training a pre-constructed double-tower model by using the image representation model training data set to obtain an image representation model; acquiring a standard positive sample data set, inputting the standard positive sample data set into an image representation model, and acquiring a standard positive sample characteristic vector set; inputting the sample to be detected into an image representation model to obtain a characteristic vector of the sample to be detected; the method solves the problems that the conventional industrial defect detection method based on machine vision has high requirements on the number of defect samples and high labeling requirements.
Description
Technical Field
The present invention relates to the field of industrial defect detection.
Background
The industrial defect detection is an indispensable link in the industrial production and manufacturing process and plays an important role in controlling the product quality. At present, most industrial manufacturers still adopt a manual mode to detect industrial defects. However, manual detection faces the problems of strong subjectivity, low efficiency, high cost and the like. The industrial defect detection method based on machine vision can overcome the subjectivity of manual detection through a standardized processing flow, avoid secondary damage to the surface of an industrial part and become an important solution for industrial automatic detection. However, the existing industrial defect detection method based on machine vision usually implements detection of defect targets and identification of defect types by learning a large number of industrial defect samples, and thus has high requirements on the number and labeling of defect samples. The patent provides an industrial defect detection method and system based on image representation, which can effectively overcome the defects of high requirement on the number of defect samples and high requirement on labeling in the conventional method and realize industrial defect detection under the condition of few samples.
Disclosure of Invention
The present invention is directed to solving, at least to some extent, one of the technical problems in the related art.
Therefore, the invention aims to provide an industrial defect sample detection method based on image representation, which is used for realizing industrial defect detection under a small number of samples.
In order to achieve the above object, a first aspect of the present invention provides an image representation-based industrial defect sample detection method, including:
acquiring an image representation model training data set, and training a pre-constructed double-tower model by using the image representation model training data set to obtain an image representation model;
acquiring a standard positive sample data set, inputting the standard positive sample data set into the image representation model, and acquiring a standard positive sample characteristic vector set;
obtaining a sample to be detected, inputting the sample to be detected into the image representation model to obtain a characteristic vector of the sample to be detected;
and determining the detection result of the sample to be detected according to the sample characteristic vector to be detected and the standard positive sample characteristic vector set.
In addition, the method for detecting the industrial defect sample based on the image representation according to the embodiment of the invention can also have the following additional technical characteristics:
further, in one embodiment of the present invention, the acquiring an image representation model training data set includes:
selecting a preset number of positive samples and negative samples at different angles according to the types of the industrial parts to be inspected, constructing a positive and negative sample pair, and recording the positive and negative sample pair
Wherein
Is a positive sample of the sample to be tested,
for negative examples, a training dataset of image representation models is formed
Further, in an embodiment of the present invention, the constructing a double-tower model, and training the double-tower model by using the image representation model training dataset to obtain an image representation model includes:
training a dataset D using the image representation model 1 Constructing training batch data, each training batch data consisting of N positive and negative sample pairs
Composition is carried out;
constructing a double-tower model by using ResNet as an original image representation model; the double-tower model is formed by positive and negative sample pairs
As input, the image representation model of the two shared parameters is respectively input to extract the characteristic vector
And optimized using the following loss function:
wherein N is the total number of positive and negative sample pairs of the current batch, l i The loss function for the ith positive and negative sample pair is shown as follows:
wherein, N is the total number of the positive and negative sample pairs of the current batch; sim (arg 1, arg 2) is a similarity measure function; t is a temperature parameter;
and using the trained double-tower model as a target image representation model.
Further, in an embodiment of the present invention, the determining the detection result of the sample to be detected according to the feature vector of the sample to be detected and the standard positive sample feature vector set includes:
similarity calculation is carried out on the sample feature vector to be detected and the standard positive sample feature vector set to obtain a similarity set, the number of similarities of which the numerical values are smaller than a similarity threshold value in the similarity set is counted, if the similarity set is smaller than the voting threshold value, the sample to be detected is a negative sample, and otherwise, the sample to be detected is a positive sample; wherein the similarity threshold and the voting threshold are derived from the defect sample detection validation dataset.
Further, in an embodiment of the present invention, the defect sample inspection verification dataset includes:
selecting a preset number of positive samples and negative samples at different angles, and constructing a defect sample detection verification data set
In order to achieve the above object, a second aspect of the present invention provides an industrial defect sample detection system based on image representation, which is characterized by comprising the following modules:
the training module is used for acquiring an image representation model training data set, and training a pre-constructed double-tower model by using the image representation model training data set to obtain an image representation model;
the data construction module is used for acquiring a standard positive sample data set, inputting the standard positive sample data set into the image representation model and acquiring a standard positive sample characteristic vector set;
the vector construction module is used for obtaining a sample to be detected, inputting the sample to be detected into the image representation model and obtaining a characteristic vector of the sample to be detected;
and the detection module is used for determining the detection result of the sample to be detected according to the sample characteristic vector to be detected and the standard positive sample characteristic vector set.
Further, in an embodiment of the present invention, the training module is further configured to:
selecting a preset number of positive samples and negative samples at different angles according to the types of the industrial parts to be inspected, constructing a positive and negative sample pair, and recording the positive and negative sample pair
Wherein
In the case of a positive sample,
for negative examples, a training dataset of image representation models is formed
Further, in one embodiment of the present invention,
further, in an embodiment of the present invention, the training module is further configured to:
training a dataset D using the image representation model 1 Constructing training batch data, each training batch data consisting of N positive and negative sample pairs
Composition is carried out;
constructing a double-tower model by using ResNet as an original image representation model; the double-tower model is formed by positive and negative sample pairs
As input, the image representation model of the two shared parameters is respectively input to extract the characteristic vector
And optimized using the following loss function:
wherein N is the total number of positive and negative sample pairs of the current batch, l i The loss function for the ith positive and negative sample pair is shown as follows:
wherein N is the total number of positive and negative sample pairs of the current batch; sim (arg 1, arg 2) is a similarity measure function; t is a temperature parameter;
and using the trained double-tower model as a target image representation model.
Further, in an embodiment of the present invention, the detecting module is further configured to:
similarity calculation is carried out on the sample feature vector to be detected and the standard positive sample feature vector set to obtain a similarity set, the number of similarities, of which the values are smaller than a similarity threshold value, in the similarity set is counted, if the number of the similarities is smaller than a voting threshold value, the sample to be detected is a negative sample, and otherwise, the sample to be detected is a positive sample; wherein the similarity threshold and the voting threshold are derived from the defect sample detection validation dataset.
In order to achieve the above object, a third aspect of the present invention provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor executes the computer program to implement the method for detecting an industrial defect sample based on image representation as described above.
The industrial defect sample detection method based on image representation comprises the steps of training an image representation model by utilizing a small number of normal samples and defect samples, extracting a standard positive sample feature vector set and a sample feature vector to be detected by utilizing the model, calculating the similarity between the sample feature vector to be detected and the standard positive sample feature vector by utilizing a similarity measurement function, and judging whether the current detection sample is an industrial defect sample by utilizing a similarity threshold value and combining a voting mechanism. The invention solves the problems of high requirement on the quantity of defect samples and high requirement on labeling of the conventional industrial defect detection method based on machine vision, and provides the high-efficiency and low-cost industrial defect sample detection method.
Drawings
The foregoing and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
fig. 1 is a schematic flow chart of an industrial defect sample detection method based on image representation according to an embodiment of the present invention.
Fig. 2 is a schematic flow chart of an image representation model training method based on a double-tower model according to an embodiment of the present invention.
Fig. 3 is a flowchart of a complete method provided by the embodiment of the present invention.
Fig. 4 is a schematic flowchart of an industrial defect sample inspection system based on image representation according to an embodiment of the present invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are illustrative and intended to be illustrative of the invention and are not to be construed as limiting the invention.
The industrial defect sample detection method based on image representation according to the embodiment of the invention is described below with reference to the accompanying drawings.
Fig. 1 is a schematic flowchart of an industrial defect sample detection method based on image representation according to an embodiment of the present invention.
As shown in fig. 1, the method for detecting industrial defect sample based on image representation comprises the following steps:
s101: acquiring an image representation model training data set, and training a pre-constructed double-tower model by using the image representation model training data set to obtain an image representation model;
s102: acquiring a standard positive sample data set, inputting the standard positive sample data set into an image representation model, and acquiring a standard positive sample characteristic vector set;
s103: acquiring a sample to be detected, inputting the sample to be detected into an image representation model to acquire a characteristic vector of the sample to be detected;
s104: and determining the detection result of the sample to be detected according to the sample characteristic vector to be detected and the standard positive sample characteristic vector set.
Further, in one embodiment of the present invention, acquiring an image representation model training data set comprises:
selecting a preset number of positive samples and negative samples at different angles according to the type of the industrial parts to be detected, constructing a positive and negative sample pair, and recording the positive and negative sample pair
Wherein
Is a positive sample of the sample to be tested,
for negative examples, a training dataset of image representation models is formed
Further, in an embodiment of the present invention, constructing a double-tower model, and training the double-tower model by using an image representation model training dataset to obtain an image representation model, includes:
training a dataset D using an image representation model 1 Constructing training batch data, each training batch data consisting of N positive and negative sample pairs
Composition is carried out;
constructing a double-tower model by using ResNet as an original image representation model; the double-tower model is formed by positive and negative sample pairs
As input, the image representation model of the two shared parameters is respectively input to extract the feature vector
And optimized using the following loss function:
wherein N is the total number of positive and negative sample pairs of the current batch, l i The loss function for the ith positive and negative sample pair is shown as follows:
wherein, N is the total number of the positive and negative sample pairs of the current batch; sim (arg 1, arg 2) is a similarity measure function; t is a temperature parameter;
and using the trained double-tower model as a target image representation model.
FIG. 2 is a flowchart of a method for training an image representation model based on a two-tower model according to the present invention.
Further, in an embodiment of the present invention, determining a detection result of a sample to be detected according to a sample feature vector to be detected and a standard positive sample feature vector set includes:
similarity calculation is carried out on the feature vector of the sample to be detected and a standard positive sample feature vector set to obtain a similarity set, the number of similarities of which the numerical values are smaller than a similarity threshold value in the similarity set is counted, if the similarity number is smaller than a voting threshold value, the sample to be detected is a negative sample, and otherwise, the sample to be detected is a positive sample; wherein the similarity threshold and the voting threshold are obtained from the defect sample detection verification dataset.
Further, in one embodiment of the present invention, a defect sample inspection verification dataset includes:
selecting a preset number of positive samples and negative samples at different angles, and constructing a defect sample detection verification data set
The above is the complete detection process of the present invention, and is specifically shown in fig. 3.
The industrial defect sample detection method based on image representation comprises the steps of training an image representation model by utilizing a small number of normal samples and defect samples, extracting a standard positive sample feature vector set and a sample feature vector to be detected by utilizing the model, calculating the similarity between the sample feature vector to be detected and the standard positive sample feature vector by utilizing a similarity measurement function, and judging whether the current detection sample is an industrial defect sample by utilizing a similarity threshold value and combining a voting mechanism. The invention solves the problems of high requirement on the number of defect samples and high requirement on labeling of the conventional industrial defect detection method based on machine vision, and provides the industrial defect sample detection method with high efficiency and low cost.
Fig. 4 is a schematic structural diagram of an industrial defect sample inspection system based on image representation according to an embodiment of the present invention.
As shown in fig. 4, the image representation-based industrial defect sample inspection system includes: a training module 100, a data construction module 200, a vector construction module 300, a detection module 400, wherein,
the training module is used for acquiring an image representation model training data set, and training a pre-constructed double-tower model by using the image representation model training data set to obtain an image representation model;
the data construction module is used for acquiring a standard positive sample data set, inputting the standard positive sample data set into the image representation model and acquiring a standard positive sample characteristic vector set;
the vector construction module is used for inputting the sample to be detected into the image representation model to obtain the characteristic vector of the sample to be detected;
and the detection module is used for determining the detection result of the sample to be detected according to the sample characteristic vector to be detected and the standard positive sample characteristic vector set.
Further, in an embodiment of the present invention, the training module is further configured to:
selecting a preset number of positive samples and negative samples at different angles according to the types of the industrial parts to be inspected, constructing a positive and negative sample pair, and recording the positive and negative sample pair
Wherein
Is a positive sample of the sample to be tested,
for negative examples, a training dataset of image representation models is formed
Further, in one embodiment of the present invention,
further, in an embodiment of the present invention, the training module is further configured to:
training a dataset D using an image representation model 1 Constructing training batch data, each training batch data consisting of N positive and negative sample pairs
Composition is carried out;
constructing a double-tower model by using ResNet as an original image representation model; the double-tower model is formed by positive and negative sample pairs
As input, the image representation model of the two shared parameters is respectively input to extract the feature vector
And optimized using the following loss function:
wherein N is the total number of positive and negative sample pairs of the current batch, l i The loss function for the ith positive and negative sample pair is shown as follows:
wherein N is the total number of positive and negative sample pairs of the current batch; sim (arg 1, arg 2) is a similarity measure function; t is a temperature parameter;
and using the trained double-tower model as a target image representation model.
Further, in an embodiment of the present invention, the detecting module is further configured to:
similarity calculation is carried out on the feature vector of the sample to be detected and a standard positive sample feature vector set to obtain a similarity set, the number of similarities of which the numerical values are smaller than a similarity threshold value in the similarity set is counted, if the similarity number is smaller than a voting threshold value, the sample to be detected is a negative sample, and otherwise, the sample to be detected is a positive sample; wherein the similarity threshold and the voting threshold are obtained from a defect sample detection verification data set.
In order to achieve the above object, a third aspect of the present invention provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor executes the computer program to implement the method for detecting an industrial defect sample based on image representation as described above.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or to implicitly indicate the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "a plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise.
Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.
Claims (10)
1. An industrial defect sample detection method based on image representation is characterized by comprising the following steps:
acquiring an image representation model training data set, and training a pre-constructed double-tower model by using the image representation model training data set to obtain an image representation model;
acquiring a standard positive sample data set, inputting the standard positive sample data set into the image representation model, and acquiring a standard positive sample characteristic vector set;
obtaining a sample to be detected, inputting the sample to be detected into the image representation model to obtain a characteristic vector of the sample to be detected;
and determining the detection result of the sample to be detected according to the sample characteristic vector to be detected and the standard positive sample characteristic vector set.
2. The method of claim 1, wherein the obtaining an image representation model training dataset comprises:
selecting a preset number of positive samples and negative samples at different angles according to the types of the industrial parts to be inspected, constructing a positive and negative sample pair, and recording the positive and negative sample pair
Wherein
Is a positive sample of the sample to be tested,
for negative examples, a training dataset of image representation models is formed
3. The method of claim 1 or 2, wherein constructing a double tower model, training the double tower model using the image representation model training dataset, resulting in an image representation model comprises:
training a dataset D using the image representation model 1 Constructing training batch data, each training batch data consisting of N positive and negative sample pairs
Forming;
constructing a double-tower model by using ResNet as an original image representation model; the double-tower model is formed by positive and negative sample pairs
As input, the image representation model of the two shared parameters is respectively input to extract the feature vector
And optimized using the following loss function:
wherein N is the total number of positive and negative sample pairs of the current batch, l i The loss function for the ith positive and negative sample pair is shown as follows:
wherein N is the total number of positive and negative sample pairs of the current batch; sim (arg 1, arg 2) is a similarity measure function; t is a temperature parameter;
and using the trained double-tower model as a target image representation model.
4. The method according to claim 1, wherein the determining the detection result of the sample to be detected according to the sample feature vector to be detected and the set of standard positive sample feature vectors includes:
similarity calculation is carried out on the sample feature vector to be detected and the standard positive sample feature vector set to obtain a similarity set, the number of similarities of which the numerical values are smaller than a similarity threshold value in the similarity set is counted, if the similarity set is smaller than the voting threshold value, the sample to be detected is a negative sample, and otherwise, the sample to be detected is a positive sample; wherein the similarity threshold and the voting threshold are derived from the defect sample detection validation dataset.
5. The method of claim 4, wherein the defect sample inspection validation dataset comprises:
selecting a preset number of positive samples and negative samples at different angles, and constructing a defect sample detection verification data set
6. An image representation based industrial defect sample detection system is characterized by comprising the following modules:
the training module is used for acquiring an image representation model training data set, and training a pre-constructed double-tower model by using the image representation model training data set to obtain an image representation model;
the data construction module is used for acquiring a standard positive sample data set, inputting the standard positive sample data set into the image representation model and acquiring a standard positive sample characteristic vector set;
the vector construction module is used for obtaining a sample to be detected, inputting the sample to be detected into the image representation model and obtaining a characteristic vector of the sample to be detected;
and the detection module is used for determining the detection result of the sample to be detected according to the sample characteristic vector to be detected and the standard positive sample characteristic vector set.
7. The system of claim 6, wherein the training module is further configured to:
selecting a preset number of positive samples and negative samples at different angles according to the types of the industrial parts to be inspected, constructing a positive and negative sample pair, and recording the positive and negative sample pair
Wherein
Is a positive sample of the sample to be tested,
for negative examples, a training dataset of image representation models is formed
8. The system of claim 6, wherein the training module is further configured to:
training a dataset D using the image representation model 1 Constructing training batch data, each training batch data consisting of N positive and negative sample pairs
Composition is carried out;
constructing a double-tower model by using ResNet as an original image representation model; the double-tower model is formed by positive and negative sample pairs
As input, the image representation model of the two shared parameters is respectively input to extract the characteristic vector
And optimized using the following loss function:
wherein N is the total number of positive and negative sample pairs of the current batch, l i The loss function for the ith positive and negative sample pair is shown as follows:
wherein N is the total number of positive and negative sample pairs of the current batch; sim (arg 1, arg 2) is a similarity measure function; t is a temperature parameter;
and using the trained double-tower model as a target image representation model.
9. The system of claim 6, wherein the detection module is further configured to:
similarity calculation is carried out on the sample feature vector to be detected and the standard positive sample feature vector set to obtain a similarity set, the number of similarities of which the numerical values are smaller than a similarity threshold value in the similarity set is counted, if the similarity set is smaller than the voting threshold value, the sample to be detected is a negative sample, and otherwise, the sample to be detected is a positive sample; wherein the similarity threshold and the voting threshold are derived from the defect sample detection validation dataset.
10. Computer device, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the image representation based industrial defect sample detection method according to any of claims 1-5 when executing the computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211279714.1A CN115713484A (en) | 2022-10-19 | 2022-10-19 | Industrial defect sample detection method and system based on image representation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211279714.1A CN115713484A (en) | 2022-10-19 | 2022-10-19 | Industrial defect sample detection method and system based on image representation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115713484A true CN115713484A (en) | 2023-02-24 |
Family
ID=85231227
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211279714.1A Pending CN115713484A (en) | 2022-10-19 | 2022-10-19 | Industrial defect sample detection method and system based on image representation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115713484A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117197027A (en) * | 2023-03-17 | 2023-12-08 | 中勍科技股份有限公司 | Electronic package optimization method and system |
-
2022
- 2022-10-19 CN CN202211279714.1A patent/CN115713484A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117197027A (en) * | 2023-03-17 | 2023-12-08 | 中勍科技股份有限公司 | Electronic package optimization method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2020173177A1 (en) | Object color difference defect detection method, device, computer device, and storage medium | |
| US11521120B2 (en) | Inspection apparatus and machine learning method | |
| CN108829878B (en) | Method and device for detecting abnormal points of industrial experimental data | |
| JP5168215B2 (en) | Appearance inspection device | |
| CN108492327B (en) | AOI image matching method, system, readable storage medium and intelligent device | |
| CN117152152B (en) | Production management system and method for detection kit | |
| JP2017219529A (en) | Appearance abnormality inspection device, method, and program | |
| CN115713484A (en) | Industrial defect sample detection method and system based on image representation | |
| US11682112B2 (en) | Inspection device and machine learning method | |
| CN113077416A (en) | Welding spot welding defect detection method and system based on image processing | |
| CN116543056A (en) | Linear camera calibration and calibration method, device and storage medium | |
| CN110266527B (en) | Sensor node fault classification alarm method and device based on spatial correlation | |
| KR101782364B1 (en) | Vision inspection method based on learning data | |
| CN113554645B (en) | Industrial anomaly detection method and device based on WGAN | |
| CN109523129A (en) | A kind of method of unmanned vehicle multi-sensor information real time fusion | |
| CN116934732A (en) | Photovoltaic module detection method and device and electronic equipment | |
| CN116188374A (en) | Socket detection method, device, computer equipment and storage medium | |
| CN115937555A (en) | Industrial defect detection algorithm based on standardized flow model | |
| CN114581785B (en) | Multi-source heterogeneous data fusion blast furnace gas flow distribution state identification method and system | |
| CN112577456B (en) | Measuring equipment point inspection method, operation control device and computer readable storage medium | |
| CN113420772A (en) | Defect detection method and device based on multi-classifier and SVDD (singular value decomposition and direct decomposition) cooperative algorithm | |
| JP2022139796A (en) | Support device and method | |
| CN109829389B (en) | Machine displacement determination method, device and computer storage medium | |
| US20220074874A1 (en) | Computer-implemented method for analysing measurement data from a measurement of an object | |
| CN114418941B (en) | Defect diagnosis method and system based on detection data of power inspection equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |