CN109760694A - A kind of intelligent travelling crane monitoring and alarming system based on machine vision - Google Patents
A kind of intelligent travelling crane monitoring and alarming system based on machine vision Download PDFInfo
- Publication number
- CN109760694A CN109760694A CN201910042473.0A CN201910042473A CN109760694A CN 109760694 A CN109760694 A CN 109760694A CN 201910042473 A CN201910042473 A CN 201910042473A CN 109760694 A CN109760694 A CN 109760694A
- Authority
- CN
- China
- Prior art keywords
- module
- electrically connected
- chip
- machine vision
- system based
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 16
- 238000012545 processing Methods 0.000 claims abstract description 15
- 102100021223 Glucosidase 2 subunit beta Human genes 0.000 claims description 12
- 101001040875 Homo sapiens Glucosidase 2 subunit beta Proteins 0.000 claims description 12
- 208000034613 Isolated polycystic liver disease Diseases 0.000 claims description 12
- 208000021033 autosomal dominant polycystic liver disease Diseases 0.000 claims description 12
- 238000003860 storage Methods 0.000 claims description 6
- 238000013507 mapping Methods 0.000 claims description 4
- 230000005540 biological transmission Effects 0.000 claims description 2
- 238000009434 installation Methods 0.000 claims 1
- 238000004422 calculation algorithm Methods 0.000 abstract description 17
- 230000008859 change Effects 0.000 abstract description 5
- 238000001514 detection method Methods 0.000 abstract description 5
- 238000000034 method Methods 0.000 description 12
- 238000010586 diagram Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 5
- 238000003491 array Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 241000287196 Asthenes Species 0.000 description 1
- 241000736199 Paeonia Species 0.000 description 1
- 235000006484 Paeonia officinalis Nutrition 0.000 description 1
- 206010039203 Road traffic accident Diseases 0.000 description 1
- 208000003443 Unconsciousness Diseases 0.000 description 1
- 238000005267 amalgamation Methods 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000007797 corrosion Effects 0.000 description 1
- 238000005260 corrosion Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 230000035772 mutation Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Landscapes
- Traffic Control Systems (AREA)
Abstract
The invention discloses a kind of intelligent travelling crane monitoring and alarming system based on machine vision, belongs to the product of scan picture and sci-tech product, including image capture module, image processing module, digital signal Buffer output module, buzzer alarm module, key drive circuit module.The achievable left and right lane line that current line lane is identified and tracked by camera, and check the range and change rate of yaw angle, drift condition of the vehicle on structured road can be supervised, it will be sounded an alarm beyond normal range (NR), the present invention carries out binaryzation to road image using improved Otsu algorithm, left and right lane line is extracted using the constraint line detection method changed based on Hough, there is strong robustness to phenomena such as partial occlusion, line interruption and noise, and meets the requirement of the lane departure warning real-time under structured road environment.
Description
Technical field
The present invention relates to intelligent travelling crane monitoring alarm technical field, specially a kind of intelligent travelling crane prison based on machine vision
Control alarm system.
Background technique
As the usability of automobile is extensive, ride safety of automobile is had been to be concerned by more and more people.Currently, driver is not intended to
Know lower deviation Frequent Accidents to propose for technical problems such as existing lane departure warning algorithm time-consuming, complexity height
A kind of real-time lane departure warning algorithm of structured road based on machine vision.It is calculated according to the improvement Otsu of road surface priori knowledge
Method, and lane line segmentation is carried out to road image；Utilize Hough change detection lane line under the conditions of straight line model and constraint, knot
Close determination of the geometrical relationship of lane line and this vehicle to multilane scene progress current vehicle Travel vehicle diatom；Kalman filter with
Track lane line, and straight line parameter prediction is carried out to discontinuous lane line or the part that is blocked；Merge yaw angle and pixel distance letter
Breath, which is realized, deviates early warning.Under the processing of internal intelligent chip, algorithm effectively can be split recognition and tracking to lane line, and
With deviateing existing lane vehicle is issued warning signal, algorithm process speed can better meet structure of about 19 frame per second
Change and deviates early warning performance requirement under road environment.
Summary of the invention
(1) the technical issues of solving
In view of the deficiencies of the prior art, the present invention provides a kind of intelligent travelling crane monitoring alarm system based on machine vision
System, solves the unconscious lower deviation Frequent Accidents of driver, so as to cause the generation problem of traffic accident.
(2) technical solution
To achieve the above object, the invention provides the following technical scheme: a kind of 1. intelligent travelling crane prisons based on machine vision
Control alarm system, including TMS320DM642 chip, SDRAM memory, image capture module, image processing module, Buffer output
Module, buzzer alarm module and key drive circuit module, it is characterised in that: the collected signal of described image acquisition module
Clamping is mounted on inside SDRAM memory, is arranged inside the SDRAM memory from the double storage organizations of each section, and internal peace
There are two staggered storage permutation, the TMS320DM642 chips to be electrically connected by data line and SDRAM memory for dress, institute
The output end for stating TMS320DM642 chip is electrically connected by conducting wire and PCLD sequence controller, the PCLD sequence controller
It changes kind of module by transmission line and memory ROM and output to be electrically connected, the output end of the memory ROM passes through conducting wire and DA
Converter is electrically connected.
Preferably, described image acquisition module includes CCD camera and digital camera, and the CCD camera passes through number
It is electrically connected according to line and converter, the digital camera is electrically connected by switching switch with SDRAM memory.
Preferably, described image processing module is electrically connected by conducting wire and SDRAM memory, the SDRAM memory
It being electrically connected by data line and TMS320DM642 chip, the TMS320DM642 chip is provided with ROI region mapping template,
The output end of the TMS320DM642 chip is electrically connected by data line and memory ROM.
Preferably, the output buffer module is electrically connected by conducting wire and PCLD sequence controller, the PCLD timing
Controller is electrically connected by control line and D/A converter, and the D/A converter is electrical by conducting wire and TMS320DM642 chip
Connection.
Preferably, the buzzer warning module includes phonetic alarm, and the TMS320DM642 chip passes through control line
It is electrically connected with the phonetic alarm on buzzer warning module.
(3) beneficial effect
The present invention provides a kind of intelligent travelling crane monitoring and alarming system based on machine vision.Have it is following the utility model has the advantages that
(1), should intelligent travelling crane monitoring and alarming system based on machine vision, by the setting of ROI region mapping template, from
And reached the calculation amount for reducing algorithm, the real-time of system is greatly improved, is calculated using the Otsu of ROI region mapping template
Method when image binaryzation, is combined with gray value flat within the scope of road surface using global threshold and obtains optimal segmenting threshold, accomplished most
Reinforce to limits Road target signature, and suppress or eliminate background information, to reinforce the robustness of road detection.
(2), it is somebody's turn to do the intelligent travelling crane monitoring and alarming system based on machine vision, and by using in TMS320DM642 chip
The Hough mutation analysis under line constraint is selected, fuzzy lane line, the combination of lane line actual situation or lane line is solved and is blocked
Under the conditions of lane line accurately extract problem.
(3), it is somebody's turn to do the intelligent travelling crane monitoring and alarming system based on machine vision, and passes through intelligent early-warning driving recording
The setting of TMS320DM642 chip, the digital processing capabilities with array processor, the algorithm of suitable system, and periphery collection
At very complete audio, video and network communication interface, it is very suitable for connecting with audio alert equipment.
Detailed description of the invention
Fig. 1 is a kind of overall procedure of the intelligent travelling crane monitoring and alarming system based on machine vision designed by this present invention
Schematic diagram；
Fig. 2 is image processing algorithm flow chart of the invention；
Fig. 3 is that ROI region of the invention maps schematic diagram；
Fig. 4 is structure of the invention road track line drawing schematic diagram；
Fig. 5 is yaw angle schematic diagram of the present invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other
Embodiment shall fall within the protection scope of the present invention.
As shown in Figure 1, the present invention provides a kind of technical solution: a kind of intelligent travelling crane monitoring alarm system based on machine vision
System, including TMS320DM642 chip, image capture module, image processing module, Buffer output module, buzzer alarm module,
Key drive circuit module, image capture module include CCD camera and digital camera, and CCD camera is by collected figure
As being used as analog signal, becomes digital signal by converter, pass through together with the collected digital signal of digital camera
Switching switch, is temporarily stored in SDRAM memory, and SDRAM and PCLD timing control signal share a clock cycle,
SDRAM is double bank structures, includes two staggered storage arrays, when TMS320DM642 chip is from a memory bank or battle array
When column access data, another has just been that read-write data are got ready, and by the close switching of the two storage arrays, is read
Efficiency can be increased exponentially；Then, TMS320DM642 chip receives instruction, handles acquired image signal, passes through
The processing of PCLD sequential control circuit, PCLD sequential control circuit change kind of a module with memory ROM and output and connect, after processed
Picture signal be stored in memory ROM；By the temporary processing of image filtering, effective digital signal is obtained, the number after processing
Word signal passes through D/A converter, obtains image binaryzation, the simulation drawing after Hough variation and Kalman filter, to judge vehicle
Whether deviate in road；If buzzer warning occurs for deviation；Conversely, buzzer siren does not sound an alarm.
It is illustrated in figure 2 image processing algorithm flow chart of the invention；Image is handled first, image processing module
It gives the image information being stored in SDRAM memory to chip TMS320DM642, chooses front part image as interested
Region, that is, ROI region utilize global threshold and road surface during image binaryzation using improved Otsu algorithm
Average gray value, which combines, in range obtains optimal segmenting threshold, and optimal segmenting threshold is expressed asThen image information after treatment is sent into memory ROM.
The solution process of optimal segmenting threshold are as follows:
1) each gray scale frequency is calculated, wherein M, N are the long width values of image, niFor number of pixels under i-gray level
2) target class and background classes gray level are calculated separately and wherein m is gray threshold, and L is total gray level
3) center in the class of target class and background classes is calculated
4) each gray level thresholding is traversed, so that inter-class variance maximum value is the global threshold that Otsu algorithm obtains, algorithm is obtained
The global threshold threshold arrived
δ2=P1(ω1-ω0)2+P2(ω2-ω0)2=P1P2(ω2-ω1)
5) it examines threshold value: choosing the road surface region of 6 pieces of 5*5 pixels of vehicle front, choose variance yields and be less than customized threshold value
Block diagram calculates its average gray value avg.Then reference threshold is set as:
Threshold_avg=acg*k k ∈ [1,2]
6) optimal segmenting threshold is sought:
7) image binaryzation: when gray value of image is greater than optimal threshold, it is believed that be target class, value 1；No person is background
Value 0.
Above-mentioned algorithm, to road image binaryzation, is then made image enhancement area pixel point and is gone using improved Otsu algorithm
It removes, the operation such as pixel connection and corrosion, so that lane line and road background segment are obvious in image and obtain single gray value vehicle
Diatom；Then retrieval tracking is carried out to lane line, chooses static region of interest ROI, coarse positioning is carried out to road area, will be known
For other area locking in front side road, the constraint line detection method for being then taken based on Hough detects lane edge,
For a plurality of candidate lane, obtain being currently located terrain vehicle diatom using the geometrical relationship of camera and road present position.Vehicle
Diatom tracking prediction uses Kalman filter, compares judgement to predicted value and measured value, obtains optimal lane parameter；Most
The point based on disappearance carries out early warning to deviation afterwards, using yaw angle and pixel distance information amalgamation mode, carries out to the two
Determine to complete to deviate early warning in threshold range.
Exporting buffer module is electrically connected between PCLD sequential control circuit, is stored in image information as data
Then output buffer passes through D/A converter, digital signal is become analog signal, change by the Hough under line constraint
After Kalman filter, the lane line in lane where extracting vehicle generates the analog image of practical lane operation, is simulated
Image calculates yaw angle φ according to lane line parameter and calculates what yaw angle φ should be kept further according to the ratio of vehicle body and road
One threshold range.
It is electrically connected between TMS320DM642 chip and the phonetic alarm of buzzer warning module, the vehicle got
Yaw angle φ issues signal if yaw angle φ, not within the scope of secure threshold, chip output generates corresponding level signal
Make buzzer warning.After alarm, resetted through key driving circuit, system is reworked.
It is illustrated in figure 3 ROI region schematic diagram；Vehicle front windshield midline and figure are mounted on according to camera
The priori knowledges such as site of road distribution, determine that the lane line region with complete display accounts for the 35% of total figure picture as in, remaining
Part is sky, both sides of the road environment, Chinese herbaceous peony cover shelter etc..In practical applications, according to user to camera position and
Angle etc. is put, and can choose different region accountings to carry out subsequent processing, thus the significantly real-time of lifting system
And reduce the complexity of algorithm calculating.
It is illustrated in figure 4 structured road lane line drawing schematic diagram；Image binaryzation is carried out to ROI, and establishes coordinate
System models carriageway surfacing image, carries out the constraint of left and right lane value range to θ, establishes parameter space two-dimensional array
(ρ, θ), all white pixel points in the region sequential search RR and LR, traverse all probable values of θ, according to ρ=xcos respectively
(θ)+ysin (θ) calculates corresponding ρ value, and the point acquired using parameter space is converted and obtains all detections into rectangular coordinate system
The straight line arrived calculates O point and all inspection vehicle straight line distance d, several current drivings of two straight lines of the shortest left and right of vertical range
In lane line L1, L2.
It is illustrated in figure 5 yaw angle schematic diagram；For the road image of shooting, middle line bottom is selected can a little to be recognized
To be current vehicle present position, i.e., quasi- vehicle point；And road end point is the intersection point of two lane lines of left and right, end point and quasi- vehicle
The level angle of point line is defined as the yaw angle in this algorithm.D is the road end point that road L1, L2 are crossed to form, and M is
Quasi- vehicle point, d1, d2 are respectively quasi- vehicle point away from left and right lane line horizontal pixel distance, and φ is the yaw angle during algorithm defines,
The content being not described in detail in this specification simultaneously belongs to the prior art well known to professional and technical personnel in the field.
It should be noted that, in this document, relational terms such as first and second and the like are used merely to a reality
Body or operation are distinguished with another entity or operation, are deposited without necessarily requiring or implying between these entities or operation
In any actual relationship or order or sequence.Moreover, the terms "include", "comprise" or its any other variant are intended to
Non-exclusive inclusion, so that the process, method, article or equipment including a series of elements is not only wanted including those
Element, but also including other elements that are not explicitly listed, or further include for this process, method, article or equipment
Intrinsic element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that
There is also other identical elements in process, method, article or equipment including the element.
It although an embodiment of the present invention has been shown and described, for the ordinary skill in the art, can be with
A variety of variations, modification, replacement can be carried out to these embodiments without departing from the principles and spirit of the present invention by understanding
And modification, the scope of the present invention is defined by the appended.
Claims (5)
1. a kind of intelligent travelling crane monitoring and alarming system based on machine vision, including TMS320DM642 chip, SDRAM memory,
Image capture module, image processing module, Buffer output module, buzzer alarm module and key drive circuit module, it is special
Sign is: the collected signal clamping of described image acquisition module is mounted on inside SDRAM memory, the SDRAM memory
Inside setting is from the double storage organizations of each section, and there are two staggered storage permutation, the TMS320DM642 chips for internal installation
It is electrically connected by data line and SDRAM memory, the output end of the TMS320DM642 chip passes through conducting wire and PCLD timing
Controller is electrically connected, and the PCLD sequence controller changes kind of module by transmission line and memory ROM and output and is electrically connected,
The output end of the memory ROM is electrically connected by conducting wire and D/A converter.
2. a kind of intelligent travelling crane monitoring and alarming system based on machine vision according to claim 1, it is characterised in that: institute
Stating image capture module includes CCD camera and digital camera, and the CCD camera is electrical by data line and converter
Connection, the digital camera are electrically connected by switching switch with SDRAM memory.
3. a kind of intelligent travelling crane monitoring and alarming system based on machine vision according to claim 1, it is characterised in that: institute
Image processing module is stated to be electrically connected by conducting wire and SDRAM memory, the SDRAM memory by data line with
TMS320DM642 chip is electrically connected, and the TMS320DM642 chip is provided with ROI region mapping template, described
The output end of TMS320DM642 chip is electrically connected by data line and memory ROM.
4. a kind of intelligent travelling crane monitoring and alarming system based on machine vision according to claim 1, it is characterised in that: institute
Output buffer module is stated to be electrically connected by conducting wire and PCLD sequence controller, the PCLD sequence controller by control line with
D/A converter is electrically connected, and the D/A converter is electrically connected by conducting wire and TMS320DM642 chip.
5. a kind of intelligent travelling crane monitoring and alarming system based on machine vision according to claim 1, it is characterised in that: institute
Stating buzzer warning module includes phonetic alarm, and the TMS320DM642 chip passes through control line and buzzer warning module
On phonetic alarm electrical connection.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910042473.0A CN109760694A (en) | 2019-01-17 | 2019-01-17 | A kind of intelligent travelling crane monitoring and alarming system based on machine vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910042473.0A CN109760694A (en) | 2019-01-17 | 2019-01-17 | A kind of intelligent travelling crane monitoring and alarming system based on machine vision |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109760694A true CN109760694A (en) | 2019-05-17 |
Family
ID=66452261
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910042473.0A Pending CN109760694A (en) | 2019-01-17 | 2019-01-17 | A kind of intelligent travelling crane monitoring and alarming system based on machine vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109760694A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111874003A (en) * | 2020-06-23 | 2020-11-03 | 安徽信息工程学院 | Vehicle driving deviation early warning method and system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN2746442Y (en) * | 2004-09-08 | 2005-12-14 | 四川华控图形科技有限公司 | Multichannel curved projection controller and image transfer card |
| CN201249721Y (en) * | 2008-06-23 | 2009-06-03 | 长安大学 | Vehicle lane departure prewarning device |
| CN101738828A (en) * | 2009-12-24 | 2010-06-16 | 北京优纳科技有限公司 | CCD high-definition intelligent industrial camera |
| CN104029680A (en) * | 2014-01-02 | 2014-09-10 | 上海大学 | Lane departure warning system and method based on monocular camera |
| CN104333681A (en) * | 2014-08-15 | 2015-02-04 | 徐云鹏 | FPGA-based microminiature CCD image collecting and processing system |
| CN107618434A (en) * | 2016-07-13 | 2018-01-23 | 芜湖优必慧新能源科技有限公司 | A kind of automobile safety system based on DPS |
-
2019
- 2019-01-17 CN CN201910042473.0A patent/CN109760694A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN2746442Y (en) * | 2004-09-08 | 2005-12-14 | 四川华控图形科技有限公司 | Multichannel curved projection controller and image transfer card |
| CN201249721Y (en) * | 2008-06-23 | 2009-06-03 | 长安大学 | Vehicle lane departure prewarning device |
| CN101738828A (en) * | 2009-12-24 | 2010-06-16 | 北京优纳科技有限公司 | CCD high-definition intelligent industrial camera |
| CN104029680A (en) * | 2014-01-02 | 2014-09-10 | 上海大学 | Lane departure warning system and method based on monocular camera |
| CN104333681A (en) * | 2014-08-15 | 2015-02-04 | 徐云鹏 | FPGA-based microminiature CCD image collecting and processing system |
| CN107618434A (en) * | 2016-07-13 | 2018-01-23 | 芜湖优必慧新能源科技有限公司 | A kind of automobile safety system based on DPS |
Non-Patent Citations (1)
| Title |
|---|
| 常婧: "《结构化道路中车道偏离预警系统研究》", 《中国优秀硕士论文全文数据库（电子期刊）》, pages 10 - 57 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111874003A (en) * | 2020-06-23 | 2020-11-03 | 安徽信息工程学院 | Vehicle driving deviation early warning method and system |
| CN111874003B (en) * | 2020-06-23 | 2021-07-20 | 安徽信息工程学院 | Vehicle driving deviation early warning method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105005771B (en) | A kind of detection method of the lane line solid line based on light stream locus of points statistics | |
| CN103914688B (en) | A kind of urban road differentiating obstacle | |
| CN109670376B (en) | Lane line identification method and system | |
| CN102708356B (en) | Automatic license plate positioning and recognition method based on complex background | |
| CN107025432B (en) | A kind of efficient lane detection tracking and system | |
| CN109460709A (en) | The method of RTG dysopia analyte detection based on the fusion of RGB and D information | |
| CN103714538B (en) | road edge detection method, device and vehicle | |
| CN104902261B (en) | Apparatus and method for the road surface identification in low definition video flowing | |
| CN107031623A (en) | A kind of road method for early warning based on vehicle-mounted blind area camera | |
| CN108805065A (en) | One kind being based on the improved method for detecting lane lines of geometric properties | |
| Jiang et al. | Lane and obstacle detection based on fast inverse perspective mapping algorithm | |
| CN112339773B (en) | Monocular vision-based non-active lane departure early warning method and system | |
| CN109190483B (en) | Lane line detection method based on vision | |
| CN108447303A (en) | The periphery visual field dangerous discernment method coupled with machine vision based on human eye vision | |
| CN106887004A (en) | A kind of method for detecting lane lines based on Block- matching | |
| CN110862033A (en) | Intelligent early warning detection method applied to coal mine inclined shaft winch | |
| CN110111283A (en) | The reminding method and system of infrared suspected target under a kind of complex background | |
| CN114898296A (en) | Bus lane occupation detection method based on millimeter wave radar and vision fusion | |
| CN109886175B (en) | Method for detecting lane line by combining straight line and circular arc | |
| CN107578012A (en) | A kind of drive assist system based on clustering algorithm selection sensitizing range | |
| CN106529404A (en) | Imaging principle-based recognition method for pilotless automobile to recognize road marker line | |
| CN110852179A (en) | Method for detecting suspicious personnel intrusion based on video monitoring platform | |
| CN107705577A (en) | A kind of real-time detection method and system based on lane line demarcation vehicle peccancy lane change | |
| CN110147748A (en) | A kind of mobile robot obstacle recognition method based on road-edge detection | |
| Liu et al. | Effective road lane detection and tracking method using line segment detector |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |