CN104386092A - Automatic train protection system and method based on image identification and multi- perception fusion - Google Patents

Automatic train protection system and method based on image identification and multi- perception fusion Download PDF

Info

Publication number
CN104386092A
CN104386092A CN201410562470.7A CN201410562470A CN104386092A CN 104386092 A CN104386092 A CN 104386092A CN 201410562470 A CN201410562470 A CN 201410562470A CN 104386092 A CN104386092 A CN 104386092A
Authority
CN
China
Prior art keywords
safety hazard
train
processing unit
image
sensor fusion
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201410562470.7A
Other languages
Chinese (zh)
Other versions
CN104386092B (en
Inventor
薛强
袁志鹏
徐建勇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Casco Signal Ltd
Original Assignee
Casco Signal Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Casco Signal Ltd filed Critical Casco Signal Ltd
Priority to CN201410562470.7A priority Critical patent/CN104386092B/en
Publication of CN104386092A publication Critical patent/CN104386092A/en
Priority to HK15105381.9A priority patent/HK1205071A1/en
Application granted granted Critical
Publication of CN104386092B publication Critical patent/CN104386092B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Landscapes

  • Train Traffic Observation, Control, And Security (AREA)

Abstract

The invention relates to an automatic train protection system and method based on image identification and multi-perception fusion. The automatic train protection system comprises an image collection unit (1), an image processing unit (2), a multi-perception fusion processing unit (3), an auxiliary perception unit (4) and an early warning result output unit (5), wherein the image collection unit (1), the image processing unit (2), the multi-perception fusion processing unit (3) and the early warning result output unit (5) are connected in sequence; the auxiliary perception unit (4) is connected with the multi-perception fusion processing unit (3). Compared with the prior art, the automatic train protection system and method based on image identification and multi-perception fusion can be adapted to complex conditions appearing in urban mass transit operation lines, the safe protection of a train is automatically realized according to the practical condition of a line, and the operation safety of urban mass transit is greatly improved.

Description

Based on automatic train protection system and the method for image recognition and sensor fusion
Technical field
The present invention relates to the train automatic protection technology in a kind of track traffic, especially relate to a kind of automatic train protection system based on image recognition and sensor fusion and method.
Background technology
The safety of urban track traffic operation depends on automatic protective system to a great extent.Under normal circumstances, the safety precaution of (comprise head-on, side blow and the three kinds of situations that knock into the back), train and track switch or other trackside equipment is at signal automatic control system (Automatic Train Control in City Rail Transit System between train and train, hereinafter referred to as ATC system) middle realization, but the automatic protection of ATC system detects based on orbit occupancy and train initiative recognition.Do not carry out by orbit occupancy and train initiative recognition the invader that detects for what occur in city rail system cloud gray model, the automatic protection of ATC system has some limitations.
For the invader (obstacle) not undertaken by orbit occupancy and train initiative recognition detecting, as fall ratchel in orbit or advertising panel, across the automobile on road junction, the people etc. fallen in track, although there is safety hazard to the operation of urban track traffic, ATC system is difficult to the safety precaution realizing train.Consider that the complexity of city rail system operation conditions is (as less in overhead or tunnel space of getting on the right track, bend more Si Ji lookout condition is poor), and by producing harm greatly and severe social influence in obstacle intrusion City Rail Transit System, need a kind of method that can realize realizing accidents such as invaders automatic protection.
Summary of the invention
Object of the present invention is exactly provide a kind of automatic train protection system based on image recognition and sensor fusion to overcome defect that above-mentioned prior art exists, the complex situations that urban track traffic circulation line occurs can be adapted to, according to circuit actual conditions, automatically realize the safety precaution of train, greatly improve urban rail transit operation security.
Object of the present invention can be achieved through the following technical solutions:
A kind of automatic train protection system based on image recognition and sensor fusion, it is characterized in that, comprise image acquisition units, graphics processing unit, sensor fusion processing unit, auxiliary perception unit and early warning result output unit, described image acquisition units, graphics processing unit, sensor fusion processing unit are connected successively with early warning result output unit, and described auxiliary perception unit is connected with sensor fusion processing unit;
The picture signal of described image acquisition units Real-time Collection train periphery running environment, and by image signal transmission in graphics processing unit, described graphics processing unit is transferred to sensor fusion processing unit after carrying out Preliminary detection to safety hazard, this sensor fusion processing unit starts auxiliary perception unit and carries out secondary perception, and the association attributes of the outside safety hazard sent by the association attributes of the outside safety hazard detected passed back auxiliary perception unit and graphics processing unit carries out convergence analysis, accurately determine every attribute of outside safety hazard further, early warning result output unit carries out security risk analysis according to every attribute of sensor fusion processing unit to outside safety hazard, determine the output situation of early warning result.
Described image acquisition units is ccd video camera or night infrared pick up camera, for the picture signal of Real-time Collection train periphery running environment, then through Video Sync Separator Chip image signal transmission in graphics processing unit.
Described graphics processing unit extracts element characteristic data dangerous in the picture signal of train periphery running environment, identifies, measure its distance, shape, relative velocity and relative acceleration, filtration deletion is carried out on the unfavorable disturbing factor that impact judges, thus realizes carrying out actv. Preliminary detection to safety hazard.
Described auxiliary perception unit is used to the outside safety hazard in aided detection train periphery running environment, and replaces image acquisition units as detection means when pick up camera cannot normally work.
Described auxiliary perception unit is one or more combination in radar, electromagnetic sensor, laser sensor, for detecting the height of outside safety hazard, distance, length, relative velocity and relative acceleration.
Based on a method for the automatic train protection system of image recognition and sensor fusion, it is characterized in that, comprise the following steps:
1) first carried out the image acquisition of train periphery running environment by image acquisition units, go to step 2);
2) graphics processing unit carries out image procossing by the picture signal sent image acquisition units, the approximate range at the outside safety hazard place of train operating safety may be affected around preliminary identification train, and measure they height, distance, shape, length, position, relative velocity and relative acceleration, if recognize safety hazard, go to step 3); Then do not return step 1) proceed image acquisition;
3) approximate range of safety hazard that sensor fusion processing unit is determined for graphics processing unit starts auxiliary perception unit and carries out secondary perception, goes to step 4);
4) association attributes of the outside safety hazard tentatively determined by image recognition that sensor fusion processing unit is sent by the association attributes of the outside safety hazard detected passed back auxiliary perception unit and graphics processing unit carries out the convergence analysis of security risk, accurately determine the association attributes of outside safety hazard further, if determine that association attributes is consistent with the association attributes of graphics processing unit identification, go to step 5), inconsistent, return step 1);
5) risk class of the exact properties determination safety hazard of the outside safety hazard that early warning result output unit exports according to sensor fusion processing unit, exports corresponding alarm message or emergency brake command, then returns step 1).
It is as follows that described graphics processing unit carries out process concrete steps to picture signal:
101) gray processing process is carried out to the view data of image acquisition units collection, remove RGB information in view data, be converted to the gray values information only having 0 ~ 255;
102) pretreatment of image: noise filtering is carried out to image;
103) gray level image is carried out to the average gray calculating of pixel, as the input judging train operation environment;
104) carry out strong and weak light running environment to judge;
105) according to step 104) in judgement, if be low light environment, then use operator 2 to carry out the binaryzation of image; If be strong light environment, then use operator 1 to carry out the binaryzation of image, wherein operator 1 and 2 has merged the image segmentation algorithm realized based on threshold value and this two kinds of methods of Image Edge-Detection, and its main difference is the value difference of different light lower threshold value;
106) by step 105) in binary map carry out Morphological scale-space, comprise view data is expanded, these the four kinds of morphologic algebraic operations of burn into opening and closing, and the filtering process to object edge information;
107) again area filling is carried out to the binary map after Morphological scale-space, closed object edge inside is filled, thus obtains the information of whole object;
108) track identification and outside safety hazard identification are carried out to the binary map finally obtained;
Rail level due to track is very smooth after train operation, then by train car light with irradiate, rail level just seems very bright, in binary map after treatment, the information of rail level just becomes two white long band, based on the falt characteristic of track, is identified by the orbit of train;
When identifying outside safety hazard to train front, using the track of train as reference baseline, Iamge Segmentation is become 5 pieces of identified regions, comprise on the left of left rail, on the right side of middle, the right rail of two rails and two rail levels, for the region recognition of two rail levels, as long as rail level whether continuously or whether rule in recognition image, if discontinuous or irregular, then there is outside safety hazard, for joint or the divergence point of track, then carry out exclusive PCR by whether there is an other continuous whole track; For other 3 regions, then identify whether large-area white portion, this area according to white portion position in the picture carry out the value of different threshold value, if there is the white portion meeting respective threshold Rule of judgment, be judged as outside safety hazard, the object simultaneously judged for the beacon on track, waveguide, sleeper, guard rail interference is got rid of by specific priori features data;
109) when judging there is an outside safety hazard, then by image measurement technology, the relative velocity of the shape of outside safety hazard, location information and train and relative acceleration information being measured, exports to sensor fusion processing unit; Then relevant information is not exported without outside safety hazard.
The described concrete treating process of sensor fusion processing unit is as follows:
201) graphics processing unit recognizes the existence of safety hazard, sensor fusion processing unit from graphics processing unit obtain the height of front safety hazard, distance, length, position, with the relative velocity of train and relative acceleration information;
202) the safety hazard angle information that This train is bound for XXX is relatively gone out by the positional information calculation of the front safety hazard obtained from graphics processing unit;
203) start the auxiliary perception unit of correspondence position according to angle information, by auxiliary perception unit obtain safety hazard distance, highly and with the relative velocity of train and relative acceleration information;
204) distance, the elevation information of safety hazard is passed back according to auxiliary perception unit, dangerous factor data with graphics processing unit is compared, if highly have also exceeded vehicle gauge requirement at same position, then can confirm the existence of safety hazard, go to step 205); If comparison data does not find safety hazard, then return step 201);
205) when being judged as there is safety hazard, relative velocity and the relative acceleration information of the distance safety hazard passed back according to auxiliary perception unit have judged whether risk of collision, this deterministic process has taken into full account the influencing factor of acceleration due to gravity under the propagation delay time of the guarantee braking ratio of vehicle, brake command, mute time of vehicle traction, the application time of emergency braking, maximum grade, in conjunction with relative velocity and relative acceleration information with safety hazard, finally obtain the distance that worst case Train may run; This distance compares with current safety hazard and train relative distance, thus judges whether this moment exists risk of collision;
If safety hazard and train have risk of collision when the worst, then export emergency brake command directly to vehicle, thus avoid colliding or reducing the injury of collision to train as much as possible; If safety hazard from train relatively away from, under present speed and acceleration/accel, driver controls train stop before colliding by applying conventional brake, then can export to the corresponding sound and light alarm information of driver, driver is reminded to note the dangerous point in front, driver takes necessary operation that train is stopped in time, thus avoids unnecessary emergency braking process; If driver does not take braking always, when judging to there is risk of collision, then automatic to vehicle enforcement emergency braking; Simultaneously alarm message also to be passed to the running scheduling personnel of trackside by in-vehicle wireless communication system and trackside repair and maintenance personnel carry out the scheduling of follow-up train and the investigation removing work of safety hazard by the information of safety hazard, guarantees follow-up train traffic safety.
Compared with prior art, the present invention can adapt to the complex situations that urban track traffic circulation line occurs, according to circuit actual conditions, automatically realizes the safety precaution of train, greatly improves urban rail transit operation security, be specially:
Make up the limitation of existing ATC system detection, add other cognition technologies by image recognition, complete the active probe to the outside safety hazard within the scope of train operation, identification, assessment to realize automatic protection;
Improve the accident of City Rail Transit System to harm train operating safety further and dispose means, realize the rapid reaction to emergency situation in urban track traffic, meet the demand of City Rail Transit System to safe operation further;
Improve during urban track traffic runs the degree of automation that the accidents such as invader are protected, manually automatic protection is upgraded to the protection of the accidents such as track invader by original by train operator, improve the degree of automation of operation, reduce human error possible in limitation in the pressure of train operator and Si Ji lookout process and operation;
By the alarm message of associated ratings, trackside dispatcher can know the safety hazard on track as early as possible, organizes follow-up train to avert danger location in time;
After trackside repair and maintenance personnel receive alarm message, can organize investigation hazard location in time, timely rid, guarantees traffic safety.
Accompanying drawing explanation
Fig. 1 is present system structural representation;
Fig. 2 is diagram of circuit of the present invention;
Fig. 3 is image processing flow figure of the present invention;
Fig. 4 is sensor fusion processing unit diagram of circuit of the present invention;
Fig. 5 is the exemplary plot after the present invention specifically installs.
Detailed description of the invention
Below in conjunction with the drawings and specific embodiments, the present invention is described in detail.
Embodiment
The present invention relates generally to the sensor fusion technology of image recognition, its basic block diagram as shown in Figure 1: in figure, 1 is image acquisition units, and 2 is graphics processing unit, and 3 is sensor fusion processing unit, and 4 is auxiliary perception unit, and 5 is early warning result output unit.
In the present invention:
Image acquisition units 1 can for ccd video camera, night infrared pick up camera or other meet the pick up camera of use scenes of the present invention, the mainly image resource of Real-time Collection train periphery running environment, then through Video Sync Separator Chip image signal transmission in graphics processing unit 2.
Graphics processing unit 2 is core processing units of carries out image identification, mainly element characteristic data dangerous in train periphery running environment are extracted, identified, its distance of preliminary surveying, shape, relative velocity, relative acceleration (being not limited to above-mentioned attribute), filtration deletion is carried out on the unfavorable disturbing factor that impact judges, thus realizes carrying out actv. Preliminary detection to safety hazard.
Sensor fusion processing unit 3 starts auxiliary perception unit 4 according to the Output rusults of graphics processing unit 2 to carry out secondary perception, and carry out convergence analysis by the association attributes of the outside safety hazard tentatively determined by image recognition sent the association attributes of the outside safety hazard detected that auxiliary perception unit 4 is passed back and graphics processing unit 2, accurately determine every attribute of outside safety hazard further.
Auxiliary perception unit 4 is used to the outside safety hazard in aided detection train periphery running environment, and when pick up camera cannot normally work (as dense fog, heavy rain etc.), replaces image acquisition units 1 as main detection means.It can be the fusion application of multiple sensors, and these sensors also can be carried out array combination by the sensors such as such as radar, electromagnetism, laser.Height, distance, length, relative velocity, the relative acceleration of what it mainly detected is outside safety hazard, but be not limited to above-mentioned attribute.
Early warning result output unit 5 carries out security risk analysis according to every attribute of sensor fusion processing unit 3 to outside safety hazard, determines the output situation of early warning result.According to Severity level, it can be the combination of sound, light, vibrations etc., also can be the alarm message of the associated ratings be presented on human-computer interaction interface, and export to the brake command of motor vehicle braking system.
Connection in Fig. 1 between unit can be wired form, also can be wireless form.
The power supply of modules is directly supplied by vehicle-mounted UPS when unexpected dead electricity, this ensure that the continuation of protection, improves the reliability of traffic safety.
2. flow process of the present invention
Composition graphs 2: operational flow diagram, the main operation sheet of the present invention is as follows:
A. first carried out the image acquisition of train periphery running environment by image acquisition units 1, turn b;
B. graphics processing unit 2 is by carrying out image procossing to the image resource that image acquisition units 1 sends, the approximate range at the outside safety hazard place of train operating safety may be affected around preliminary identification train, and their height of preliminary surveying, distance, shape, length, position, relative velocity, relative acceleration, but be not limited to above-mentioned attribute, if recognize safety hazard, turn c, then do not turn a and proceed image acquisition;
C. the approximate range of safety hazard that sensor fusion processing unit 3 is determined for graphics processing unit 2 starts auxiliary perception unit 4 and carries out secondary perception, turns d;
D. sensor fusion processing unit 3 carries out the convergence analysis of security risk by the association attributes of the outside safety hazard tentatively determined by image recognition sent the association attributes of the outside safety hazard detected that auxiliary perception unit 4 is passed back and graphics processing unit 2, accurately determine the association attributes of outside safety hazard further, if determine that association attributes is consistent with the association attributes of graphics processing unit identification, turn e, inconsistent, turn a;
E. the risk class of the exact properties determination safety hazard of outside safety hazard that exports according to sensor fusion processing unit 3 of early warning result output unit 5, exports corresponding alarm message or emergency brake command, then turns a.
Composition graphs 3: image processing flow figure, the main operation sheet of image procossing of the present invention is as follows:
A. because train traffic safety is higher to image requirement of real-time, for reducing the data volume of subsequent treatment, gray processing process is carried out to the view data that image acquisition units 1 gathers, removes RGB information in view data, be converted to the gray values information only having 0 ~ 255;
B. the pretreatment of image: in order to the basic noise in removal of images, strengthens picture contrast, improves picture quality, so that subsequent treatment and analysis, first carries out noise filtering to image, and relatively more conventional has gaussian filtering, medium filtering, mean filter etc.;
C. gray level image is carried out to the average gray calculating of pixel, as the input judging train operation environment;
D. carry out strong and weak light running environment to judge.The running environment of train is divided into tunnel, overhead and ground.When overhead and ground handling, strong and weak luminous environment is determined by day and night.During daytime, whole gray-scale map brightness ratio is brighter, so its average gray value is less; And the brightness ratio of whole gray-scale map is comparatively dark during night, so its average gray value is larger.And train operation is when tunnel environment, similar with night.Determine the threshold value of the light running environment that can more accurately tell the men from the boys, thus carry out the judgement of the strong and weak luminous environment of train operation;
E. according to the judgement in d, if be low light environment (night or tunnel), then operator 2 is used to carry out the binaryzation of image; If be strong light environment (daytime), then operator 1 is used to carry out the binaryzation of image.Operator 1 and 2 has merged the image segmentation algorithm realized based on threshold value and this two kinds of methods of Image Edge-Detection, and their main difference is the value difference of different light lower threshold value.And the binaryzation of so-called image, according to corresponding algorithm, the gray value of the pixel in gray-scale map is got 0 or 255 exactly, thus make original gray-scale map only have black and white two kinds of colors, and then the irrelevant information in image is carried out filtering;
F. the binary map in e is carried out Morphological scale-space.Morphological scale-space expands to view data exactly, these the four kinds of morphologic algebraic operations of burn into opening and closing, thus makes the marginal information of objects in images more outstanding.Here Morphological scale-space also comprises the filtering process to object edge information, thus makes marginal information more level and smooth;
G. again area filling is carried out to the binary map after Morphological scale-space.So-called area filling is exactly filled closed object edge inside, thus obtains the information of whole object;
H. track identification and outside safety hazard identification are carried out to the binary map finally obtained.Rail level due to track is very smooth after train operation, then by the car light of train to irradiate, rail level just seems very bright, so in binary map after treatment, the information of rail level just becomes two white long band.Again based on the falt characteristic of track, so be easy to the orbit of train to identify.When identifying outside safety hazard to train front, using the track of train as with reference to baseline, Iamge Segmentation is become 5 pieces of identified regions, both on the left of left rail, on the right side of middle, the right rail of two rails and two rail levels.For the region recognition of two rail levels, as long as rail level whether continuously or whether rule in recognition image, if discontinuous or irregular, then there is outside safety hazard, for joint or the divergence point of track, then can carry out exclusive PCR by whether there is an other continuous whole track.For other 3 regions, then identify whether large-area white portion, this area can according to white portion position in the picture carry out the value object of the same size (size can be shown in the picture because of distant relationships different) of different threshold value, if there is the white portion meeting respective threshold Rule of judgment, be judged as outside safety hazard, the object that the interference simultaneously judged for other influences such as the beacon on track, waveguide, sleeper, guard rails judges can be got rid of by specific priori features data.When carrying out outside safety hazard identification to train two side areas, then main using the region within the scope of track clearance as identified region, if there is large-area white portion, be judged as outside safety hazard;
I. when judging there is an outside safety hazard, then by image measurement technology, the information such as the relative velocity of the shape of outside safety hazard, location information and train and relative acceleration being measured, exports to sensor fusion processing unit 3; Then relevant information is not exported without outside safety hazard.
It should be noted that; merely illustrate a kind of feasible image processing method herein; do not refer to that the present invention only uses this image processing method, and be not used to limit the present invention, any image processing method that can realize object of the present invention all belongs to protection scope of the present invention.
Composition graphs 4: sensor fusion processing unit diagram of circuit, the main operation sheet of sensor fusion process of the present invention is as follows:
A. graphics processing unit recognizes the existence of safety hazard, sensor fusion processing unit from graphics processing unit obtain the height of front safety hazard, distance, length, position, with the information such as the relative velocity of train and relative acceleration;
B. the safety hazard angle information that This train is bound for XXX is relatively gone out by the positional information calculation of the front safety hazard obtained from graphics processing unit;
C. the aiding sensors of correspondence position is started according to angle information, aiding sensors can be the sensors such as radar, electromagnetism, laser, also these sensors can be carried out array combination, can require to the difference of gauge the sensor placing relevant position according to vehicle correspondence position, eventually through these aiding sensors obtain safety hazard distance, highly and with the information such as the relative velocity of train and relative acceleration;
D. pass distance, the elevation information of safety hazard back according to aiding sensors, the dangerous factor data with image procossing is compared, if highly have also exceeded vehicle gauge requirement at same position, then can confirm the existence of safety hazard, turn e; If comparison data does not find safety hazard, then turn a;
E., when being judged as there is safety hazard, relative velocity and the relative acceleration information of the distance safety hazard passed back according to aiding sensors have judged whether risk of collision.This deterministic process has taken into full account the factors such as the effect of acceleration due to gravity under the propagation delay time of the guarantee braking ratio of vehicle, brake command, mute time of vehicle traction, the application time of emergency braking, maximum grade, in conjunction with relative velocity and relative acceleration information with safety hazard, finally obtain the distance that worst case Train may run.This distance compares with current safety hazard and train relative distance, thus judges whether this moment exists risk of collision.If safety hazard and train have risk of collision when the worst, then export emergency brake command directly to vehicle, thus avoid colliding or reducing the injury of collision to train as much as possible; If safety hazard from train relatively away from, under present speed and acceleration/accel, driver can control train and stop before colliding by applying conventional brake, then can export to the corresponding sound and light alarm information of driver, driver is reminded to note the dangerous point in front, driver takes necessary operation that train is stopped in time, thus avoids unnecessary emergency braking process; If driver does not take braking always, when judging to there is risk of collision, then automatic to vehicle enforcement emergency braking.Simultaneously alarm message also can be passed to the running scheduling personnel of trackside by in-vehicle wireless communication system and trackside repair and maintenance personnel carry out the scheduling of follow-up train and the investigation removing work of safety hazard by the information of safety hazard, guarantees follow-up train traffic safety.
The running environment different according to them and the requirement to safety precaution, the present invention can carry out different configurations.
Composition graphs 5, tell about a kind of exemplary embodiments of the present invention:
1) Fig. 5 essentially describes present device main annexation ON TRAINS, wherein 5 directly can export brake command to the brake system of train, thus realize train snap catch in case of emergency;
2) according to the running environment of track traffic, the orbital region of train operation ahead is attached most importance to search coverage.Camera is arranged on driver's cab top, and angle is that level is on the lower side, effectively can expand the image acquisition region of pick up camera like this, accomplishes as far as possible detection ahead of time.It should be noted that number and the installation site of camera can be determined according to actual Detection results, should not become the circumscribed factor of restriction the present invention;
3) require respectively to place an aiding sensors in driver's cab left, center, right to the difference of gauge according to vehicle diverse location, middle sensor is arranged on bottom, be mainly used in the obstacle between detection two rail, both sides be then position, lower middle, main realization to the Tracking Recognition of the safety hazard outside track, thus effectively detects the association attributeses such as the height of outside safety hazard fast; It should be noted that number and the installation site of sensor are determined according to the actual Detection results of sensor, should not become the circumscribed factor of restriction the present invention;
4) image recognition is carried out by graphics processing unit 2, the characteristic data of key element dangerous in front track bounds is extracted, identified, carry out secondary identification by auxiliary perception unit 4 and confirm dangerous key element, the outside safety hazard association attributes of precise evaluation is carried out by sensor fusion processing unit 3, come safety hazard security risk analysis by early warning result output unit 5, and point out the emergency brake command of reporting to the police or directly exporting vehicle to driver, thus complete the automatic protection to train operation ahead;
5) above-described embodiment and be not used to limit the present invention; every content not departing from technical solution of the present invention; according to technical spirit of the present invention to any amendment made for any of the above embodiments, equivalent variations and modification, all still belong in the scope of technical solution of the present invention protection.

Claims (8)

1. the automatic train protection system based on image recognition and sensor fusion, it is characterized in that, comprise image acquisition units (1), graphics processing unit (2), sensor fusion processing unit (3), auxiliary perception unit (4) and early warning result output unit (5), described image acquisition units (1), graphics processing unit (2), sensor fusion processing unit (3) are connected successively with early warning result output unit (5), and described auxiliary perception unit (4) is connected with sensor fusion processing unit (3);
The picture signal of described image acquisition units (1) Real-time Collection train periphery running environment, and by image signal transmission in graphics processing unit (2), sensor fusion processing unit (3) is transferred to after described graphics processing unit (2) carries out Preliminary detection to safety hazard, this sensor fusion processing unit (3) starts auxiliary perception unit (4) and carries out secondary perception, and by carrying out convergence analysis to the association attributes of the outside safety hazard that association attributes and the graphics processing unit (2) of the outside safety hazard detected that auxiliary perception unit (4) is passed back send, accurately determine every attribute of outside safety hazard further, early warning result output unit (5) carries out security risk analysis according to every attribute of sensor fusion processing unit (3) to outside safety hazard, determine the output situation of early warning result.
2. a kind of automatic train protection system based on image recognition and sensor fusion according to claim 1, it is characterized in that, described image acquisition units (1) is ccd video camera or night infrared pick up camera, for the picture signal of Real-time Collection train periphery running environment, then through Video Sync Separator Chip image signal transmission in graphics processing unit (2).
3. a kind of automatic train protection system based on image recognition and sensor fusion according to claim 1, it is characterized in that, described graphics processing unit (2) extracts element characteristic data dangerous in the picture signal of train periphery running environment, identifies, measure its distance, shape, relative velocity and relative acceleration, filtration deletion is carried out on the unfavorable disturbing factor that impact judges, thus realizes carrying out actv. Preliminary detection to safety hazard.
4. a kind of automatic train protection system based on image recognition and sensor fusion according to claim 1, it is characterized in that, described auxiliary perception unit (4) is used to the outside safety hazard in aided detection train periphery running environment, and replaces image acquisition units (1) when pick up camera cannot normally work as detection means.
5. a kind of automatic train protection system based on image recognition and sensor fusion according to claim 4, it is characterized in that, described auxiliary perception unit (4) is radar, one or more combination in electromagnetic sensor, laser sensor, for detecting the height of outside safety hazard, distance, length, relative velocity and relative acceleration.
6., as claimed in claim 1 based on a method for the automatic train protection system of image recognition and sensor fusion, it is characterized in that, comprise the following steps:
1) first carried out the image acquisition of train periphery running environment by image acquisition units (1), go to step 2);
2) graphics processing unit (2) is by carrying out image procossing to the picture signal that image acquisition units (1) sends, the approximate range at the outside safety hazard place of train operating safety may be affected around preliminary identification train, and measure they height, distance, shape, length, position, relative velocity and relative acceleration, if recognize safety hazard, go to step 3); Then do not return step 1) proceed image acquisition;
3) approximate range of safety hazard that sensor fusion processing unit (3) is determined for graphics processing unit (2) starts auxiliary perception unit (4) and carries out secondary perception, goes to step 4);
4) sensor fusion processing unit (3) carries out the convergence analysis of security risk by the association attributes of the outside safety hazard tentatively determined by image recognition sent the association attributes of the outside safety hazard detected that auxiliary perception unit (4) is passed back and graphics processing unit (2), accurately determine the association attributes of outside safety hazard further, if determine that association attributes is consistent with the association attributes that graphics processing unit (2) identifies, go to step 5), inconsistent, return step 1);
5) risk class of the exact properties determination safety hazard of the outside safety hazard that early warning result output unit (5) exports according to sensor fusion processing unit (3), export corresponding alarm message or emergency brake command, then return step 1).
7. the method for a kind of automatic train protection system based on image recognition and sensor fusion according to claim 5, is characterized in that, it is as follows that described graphics processing unit (2) carries out process concrete steps to picture signal:
101) gray processing process is carried out to the view data that image acquisition units (1) gathers, remove RGB information in view data, be converted to the gray values information only having 0 ~ 255;
102) pretreatment of image: noise filtering is carried out to image;
103) gray level image is carried out to the average gray calculating of pixel, as the input judging train operation environment;
104) carry out strong and weak light running environment to judge;
105) according to step 104) in judgement, if be low light environment, then use operator 2 to carry out the binaryzation of image; If be strong light environment, then use operator 1 to carry out the binaryzation of image, wherein operator 1 and 2 has merged the image segmentation algorithm realized based on threshold value and this two kinds of methods of Image Edge-Detection, and its main difference is the value difference of different light lower threshold value;
106) by step 105) in binary map carry out Morphological scale-space, comprise view data is expanded, these the four kinds of morphologic algebraic operations of burn into opening and closing, and the filtering process to object edge information;
107) again area filling is carried out to the binary map after Morphological scale-space, closed object edge inside is filled, thus obtains the information of whole object;
108) track identification and outside safety hazard identification are carried out to the binary map finally obtained;
Rail level due to track is very smooth after train operation, then by train car light with irradiate, rail level just seems very bright, in binary map after treatment, the information of rail level just becomes two white long band, based on the falt characteristic of track, is identified by the orbit of train;
When identifying outside safety hazard to train front, using the track of train as reference baseline, Iamge Segmentation is become 5 pieces of identified regions, comprise on the left of left rail, on the right side of middle, the right rail of two rails and two rail levels, for the region recognition of two rail levels, as long as rail level whether continuously or whether rule in recognition image, if discontinuous or irregular, then there is outside safety hazard, for joint or the divergence point of track, then carry out exclusive PCR by whether there is an other continuous whole track; For other 3 regions, then identify whether large-area white portion, this area according to white portion position in the picture carry out the value of different threshold value, if there is the white portion meeting respective threshold Rule of judgment, be judged as outside safety hazard, the object simultaneously judged for the beacon on track, waveguide, sleeper, guard rail interference is got rid of by specific priori features data;
109) when judging there is outside safety hazard, then pass through image measurement technology, the relative velocity of the shape of outside safety hazard, location information and train and relative acceleration information are measured, exports to sensor fusion processing unit (3); Then relevant information is not exported without outside safety hazard.
8. the method for a kind of automatic train protection system based on image recognition and sensor fusion according to claim 5, is characterized in that, the concrete treating process of described sensor fusion processing unit (3) is as follows:
201) graphics processing unit (2) recognizes the existence of safety hazard, sensor fusion processing unit (3) from graphics processing unit (2) obtain the height of front safety hazard, distance, length, position, with the relative velocity of train and relative acceleration information;
202) the safety hazard angle information that This train is bound for XXX is relatively gone out by the positional information calculation of the front safety hazard obtained from graphics processing unit (2);
203) start the auxiliary perception unit (4) of correspondence position according to angle information, by auxiliary perception unit (4) obtain safety hazard distance, highly and with the relative velocity of train and relative acceleration information;
204) distance, the elevation information of safety hazard is passed back according to auxiliary perception unit (4), dangerous factor data with graphics processing unit (2) is compared, if highly have also exceeded vehicle gauge requirement at same position, then can confirm the existence of safety hazard, go to step 205); If comparison data does not find safety hazard, then return step 201);
205) when being judged as there is safety hazard, relative velocity and the relative acceleration information of the distance safety hazard passed back according to auxiliary perception unit (4) have judged whether risk of collision, this deterministic process has taken into full account the influencing factor of acceleration due to gravity under the propagation delay time of the guarantee braking ratio of vehicle, brake command, mute time of vehicle traction, the application time of emergency braking, maximum grade, in conjunction with relative velocity and relative acceleration information with safety hazard, finally obtain the distance that worst case Train may run; This distance compares with current safety hazard and train relative distance, thus judges whether this moment exists risk of collision;
If safety hazard and train have risk of collision when the worst, then export emergency brake command directly to vehicle, thus avoid colliding or reducing the injury of collision to train as much as possible; If safety hazard from train relatively away from, under present speed and acceleration/accel, driver controls train stop before colliding by applying conventional brake, then can export to the corresponding sound and light alarm information of driver, driver is reminded to note the dangerous point in front, driver takes necessary operation that train is stopped in time, thus avoids unnecessary emergency braking process; If driver does not take braking always, when judging to there is risk of collision, then automatic to vehicle enforcement emergency braking; Simultaneously alarm message also to be passed to the running scheduling personnel of trackside by in-vehicle wireless communication system and trackside repair and maintenance personnel carry out the scheduling of follow-up train and the investigation removing work of safety hazard by the information of safety hazard, guarantees follow-up train traffic safety.
CN201410562470.7A 2014-10-21 2014-10-21 Automatic train protection system and method based on image identification and multi- perception fusion Active CN104386092B (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201410562470.7A CN104386092B (en) 2014-10-21 2014-10-21 Automatic train protection system and method based on image identification and multi- perception fusion
HK15105381.9A HK1205071A1 (en) 2014-10-21 2015-06-05 An automatic train protection system and method based on image recognition and multi perception fusion

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201410562470.7A CN104386092B (en) 2014-10-21 2014-10-21 Automatic train protection system and method based on image identification and multi- perception fusion

Publications (2)

Publication Number Publication Date
CN104386092A true CN104386092A (en) 2015-03-04
CN104386092B CN104386092B (en) 2017-02-22

Family

ID=52604080

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201410562470.7A Active CN104386092B (en) 2014-10-21 2014-10-21 Automatic train protection system and method based on image identification and multi- perception fusion

Country Status (2)

Country Link
CN (1) CN104386092B (en)
HK (1) HK1205071A1 (en)

Cited By (31)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105083324A (en) * 2015-07-28 2015-11-25 陕西西北铁道电子有限公司 Method and device for locomotive anticollision using on-board optical detection mechanism combining with auxiliary image pickup
CN105398471A (en) * 2014-09-08 2016-03-16 通用电气公司 Optical Route Examination System And Method
CN106043350A (en) * 2016-05-27 2016-10-26 同济大学 Independent control braking system used for tramcar early warning avoidance
CN106364530A (en) * 2016-10-14 2017-02-01 王志伟 Train control system and method based on radar
CN106585670A (en) * 2016-12-09 2017-04-26 交控科技股份有限公司 Video-based urban rail transit ahead train detection system and method
CN106741010A (en) * 2016-12-21 2017-05-31 吴中区穹窿山倪源交通器材经营部 A kind of anti-collision early warning method of rail transit train
CN106926872A (en) * 2017-03-29 2017-07-07 宁夏宁信信息科技有限公司 Inter-city passenger rail line security control method and system
CN106956687A (en) * 2015-03-31 2017-07-18 江苏理工学院 Subway traffic conflict early warning method based on robust strategy
CN107399342A (en) * 2017-06-30 2017-11-28 北京中智润邦科技有限公司 A kind of railway circumference, which is artificially invaded, determines method, apparatus and system
CN108027615A (en) * 2015-09-11 2018-05-11 罗伯特·博世有限公司 Method and apparatus for running vehicle
CN108082215A (en) * 2016-11-23 2018-05-29 比亚迪股份有限公司 Track traffic method for safety monitoring and monitoring system and track train
CN108583622A (en) * 2018-04-02 2018-09-28 交控科技股份有限公司 Recognition methods, device, equipment and the medium of rail traffic situation
CN108583620A (en) * 2018-04-02 2018-09-28 交控科技股份有限公司 The processor and early warning system of train assistance tracking early warning
CN109311495A (en) * 2016-06-13 2019-02-05 普拉塞-陶伊尔铁路机械出口股份有限公司 For safeguarding the method and system of the vehicle line of rail vehicle
CN109598187A (en) * 2018-10-15 2019-04-09 西北铁道电子股份有限公司 Obstacle recognition method, differentiating obstacle and railcar servomechanism
CN109664854A (en) * 2017-10-17 2019-04-23 杭州海康威视数字技术股份有限公司 A kind of automobile method for early warning, device and electronic equipment
CN109720380A (en) * 2017-10-31 2019-05-07 交控科技股份有限公司 For hiding the initiative recognition system and hiding train method for removing that train excludes
CN109977748A (en) * 2019-01-10 2019-07-05 浙江众合科技股份有限公司 A kind of train obstacle detection method of more integration technologies
CN110023171A (en) * 2016-12-07 2019-07-16 西门子移动有限责任公司 Method for distinguishing, equipment and rail vehicle, especially rolling stock are known for the dangerous situation in rail traffic, especially in railway traffic
CN110248858A (en) * 2016-12-07 2019-09-17 西门子交通有限责任公司 For in rail traffic the image analysis based on track, in particular for method, equipment and the rail vehicle of the image analysis based on rail in railway traffic, especially rolling stock
CN110712665A (en) * 2019-10-22 2020-01-21 深圳市微度数字技术有限公司 Intelligent auxiliary device for safe driving of locomotive
CN111032428A (en) * 2017-08-28 2020-04-17 黑拉有限责任两合公司 Method for operating at least one light module of a lighting unit of a vehicle, lighting unit, computer program product and computer-readable medium
CN111114561A (en) * 2019-12-19 2020-05-08 中铁轨道交通装备有限公司 Tramcar auxiliary driving system based on ADAS technology and control method
CN111824213A (en) * 2019-04-18 2020-10-27 中车株洲电力机车研究所有限公司 Automatic driving method and system for heavy-duty locomotive
CN112061180A (en) * 2020-08-28 2020-12-11 交控科技股份有限公司 Turnout side impact protection method and system
CN112298285A (en) * 2019-07-26 2021-02-02 比亚迪股份有限公司 Train control system, train control method and train
CN112441075A (en) * 2019-08-30 2021-03-05 比亚迪股份有限公司 Rail transit external environment sensing system and method and rail transit equipment
TWI778149B (en) * 2017-10-23 2022-09-21 日商大福股份有限公司 Vehicle
CN115331392A (en) * 2022-08-17 2022-11-11 卡斯柯信号有限公司 Vehicle-mounted autonomous disaster prevention early warning protection system based on Beidou short message
CN116461571A (en) * 2022-09-20 2023-07-21 江苏中车数字科技有限公司 Obstacle sensing system based on vision, radio frequency positioning and multi-line laser fusion
CN116834802A (en) * 2023-07-03 2023-10-03 湖北空间智能技术有限公司 Method, equipment and storage medium for detecting and positioning invaded objects in operation train track

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070170315A1 (en) * 2006-01-20 2007-07-26 Gedalyahu Manor Method of detecting obstacles on railways and preventing train accidents
CN201296263Y (en) * 2008-10-17 2009-08-26 陈立 A train operation ahead dynamic monitoring system
CN103287461A (en) * 2012-02-28 2013-09-11 哈尔滨精达自动化技术开发有限责任公司 Railway vehicle-mounted device for monitoring foreign matter invasion
JP2014012457A (en) * 2012-07-04 2014-01-23 Panasonic Corp Obstacle-within-railroad-crossing detection supporting system

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070170315A1 (en) * 2006-01-20 2007-07-26 Gedalyahu Manor Method of detecting obstacles on railways and preventing train accidents
CN201296263Y (en) * 2008-10-17 2009-08-26 陈立 A train operation ahead dynamic monitoring system
CN103287461A (en) * 2012-02-28 2013-09-11 哈尔滨精达自动化技术开发有限责任公司 Railway vehicle-mounted device for monitoring foreign matter invasion
JP2014012457A (en) * 2012-07-04 2014-01-23 Panasonic Corp Obstacle-within-railroad-crossing detection supporting system

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
李斌等: "基于机器视觉的智能车辆障碍物检测方法研究", 《公路交通科技》 *
黄伟: "基于雷达和机器视觉的车辆前方障碍物检测系统设计与实现", 《中国优秀硕士学位论文全文数据库 信息科技辑》 *

Cited By (40)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105398471A (en) * 2014-09-08 2016-03-16 通用电气公司 Optical Route Examination System And Method
CN105398471B (en) * 2014-09-08 2019-02-19 通用电气公司 Optical line inspection system and method
CN106956687A (en) * 2015-03-31 2017-07-18 江苏理工学院 Subway traffic conflict early warning method based on robust strategy
CN105083324A (en) * 2015-07-28 2015-11-25 陕西西北铁道电子有限公司 Method and device for locomotive anticollision using on-board optical detection mechanism combining with auxiliary image pickup
CN108027615A (en) * 2015-09-11 2018-05-11 罗伯特·博世有限公司 Method and apparatus for running vehicle
CN108027615B (en) * 2015-09-11 2021-08-10 罗伯特·博世有限公司 Method and device for operating a vehicle
CN106043350A (en) * 2016-05-27 2016-10-26 同济大学 Independent control braking system used for tramcar early warning avoidance
CN109311495A (en) * 2016-06-13 2019-02-05 普拉塞-陶伊尔铁路机械出口股份有限公司 For safeguarding the method and system of the vehicle line of rail vehicle
CN106364530A (en) * 2016-10-14 2017-02-01 王志伟 Train control system and method based on radar
CN108082215A (en) * 2016-11-23 2018-05-29 比亚迪股份有限公司 Track traffic method for safety monitoring and monitoring system and track train
CN110023171A (en) * 2016-12-07 2019-07-16 西门子移动有限责任公司 Method for distinguishing, equipment and rail vehicle, especially rolling stock are known for the dangerous situation in rail traffic, especially in railway traffic
CN110248858A (en) * 2016-12-07 2019-09-17 西门子交通有限责任公司 For in rail traffic the image analysis based on track, in particular for method, equipment and the rail vehicle of the image analysis based on rail in railway traffic, especially rolling stock
CN106585670B (en) * 2016-12-09 2018-04-17 交控科技股份有限公司 To train detection systems and method before a kind of urban track traffic based on video
CN106585670A (en) * 2016-12-09 2017-04-26 交控科技股份有限公司 Video-based urban rail transit ahead train detection system and method
CN106741010A (en) * 2016-12-21 2017-05-31 吴中区穹窿山倪源交通器材经营部 A kind of anti-collision early warning method of rail transit train
CN106926872A (en) * 2017-03-29 2017-07-07 宁夏宁信信息科技有限公司 Inter-city passenger rail line security control method and system
CN107399342A (en) * 2017-06-30 2017-11-28 北京中智润邦科技有限公司 A kind of railway circumference, which is artificially invaded, determines method, apparatus and system
CN111032428A (en) * 2017-08-28 2020-04-17 黑拉有限责任两合公司 Method for operating at least one light module of a lighting unit of a vehicle, lighting unit, computer program product and computer-readable medium
CN111032428B (en) * 2017-08-28 2023-08-04 黑拉有限责任两合公司 Method for controlling at least one light module of a lighting unit of a vehicle, lighting unit, computer program product and computer-readable medium
CN109664854A (en) * 2017-10-17 2019-04-23 杭州海康威视数字技术股份有限公司 A kind of automobile method for early warning, device and electronic equipment
CN109664854B (en) * 2017-10-17 2020-11-03 杭州海康威视数字技术股份有限公司 Automobile early warning method and device and electronic equipment
TWI778149B (en) * 2017-10-23 2022-09-21 日商大福股份有限公司 Vehicle
CN109720380A (en) * 2017-10-31 2019-05-07 交控科技股份有限公司 For hiding the initiative recognition system and hiding train method for removing that train excludes
CN109720380B (en) * 2017-10-31 2020-11-13 交控科技股份有限公司 Active recognition system for hidden train removal and hidden train removal method
US10668936B2 (en) 2018-04-02 2020-06-02 Traffic Control Technology Co., Ltd Processor and system for train assistance tracking and early-warning
CN108583620B (en) * 2018-04-02 2019-08-30 交控科技股份有限公司 The processor and early warning system of train assistance tracking early warning
CN108583620A (en) * 2018-04-02 2018-09-28 交控科技股份有限公司 The processor and early warning system of train assistance tracking early warning
CN108583622A (en) * 2018-04-02 2018-09-28 交控科技股份有限公司 Recognition methods, device, equipment and the medium of rail traffic situation
CN109598187A (en) * 2018-10-15 2019-04-09 西北铁道电子股份有限公司 Obstacle recognition method, differentiating obstacle and railcar servomechanism
CN109977748B (en) * 2019-01-10 2021-07-20 浙江众合科技股份有限公司 Train obstacle detection method based on multi-fusion technology
CN109977748A (en) * 2019-01-10 2019-07-05 浙江众合科技股份有限公司 A kind of train obstacle detection method of more integration technologies
CN111824213A (en) * 2019-04-18 2020-10-27 中车株洲电力机车研究所有限公司 Automatic driving method and system for heavy-duty locomotive
CN112298285A (en) * 2019-07-26 2021-02-02 比亚迪股份有限公司 Train control system, train control method and train
CN112441075A (en) * 2019-08-30 2021-03-05 比亚迪股份有限公司 Rail transit external environment sensing system and method and rail transit equipment
CN110712665A (en) * 2019-10-22 2020-01-21 深圳市微度数字技术有限公司 Intelligent auxiliary device for safe driving of locomotive
CN111114561A (en) * 2019-12-19 2020-05-08 中铁轨道交通装备有限公司 Tramcar auxiliary driving system based on ADAS technology and control method
CN112061180A (en) * 2020-08-28 2020-12-11 交控科技股份有限公司 Turnout side impact protection method and system
CN115331392A (en) * 2022-08-17 2022-11-11 卡斯柯信号有限公司 Vehicle-mounted autonomous disaster prevention early warning protection system based on Beidou short message
CN116461571A (en) * 2022-09-20 2023-07-21 江苏中车数字科技有限公司 Obstacle sensing system based on vision, radio frequency positioning and multi-line laser fusion
CN116834802A (en) * 2023-07-03 2023-10-03 湖北空间智能技术有限公司 Method, equipment and storage medium for detecting and positioning invaded objects in operation train track

Also Published As

Publication number Publication date
CN104386092B (en) 2017-02-22
HK1205071A1 (en) 2015-12-11

Similar Documents

Publication Publication Date Title
CN104386092A (en) Automatic train protection system and method based on image identification and multi- perception fusion
CN109977748B (en) Train obstacle detection method based on multi-fusion technology
CN110164183A (en) A kind of safety assistant driving method for early warning considering his vehicle driving intention under the conditions of truck traffic
CN110155045A (en) A kind of vehicle promptly accelerates Self-Protection Subsystem and its self-protection method
CN110077399A (en) A kind of vehicle collision avoidance method merged based on roadmarking, wheel detection
CN109272776A (en) A kind of rear-end collision prevention early warning system based on image procossing
CN107867283A (en) Integrated form FCW/ACC/AEB systems and vehicle based on forecast model
CN104346955A (en) Man-vehicle communication-based pedestrian collision avoiding method and collision avoiding system
CN106114502A (en) A kind of intelligent automobile aid system
CN102991504A (en) Device and method for determining and prewarning lane change safety of drivers
CN103832434A (en) Traveling safety control system and method
CN104648389B (en) Automobile safety protective method and system
CN205768952U (en) Night intelligence auxiliary driving device
CN109001743A (en) Tramcar anti-collision system
CN112606831A (en) Anti-collision warning information external interaction method and system for passenger car
CN105083328A (en) Method and device for locomotive anticollision using on-board optical detection combining with wireless bridge communication
CN109677377A (en) A kind of road vehicle limit for height finds out system and the anti-collision protection device that actively brakes
CN105438147A (en) Locomotive collision avoidance method combining locomotive-mounted optical detection and point-to-point communication
CN114387785A (en) Safety management and control method and system based on intelligent highway and storable medium
CN207579859U (en) A kind of vehicle collision prevention system based on cognition technology
CN114537374A (en) Vehicle front anti-collision system based on travelable area
CN208847836U (en) Tramcar anti-collision system
CN105083326A (en) Method and device for locomotive anticollision using optical detection mechanism to track steel rail track
CN116534085A (en) Railway shunting operation system and implementation method thereof
CN111681383A (en) Railway locomotive running ground environment sensing technology

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 1205071

Country of ref document: HK

C14 Grant of patent or utility model
GR01 Patent grant
REG Reference to a national code

Ref country code: HK

Ref legal event code: GR

Ref document number: 1205071

Country of ref document: HK