CN106157662B - The limitation speed display device of vehicle - Google Patents
The limitation speed display device of vehicle Download PDFInfo
- Publication number
- CN106157662B CN106157662B CN201610302675.0A CN201610302675A CN106157662B CN 106157662 B CN106157662 B CN 106157662B CN 201610302675 A CN201610302675 A CN 201610302675A CN 106157662 B CN106157662 B CN 106157662B
- Authority
- CN
- China
- Prior art keywords
- speed
- limitation speed
- display
- auxiliary
- limitation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012545 processing Methods 0.000 claims description 56
- 238000001514 detection method Methods 0.000 abstract description 10
- 238000000034 method Methods 0.000 description 5
- 238000003860 storage Methods 0.000 description 5
- 230000014759 maintenance of location Effects 0.000 description 4
- 230000033228 biological regulation Effects 0.000 description 2
- 238000012790 confirmation Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06V20/582—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of traffic signs
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/09—Arrangements for giving variable traffic instructions
- G08G1/0962—Arrangements for giving variable traffic instructions having an indicator mounted inside the vehicle, e.g. giving voice messages
- G08G1/0967—Systems involving transmission of highway information, e.g. weather, speed limits
- G08G1/096733—Systems involving transmission of highway information, e.g. weather, speed limits where a selection of the information might take place
- G08G1/09675—Systems involving transmission of highway information, e.g. weather, speed limits where a selection of the information might take place where a selection from the received information takes place in the vehicle
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60Q—ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL
- B60Q9/00—Arrangement or adaptation of signal devices not provided for in one of main groups B60Q1/00 - B60Q7/00, e.g. haptic signalling
Abstract
The present invention relates to the limitation speed display device (10) of vehicle.The device possesses the display (13) of display limitation speed.Limitation speed of the view data that the device is based on being got by image acquiring device (11) shown by come the traffic sign in detection image (60).In the case that traffic sign in the picture is the mark (52) with auxiliary sign, when the limitation speed Vsub_new detected, which is more than, limits speed in the speed Vst that display is shown as standard, the limitation speed which makes to detect is shown in display as standard limitation speed.On the other hand, in the case that even if traffic sign in the picture is the mark with auxiliary sign, when the limitation speed detected is to limit speed below the speed that display is shown as standard, the limitation speed that the device does not also make to detect is shown in display as standard limitation speed, and maintains the display for the speed that display limits speed as standard and just showing.
Description
Technical field
The present invention relates to limitation speed shown on occupant (predominantly driver) display traffic sign for vehicle
" the limitation speed display device of vehicle ".
Background technology
The friendship of the display information related with the limitation speed for the vehicle travelled on the road is provided with road
Logical mark (road sign).Record what a kind of limitation speed to shown by the traffic sign was shown in patent document 1
The limitation speed display device of vehicle.The device (hereinafter referred to as " existing apparatus ".) clapped based on the camera by being equipped on vehicle
The data of the image of the landscape for the vehicle front taken the photograph identify the traffic sign in image, and the traffic sign that this is identified
The driver being shown in vehicle is capable of the display that the position of visual confirmation is set.
Also, as traffic sign, exist be accompanied with show limitation speed and applied condition (weather, vehicle and
Period etc.) board (auxiliary sign) traffic sign (hereinafter referred to as " band auxiliary sign traffic sign ".).In patent document 2
Record whether a kind of limitation speed judged shown by the traffic sign with auxiliary sign is applied to be provided with this with auxiliary
The landmark identification device of the vehicle travelled is helped on the road of the traffic sign of mark.
Patent document 1：Japanese Unexamined Patent Publication 2010-205160 publications
Patent document 2：Japanese Unexamined Patent Publication 2010-282278 publications
However, for example when limitation speed that situation of the display suitable for snowfall is provided with road is with auxiliary sign
In the case of traffic sign, almost the road be both provided with display suitable for be not situation snowfall limitation speed friendship
Logical mark.That is, it is provided with most cases in the road for being provided with the traffic sign with auxiliary sign and is not accompanied with auxiliary mark
The common traffic sign (hereinafter referred to as " standard traffic mark " of will.).Moreover, in general, the traffic mark with auxiliary sign
The limitation speed shown in will is less than the limitation speed shown by standard traffic mark.
Also it is possible to limitation speed display device is produced due to some reasons (for example, by reasons such as other occlusions)
And the detection of standard traffic mark is caused to fail, as a result, the limitation speed shown by limitation speed display device is with originally should
Situation of the limitation speed of the application compared to excessively small speed.Also, due to auxiliary sign is typically small and its described in
Have that the situation of many words is more, although so may also produce limitation speed display device detects the traffic with auxiliary sign
Mark, but the situation of the content of the condition (applicable elements) shown by the auxiliary sign can not be detected.
If considering case above, at the time of the limitation speed shown by the traffic sign with auxiliary sign is detected,
In the case that the limitation detected the speed is more than " the limitation speed shown by limitation speed display device ", detect this
Limitation speed be shown in limitation speed display device and can make " although the speed that limitation speed than that should apply is small
But the possibility that the speed of the closer limitation speed that should be applied " is shown in limitation speed display device is higher.
Also, according to national difference, it " be not limitation with showing place name etc. on the board of auxiliary sign same shape to have yet
The situation of the word of the application conditions of speed ".The traffic sign with board is identified as the traffic sign with auxiliary sign sometimes.
If in this case, not using the limitation speed shown by the traffic sign with board as the speed for showing limitation speed display device
The candidate of degree, the then possibility that correctly limitation speed is not displayed become higher.
The content of the invention
The present invention completes to tackle the problem.That is, can it is an object of the present invention to provide one kind
Display is " although the small speed of limitation speed than that should apply but approach the speed for the limitation speed that should be applied
" the limitation speed display device of vehicle " of the possibility higher of degree " or " the limitation speed that should be applied ".
" the limitation speed display device (hereinafter referred to as " apparatus of the present invention ") of vehicle " of the present invention is equipped on car
, possess：
Display, for showing the information related with the limitation speed for the road that the vehicle is travelling；
Image acquiring device, the landscape for shooting the direct of travel of the vehicle obtain the view data of the landscape；And
Processing unit, the limitation shown by the traffic sign in the landscape photographed is detected based on described image data
Speed, and the limitation speed detected is shown in the display as standard limitation speed.
The processing unit shown based on the view data got described in the institute of limitation speed that detects
State traffic sign whether be the mark with auxiliary sign judgement.
Also, it is not the band auxiliary when obtaining the traffic sign in the case where newly detecting the limitation speed
During this judgement of the mark of mark, the processing unit makes the limitation speed newly detected be shown as standard limitation speed
It is shown in the display.
Thereby, it is possible to by be not shown by the traffic sign (standard traffic mark) of the mark with auxiliary sign limitation speed
Degree is shown in display as standard limitation speed.
Also, it is the band auxiliary mark when obtaining the traffic sign in the case where newly detecting the limitation speed
During this judgement of the mark of will, it is more than in the limitation speed newly detected and limits speed as the standard in the display
During the speed spent and just shown, the processing unit makes the limitation speed that this is newly detected be shown in this as standard limitation speed
Display.
Thereby, it is possible to will be " although the speed for the limitation speed for being less than applying but closer should answer
By the use of limitation speed speed " or " the limitation speed that should be applied " as standard limitation speed be shown in limitation speed
The possibility of display device becomes higher.
Also, when even to obtain the traffic sign be described this judgement of the mark with auxiliary sign, also described
When the limitation speed newly detected is below the speed for limiting speed as the standard in the display and just showing, do not make
The limitation speed newly detected is shown in the display as standard limitation speed, and maintains the display as the mark
The display for the speed that standard limits speed and just shows.
It is to limit speed as standard in display and just showing in the limitation speed shown by the mark with auxiliary sign
Speed below in the case of, it is unclear that limitation speed shown by the mark with auxiliary sign and in display as standard
Which of limitation speed and the speed that just shows is speed closer to " the limitation speed that should be applied ".Therefore, exist
" the limitation speed shown by the mark with auxiliary sign " newly detected is to limit speed as standard in display and just showing
When below the speed shown, display is shown in by the limitation speed for not making newly to detect, can be prevented it is not clear whether more connecing
The speed for the limitation speed that should closely apply is shown in display as standard limitation speed.
Also, it is configured to also show auxiliary limitation speed in addition to showing the standard limitation speed in the display
In the case of degree, the processing unit is configured in the case where newly detecting the limitation speed, when obtaining the traffic
Mark is described this judgement of mark with auxiliary sign and the limitation speed newly detected is in the display conduct
When below the speed that the auxiliary limits speed and just shows, the limitation speed that this is newly detected is set to limit speed as the auxiliary
It is shown in the display.
Thus, it is as auxiliary in display in " the limitation speed shown by the mark with auxiliary sign " newly detected
When below the speed for limiting speed and just showing, limitation speed that this is newly detected can be made to be shown in as auxiliary limitation speed
Display.
Alternatively, it is configured to also show auxiliary limitation speed in addition to showing the standard limitation speed in the display
In the case of degree, the processing unit is configured in the case where newly detecting the limitation speed, when obtaining the traffic
Mark is described this judgement of mark with auxiliary sign and identifies that the auxiliary sign of the mark with auxiliary sign is shown
During the content shown, no matter the limitation speed newly detected whether be the display as it is described auxiliary limitation speed and
Below the speed just shown, the limitation speed that this is newly detected all is set to be shown in the display as auxiliary limitation speed.
Thus, in the case where identifying the content shown by the auxiliary sign of the mark with auxiliary sign, can make new
The limitation speed detected is shown in display as auxiliary limitation speed.
On the other hand, processing unit can also be configured in the case where newly detecting the limitation speed, described when obtaining
Traffic sign be it is described with auxiliary sign indicate this judgement and it is unidentified go out the mark with auxiliary sign auxiliary mark
During content shown by will, the limitation speed newly detected be the display as the auxiliary limitation speed and
When below the speed just shown, the limitation speed that this is newly detected is set to be shown in the display as auxiliary limitation speed.
Thus, it is unidentified go out the mark with auxiliary sign auxiliary sign shown by content in the case of, only new
" the limitation speed shown by the mark with auxiliary sign " detected is just being shown as auxiliary limitation speed in display
Speed below when, the limitation speed that can make newly to detect is shown in display.
Also, the processing unit can also be configured in the case where newly detecting the limitation speed, described when obtaining
Traffic sign is described this judgement of the mark with auxiliary sign and is not shown as the auxiliary limit in the display
During the speed of speed processed, the limitation speed that this is newly detected is set to be shown in the display as auxiliary limitation speed.
Thus, when not showing auxiliary limitation speed over the display, " the mark with auxiliary sign newly detected can be made
Limitation speed shown by will " is shown in display as auxiliary limitation speed.
With reference to following attached drawing and according to the explanation of described the embodiments of the present invention, it is easier to understand the present invention
Other purposes, other features and bonus.
Brief description of the drawings
Fig. 1 is the summary construction diagram of the limitation speed display device of the vehicle involved by embodiments of the present invention.
Fig. 2 is the figure for representing the image (shooting image) that camera as shown in Figure 1 photographs.
Fig. 3 (A) shows standard traffic mark, and Fig. 3 (B) shows the traffic sign with auxiliary sign.
Fig. 4 is the flow chart of the program performed by the processing unit (CPU) represented shown in Fig. 1.
Fig. 5 is the flow chart of the program performed by the processing unit (CPU) represented shown in Fig. 1.
Fig. 6 is for being illustrated to display of limitation speed of limitation speed display device progress shown in Fig. 1 etc.
Figure.
Fig. 7 is for being illustrated to display of limitation speed of limitation speed display device progress shown in Fig. 1 etc.
Figure.
Fig. 8 is the flow chart of the program performed by the processing unit (CPU) represented shown in Fig. 1.
Fig. 9 is the flow chart of the program performed by the processing unit (CPU) represented shown in Fig. 1.
Embodiment
Hereinafter, " the limitation speed display device of vehicle " involved by embodiments of the present invention is said referring to the drawings
It is bright.
＜ forms ＞
Fig. 1 shows the limitation speed display device (hereinafter referred to as " implementation " involved by embodiments of the present invention.)
10 schematic configuration.This implementation 10 is equipped on vehicle (automobile, this vehicle).This implementation 10 includes camera (figure
As acquisition device) 11, processing unit 12 and display 13.
Camera 11 is fixed in bracket of room mirror (so-called Room Mirror) of Vehicular body front etc..Camera 11
Optical axis is consistent with car body anteroposterior direction (vehicle traveling direction) in the state of camera 11 is fixed in car body.Therefore, take a picture
Machine 11 is as shown in Fig. 2, the image (image of the landscape can be obtained by shooting the landscape of vehicle traveling direction (vehicle front)
Data) 60.If camera 11 receives shooting instruction from processing unit 12, the landscape of vehicle front is shot.Camera 11 is for example
It is CCD camera.
Road periphery (side of road and the top of road etc.) is provided with " showing for vehicle shown in Fig. 3
The traffic sign 51 and 52 of maximum limit speed (hereinafter referred to as " limitation speed ") etc. ".Traffic mark shown in Fig. 3 (A)
Will 51 is that do not possess to show the auxiliary of the condition that the limitation speed (standard limitation speed) shown by the traffic sign 51 is applied
Help the traffic sign (standard traffic mark) of mark.On the other hand, the traffic sign 52 shown in Fig. 3 (B) is that possess to show the friendship
Traffic sign (the band of the auxiliary sign 52a for the condition that limitation speed (auxiliary limitation speed) shown by logical mark 52 is applied
The traffic sign of auxiliary sign, the mark with auxiliary sign).Should " condition applied " for example include weather, vehicle and
The conditions such as period.
Processing unit 12 is the electronics electricity with the known microcomputer for including RAM (storage part), ROM and CPU etc.
Road device (ECU).As be described in detail later, processing unit 12 passes through image 60 (view data, the bat to being got by camera 11
Take the photograph image, picture signal) handled, to detect (identification) traffic sign 51 and/or 52.
Also, processing unit 12 is as be described in detail later, when the feelings that standard traffic mark 51 is detected in shooting image 60
Under condition, the limitation velocity measuring (identification) shown by standard traffic mark 51 that this is detected limits speed Vst_ for standard
New, and the standard limitation speed Vst_new for determining whether to detect this is shown in display 13.Also, processing unit 12 such as with
It is described in detail like that, in the case of the traffic sign 52 with auxiliary sign is detected in shooting image 60, also detects this afterwards
The traffic sign 52 with auxiliary sign shown by limitation velocity measuring (identification) be auxiliary limitation speed Vsub_new, and sentence
The fixed auxiliary for detecting whether this limitation speed Vsub_new is shown in display 13.
Display 13 is arranged at the position (for example, in instrument group panel) for being capable of visual confirmation from the driver's seat of vehicle.
As shown in Figure 1, display 13 includes the first display unit 13a and the display for showing main limitation speed (standard limitation speed)
Second display unit 13b of the limitation speed (auxiliary limitation speed) of auxiliary.Each display unit 13a and 13b is respectively according to processing unit
12 display is indicated to show standard limitation speed and auxiliary limitation speed.
Vehicle speed sensor 21 detects the speed (speed) of vehicle, and exports the signal for representing speed SPD.Sensed from speed
The signal that device 21 exports is transfused to processing unit 12.Processing unit 12 obtains speed SPD based on the signal that this is transfused to.
The alarm that alarm 22 makes a sound in response to the instruction from processing unit 12.
＜ acts on ＞
Next, the work of this implementation 10 to forming as described illustrates.Pass through on the reality of work
The CPU (hreinafter referred to as " CPU ") that the processing unit 12 of this implementation 10 possesses is according to the program (instruction) for being stored in ROM
Operations described below is performed to realize.
In this example, when ignition switch is turned on, the data Vst and auxiliary limitation speed of standard limitation speed
The data Vsub of degree is eliminated from RAM.That is, as standard limit speed and auxiliary limitation speed data and " expression be not
The data of effective situation " are stored in the predetermined region of RAM as these data.
CPU is often just held repeatedly in the case where ignition switch is turned on by the stipulated time (for example, 0.1 second)
With the processing of the program shown in flow chart in row Fig. 4.Therefore, at the time of if as regulation, CPU is since the step 400 of Fig. 4
Processing, carries out the processing of step 405 discussed below and step 410 successively.
Step 405：CPU sends out shooting instruction shot camera 11 to camera 11, obtains by camera 11
The shooting image (view data) 60 photographed.
Step 410：CPU is using known method (for example, referring to patent document 1.) bat to being got by step 405
Take the photograph image 60 to be handled, whether the data judging based on the shooting image 60 after the processing is in shooting image 60 comprising " aobvious
Show the traffic sign 51 and/or 52 " of limitation speed.Also, CPU includes standard traffic mark in shooting image 60 is determined as
In the situation (CPU identifies that (detection) goes out the situation of standard traffic sign 51 in shooting image 60) of will 51, by the standard traffic
Limitation velocity measuring shown by mark 51 limits speed Vst_new for standard.Also, CPU is in shooting image 60 is determined as
(CPU identifies that (detection) goes out the friendship with auxiliary sign to situation comprising the traffic sign 52 with auxiliary sign in shooting image 60
The situation of logical mark 52) under, it is auxiliary limitation speed by the limitation velocity measuring shown by the traffic sign 52 with auxiliary sign
Spend Vsub_new.
Next, CPU enters step 415, based on the judgement in step 410 as a result, judging whether wrapped in shooting image 60
Containing " the standard traffic mark 51 of display limitation speed ".In the case of standard traffic mark 51 being included in shooting image 60, CPU
It is determined as "Yes" in the step 415, carries out the processing of step 420 discussed below and step 422 successively.
Step 420：CPU sends out the standard limitation speed newly detected in step 410 for showing the first display unit 13a
The instruction (display instruction) of Vst_new is spent, and sends out the display knot for being used to make the auxiliary limitation speed in the second display unit 13b
The instruction (non-display instruction) of beam.As a result, the standard limitation speed Vst_new newly detected is shown in the first display unit 13a,
And the display of the auxiliary limitation speed in the second display unit 13b is stopped.
Step 422：CPU is using the standard newly detected in step 410 limitation speed Vst_new as " in the first display
RAM is arrived in the data Vst " storages for the standard limitation speed that portion 13a is shown, and will be " in the auxiliary limit that the second display unit 13b is shown
The data Vsub " of speed processed is eliminated from RAM.Afterwards, CPU enters step 425.
On the other hand, in the case of not including standard traffic mark 51 in shooting image 60, CPU sentences in step 415
It is set to "No", is directly entered step 425.
If entering step 425, CPU is based on the judgement in step 410 as a result, judging whether included in shooting image 60
Traffic sign 52 with auxiliary sign.When being included in shooting image 60, " display limits the traffic mark with auxiliary sign of speed
In the case of will 52 ", CPU is determined as "Yes" and enters step 430 in step 425, carries out in Fig. 5 by shown in flow chart
The processing of program (sub-process).
Therefore, if CPU enters step 430, CPU and enters step 505 from step 500 start to process of Fig. 5, judgement is
The auxiliary newly detected in the no step 410 for showing standard limitation speed and Fig. 4 in the first display unit 13a limits speed
Whether Vsub_new is more than the standard limitation speed Vst shown in the first display unit 13a.
When CPU perform step 505 processing at the time of, the first display unit 13a show standard limitation speed and
The auxiliary limitation speed Vsub_new newly detected is more than the first display unit 13a standard limitation speed Vst's shown
In the case of, CPU is determined as "Yes" in the step 505, carries out the place of step 510 discussed below and step 512 successively
Reason.
Step 510：CPU, which is sent out, to be used to make the first display unit 13a be shown in the auxiliary newly detected in the step 410 of Fig. 4
The instruction (display instruction) of speed Vsub_new is limited, and sends out and is used to make the auxiliary in the second display unit 13b to limit speed
Display terminate instruction (non-display instruction).As a result, the auxiliary limitation speed Vsub_new newly detected is displayed on
First display unit 13a, and the display of the auxiliary limitation speed in the second display unit 13b is stopped.
Step 512：The auxiliary limitation speed Vsub_new that CPU is newly detected described in the first display unit 13a as " showing
RAM is arrived in the data Vst " storages for the standard limitation speed shown, and " will limit speed in the auxiliary that the second display unit 13b is shown
Data Vsub " from RAM eliminate.Afterwards, CPU returns to the program of Fig. 4 via step 595.
On the other hand, at the time of the processing of step 505 is performed in CPU, do not show that standard limits on the first display unit 13a
Speed processed or the auxiliary limitation speed Vsub_new newly detected in the step 410 of Fig. 4 are less than in the first display unit 13a
In the case of the standard limitation speed Vst of display, CPU is determined as "No" in the step 505 and enters step 515.
If entering step 515, CPU determines whether to show auxiliary limitation speed and in Fig. 4 in the second display unit 13b
Step 410 in the auxiliary limitation speed Vsub_new that newly detects whether be that auxiliary show in the second display unit 13b limits
More than speed Vsub.
When CPU perform step 515 processing at the time of, the second display unit 13b do not show auxiliary limitation speed or
The auxiliary limitation speed Vsub_new newly detected in the step 410 of Fig. 4 is less than the auxiliary limit shown in the second display unit 13b
In the case of speed Vsub processed, CPU is determined as "No" in the step 515, carry out successively step 520 discussed below and
The processing of step 522.
Step 520：CPU, which is sent out, to be used to make the second display unit 13b be shown in the auxiliary newly detected in the step 410 of Fig. 4
Limit the instruction (display instruction) of speed Vsub_new.As a result, the auxiliary limitation speed Vsub_new newly detected is shown
It is shown in the second display unit 13b.
Step 522：CPU is using the auxiliary newly detected in the step 410 of Fig. 4 limitation speed Vsub_new as " the
RAM is arrived in the data Vsub " storages for the auxiliary limitation speed that two display unit 13b are shown.Afterwards, CPU returns to figure via step 595
4 program.
On the other hand, at the time of the processing of step 515 is performed in CPU, auxiliary limitation is shown in the second display unit 13b
Speed and the auxiliary newly detected in the step 410 of Fig. 4, which limit speed Vsub_new, to be shown in the second display unit 13b
In the case of auxiliary limitation more than speed Vsub, CPU is determined as "Yes" in the step 515, and Fig. 4 is returned to via step 595
Program.In this case, the display of the auxiliary limitation speed in the second display unit 13b is not changed, continue the second display
Current display in portion 13b.
Wherein, in the case of band auxiliary sign traffic sign 52 is not included in shooting image 60, CPU is Fig. 4 the step of
It is determined as "No" in 425 and is directly accessed step 495, temporarily terminates this program.It is in this case, auxiliary in the second display unit 13b
Help the display of limitation speed not to be changed, continue the current display in the second display unit 13b.
Above is the work of this implementation 10.According to this implementation 10, such as when vehicle 30 is in shown in Fig. 6
In the case of being travelled on the road of situation, the display of the limitation speed in display 13 carries out as described below.
In the example shown in Fig. 6, it is provided with the left of road and " shows that the band of the auxiliary limitation speed of 100km/h is auxiliary
Help the traffic sign 52 " of mark.In which it is assumed that at the time of vehicle 30 is just in the nearby traveling of traffic sign 52, it is aobvious first
The portion 13a of showing shows " the standard limitation speed of 80km/h ", and " the auxiliary limitation speed of 60km/h is shown in the second display unit 13b
The situation of degree ".
If include traffic sign 52 before vehicle 30 and then close to traffic sign 52, the shooting of camera 11 of the vehicle 30 and exist
Interior landscape.At this time, this implementation 10 identifies (detection) traffic sign 52, and it is band auxiliary mark to be determined as the traffic sign 52
The auxiliary limitation speed (=100km/h) that the traffic sign of will and detecting is shown on the traffic sign 52.
It is additionally, since the auxiliary limitation speed (=100km/h) detected and is more than what is shown in the first display unit 13a
Standard limitation speed (=80km/h), so this implementation 10 will be used to make in transverse direction of the vehicle 30 by traffic sign 52
First display unit 13a show described in detect auxiliary limitation speed (=100km/h) instruction sent out to display 13.
That is, CPU is determined as "No" in the step 415 of Fig. 4, is determined as in step 425 and the step 505 of Fig. 5
"Yes", carries out the processing of step 510.As a result, it is switched in the first display unit 13a standard limitation speed shown from 80km/h
100km/h。
In this way, according to this implementation 10, when the first display unit 13a shows standard limitation speed, when new inspection
The auxiliary limitation speed Vsub_new measured be in the case where standard that the first display unit 13a show limits more than speed Vst,
The auxiliary limitation speed Vsub_new newly detected is also shown in the first display unit 13a.
On the other hand, according to this implementation 10, such as when vehicle 30 travels on the road in the situation shown in Fig. 7
In the case of, the display of the limitation speed in display 13 carries out as described below.
In the example shown in Fig. 7, " the band auxiliary of the auxiliary limitation speed of display 40km/h is provided with the left of road
The traffic sign 52 " of mark.In which it is assumed that at the time of vehicle 30 is just in the nearby traveling of traffic sign 52, in the first display
Portion 13a shows " the standard limitation speed of 80km/h ", is shown " auxiliary of 60km/h limits speed " in the second display unit 13b
Situation.
If include traffic sign 52 before vehicle 30 and then close to traffic sign 52, the shooting of camera 11 of the vehicle 30 and exist
Interior landscape.At this time, this implementation 10 identifies (detection) traffic sign 52, and it is band auxiliary mark to be determined as the traffic sign 52
The auxiliary limitation speed (=40km/h) that the traffic sign of will and detecting is shown on the traffic sign 52.
Moreover, because the auxiliary limitation speed (=40km/h) detected is the standard limit shown in the first display unit 13a
Speed (=80km/h) processed limits speed (=60km/h) below, less than the auxiliary shown in the second display unit 13b, so this reality
Apply device 10 in transverse direction of the vehicle 30 by traffic sign 52 by for make the second display unit 13b show described in detect it is auxiliary
The instruction of limitation speed (=40km/h) is helped to be sent out to display 13.
That is, CPU is determined as "No" in the step 415 of Fig. 4, is determined as "Yes" in step 425, in the step 505 of Fig. 5
And it is determined as "No" in step 515, carry out the processing of step 520.As a result, the auxiliary limitation shown in the second display unit 13b
Speed is switched to 40km/h from 60km/h.
In this way, according to this implementation 10, the auxiliary limitation speed Vsub_new newly detected is in the first display unit 13a
Standard limitation below the speed Vst of display, therefore, is not shown in even in the auxiliary limitation speed Vsub_new newly detected
During the first display unit 13a, the auxiliary shown in the second display unit 13b is less than in the auxiliary limitation speed Vsub_new newly detected
In the case of limiting speed Vsub, the auxiliary newly detected limitation speed Vsub_new is also shown in the second display unit 13b.
However, implementation 10 can also be configured to only in the auxiliary mark of traffic sign 52 of the None- identified with auxiliary sign
In the case of the content of will 52a, the sub-process carried out in the step 430 of Fig. 4 is performed.In this case, implementation 10 performs
With the program shown in flow chart in Fig. 8.
The program of Fig. 8 except the program of Fig. 4 has been added step 827 to step 829 point in addition to, the program phase with Fig. 4
Together, so the explanation for the step of omitting to beyond step 827 to step 829.
If being determined as "Yes" in the step 425 of Fig. 8, the CPU of implementation 10 enters step 827.If CPU enters step
Rapid 827, then determine whether to identify that (detection) go out that the shooting image 60 got in step 405 included with auxiliary sign
The content of the auxiliary sign 52a of traffic sign 52.
In the case of the content for going out the auxiliary sign 52a of the traffic sign 52 with auxiliary sign in unidentified (detection), CPU
It is determined as "No" in the step 827 and enters step 430, performs the program of the Fig. 5 illustrated before.
On the other hand, go out in identification (detection) in shown by the auxiliary sign 52a of the traffic sign 52 with auxiliary sign
In the case of appearance, CPU is determined as "Yes" in step 827, carries out step 828 discussed below and step 829 successively
Processing.
Step 828：CPU sends out the auxiliary limitation speed newly detected in step 410 for showing the second display unit 13b
Spend the instruction (display instruction) of Vsub_new.As a result, the auxiliary limitation speed Vsub_new newly detected is shown in second and shows
Show portion 13b.
Step 829：CPU is using the auxiliary newly detected in step 410 limitation speed Vsub_new as " in the second display
RAM is arrived in the data Vsub " storages for the auxiliary limitation speed that portion 13b is shown.Afterwards, CPU enters step 895, temporarily terminates this journey
Sequence.
Thereby, it is possible to only in the case where identifying the content of auxiliary sign 52a of the traffic sign 52 with auxiliary sign,
The auxiliary for making newly to detect limits speed Vsub_new and is shown in the second display unit 13b.
Also, the CPU of processing unit 12 is in the case where ignition switch is turned on, often by the stipulated time (for example,
0.1 second) just perform repeatedly in Fig. 9 with the program shown in flow chart.Therefore, at the time of if as regulation, steps of the CPU from Fig. 9
Rapid 900 start to process and enter step 905, obtain " speed SPD " and " standard limitation show in the first display unit 13a is fast
Spend Vst ".Speed SPD is obtained based on the signal exported from vehicle speed sensor 21.
Next, CPU enters step 910, whether the speed SPD for judging to get in step 905 is more than in step 905
In the standard limitation speed Vst that gets.
At the time of CPU performs the processing of step 910, in the case that speed SPD is more than standard limitation speed Vst, CPU
It is determined as "Yes" in the step 910 and enters step 915, sends out the instruction for being used for making alarm 22 work.Afterwards, CPU into
Enter step 995, temporarily terminate this program.
On the other hand, at the time of CPU performs the processing of step 910, when speed SPD is standard limitation below speed Vst
In the case of, CPU is determined as "No" in the step 910, is directly entered step 995, temporarily terminates this program.
Program more than, when speed SPD, which is more than, limits speed Vst in the standard that the first display unit 13a is shown, leads to
Cross alarm 22 and send the alarm tone for informing that speed SPD is more than this case that standard limitation speed Vst.
Wherein, if the implementation 10 is more than the standard shown in the first display unit 13a by speed SPD limits speed
Degree Vst then makes a sound from alarm 22, to inform that driver's speed SPD is more than standard limitation speed Vst.However, implement dress
Put 10 can also be by showing that speed SPD is more than the situation that standard limits speed Vst, to inform driver's car in display 13
Fast SPD is more than standard limitation speed Vst.That is, can also be configured in the present invention, by including in sound, display 13
The alarm unit of display, to inform that driver's speed SPD is more than standard limitation speed Vst.
As described above, involved by embodiments of the present invention " the limitation speed display device 10 " of vehicle possesses
Display (Display) 13, image acquiring device (camera) 11 and processing unit 12.
Processing unit 12 is clapped based on the view data got by image acquiring device 11 to detect by image acquiring device 11
Traffic sign (standard traffic mark and/or the traffic sign with auxiliary sign) 51 or 52 in the landscape taken the photograph is clapped
The limitation speed (standard limits speed or auxiliary limitation speed) shown by traffic sign is taken the photograph, and this is detected according to condition
Limitation speed Vst_new or Vsub_new as standard limitation speed be shown in display 13.
Also, processing unit 12 carry out shooting traffic sign whether be the mark 52 with auxiliary sign judgement (the step of Fig. 4
405th, step 410 and step 425).
Here, it is new detect limitation speed in the case of, when obtaining showing the shooting traffic sign of the limitation speed not
(it is determined as "Yes" in the step 415 of Fig. 4) when being mark 52 (being standard traffic mark 51) this judgement with auxiliary sign, locates
Limitation speed (the standard limitation speed) Vst_new that reason portion 12 makes newly to detect is shown in display 13 as standard limitation speed
(the step 420) of Fig. 4.
Also, in the case where newly detecting limitation speed, when the shooting traffic sign for obtaining showing the limitation speed is
(it is determined as "Yes" in the step 425 of Fig. 4) during this judgement of the mark 52 with auxiliary sign, in the limitation speed newly detected
(Fig. 5 when (auxiliary limitation speed) Vsub_new is more than the speed Vst for limiting speed on the monitor 13 as standard and just showing
Step 505 in be determined as "Yes"), the limitation speed Vsub_new that processing unit 12 makes newly to detect as standard limitation speed show
It is shown in the 13 (step 510) of Fig. 5 of display.
On the other hand, in the case where newly detecting limitation speed, even if when the shooting for obtaining showing the limitation speed is handed over
Logical mark (is determined as "Yes") when being this judgement of the mark 52 with auxiliary sign in the step 425 of Fig. 4, also newly detecting
(the step of Fig. 5 when limitation speed Vsub_new is below the speed Vst for limiting speed as standard in display 13 and just showing
It is determined as "No" in 505), the limitation speed Vsub_new that processing unit 12 does not make newly to detect is shown in as standard limitation speed
Display 13, and maintain the display for the speed that display 13 limits speed as standard and just showing.
Also, display 13 is configured to also show auxiliary limitation speed in addition to display standard limits speed.
Here, in the case where newly detecting limitation speed, when the shooting traffic sign for obtaining showing the limitation speed is
This judgement (being determined as "Yes" in the step 425 of Fig. 4) of mark 52 with auxiliary sign and the limitation speed newly detected
Vsub_new is (to sentence in below the speed Vsub that display 13 is just being shown as auxiliary limitation speed in the step 515 of Fig. 5
It is set to "No"), the limitation speed Vsub_new that processing unit 12 makes newly to detect is shown in display 13 as auxiliary limitation speed
(the step 520) of Fig. 5.
Alternatively, in the case where newly detecting limitation speed, when the shooting traffic sign for obtaining showing the limitation speed is
This judgement (being determined as "Yes" in the step 425 of Fig. 8) of mark 52 with auxiliary sign and identify the mark with auxiliary sign
(it is determined as "Yes" in the step 827 of Fig. 8) during content shown by the auxiliary sign 52a of will 52, no matter processing unit 12 is new detection
To limitation speed Vsub_new whether be as auxiliary limitation speed and below the speed Vsub that is just showing, all in display 13
The limitation speed Vsub_new for making newly to detect is shown in display 13 as auxiliary limitation speed.
On the other hand, in the case where newly detecting limitation speed, when the shooting traffic mark for obtaining showing the limitation speed
Will be this judgement (being determined as "Yes" in the step 425 of Fig. 8) of the mark 52 with auxiliary sign and it is unidentified go out band auxiliary mark
(it is determined as "No" in the step 827 of Fig. 8) during content shown by the auxiliary sign 52a of the mark 52 of will, what is newly detected
Limitation speed Vsub_new is the (step of Fig. 5 in below the speed Vsub that display 13 is just being shown as auxiliary limitation speed
It is determined as "No" in rapid 515), the limitation speed Vsub_new that processing unit 12 makes newly to detect is shown in as auxiliary limitation speed
13 (the step 520) of Fig. 5 of display.
Also, in the case where newly detecting limitation speed, when the shooting traffic sign for obtaining showing the limitation speed is
This judgement (being determined as "Yes" in the step 425 of Fig. 4) of mark 52 with auxiliary sign and work is not shown in display 13
(it is determined as "No" in the step 515 of Fig. 5) when limiting the speed of speed for auxiliary, processing unit 12 makes the limitation speed newly detected
Vsub_new is shown in the 13 (step 520) of Fig. 5 of display as auxiliary limitation speed.
Therefore, limitation speed display device 10 involved according to the embodiment of the present invention, can show " although small
In the speed for the limitation speed that should be applied but the speed of the closer limitation speed that should be applied " or it is " original
The possibility of the limitation speed that should be applied " becomes higher.
In addition, the invention is not limited in the embodiment, various modifications example can be used within the scope of the invention.
For example, display 13 can also only have the first display unit 13a.In this case, it is directed to the second display unit in each program
The submitting and processing of the instruction of 13b are omitted.
Symbol description
10 ... limitation speed display devices；11 ... cameras；12 ... processing units；13 ... displays；13a ... first is shown
Portion；The second display units of 13b ...；21 ... vehicle speed sensor；22 ... alarms；30 ... vehicles；51 ... standard traffic marks；52 ... bands
The traffic sign (mark with auxiliary sign) of auxiliary sign；52a ... auxiliary signs；60 ... shooting images；Vst_new ... is newly examined
The standard limitation speed measured；Vst ... standards limit speed；The auxiliary limitation speed that Vsub_new ... is newly detected；Vsub…
Auxiliary limitation speed.
Claims (4)
1. a kind of limitation speed display device of vehicle, is equipped on vehicle, and possess：
Display, for showing the information related with the limitation speed for the road that the vehicle is travelling；
Image acquiring device, the landscape for shooting the direct of travel of the vehicle obtain the view data of the landscape；And
Processing unit, the limitation speed shown by the traffic sign in the landscape photographed is detected based on described image data
Degree, makes the limitation speed detected be shown in the display as standard limitation speed,
In the limitation speed display device, the processing unit is configured to：
Whether the traffic sign of the limitation speed detected described in being shown based on the view data got
It is the judgement of the mark with auxiliary sign,
In the case where newly detecting the limitation speed,
When it is not described this judgement of the mark with auxiliary sign to obtain the traffic sign, make the limitation newly detected
Speed is shown in the display as standard limitation speed,
When it is described this judgement of the mark with auxiliary sign to obtain the traffic sign, in the limitation speed newly detected
Degree is more than the limitation speed in the speed that the display is just being shown as standard limitation speed, making this newly detect
The display is shown in as standard limitation speed,
Even to obtain the traffic sign be described this judgement of the mark with auxiliary sign, also newly detected described
When limitation speed is below the speed for limiting speed as the standard in the display and just showing, this is not set newly to detect
Limitation speed as the standard limitation speed be shown in the display, and maintain the display as the standard limit speed
And the display of the speed just shown.
2. the limitation speed display device of vehicle according to claim 1, wherein,
The display is configured to also show auxiliary limitation speed in addition to showing the standard limitation speed,
The processing unit is configured in the case where newly detecting the limitation speed, is described when obtaining the traffic sign
This judgement of mark and the limitation speed newly detected with auxiliary sign are as the auxiliary limit in the display
Speed processed and when below the speed that just shows, make the limitation speed that this is newly detected as auxiliary limitation speed and be shown in this and is aobvious
Show device.
3. the limitation speed display device of vehicle according to claim 1, wherein,
The display is configured to also show auxiliary limitation speed in addition to showing the standard limitation speed,
The processing unit is configured to：
Newly detect it is described limitation speed in the case of, when obtain the traffic sign be the mark with auxiliary sign this
One judges and when identifying the content shown by the auxiliary sign of the mark with auxiliary sign, makes the limit newly detected
Speed processed is shown in the display as auxiliary limitation speed,
Newly detect it is described limitation speed in the case of, when obtain the traffic sign be the mark with auxiliary sign this
One judge and it is unidentified go out the mark with auxiliary sign auxiliary sign shown by content when, newly detected described
When limitation speed is below the speed that the display is just being shown as the auxiliary limitation speed, make what this was newly detected
Limitation speed is shown in the display as auxiliary limitation speed.
4. according to the limitation speed display device described in claim 2 or 3, wherein,
The processing unit is configured in the case where newly detecting the limitation speed, is described when obtaining the traffic sign
This judgement of mark with auxiliary sign and the display do not show as it is described auxiliary limitation speed speed when,
The limitation speed that this is newly detected is set to be shown in the display as auxiliary limitation speed.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2015096784A JP6187534B2 (en) | 2015-05-11 | 2015-05-11 | Vehicle regulation speed display device |
| JP2015-096784 | 2015-05-11 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106157662A CN106157662A (en) | 2016-11-23 |
| CN106157662B true CN106157662B (en) | 2018-04-27 |
Family
ID=57208885
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610302675.0A Active CN106157662B (en) | 2015-05-11 | 2016-05-09 | The limitation speed display device of vehicle |
Country Status (4)
| Country | Link |
|---|---|
| US (1) | US10227038B2 (en) |
| JP (1) | JP6187534B2 (en) |
| CN (1) | CN106157662B (en) |
| DE (1) | DE102016107530B4 (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP7327047B2 (en) * | 2019-09-27 | 2023-08-16 | スズキ株式会社 | Vehicle driving support system |
| JP7251848B1 (en) | 2022-02-25 | 2023-04-04 | アトミクス株式会社 | Traffic safety facility management system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN201268230Y (en) * | 2008-09-22 | 2009-07-08 | 交通部公路科学研究所 | Speed limit sign information vehicle auxiliary drive system based on computer vision |
| JP2010205160A (en) * | 2009-03-05 | 2010-09-16 | Denso Corp | Method for notifying speed-limit sign recognition result |
| CN102568236A (en) * | 2010-12-08 | 2012-07-11 | 罗伯特·博世有限公司 | Method and device for recognizing road signs and comparing with road signs information |
| CN103794049A (en) * | 2012-10-30 | 2014-05-14 | 罗伯特·博世有限公司 | Data processing device and method for such a data processing device |
| JP2014522049A (en) * | 2011-08-04 | 2014-08-28 | コンティ テミック マイクロエレクトロニック ゲゼルシャフト ミット ベシュレンクテル ハフツング | Traffic sign recognition method |
Family Cites Families (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6681195B1 (en) * | 2000-03-22 | 2004-01-20 | Laser Technology, Inc. | Compact speed measurement system with onsite digital image capture, processing, and portable display |
| US7526103B2 (en) * | 2004-04-15 | 2009-04-28 | Donnelly Corporation | Imaging system for vehicle |
| DE102007034505A1 (en) * | 2007-07-24 | 2009-01-29 | Hella Kgaa Hueck & Co. | Method and device for traffic sign recognition |
| DE602007012687D1 (en) | 2007-08-17 | 2011-04-07 | Magneti Marelli Spa | Method and system for detecting traffic signs with additional panels |
| US8725394B2 (en) * | 2007-10-09 | 2014-05-13 | Siemens Corporation | Multi-modal speed limit assistant |
| JP5439890B2 (en) * | 2009-03-25 | 2014-03-12 | 富士通株式会社 | Image processing method, image processing apparatus, and program |
| JP5361541B2 (en) * | 2009-06-02 | 2013-12-04 | 三菱電機株式会社 | Sign recognition device |
| DE102010039634B4 (en) | 2010-08-23 | 2013-05-08 | Ford Global Technologies, Llc | Arrangement and method for traffic sign recognition |
| US20130271292A1 (en) * | 2011-10-09 | 2013-10-17 | James Andrew McDermott | Driver Alert and Monitoring System |
| US20140327772A1 (en) * | 2013-05-03 | 2014-11-06 | Magna Electrics Inc. | Vehicle vision system with traffic sign comprehension |
| JP6200333B2 (en) * | 2014-01-16 | 2017-09-20 | 本田技研工業株式会社 | Road sign recognition device and road sign recognition method |
| JP6323074B2 (en) * | 2014-03-06 | 2018-05-16 | 株式会社デンソー | Display object information display device |
| KR101737791B1 (en) * | 2014-10-23 | 2017-05-19 | 현대모비스 주식회사 | Traffic sign recognizing apparatus for vehicle and operating method threrof. |
| JP6062473B2 (en) * | 2015-03-17 | 2017-01-18 | 本田技研工業株式会社 | Road sign judgment device and road sign judgment method |
-
2015
- 2015-05-11 JP JP2015096784A patent/JP6187534B2/en active Active
-
2016
- 2016-04-06 US US15/092,042 patent/US10227038B2/en active Active
- 2016-04-22 DE DE102016107530.2A patent/DE102016107530B4/en active Active
- 2016-05-09 CN CN201610302675.0A patent/CN106157662B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN201268230Y (en) * | 2008-09-22 | 2009-07-08 | 交通部公路科学研究所 | Speed limit sign information vehicle auxiliary drive system based on computer vision |
| JP2010205160A (en) * | 2009-03-05 | 2010-09-16 | Denso Corp | Method for notifying speed-limit sign recognition result |
| CN102568236A (en) * | 2010-12-08 | 2012-07-11 | 罗伯特·博世有限公司 | Method and device for recognizing road signs and comparing with road signs information |
| JP2014522049A (en) * | 2011-08-04 | 2014-08-28 | コンティ テミック マイクロエレクトロニック ゲゼルシャフト ミット ベシュレンクテル ハフツング | Traffic sign recognition method |
| CN103794049A (en) * | 2012-10-30 | 2014-05-14 | 罗伯特·博世有限公司 | Data processing device and method for such a data processing device |
Also Published As
| Publication number | Publication date |
|---|---|
| JP6187534B2 (en) | 2017-08-30 |
| DE102016107530B4 (en) | 2019-04-18 |
| JP2016212692A (en) | 2016-12-15 |
| DE102016107530A1 (en) | 2016-11-17 |
| US10227038B2 (en) | 2019-03-12 |
| CN106157662A (en) | 2016-11-23 |
| US20160332566A1 (en) | 2016-11-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106157661B (en) | The limitation speed display device of vehicle | |
| KR101680023B1 (en) | Method for combining a road sign recognition system and lane detection system of a motor vehicle | |
| CN105989725B (en) | Road sign judgment means | |
| JP6447958B2 (en) | Sign recognition display device | |
| WO2015155029A1 (en) | Locating a head-mounted display (hmd) in a vehicle | |
| JP4742953B2 (en) | Image processing apparatus, image display system, and program | |
| CN108437896B (en) | Vehicle driving assistance method, device, equipment and storage medium | |
| JP6496018B2 (en) | Traffic sign validity confirmation system and method | |
| US20160267336A1 (en) | Method for calibrating a camera for a gaze direction detection in a vehicle, device for a motor vehicle having a camera and at least one further element, and computer program product | |
| CN107085963A (en) | Driving auxiliary device for vehicle | |
| JP2001304876A (en) | Method for storing/reproducing image by on-vehicle camera | |
| CN106157662B (en) | The limitation speed display device of vehicle | |
| CN108621939A (en) | The control method of the display system of vehicle and the display system of vehicle | |
| JP2008082932A (en) | Navigation device, its control method, and control program | |
| CN113743312A (en) | Image correction method and device based on vehicle-mounted terminal | |
| JP6536893B2 (en) | Indicator recognition display | |
| JP4917609B2 (en) | Data recording apparatus, data recording method, data recording program, and computer-readable recording medium | |
| JP6288204B1 (en) | Vehicle speed limit detection device | |
| JP7183729B2 (en) | Imaging abnormality diagnosis device | |
| JP2004064409A (en) | Information recording device, information recording method, and information recording program | |
| JP3948018B2 (en) | Information recording apparatus, information recording method, and information recording program | |
| JP6331232B2 (en) | Vehicle speed limit detection device | |
| JP4857159B2 (en) | Vehicle driving support device | |
| JP7030000B2 (en) | Information processing methods, information processing systems, and programs | |
| JP7033474B2 (en) | Analytical equipment, analysis methods, programs, and storage media |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |