US11059489B2 - Methods and systems for detecting road surface conditions - Google Patents
Methods and systems for detecting road surface conditions Download PDFInfo
- Publication number
- US11059489B2 US11059489B2 US16/169,149 US201816169149A US11059489B2 US 11059489 B2 US11059489 B2 US 11059489B2 US 201816169149 A US201816169149 A US 201816169149A US 11059489 B2 US11059489 B2 US 11059489B2
- Authority
- US
- United States
- Prior art keywords
- road surface
- reflections
- doppler
- beams
- histogram
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active, expires
Links
- 238000000034 method Methods 0.000 title claims abstract description 64
- 238000012545 processing Methods 0.000 claims description 27
- 230000005540 biological transmission Effects 0.000 claims description 18
- 230000008859 change Effects 0.000 claims description 9
- 238000001914 filtration Methods 0.000 claims description 2
- 230000004044 response Effects 0.000 description 12
- 238000004590 computer program Methods 0.000 description 9
- 230000008569 process Effects 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 230000006399 behavior Effects 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000003993 interaction Effects 0.000 description 3
- 238000000926 separation method Methods 0.000 description 3
- 238000007619 statistical method Methods 0.000 description 3
- 239000000969 carrier Substances 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 230000007480 spreading Effects 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 238000010420 art technique Methods 0.000 description 1
- 239000010426 asphalt Substances 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000010363 phase shift Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/88—Radar or analogous systems specially adapted for specific applications
- G01S13/93—Radar or analogous systems specially adapted for specific applications for anti-collision purposes
- G01S13/931—Radar or analogous systems specially adapted for specific applications for anti-collision purposes of land vehicles
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
- B60W40/02—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to ambient conditions
- B60W40/06—Road conditions
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
- B60W40/02—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to ambient conditions
- B60W40/06—Road conditions
- B60W40/064—Degree of grip
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
- B60W40/02—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to ambient conditions
- B60W40/06—Road conditions
- B60W40/068—Road friction coefficient
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B15/00—Measuring arrangements characterised by the use of electromagnetic waves or particle radiation, e.g. by the use of microwaves, X-rays, gamma rays or electrons
- G01B15/08—Measuring arrangements characterised by the use of electromagnetic waves or particle radiation, e.g. by the use of microwaves, X-rays, gamma rays or electrons for measuring roughness or irregularity of surfaces
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/02—Systems using reflection of radio waves, e.g. primary radar systems; Analogous systems
- G01S13/06—Systems determining position data of a target
- G01S13/42—Simultaneous measurement of distance and other co-ordinates
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/02—Systems using reflection of radio waves, e.g. primary radar systems; Analogous systems
- G01S13/50—Systems of measurement based on relative movement of target
- G01S13/58—Velocity or trajectory determination systems; Sense-of-movement determination systems
- G01S13/583—Velocity or trajectory determination systems; Sense-of-movement determination systems using transmission of continuous unmodulated waves, amplitude-, frequency-, or phase-modulated waves and based upon the Doppler effect resulting from movement of targets
- G01S13/584—Velocity or trajectory determination systems; Sense-of-movement determination systems using transmission of continuous unmodulated waves, amplitude-, frequency-, or phase-modulated waves and based upon the Doppler effect resulting from movement of targets adapted for simultaneous range and velocity measurements
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/02—Systems using reflection of radio waves, e.g. primary radar systems; Analogous systems
- G01S13/50—Systems of measurement based on relative movement of target
- G01S13/58—Velocity or trajectory determination systems; Sense-of-movement determination systems
- G01S13/60—Velocity or trajectory determination systems; Sense-of-movement determination systems wherein the transmitter and receiver are mounted on the moving object, e.g. for determining ground speed, drift angle, ground track
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/88—Radar or analogous systems specially adapted for specific applications
- G01S13/89—Radar or analogous systems specially adapted for specific applications for mapping or imaging
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/02—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00
- G01S7/35—Details of non-pulse systems
- G01S7/352—Receivers
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2420/00—Indexing codes relating to the type of sensors based on the principle of their operation
- B60W2420/40—Photo, light or radio wave sensitive means, e.g. infrared sensors
- B60W2420/408—Radar; Laser, e.g. lidar
-
- B60W2420/52—
-
- G01S2007/356—
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/02—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00
- G01S7/35—Details of non-pulse systems
- G01S7/352—Receivers
- G01S7/356—Receivers involving particularities of FFT processing
Definitions
- roads which are wet and/or are covered with ice and/or snow have a surface friction which is relatively low compared with the surface friction of a road which is dry. Consequently, a vehicle travelling on a road having road surface conditions which result in relatively low surface friction, may require longer stopping distances (e.g. as compared to stopping distances required on a dry road), and may generally be at greater risk for experiencing vehicle slippage or the like while the vehicle is maneuvering (e.g. travelling around a corner, stopping, etc. . . . ). Thus, the existence of road surface conditions which result in relatively low surface friction can result in increased vehicle accidents.
- Prior art techniques attempt to determine road surface conditions by comparing ratios of powers received from vertically and horizontally transmitted waveforms. The techniques further determine coefficients of reflections to determine road surface conditions from said comparison. However, such techniques result in a large variation of results for rough asphalt road surface conditions.
- aspects of the present disclosure relate to methods and systems for detecting road surface conditions.
- a method for detecting road surface conditions.
- the method comprises transmitting one or more radio frequency (RF) signals at a road surface and receiving at least portions of RF signals reflected from the road surface.
- the RF reflections of the one or more transmitted RF signals are received in two or more receive channels.
- the received reflected signals are used to determine a road surface condition based upon a doppler signature of the received RF reflections.
- RF radio frequency
- the one or more RF signals can be transmitted using one or more sensors.
- the method can include directing the one or more RF signals in front of and/or behind a vehicle traveling on a road surface.
- the method can further comprise controlling the transmission of the RF signals through one or more beams such that each beam has a field of view (FoV) of about 6 degrees.
- the FoV is selected such that it is wide enough to accumulate enough data in the histograms and narrow enough to limit the spread of normalized doppler. Specifically, if the selected FoV is too large, unwanted doppler will be accumulated; if the FoV is too narrow, not enough data will be available to the sensor to accurately determine road surface conditions.
- the method can also comprise generating an RF waveform for transmission via the one or more RF beams.
- the RF waveform can comprise a plurality of chirp signals (or more simply “chirps”).
- the method can further comprise performing Frequency Fourier Transform (FFT) processing of the RF reflections provided to each of the two or more receive channels.
- FFT Frequency Fourier Transform
- the method also comprises determining range, doppler, phase difference and magnitude based on the FFT processing.
- the method further comprises determining angle and azimuth of the RF reflections based on the phase difference.
- the method can further comprise filtering the received RF reflections to remove those RF reflections that originate outside the FoV.
- FFT Frequency Fourier Transform
- the method can additionally comprise doppler shifting the filtered RF reflections to determine a doppler speed.
- the method also comprises generating a normalized doppler signal that is a function of the doppler speed and a vehicle speed. Further, the method comprises generating first and second histograms of the normalized doppler signal. The first histogram comprises data received from those RF reflections originating from a near region of the road surface. The second histogram comprises data received from those RF reflections originating from a far region of the road surface.
- the method also comprises determining the road surface condition based on a quality factor (Q-factor) of each histogram.
- Q-factor quality factor
- a road surface detector comprises an RF transmitter, an RF receiver, and one or more processors.
- the transmitter is configured to transmit one or more radio frequency (RF) signals at a road surface.
- the receiver is configured to receive at least portions of the RF signals reflected from the road. Such reflected RF signals are received through a receive antenna and provided to two or more receive channels.
- the one or more processors are configured to process the signals provided thereto to determine a road surface condition based upon a doppler signature of the received RF reflections.
- the transmitter can comprise comprises one or more spaced apart sensors configured to transmit the one or more RF beams.
- the transmitter can further be configured to direct the one or more RF beams in front of and/or behind a vehicle traveling on a road surface.
- the transmitter can also be configured to control the transmission of the one or more beams such that each beam has a field of view (FoV) of 6 degrees.
- the FoV is selected such that it is wide enough to accumulate enough data in the histograms and narrow enough to limit the spread of normalized doppler. Specifically, if the selected FoV is too large, unwanted doppler will be accumulated; if the FoV is too narrow, not enough data will be available to the sensor to accurately determine road surface conditions.
- the transmitter can also be configured to generate an RF waveform for transmission via the one or more RF beams.
- the RF waveform comprises a plurality of chirps with each chirp having a transmit duration followed by a pause period.
- Each chirp can have a chirp slope that is a function of each chirp's frequency change over the transmit duration.
- a single data cycle count can comprise sixty-four (64) chirps. Each data cycle count has a predetermined duration. The predetermined duration can be 45 ms.
- the one or more processors can further be configured to perform Frequency Fourier Transform (FFT) processing of the RF reflections on each of the two or more receive channels.
- FFT Frequency Fourier Transform
- the one or more processors are also configured to determine range, doppler, phase difference and magnitude based on the FFT processing.
- the one or more processor are configured to determine angle and azimuth of the RF reflections based on the phase difference.
- the one or more processor can further be configured to filter the received RF reflections to remove those RF reflections that originate outside the FoV.
- the one or more processors can additionally be configured to doppler shift the filtered RF reflections to determine a doppler speed. They can also be configured to generate a normalized doppler signal that is a function of the doppler speed and a vehicle speed. The one or more processors are also configured to generate first and second histograms of the normalized doppler signal. The first histogram comprises data received from those RF reflections originating from a near region of the road surface. The second histogram comprises data received from those RF reflections originating from a far region of the road surface. The one or more processors are configured to determine the road surface condition based a quality (Q) factor of each histogram.
- Q quality
- FIG. 1 illustrates an environment in which a vehicle is traveling on a road having a road surface condition.
- FIG. 2 is a block diagram of a radar sensor according to embodiments described herein.
- FIG. 3 illustrates a vehicle having two sensors disposed thereon with each sensor capable of detecting a road surface condition according to embodiments described herein.
- FIG. 3A is a block diagram of another road sensor according to embodiments described herein.
- FIG. 4 is a plot of transmit frequency vs. time which illustrates transmit signal waveforms provided by a sensor capable of detecting road surface conditions.
- FIG. 5 is a flow diagram of a method for detecting a road surface condition according to embodiments described herein.
- FIG. 6 is a histogram of normalized doppler signatures used by a radar sensor to detect a road surface condition according to embodiments described herein.
- FIG. 7 is a diagram of a vehicle having a sensor disposed thereon which illustrates near and far regions from which the sensor analyzes doppler signatures to detect a road surface condition according to embodiments described herein.
- FIGS. 8-8B are a series of histograms which illustrate a method of processing histograms to detect a road surface condition according to embodiments described herein.
- FIGS. 9 and 9A are example histograms that indicate a dry road condition according to embodiments described herein.
- FIGS. 10 and 10A are example histograms that indicate a wet road condition according to embodiments described herein.
- FIGS. 11 and 11A are example histograms that indicate snow covered road conditions according to embodiments described herein.
- Vehicles can include collision avoidance systems designed to prevent or reduce a severity of a collision. Such systems typically use one or more of radar, laser (LIDAR), and cameras (employing image recognition) to detect an imminent crash. Some of these systems provide a warning to the vehicle's driver and other systems act autonomously without driver input (e.g. by braking, steering, or both). Collision avoidance by braking is appropriate at low vehicle speeds (e.g. below about 31 mph), while collision avoidance by steering may be more appropriate at higher vehicle speeds if vehicle travel lanes are clear. For example, autonomously braking a vehicle traveling below 31 mph may not be the best decision if the road surface is wet or snowy or otherwise has a surface friction which is relatively low compared with the surface friction of a road which is dry.
- Embodiments of the present disclosure relate to methods and systems for determining a road surface condition with a radar sensor.
- the methods and systems generate “normalized” doppler histograms and utilize such normalized Doppler histograms to calculate multiple parameters.
- the parameters may include, but are not limited to, radar cross section (RCS), quality (Q)-factor, update rate of the histograms, and doppler spreading across bins of the histogram.
- the methods and systems described herein analyze the histograms and parameters to determine a road surface condition.
- FIG. 1 illustrates an environment 100 in which a vehicle 105 is traveling along a surface 110 a of a road 110 .
- the vehicle is equipped with one or more sensors 120 with at least one of the sensors 120 capable of detecting at least one environmental condition of the road 110 and in particular a condition of road surface 110 a (i.e. a road surface conditions).
- at least one sensor can detect a plurality of road surface conditions including, but not limited to, dry, wet, icy, and/or snowy road surface conditions.
- the sensor may be provided as a multi-beam sensor which utilizes multiple chirp signals in a frequency modulated continuous wave (FMCW) system which may be the same as or similar to the type described in U.S. Pat. No. 7,071,868 assigned to the assignee of the present application, and which is incorporated herein by reference in its entirety.
- FMCW frequency modulated continuous wave
- the FMCW sensor may utilize one or more transmit beams and two or more received beams.
- a sensor 120 capable of detecting road surface conditions may operate at a transmit frequency of about 24 GHz or about 79 GHz. Other frequencies or ranges of frequencies may, of course, also be used.
- Sensors 120 can be disposed on any portion of vehicle 105 which allows the sensor to direct a radio frequency (RF) transmit signal toward road surface 110 a such that at least portions of the RF signals reflect off road surface 110 a and can be received via receive antenna in sensor 120 . Such signals are subsequently from the receive antenna to a receiver coupled to the receive antenna.
- sensors 120 are preferably positioned to allow radar beams to generate a large doppler signature (and ideally, the largest possible doppler signature).
- the greatest doppler radar signature for a moving vehicle is behind (and ideally directly behind) or in front of (and ideally in front of) the vehicle.
- the sensor 120 determines road conditions based on short-term, near-range integrated statistics of the doppler signature of the road surface behind or in front of the vehicle 105 .
- vehicle 105 can also include a collision avoidance system (not shown) that uses road surface condition information when making decisions related to avoid collisions.
- vehicle can also include driver assistance systems (not shown) that use road surface condition information to adjust vehicle driving parameters (e.g., slowing cruise control speeds if the road becomes wet).
- FIG. 1 Although two sensors 120 capable of detecting road surface conditions are shown in FIG. 1 , after reading the disclosure provided herein, a skilled artisan understands that one or more than two sensors 120 can be used to detect road surface conditions.
- a sensor 220 capable of detecting road surface conditions includes a transmit antenna 222 coupled to a transmit signal path which includes a waveform generator 260 .
- Transmit antenna 222 may include one or more antenna elements 250 and may be capable of generating one or more transmit beams through which one or more RF signals generated by transmit path and waveform generator 260 are transmitted (i.e. are emitted) via transmit antenna 222 .
- Receive antenna 252 may include one or more antenna elements 252 a - 252 N and may be capable of generating one or more receive beams through which at least portions of one or more RF signals reflected from a road surface (e.g. road surface 110 a in FIG. 1 ) may be received.
- a road surface e.g. road surface 110 a in FIG. 1
- Multi-channel receiver 265 receives the signals provided thereto, appropriately processes the signals and provides an output signal to a road surface detection processor 270 .
- the waveform generator 260 generates a waveform (e.g., the waveform 400 of FIG. 4 ) and transmitter 250 transmits the waveform toward a road surface 110 a (e.g. the road surface of FIG. 1 ) via transmit antenna 222 .
- a waveform e.g., the waveform 400 of FIG. 4
- transmitter 250 transmits the waveform toward a road surface 110 a (e.g. the road surface of FIG. 1 ) via transmit antenna 222 .
- the multi-channel receiver 265 preferably includes two or more processing channels for processing the reflected RF signals.
- the multi-channel receiver 265 and signal processor(s) 270 processes the radar response signals to detect a road surface's condition according to the method 500 of FIG. 5 .
- FIG. 3 illustrates a vehicle 305 that is equipped with two radar sensors 320 a , 320 b configured to detect road surface conditions. Although two sensors are shown, after reading the disclosure provided herein, a skilled artisan understands that the vehicle 305 can be equipped with one sensor or more than two sensors that are configured to detect road surface conditions.
- the radar sensors 320 a , 320 b are respectively positioned on right and left sides of a rear bumper area 306 of the vehicle 305 (with right and left here defined with respect to a person standing behind the rear of the vehicle and facing the front of the vehicle—i.e. facing the direction of front bumper 307 ).
- the sensors can be disposed anywhere on the vehicle where they are able to obtain doppler signatures from a road surface (e.g. road surface 110 a in FIG. 1 ).
- the sensors are preferably positioned on the vehicle in a location which allows the sensor to obtain the a relatively large doppler signature from the road surface (and ideally, which allows the sensor to obtain the largest possible doppler signature from the road surface).
- a preferred location is in a front bumper area 307 and/or a rear bumper area 306 of the vehicle 305 .
- radar sensor 320 a is capable of switching between a plurality of, here four (4), transmit beams formed by a transmit antenna. It should be understood that the transmit beams may be used for any number of vehicle systems including vehicle control systems including, but not limited to: collision avoidance systems, object detection systems or other vehicle systems.
- one of the four beams, here the fourth beams 325 a , 326 d , of each respective radar sensor are used to detect road surface conditions.
- Each of the fourth beams have a field of view (FoV) that covers an angular range extending from about 158 degrees to about 217 degrees (as shown in FIG. 3 ).
- some sensors may utilize only a single beam which may be configured to have FoVs extending from about 177 degrees to about 183 degrees (i.e., a beam width of six (6) degrees) as shown in FIG. 3 and indicated as 330 a - b .
- the FoV is selected such that it is wide enough to accumulate enough data in the histograms and narrow enough to limit the spread of normalized doppler. Specifically, if the selected FoV is too large, unwanted doppler will be accumulated; if the FoV is too narrow, not enough data will be available to the sensor to accurately determine road surface conditions.
- the representative sensor 320 a comprises a transmit control processor 353 , TX/RX chip 351 , and road surface detector processor 354 .
- the transmit control processor 353 generates control signals for transmitting a radar transmit waveform (e.g., the waveform 400 of FIG. 4 ).
- the tuning circuit 357 receives the control signals and provides a tuning signal to an oscillator 362 (e.g., a voltage-controlled oscillator (VCO)) of the T X/RX chip 351 .
- the VCO 365 generates a corresponding chirp signal from the tuning signal.
- a power amplifier 363 a amplifies the transmission chirp signal and RF switch 370 selects a transmit beam (e.g., the beam 325 d of FIG. 3 ) for transmitting the transmission chirp signal.
- a phase network 371 introduces a phase shift to the antenna elements such that the antenna forms a desired antenna beam through which is transmitted the transmission chirp signal.
- a receive antenna 352 comprising antenna elements 352 a , 352 b then receives radar response signals (i.e., RF reflections of the transmission chirp signal).
- Antenna 352 is coupled to a dual-channel receiver.
- Each receiver channel comprises respective low noise amplifiers (LNAs) 373 a , 373 b and mixers 374 a , 374 b for processing the radar response signals as is generally known (i.e. the LNAs 373 a , 373 b amplify the radar response signals provided thereto while mixers 374 a , 374 b mix (or down convert) the amplified radar response signals with a local oscillator frequency signal provided by power amplifier 363 b and signal splitter 381 .
- LNAs low noise amplifiers
- mixers 374 a , 374 b mix (or down convert) the amplified radar response signals with a local oscillator frequency signal provided by power amplifier 363 b and signal splitter 381 .
- the local oscillator (LO) frequency signal is equivalent to the transmission chirp signal.
- Mixers 374 a , 384 b thus produce an intermediate frequency (IF) signal which is provided to an IF filter 383 .
- the down converted, IF-filtered radar response signals are then converted to digital signals (e.g. a digital bit stream) by analog-to-digital converters (ADCs) 384 a , 384 b .
- ADCs analog-to-digital converters
- the road surface detector processor 354 processes the digital signals to determine a condition of a road surface (e.g., the road surface 110 a of FIG. 1 ).
- the digital signal processor 354 can determine a condition of the road surface according to the method 500 of FIG. 5 .
- an example radar transmit waveform 400 used to detect road surface conditions comprises a plurality of chirp signals 401 (or more simply “chirps).
- the chirp waveform 400 is presented as a graph 411 that plots frequency 402 over time 404 .
- a single transmission waveform comprises sixty-four (64) chirps.
- the time to transmit the entire waveform can be forty-five (45) milli-seconds.
- Each of the chirps 401 is transmitted during a transmission window 407 that comprises a transmit time period (or duration) 408 followed by a pause time period (or duration) 409 . Accordingly, each of the chirps 401 has a slope that is a function each chirp's frequency change over the transmit duration 409 .
- a radar sensor processes return signals of the waveform 400 to detect a road surface condition as further described herein. Accordingly, the waveform 400 is designed to have a chirp linearity, number of chirps, and chirp slope to enable the radar sensor to resolve doppler.
- a method 500 for detecting a road surface condition begins as shown in processing block 504 in which a sensor receives one or more RF return signals.
- return signals are processed by an RF receiver (e.g. the RF receiver described above in conjunction with FIG. 3A ) as is generally known.
- the receiver receives radar return signals via two processing channels (e.g., the processing channels of the multi-channel receiver 265 of FIG. 2 ).
- the receiver appropriately down converts and digitizes (e.g. via a downconverter and analog to digital converter) the signals provided at the receiver input and provides a digital signal (e.g. a stream of bits) at an output thereof.
- processing block 505 a Fast Fourier Transform (FFT) is performed on the digitized radar response signals provided by each of the processing channels.
- FFT Fast Fourier Transform
- Processing block 505 thus produces FFT response signals that are, at processing block 510 , filtered such that data is restricted to be sourced directly from those response signals that originate directly behind host.
- the method 500 can include implementing one or more filters that filter out signals associated with zero (0) doppler. Additionally, a filter removes those signals originating from outside a limited FoV (e.g., the FoV 330 a , 330 b of FIG. 3 ). For example, the filter determines an angle of arrival based on a phase difference of radar return signals received at each of two receiving elements (e.g., elements 225 a - b of FIG. 2 ). Thus, those signals having an angle of arrival outside the limited FoV are eliminated (i.e. are not used in the generation of histograms as will be described in detail further below).
- a filter removes those signals originating from outside a limited FoV (e.g., the FoV 330 a , 330 b of FIG. 3 ). For example, the filter determines an angle of arrival based on a phase difference of radar return signals received at each of two receiving elements (e.g., elements 225 a - b of FIG. 2 ). Thus, those
- the method 500 includes doppler shifting the filtered RF response signals to limit doppler ambiguity and generate a doppler shifted signal.
- the doppler is shifted based on a Doppler Nyquist and a Doppler Sample Rate because the doppler, due to static infrastructure along a road, wraps around edges of the FFT signal. Accordingly, doppler is shifted at an integer times the Doppler Sample Rate based on a vehicle speed.
- the Doppler Nyquist is 1 ⁇ 2* Doppler Sample Rate. Accordingly, those radar return signals originating from behind or in front of the vehicle should normalize to a value of one (1) because the road surface speed corresponds to the vehicle's speed. When road conditions change, the behavior of doppler signature changes e.g. (wet vs dry vs snow).
- Processing then proceeds to processing block 520 which includes normalizing the doppler shifted signal based upon the vehicle's speed. Specifically, the doppler shifted signal is divided by the speed of the vehicle to generate a normalized doppler signal.
- processing then proceeds to processing block 525 which includes dividing the normalized doppler signature signal based on a location source of the received radar return signals.
- the signal is divided into near range and far range signals.
- the near range signals include data corresponding to the radar response signals emanating from a first region proximate the vehicle.
- the far range signals include data corresponding to the radar response signals emanating from a second region further away from the vehicle than the first region. The near and far regions are further described with respect to FIGS. 6-7 .
- the method 500 includes generating near range histograms and far range histograms that identify doppler spread within each of the regions.
- the normalized doppler data is integrated into a histogram by counting+1 for every data point taken from the FFT that passes a magnitude threshold.
- An x-axis of the histogram is the normalized doppler and the y-axis is a current count for that normalized Doppler bin collection.
- Each histogram is decayed to maintain a refresh rate of about 3 seconds of data.
- the method 500 includes performing statistical analysis on each of the histograms to determine a Q-factor for each histogram.
- a higher Q value represents a lower noise component, resulting in a predictable outcome.
- the method 500 includes determining a road surface condition.
- the histogram 600 includes a first histogram region 605 a and a second histogram region 605 b .
- Each region corresponds to locations where there is a separation of data. The separation of data can be based on regions from which radar return signals originate.
- the first histogram region 605 a can correspond to data originating from a first region (e.g., a region proximate the vehicle).
- the second histogram region 605 b can correspond to data originating from a second region (e.g., a region farther away from the vehicle than the first region).
- a vehicle 704 is equipped with a sensor 720 capable of detecting road surface conditions (e.g., the radar sensor 220 of FIG. 2 ).
- Sensor 720 transmits a waveform (e.g., the waveform 400 of FIG. 4 ), and processes waveform radar return signals (e.g., according to method 500 of FIG. 5 ).
- the detector 720 generates a histogram (e.g., the histogram 600 of FIG.
- first histogram region 605 a corresponds to data originating from radar return signals in the near region 770 .
- the second histogram region 605 b corresponds to data originating from radar return signals in the near region 780 .
- Each of the first and second histograms 605 a - b are then processed according to method 800 of FIG. 8 .
- FIGS. 8-8B are a series of plots which illustrate the statistical analysis that takes place in processing blocks 535 a , 535 b of the method 500 of FIG. 5 .
- a generated histogram 830 has a series of peak regions and a series of valleys.
- the histogram data is “smoothed” to generate smoothed curve 835 .
- the smoothed curve 835 can be generated using a variety of known smoothing techniques including but not limited to Savitky Golay windowing.
- the method further includes locating a highest peak 836 of the smoothed curve 835 within the histogram 830 . This may be accomplished, for example, by using a Center of Mass (CoM) computation, and then averaging of all non-zero histogram data points (e.g., data above non-zero line 850 ). Other techniques may, of course, also be used.
- CoM Center of Mass
- a baseline distance, B, of a portion of the smoothed curve 835 that included the highest peak 836 is then determined.
- the baseline distance corresponds to a distance between the first intersection points of smoothed curve 835 with the line 850 that are to the left and right of the highest peak 836 .
- the method determines a peak distance A that is a distance between the highest peak 836 and the line 850 .
- a Q-factor of the histogram data is then determined as A divided by B (i.e. A/B).
- the method 500 includes determining the road condition based on the Q-factor.
- the method 500 includes looking at the relationship between Q-factor of each histogram region (i.e., near and far).
- the method 500 , at 540 can also include looking at the 1 st derivative of the Q factor to see how dynamic the behavior is (i.e., change over time). For example, significant change can indicate a wet road condition.
- a near region histogram 905 a ( FIG. 9 ) and a far region histogram 905 b ( FIG. 9A ) depict a dry road condition.
- a wet road condition produces a near region histogram 1000 a ( FIG. 10 ) and a far region histogram 1000 b ( FIG. 10A ).
- a road is wet, packets of water in crevices or cracks of the road surface cause the behavior of doppler in the near and far regions to vary significantly.
- the histograms 1000 a - b have a higher Q factor and present more dynamic Q-factor change over time as compared to a dry road condition.
- a snow covered road condition produces a near region histogram 1100 a ( FIG. 11 ) and a far region histogram 1100 b ( FIG. 11A ).
- the near and far region histograms 1100 a - b demonstrate dynamic Q-factor behavior due to fluctuating spread of doppler across the histogram bins.
- the above-described systems and methods can be implemented in digital electronic circuitry, in computer hardware, firmware, and/or software.
- the implementation can be as a computer program product.
- the implementation can, for example, be in a machine-readable storage device, for execution by, or to control the operation of, data processing apparatus.
- the implementation can, for example, be a programmable processor, a computer, and/or multiple computers.
- a computer program can be written in any form of programming language, including compiled and/or interpreted languages, and the computer program can be deployed in any form, including as a stand-alone program or as a subroutine, element, and/or other unit suitable for use in a computing environment.
- a computer program can be deployed to be executed on one computer or on multiple computers at one site.
- Method steps can be performed by one or more programmable processors executing a computer program to perform functions of the invention by operating on input data and generating output. Method steps can also be performed by and an apparatus can be implemented as special purpose logic circuitry.
- the circuitry can, for example, be a FPGA (field programmable gate array) and/or an ASIC (application-specific integrated circuit).
- Subroutines and software agents can refer to portions of the computer program, the processor, the special circuitry, software, and/or hardware that implement that functionality.
- processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer.
- a processor receives instructions and data from a read-only memory or a random access memory or both.
- the essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data.
- a computer can include, can be operatively coupled to receive data from and/or transfer data to one or more mass storage devices for storing data (e.g., magnetic, magneto-optical disks, or optical disks).
- Data transmission and instructions can also occur over a communications network.
- Information carriers suitable for embodying computer program instructions and data include all forms of non-volatile memory, including by way of example semiconductor memory devices.
- the information carriers can, for example, be EPROM, EEPROM, flash memory devices, magnetic disks, internal hard disks, removable disks, magneto-optical disks, CD-ROM, and/or DVD-ROM disks.
- the processor and the memory can be supplemented by, and/or incorporated in special purpose logic circuitry.
- the above described techniques can be implemented on a computer having a display device.
- the display device can, for example, be a cathode ray tube (CRT) and/or a liquid crystal display (LCD) monitor.
- CTR cathode ray tube
- LCD liquid crystal display
- the interaction with a user can, for example, be a display of information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer (e.g., interact with a user interface element).
- Other kinds of devices can be used to provide for interaction with a user.
- Other devices can, for example, be feedback provided to the user in any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback).
- Input from the user can, for example, be received in any form, including acoustic, speech, and/or tactile input.
- the above described techniques can be implemented in a distributed computing system that includes a back-end component.
- the back-end component can, for example, be a data server, a middleware component, and/or an application server.
- the above described techniques can be implemented in a distributing computing system that includes a front-end component.
- the front-end component can, for example, be a client computer having a graphical user interface, a Web browser through which a user can interact with an example implementation, and/or other graphical user interfaces for a transmitting device.
- the components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), the Internet, wired networks, and/or wireless networks.
- LAN local area network
- WAN wide area network
- the Internet wired networks, and/or wireless networks.
- the system can include clients and servers.
- a client and a server are generally remote from each other and typically interact through a communication network.
- the relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
- Packet-based networks can include, for example, the Internet, a carrier internet protocol (IP) network (e.g., local area network (LAN), wide area network (WAN), campus area network (CAN), metropolitan area network (MAN), home area network (HAN)), a private IP network, an IP private branch exchange (IPBX), a wireless network (e.g., radio access network (RAN), 802.11 network, 802.16 network, general packet radio service (GPRS) network, HiperLAN), and/or other packet-based networks.
- IP carrier internet protocol
- LAN local area network
- WAN wide area network
- CAN campus area network
- MAN metropolitan area network
- HAN home area network
- IP network IP private branch exchange
- wireless network e.g., radio access network (RAN), 802.11 network, 802.16 network, general packet radio service (GPRS) network, HiperLAN
- GPRS general packet radio service
- HiperLAN HiperLAN
- Circuit-based networks can include, for example, the public switched telephone network (PSTN), a private branch exchange (PBX), a wireless network (e.g., RAN, bluetooth, code-division multiple access (CDMA) network, time division multiple access (TDMA) network, global system for mobile communications (GSM) network), and/or other circuit-based networks.
- PSTN public switched telephone network
- PBX private branch exchange
- CDMA code-division multiple access
- TDMA time division multiple access
- GSM global system for mobile communications
- the transmitting device can include, for example, a computer, a computer with a browser device, a telephone, an IP phone, a mobile device (e.g., cellular phone, personal digital assistant (PDA) device, laptop computer, electronic mail device), and/or other communication devices.
- the browser device includes, for example, a computer (e.g., desktop computer, laptop computer) with a world wide web browser (e.g., Microsoft® Internet Explorer® available from Microsoft Corporation, Mozilla® Firefox available from Mozilla Corporation).
- the mobile computing device includes, for example, a Blackberry®.
- Comprise, include, and/or plural forms of each are open ended and include the listed parts and can include additional parts that are not listed. And/or is open ended and includes one or more of the listed parts and combinations of the listed parts.
Landscapes
- Engineering & Computer Science (AREA)
- Remote Sensing (AREA)
- Radar, Positioning & Navigation (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Electromagnetism (AREA)
- Automation & Control Theory (AREA)
- Mathematical Physics (AREA)
- Transportation (AREA)
- Mechanical Engineering (AREA)
- Traffic Control Systems (AREA)
- Radar Systems Or Details Thereof (AREA)
Abstract
Description
F pri·λ/2=λ/((T chirp +T pause)·2), (EQ. 1)
-
- Fpri represents the frequency of the pulse repetition interval;
- λ is a wavelength of the transmit signal frequency;
- Tchirp is a chirp transmission duration; and
- Tpause is a pause period (i.e. a period of time) following each chip transmission duration.
Claims (14)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US16/169,149 US11059489B2 (en) | 2018-10-24 | 2018-10-24 | Methods and systems for detecting road surface conditions |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US16/169,149 US11059489B2 (en) | 2018-10-24 | 2018-10-24 | Methods and systems for detecting road surface conditions |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| US20200130697A1 US20200130697A1 (en) | 2020-04-30 |
| US11059489B2 true US11059489B2 (en) | 2021-07-13 |
Family
ID=70328202
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| US16/169,149 Active 2039-10-17 US11059489B2 (en) | 2018-10-24 | 2018-10-24 | Methods and systems for detecting road surface conditions |
Country Status (1)
| Country | Link |
|---|---|
| US (1) | US11059489B2 (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP7264086B2 (en) * | 2020-02-27 | 2023-04-25 | トヨタ自動車株式会社 | Vehicle control device and vehicle control method |
| JP7584859B2 (en) * | 2020-09-10 | 2024-11-18 | Jrcモビリティ株式会社 | Apparatus and method for detecting moving speed |
| US11463842B2 (en) * | 2020-10-07 | 2022-10-04 | Huawei Technologies Co., Ltd. | Method and apparatus for chirp signal-based pose estimation |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060125679A1 (en) * | 2004-11-26 | 2006-06-15 | Omron Corporation | Image processing system for mounting to a vehicle |
| US20170315229A1 (en) * | 2016-03-24 | 2017-11-02 | RFNAV, Inc. | Low Cost 3D Radar Imaging and 3D Association Method from Low Count Linear Arrays for All Weather Autonomous Vehicle Navigation |
| US20180217231A1 (en) * | 2017-01-27 | 2018-08-02 | Massachusetts Institute Of Technology | Determining surface characteristics |
| US20190072669A1 (en) * | 2017-09-07 | 2019-03-07 | Magna Electronics Inc. | Vehicle radar sensing system with surface segmentation using interferometric statistical analysis |
| US20200017083A1 (en) * | 2016-09-22 | 2020-01-16 | Omniklima Ab | Method and arrangement for determining a condition of a road surface |
-
2018
- 2018-10-24 US US16/169,149 patent/US11059489B2/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060125679A1 (en) * | 2004-11-26 | 2006-06-15 | Omron Corporation | Image processing system for mounting to a vehicle |
| US20170315229A1 (en) * | 2016-03-24 | 2017-11-02 | RFNAV, Inc. | Low Cost 3D Radar Imaging and 3D Association Method from Low Count Linear Arrays for All Weather Autonomous Vehicle Navigation |
| US20200017083A1 (en) * | 2016-09-22 | 2020-01-16 | Omniklima Ab | Method and arrangement for determining a condition of a road surface |
| US20180217231A1 (en) * | 2017-01-27 | 2018-08-02 | Massachusetts Institute Of Technology | Determining surface characteristics |
| US20190072669A1 (en) * | 2017-09-07 | 2019-03-07 | Magna Electronics Inc. | Vehicle radar sensing system with surface segmentation using interferometric statistical analysis |
Non-Patent Citations (1)
| Title |
|---|
| Häkli et al., "Road Surface Condition Detection using 24 GHz Automotive Radar Technology;" 14th International Radar Symposium (IRS), vol. 2; Jan. 2013; 6 Pages. |
Also Published As
| Publication number | Publication date |
|---|---|
| US20200130697A1 (en) | 2020-04-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102186191B1 (en) | Radar sensing with interference suppression | |
| US11693085B2 (en) | FMCW radar with interference signal suppression | |
| US9746546B2 (en) | Method and device for sensing surrounding environment based on frequency modulated continuous wave radar | |
| US10036805B2 (en) | Method and apparatus for detecting surrounding environment based on sensing signals of frequency-modulated continuous wave radar and continuous wave radar | |
| JP4007498B2 (en) | Automotive radar equipment | |
| US6369747B1 (en) | Radar apparatus | |
| KR20190096291A (en) | Rader sensing with phase correction | |
| US6888494B2 (en) | FMCW radar system | |
| US10234541B2 (en) | FMCW radar device | |
| EP0906581B1 (en) | Vehicle radar system | |
| JP5595496B2 (en) | Radar equipment | |
| US11994576B2 (en) | Radar systems and methods | |
| US10429494B2 (en) | Method and apparatus for detecting target object | |
| US10823846B2 (en) | Object detection method and object detection device | |
| EP1318415B1 (en) | Mispairing determination method for FM-CW radar | |
| CN110873877A (en) | Method and device for determining target motion trajectory | |
| JP2001242242A (en) | Millimeter wave radar device with detection performance improvement function | |
| US9046607B2 (en) | Radar system and detection method | |
| US11059489B2 (en) | Methods and systems for detecting road surface conditions | |
| JP2006010410A (en) | Target detection device | |
| JP3743181B2 (en) | Pulse Doppler radar device | |
| JP2006058135A (en) | Moving object detection apparatus and moving object detection method | |
| CN104977566A (en) | Signal processing method and device for a frequency modulation continuous wave radar sensing system | |
| JP6275370B2 (en) | Radar equipment | |
| JPH11271431A (en) | FMCW radar equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
| AS | Assignment |
Owner name: VALEO RADAR SYSTEMS, INC., NEW HAMPSHIRE Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:INSANA, DAVID;REEL/FRAME:047357/0388 Effective date: 20181023 |
|
| STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
| STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
| STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
| STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT RECEIVED |
|
| STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED |
|
| STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
| MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 4 |