WO2003029045A2 - Multi-technology object detection system and method - Google Patents

Multi-technology object detection system and method Download PDF

Info

Publication number
WO2003029045A2
WO2003029045A2 PCT/US2002/031213 US0231213W WO03029045A2 WO 2003029045 A2 WO2003029045 A2 WO 2003029045A2 US 0231213 W US0231213 W US 0231213W WO 03029045 A2 WO03029045 A2 WO 03029045A2
Authority
WO
WIPO (PCT)
Prior art keywords
sensors
vehicle
sensor output
processor
detection
Prior art date
Application number
PCT/US2002/031213
Other languages
French (fr)
Other versions
WO2003029045A3 (en
Inventor
Stephen P. Hebeisen
Heyward Williams
Hugh Bishop
Original Assignee
Sense Technologies, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sense Technologies, Inc. filed Critical Sense Technologies, Inc.
Publication of WO2003029045A2 publication Critical patent/WO2003029045A2/en
Publication of WO2003029045A3 publication Critical patent/WO2003029045A3/en

Links

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S15/00—Systems using the reflection or reradiation of acoustic waves, e.g. sonar systems
    • G01S15/88—Sonar systems specially adapted for specific applications
    • G01S15/93—Sonar systems specially adapted for specific applications for anti-collision purposes
    • G01S15/931—Sonar systems specially adapted for specific applications for anti-collision purposes of land vehicles
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60Q—ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL
    • B60Q9/00—Arrangement or adaptation of signal devices not provided for in one of main groups B60Q1/00 - B60Q7/00, e.g. haptic signalling
    • B60Q9/008—Arrangement or adaptation of signal devices not provided for in one of main groups B60Q1/00 - B60Q7/00, e.g. haptic signalling for anti-collision purposes
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/86—Combinations of radar systems with non-radar systems, e.g. sonar, direction finder
    • G01S13/862—Combination of radar systems with sonar systems
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/88—Radar or analogous systems specially adapted for specific applications
    • G01S13/93—Radar or analogous systems specially adapted for specific applications for anti-collision purposes
    • G01S13/931—Radar or analogous systems specially adapted for specific applications for anti-collision purposes of land vehicles
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/02—Systems using reflection of radio waves, e.g. primary radar systems; Analogous systems
    • G01S13/50—Systems of measurement based on relative movement of target
    • G01S13/52—Discriminating between fixed and moving objects or between objects moving at different speeds
    • G01S13/56—Discriminating between fixed and moving objects or between objects moving at different speeds for presence detection
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/88—Radar or analogous systems specially adapted for specific applications
    • G01S13/93—Radar or analogous systems specially adapted for specific applications for anti-collision purposes
    • G01S13/931—Radar or analogous systems specially adapted for specific applications for anti-collision purposes of land vehicles
    • G01S2013/9314—Parking operations
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/88—Radar or analogous systems specially adapted for specific applications
    • G01S13/93—Radar or analogous systems specially adapted for specific applications for anti-collision purposes
    • G01S13/931—Radar or analogous systems specially adapted for specific applications for anti-collision purposes of land vehicles
    • G01S2013/9315—Monitoring blind spots
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/88—Radar or analogous systems specially adapted for specific applications
    • G01S13/93—Radar or analogous systems specially adapted for specific applications for anti-collision purposes
    • G01S13/931—Radar or analogous systems specially adapted for specific applications for anti-collision purposes of land vehicles
    • G01S2013/9317—Driving backwards
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/88—Radar or analogous systems specially adapted for specific applications
    • G01S13/93—Radar or analogous systems specially adapted for specific applications for anti-collision purposes
    • G01S13/931—Radar or analogous systems specially adapted for specific applications for anti-collision purposes of land vehicles
    • G01S2013/932—Radar or analogous systems specially adapted for specific applications for anti-collision purposes of land vehicles using own vehicle data, e.g. ground speed, steering wheel direction
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/88—Radar or analogous systems specially adapted for specific applications
    • G01S13/93—Radar or analogous systems specially adapted for specific applications for anti-collision purposes
    • G01S13/931—Radar or analogous systems specially adapted for specific applications for anti-collision purposes of land vehicles
    • G01S2013/9324—Alternative operation using ultrasonic waves
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/88—Radar or analogous systems specially adapted for specific applications
    • G01S13/93—Radar or analogous systems specially adapted for specific applications for anti-collision purposes
    • G01S13/931—Radar or analogous systems specially adapted for specific applications for anti-collision purposes of land vehicles
    • G01S2013/9327—Sensor installation details
    • G01S2013/93272—Sensor installation details in the back of the vehicles
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/88—Radar or analogous systems specially adapted for specific applications
    • G01S13/93—Radar or analogous systems specially adapted for specific applications for anti-collision purposes
    • G01S13/931—Radar or analogous systems specially adapted for specific applications for anti-collision purposes of land vehicles
    • G01S2013/9327—Sensor installation details
    • G01S2013/93275—Sensor installation details in the bumper area
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S15/00—Systems using the reflection or reradiation of acoustic waves, e.g. sonar systems
    • G01S15/88—Sonar systems specially adapted for specific applications
    • G01S15/93—Sonar systems specially adapted for specific applications for anti-collision purposes
    • G01S15/931—Sonar systems specially adapted for specific applications for anti-collision purposes of land vehicles
    • G01S2015/937—Sonar systems specially adapted for specific applications for anti-collision purposes of land vehicles sensor installation details
    • G01S2015/938—Sonar systems specially adapted for specific applications for anti-collision purposes of land vehicles sensor installation details in the bumper area

Definitions

  • the invention relates to object detection systems including but not limited to vehicular warning systems, and more particularly, to devices mountable on a vehicle for warning the vehicle operator of obstacles. Still more particularly, this invention relates to an improved programmable warning system and method using a plurality of different sensing technologies (e.g., microwave and ultrasonic detection) to detect objects.
  • sensing technologies e.g., microwave and ultrasonic detection
  • a system called GUARDIAN ALERT TM that has been marketed and manufactured by Sense Technologies, Inc. is capable of warning a driver of the existence of objects within a defined area behind the vehicle when the vehicle is engaged in reverse gear.
  • the GUARDIAN ALERT TM Backup System is a back-up obstacle detection/collision warning system. This system is capable of warning a driver in the event that there is an obstacle in a predefined zone directly behind a rearwardly moving vehicle that could present an accident threatening circumstance. Drivers of motor vehicles have always had a certain degree of difficulty backing up due to their inability to see all obstacles while operating their vehicles in reverse.
  • the GUARDIAN ALERT TM Backup System is comprised of a transceiver, an antenna, an intermediate frequency amplifier, a data acquisition and analysis circuit and an audio-visual alarm.
  • the components are mounted in an environmentally sealed, high impact, plastic housing.
  • the unit is adapted for mounting at the rearward end of the vehicle in order to direct its wave output rearwardly.
  • the internal working components of the GUARDIAN ALERT TM Backup System are smaller than a pack of cigarettes.
  • the unit does not materially alter the exterior design of the vehicle and can, in fact, mount inconspicuously within the hidden, low-impact crash foam found behind the cosmetic bumpers of today's typical motor vehicles.
  • the circuit of the device is adapted for electrical connection to the back-up light circuit of a vehicle for activation only when the vehicle transmission is engaged in reverse gear.
  • the GUARDIAN ALERT TM Backup System employs microwave radar technology that applies the Doppler Shift Principle to detect the presence of a target within a certain predefined range that is moving with respect to the vehicle. Electromagnetic wave signals are projected by the transceiver rearwardly and return wave signals reflected from any object within the transceiver's detection range behind the vehicle which are supplied back to the transceiver through an antenna connection. An object that moves closer to the unit, due either to the movement of the object itself or the rearward motion of the vehicle, sends back a signal that is shifted slightly in frequency which is referred to as the Doppler Shift.
  • the resultant Doppler Shift signal is amplified and detected by the data acquisition and analysis circuit which ultimately triggers the system's visual and audio alarm.
  • a visual and audio display provides an indication as to the relative distance from the detected object, and most likely would be located inside the vehicle.
  • the effective range of the system is adjustable to accommodate the needs of different-sized vehicles for blind area coverage.
  • the GUARDIAN ALERT TM Backup System immediately sounds an alarm when any moving or stationary object is detected within the transceiver range behind a rearwardly moving vehicle on which it is installed. Even when the vehicle is stationary, but with back-up lights activated, the device will signal the presence of any moving object within the transceiver range. Because the system operates by microwave radar, it is unaffected by sound, noise, light, or weather conditions such as snow, rain, heat, fog or cold. See, for example, U.S. Patent No. RE34,773; U.S. Patent No. 4,797,673; U.S. Patent No. 4,864,298; and U.S. Patent No. 5,028,920.
  • the infrared systems can be both active or passive sensors where passive systems do not employ their own transmission source for infrared energy.
  • Active infrared sensors project infrared beams and detect reflected infrared light from objects in the scanned zone behind the vehicle, which activates an audio or visual alarm.
  • Infrared systems may not be particularly reliable for detecting objects in the rear blind area. Bright sunlight and reflections from bright objects could trigger false alarms and their range depends upon the reflectivity of the detected object.
  • the typically narrow beam pattern usually necessitates the use of arrays of sensors to scan the desired zone, increasing system cost and complexity and decreasing reliability.
  • Ultrasonic systems work on the principle of emitting an ultrasonic sound burst and detecting a reflected energy wave returned to the source by contact with an object in the detection zone, again activating an alarm.
  • Ultrasonic sensors can be extremely cost-effective and offer presence sensing capabilities (the ability to detect objects whether they are moving or stationary) with range information.
  • Ultrasonic sensors generally employ a round trip time of flight detection system in order to detect obstacles and the range to the obstacles.
  • the detection range is more limited than microwave radar due to speed of sound issues, attenuation of sound energy issues and overall accuracy issues.
  • the accuracy of the ultrasonic system can be easily compromised by the humidity in the environment. Also, ultrasonic noise such as tire rubbing can false trip these sensors as well.
  • Other areas where ultrasonic technology has some problems includes build-up of dust, dirt, leaves, and various environmental phenomena such as snow over the transducers which prevents the system from receiving reflected ultrasonic energy.
  • ultrasonic sensors typically must be aligned to not see the ground in order to work effectively. However, this leaves them vulnerable to missing objects such as bricks and cement parking blocks along with children that have fallen or are hiding near the ground surface. Additionally, ultrasonic sensing does not inherently qualify moving vs. stationary targets since it is a presence sensing technology. Additionally, the round trip time for ultrasonic target information tends to limit processing time in low cost processors and thereby the maximum vehicle/object speed at which the solution can work.
  • microwave-based Doppler radar detection is the optimal sensing technology to use.
  • microwave technology is an electromagnetic media which is fairly impervious to the environmental conditions and is readily available in fairly low cost forms such as Doppler motion sensors used for security intrusion alarms and other commercial applications.
  • Doppler is best at identifying relative object movement towards or away from the sensor's radiating orientation. It is also capable of distinguishing direction of movement in addition to relative speed.
  • microwave technology is not perfect for every sort of obstacle detection process.
  • microwave radar can sometimes have difficulty detecting objects with low dielectric constant and radar signature Uke plastic pipes, chain link fences with link spacings larger than the transmitted wavelength and shrubs.
  • Microwave-based systems can also sometimes have difficulty detecting smaller or lower dielectric material objects in front of large metallic objects like chain link fences in front of a metallic warehouse wall.
  • microwave-based systems can at times detect obstacles within its detection range that are not obstacles for the vehicle. For instance, microwave systems can detect items such as sewer pipes that are laying beneath the pavement surface which clearly would not pose a threat for collision to the vehicle. Therefore, sensing technologies such as ultrasonic may work under some limited circumstances where microwave sensing is not completely effective.
  • One aspect of the invention preferably uses at least two sensor technologies and at least one sensor transducer for each technology to cost effectively and robustly detect objects within a predefined detection pattern.
  • the sensor technologies are complementary in their detection capabilities so that each has a unique capability that allows it to detect characteristics about the intended objects that the other technology cannot.
  • the sensor technologies or a portion of the technologies also preferably have some cross-over in their capabilities over some aspects of the detection pattern so that the detection of objects by one technology can be verified by the other technology in order to minimize falsely identifying objects within the detection pattern.
  • another aspect of the invention relies on at least one of the sensor technologies to have ranging information on objects that are detected.
  • the sensor system utilizes the ranging information to help determine based on the distance between the object and the sensor how the various sensor signals will be applied to determine the likelihood that an object has been detected, the urgency of the situation, whether or not to activate an alert based on the situation and the characteristics of that alert.
  • Figures 1 A-1C show exemplary relationships of individual sensor element detection patterns to the Stop, Alert and further detection zones.
  • Figure 2 shows an example illustrative dual-sensor technology sensing system wherein four ultrasonic sensor elements and two microwave sensor elements are arranged across the rear bumper on a vehicle.
  • Figure 3 shows an example functional block diagram applying dual- sensor technologies
  • Figure 3A shows an example illustrative schematic circuit diagram
  • FIGS 4 and 5A-5C are flowcharts of example software controlled processes performed by the illustrative embodiment.
  • Figures 1 and 2 show an example dual-technology vehicle warning system 100 provided by a preferred embodiment of this invention.
  • System 100 is mounted within the rear bumper 102 of a vehicle 104 and automatically activates when reverse gear is selected.
  • System 100 includes two visible ultrasonic sensors 106 and one "bidden" next-generation Doppler Microwave sensor 108 mounted behind the vehicle's rear bumper 102 face.
  • the ultrasonic sensors 106 are preferably mounted in the bumper 102 corners 110 and provide detection in the example embodiment to 2 meters (near field) of any object, including stationary and moving.
  • the radar sensor 108 in the illustrative embodiment is configured to detect any object the distance to which is diminishing, because either the vehicle 104 is moving toward the object, the object is moving toward the vehicle or both.
  • Sensor 108 in the illustrative embodiment detects out to 6 meters (far field) depending on vehicle speed for safety and collision avoidance.
  • the example embodiment integrated system 100 reports any object, stationary or moving, microwave and/or ultrasonic, within 2 meters, so the driver does not begin to reverse the vehicle 104 without the system's notification. This detection range will minimize the false detection algorithms in order to improve response time for driver reaction due to the shorter stopping distance. For stationary targets detected here while the vehicle is in reverse with the brake applied may/may not squelch the audible alarm but keep the visual alarm in order to minimize annoyance to the driver.
  • Figure IB shows both micrpw preparedaye a ⁇ ltr sqpic Ret ⁇
  • Figure 1C shows both microwave and ultrasonic
  • the system 100 also changes its notification to the driver as the object comes nearer the vehicle, giving the driver additional alerting assistance.
  • the exemplary system 100 is designed to supplement other safety practices by enhancing the driver's awareness of difficult-to-see areas.
  • the illustrative system 100 does not take any automatic action to prevent accidents.
  • Responsibility for the safe operation of the vehicle remains with the driver. To this extent it is ideally important for the display to mount so that the driver does not alter their reversing habits in order to view its output.
  • natural mounting opportunities present themselves in or about the rear view and side view mirrors, and also in various positions around the rear window of the vehicle if that is where the driver looks while reversing.
  • the illustrative system 100 offers several benefits including for example:
  • System 100 preferably uses at least two sensor technologies and at least one sensor transducer for each technology to cost effectively and robustly detect objects within a predefined detection pattern.
  • the sensor technologies are complementary in their detection capabilities so that each has a unique capability that allows it to detect characteristics about the intended objects that the other technology cannot.
  • the sensor technologies or a portion of the technologies also preferably have some cross-over in their capabilities over some aspects of the detection pattern so that the detection of objects by one technology can be verified by the other technology in order to minimize falsely identifying objects within the detection pattern.
  • illustrative system 100 relies on at least one of the sensor technologies to have ranging information on objects that are detected.
  • the sensor system 100 utilizes the ranging information to help determine based on the distance between the object and the sensor how the various sensor signals will be applied to determine the likelihood that an object has been detected, the urgency of the situation, whether or not to activate an alert based on the situation and the characteristics of that alert.
  • the number of sensor elements for each sensing technology is not explicitly limited and generally will be governed more by application detection pattern and cost. Additionally, the distances provided in the detection pattern are merely given distances for illustration. In preferred implementations, the distances will be tailored to the application. In addition, the detection range distances are not necessarily static and may correlate more to an urgency to detect that takes into account more than just the distance to the object.
  • Illustrative system 100 thus has at least two sensor technologies with their included transducers and antennas for providing an electrical signal that describes each technology's view of the environment and for focusing each technology's view of the environment into a specific area, respectively.
  • the actual number of sensor elements can vary.
  • the primary intention of the diagram is to illustrate that each sensor technology can have more than one sensor element and to show one example illustrative configuration.
  • illustrative system 100 further includes signal conditioning circuitry 120, 122 for each transducer that conditions the signals coming from the receivers within the transducers for analysis by a controller 124.
  • a controller 124 accepts the conditioned signals from each sensor technology, applies predefined logic to determine how the information is used to not only detect objects within the detection area but also assess whether the objects should be identified, the nature of any identification via a display or alarm, and outputs the signal required to drive the display or alarm appropriately.
  • a display or alarm 126 accepts the signal from the controller 124 and activates the appropriate indication.
  • the indication can be one or a combination of several methods typical to the industry including audible indication (i.e. buzzers, beepers and speakers), visual indication (i.e., LEDs, LCDs and lamps) and vibrational methods.
  • a power supply and conditioning circuit transforms and conditions the power for activating the circuitry identified for each of the functions above.
  • connection and conditioning circuitry (not shown) is preferably used to help determine the status of the environment to which the sensing system 100 is attached. For example, if the sensing system 100 is attached to a vehicle 102, status information based on brake activation, transmission status (Park, Reverse, Neutral, Forward - 1/2/3/OD), vehicle speed and/or steering orientation can be used to alter operational modes of the system in order to power or enable the output on the system, alter detection pattern width and range, activate and even to help discriminate object characteristics.
  • a security sensing device mounted within a building where the status of the building is brought to the sensor such as occupied, day vs. night, time of day, HVAC status, security panel status (alarm/standby).
  • the sensor technologies 106, 108 will mount on or about the rear of the vehicle 102 so that the detection patterns cover the area that the vehicle would typically reverse over.
  • the display/alarm 126 will be expected to mount so that the operator of the vehicle 102 can monitor its alert activity without changing his or her driving habits. In this fashion, this sensing system 100 will act as a driver's aid.
  • the display 126 may or may not be placed within the passenger area of the vehicle.
  • the rest of the electronics can be configured in any number of topologies ranging from centralized (everything in one place) to distributed (each sensing technology operates as an independent sub-system whereby its electronics and output status is fed to a central controller) to a hybrid approach depending on the application.
  • Microwave technology is an electromagnetic media which is fairly impervious to the environmental conditions and is readily available in fairly low cost forms such as Doppler motion sensors used for security intrusion alarms and other commercial applications.
  • Doppler is best at identifying relative object movement towards or away from the sensor's radiating orientation. It is capable of distinguishing direction of movement in addition to relative speed.
  • Microwave technology can have difficulty detecting objects with low dielectric constant and radar signature like plastic pipes, chain link fences with link spacings larger than the transmitted wavelength and shrubs. It has difficulty detecting smaller or lower dielectric material objects in front of large metallic objects like warehouse walls.
  • a microwave Doppler FSK Ranging transducer will provide cost effective discrimination of target range rather than relying on signal level which changes with the radar signature of targets (size, shape and composition). This ranging information will be complemented with the ultrasonic system and its range measurement capabilities.
  • Ultrasonic sensors employ a round trip time of flight detection system in order to detect obstacles and the range to the obstacles.
  • the detection range is more limited than Microwave Radar and will therefore be used for closer ranges.
  • the accuracy of the ultrasonic system can be more easily compromised by the humidity in the environment and thus by limiting the application range the sensor system hmits the tolerance on the measurement information.
  • Ultrasonic noise such as tire rubbing can false trip these sensors as well.
  • Other areas where ultrasonic technology has some problems includes buildup of dust, dirt, leaves, and various environmental phenomena such as snow over the transducers which prevents the system from receiving reflected ultrasonic energy.
  • ultrasonic sensors typically must be aligned to not see the ground in order to work effectively. However, this leaves them vulnerable to missing objects such as bricks and cement parking blocks along with children that have fallen or are hiding. Microwave Doppler will be applied to help minimize this vulnerability.
  • Ultrasonic sensing does not inherently qualify moving vs. stationary targets. It is a Presence Sensing Technology. Although processing routines could potentially be used to detect changes in distance in order to qualify movement on targets, a motion sensing system should be employed that can see to the ground and therefore a separate motion detection sub-system is required for alignment or detection pattern reasons. Therefore, the opportunity is there to utilize a second technology (microwave) for motion sensing to complement the technological strengths of the ultrasonic technology.
  • microwave microwave
  • the round trip time for ultrasonic target information tends to limit processing time in low cost processors and thereby the maximum vehicle/object speed at which the solution can work.
  • a second technology such as microwave Doppler can bring about a complementary capability.
  • the different sensor technologies 106, 108 used by the illustrative embodiment described herein have niches of capabilities and yet also have overlapping capabiUties as well.
  • Illustrative sensing system 100 employs the two sensor technologies 106, 108 as foUows.
  • Sensing system 100 in the example segments the detection zone into at least two priorities based on range from the vehicle while providing the opportunity for having many more levels of differentiation at farther ranges as appUcations require.
  • the two closest detection zones in order of increasing distance from the vehicle wiU be referred to as the Stop Zone and the Alert Zone.
  • the size of these detection zones will vary by appUcation to various types of vehicles. For instance, the Stop Zone for an economy passenger vehicle will most likely be considerably shorter than that for large earth moving dump trucks and may typicaUy faU in the 3 foot range.
  • the exemplary sensor sub-systems are configured to completely cover the width of the vehicle independently so that their detection zones roughly overlap close to the vehicle 102. It is expected that the range for the ultrasonic sensing system 106 wiU be shorter than that of the microwave sensing system 108. However, it is desired that the ultrasonic sensing system 106 at least cover through the Alert Zone. Meanwhile, the microwave sensing system 108 can reasonably extend farther in range beyond the Alert Zone to accommodate more detection ranges that might be required for larger commercial trucking or earth moving equipment appUcations, but these ranges are considered to be less critical in the need for the driver's attention due to the availability of stopping distance.
  • the sensing system may or may not be activated at all times on the vehicle 102. However, the display/alarm 126 will not be enabled until the vehicle transmission is placed into reverse in one illustrative embodiment.
  • the system 100 is preferably configured so that the display/alarm 126 always indicates the most pressing or highest priority alarm if more than one target is detected.
  • Priority can be estabUshed in many ways by the controUer 124 but should relate to the need for driver attention: e.g. object range, object range + closing rate and object type (i.e., human/non-human).
  • object type i.e., human/non-human.
  • the priority actually changes based on the range to the object or zone that the object faUs within.
  • Objects falUng within the Stop Zone always represent the highest priority which is a range priority.
  • Objects detected in the Alert Zone but having extreme relative closing rates or having been identified as a human target can become associated with the Stop Zone and thereby become the highest priority.
  • the range that is closest to the vehicle will apply both microwave Doppler and ultrasonic presence detection algorithms in an OR'd fashion so that an alert is activated by the controller 124 if either the ultrasonic or microwave system detects objects within this range. Therefore, combining microwave and ultrasonic detection provides the vehicle 102 with a warning on stationary targets before the vehicle starts to move using primarily the ultrasonic technology. As the vehicle 102 begins to reverse or if an object moves within this range before the vehicle 102 reverses, the microwave sensor sub-system wiU detect objects that appear within range even if they are close to the ground while the ultrasonic sensor sub-system wiU complement the microwave signals on objects within this range that are considerably higher than the ground.
  • a detection made by the ultrasonic sub-system while the vehicle is in reverse and stationary wiU minimize the averaging routines appUed to the microwave system for objects detected within the same vicinity as the vehicle or object starts to move without fear of false detecting. This will aUow the ultrasonic alert to further quicken the response of the microwave system.
  • the system 100 is activated even when the vehicle is not in reverse AND if a vehicle has the transmission go from a forward gear to a park or neutral gear, then the sensor system 100 will automatically save into memory the range information found on objects detected by both the microwave and ultrasonic system regardless of direction of movement even though the alarm is not enabled as long as the sensor is powered/activated during these times. If the vehicle 102 is subsequently placed into reverse, the sensing system wiU reload the information on these targets and for objects detected within these ranges the averaging counters will be reduced in order to minimize the response time to alert.
  • the sensing system 100 may or may not be configured to quatify objects by their direction of motion with respect to the sensor. Typically, it would be safer within the Stop Zone if there were no direction of motion (DOM) quaUfication required as opposed to only signaUng an alarm if there were a closing distance between the object and sensor.
  • DOM direction of motion
  • Zone will also employ both microwave and ultrasonic sensing technologies. However, within this zone the detection made by both technologies 106, 108 is AND'ed together in the example embodiment in order to minimize false alarms. For example, before the system 100 enables an alarm for this range, the controller 124 must have confirmation of the target by both the microwave and ultrasonic sensor subsystems 106, 108. Thus, the system 100 does not alarm if the vehicle is in Reverse but stationary while a stationary object presents itself within this zone.
  • the controller 124 will apply this information to the microwave detection system 108 and once again minimize the averaging counter required to acknowledge a target because the fear of false detection has been reduced by the redundancy of the ultrasonic sensor sub-system verifying the object.
  • the microwave Doppler signals will not trigger an alarm by themselves in the exemplary system within this zone which will enable the system 100 to minimize the number of false detects on items buried within the ground structure such as sewer mains, pipes and steel support rods.
  • Environmental elements wiU also be further discriminated against by both systems as the lack of complementary microwave detections will eUminate falsing in high wind conditions and on other sources of ultrasonic noise typically "seen” by ultrasonic sensors, while rain and other hydrometeors can be eUminated from the higher frequency microwave technology signals as the ultrasonic system 106does not duplicate detection of these items.
  • the sensing system 100 may or may not be configured within this zone to alert on moving objects only if there is a closing distance between the vehicle and object. Typically, since the Alert Zone should be trying to minimize falsing it would make sense to apply DOM quaUfication that requires a closing distance.
  • a closing rate assessment is also made on detected objects. If this is greater than a certain speed, then the object is considered to be in the Stop Zone because the time to collision is expected to be approaching the expected stopping time for the vehicle and/or the object.
  • microwave Doppler In detection ranges that are estabUshed beyond the Alert Zone, typically only microwave Doppler would be employed due to Umitations on the ultrasonic subsystem in range. These ranges would only be considered to be warnings to the vehicle operator. Most likely direction of motion (DOM) quaUfication would be employed here once again in order to ⁇ iinimize nuisance alarms.
  • a closing rate assessment may also be activated in order to determine if the object should be considered within the Alert or even Stop Zones.
  • Figure 3 A is a circuit diagram of an example illustrative circuit diagram.
  • the Figure 3A circuit diagram provides an additional input to accommodate the signal output from an ultrasonic sensing system 106 as well as an input from a motion detector 108 such as for example a microwave sensor using the Doppler shift principle.
  • software is used to combine these two signals together to provide a correct response to the display 126. While it may be possible to integrate the individual sensing systems further, this illustrative and exemplary embodiment lets each sensing system 106, 108 operate independently and then combines the two outputs together using an algorithm to be described below.
  • the exemplary circuit diagram shown in Figure 3A is similar to that disclosed in U.S. Patent appUcation serial number 09/906,790 entitled “Programmable Microwave Backup Warning System and Method” filed July 18, 2001 (incorporated herein by reference) but has an additional input 500 for receiving the output of a ultrasonic presence detector 106.
  • This input 500 is processed using a schottky inverter 502 in order to provide a degree of hysteresis in order to reduce system response to transient inputs.
  • a microprocessor 124 is also capable of driving the input/output connector 500 as necessary.
  • Figures 4 and 5A-5C show illustrative routines which identify whether an object is stationary or moving and operate by considering the vehicles transmission mode, the braking status, vehicle velocity information (if available), presence detection versus motion detection signals and the nature of the Doppler information from the microwave sensor sub-system. This information will help to identify special driving patterns as detailed further below, and also human recognition routines also further detailed below.
  • Figure 4 shows a flowchart of an iUustrative "main" software routine performed by controller 124 to sense the presence of objects such as obstacles.
  • system 100 is configured to be powered on whenever power is appUed to a vehicle's reverse Ughts. A self-test is performed at power-up in the example embodiment but could also be considered to be employed periodically while the sensor is operating.
  • System 100 in this particular example is configured to be direction-of-motion conscious, with presence detection being performed by the ultrasonic sensor 106 and motion detection being performed by the microwave sensor 108 in this example.
  • the exemplary and iUustrative "main" routine shown in Figure 4 begins by testing whether a presence detect sensor 106 (e.g., an ultrasonic type sensor in the preferred exemplary embodiment) has detected the existence of an object (decision block 402). If the presence of an object has been detected by the presence detector 106 ("yes" exit to decision block 402), the exemplary and illustrative routine 400 uses additional information from the presence detector 106 (and possibly also the motion detector 108, such as, for example, a microwave sensor using the Doppler shift principle), to determine whether the obstacle that has been detected is at a short range (decision block 404), a mid range (decision block 406), or a far range (decision block 408).
  • a presence detect sensor 106 e.g., an ultrasonic type sensor in the preferred exemplary embodiment
  • the exemplary and illustrative routine 400 uses additional information from the presence detector 106 (and possibly also the motion detector 108, such as, for example, a microwave sensor using the Doppler shift
  • the "stop" alarm generation routine (the highest priority of three different priority alarms in the exemplary iUustrative embodiment) is shown in more detail in Figure 5C.
  • generation of the "stop" alarm involves enabling the stop alarm, resetting the stop output counter, and resetting the driver mode counter (block 412), and then powering the alarm (block 414) to actuaUy generate an audible and/or visual alarm. Control then returns to the main loop 416 of main routine 400 to continue the sensing and monitoring process.
  • routine 400 determines whether motion has also been detected and logged (decision block 418; see also decision block 450). If both motion and presence have been detected (“yes" exit to decision block 418), the exemplary routine 400 determines whether the motion detector 108 indicates that the object is closing with the vehicle (i.e., is moving closer relative to the vehicle) (decision block 420). If decision block 420 reveals that the object is closing, then an additional test performed by the exemplary and illustrative routine 400 determines whether the closing velocity is greater than a certain predetermined velocity threshold (decision block 422).
  • the "stop” alarm event is generated (block 410) to inform the vehicle operator that a colUsion is imminent. Otherwise, if the closing velocity is less than a predetermined threshold velocity ("no" exit to decision block 422), then a lower priority "alert” alarm event is generated instead (block 424).
  • An illustrative "alert" alarm routine in the exemplary embodiment is shown in Figure 5B. In this example, the system 100 determines whether an alarm is already enabled (decision block 426). If the alarm is already enabled ("yes" exit to decision block 426), then the exemplary system 100 determines whether the alarm event that is being generated corresponds to "stop" (decision block 428).
  • the exemplary embodiment If a "stop" alarm event is already being generated ("yes” exit to decision block 428), then the exemplary embodiment resets the alert output counter (block 430) and returns (block 432). Otherwise, if the alarm is not already enabled or is not already in a "stop” type alarm mode ("no" exits to decision blocks 426, 428, respectively), the exemplary system enables the "alert” alarm, resets the alert output counter, and resets the driver mode counter (block 434) and then powers the alarm to provide a corresponding alert indication (block 436).
  • routine 400 performs certain checks to determine whether the obstacle may be a human being (block 438). By taking into account whether the vehicle is moving, and by looking at the variabiUty in DOM information and consistency of Doppler velocity, an estimate can be made as to whether a potential object is human or not. The reason for doing so is quite simply due to the fact that in most cases the microwave sensor 108 may be mounted so that it is looking about knee height on a human.
  • the gate of the typical human stride does not always provide a consistent enough DOM indication to estabUsh detection on an object. Therefore, if a microwave Doppler signal has been picked up within the alert zone but is avoiding detection the controller 124 will apply an analysis routine to see if the sensor is picking up a human that is crossing the detection pattern or a human that has a particularly labored gate. If the object is determined to be a human within the alert zone, then DOM quaUfication wiU be removed from consideration and the controUer will activate the Display/Alarm.
  • routine 400 If the object is a person ("yes" exit to decision block 438), then in the exemplary embodiment, the routine 400 essentially ignores or discounts the fact that the object appears to be moving away relative to the vehicle and nevertheless performs the velocity threshold check of decision block 422 anyway in order to generate a "stop" or an "alert” alarm. Thus, in the exemplary embodiment, routine 400 will generate some sort of an alarm whenever a person is detected within the mid range irrespective of the relative direction and velocity of motion. This is appropriate to minimize injury and dataUties to people, and also reflects the fact that human beings (unlike inanimate objects) will generally set off a motion detector even if they are standing very still because there is nearly always some movement of body position.
  • routine 400 determines whether there was prior far motion detected (decision block 440). If there was no prior far motion detected ("no" exit to decision block 440), then control preferably returns to main loop 416. On the other hand, if prior far motion was detected ("yes" exit to decision block 440), then in the exemplary embodiment, routine 400 determines whether the vehicle is stationary ⁇ meaning that the object is moving as opposed to the vehicle (decision block 442).
  • determining whether the prior far motion was based on vehicle movement or obstacle movement can be determined by sensing, for example, whether the vehicle brake is being appUed. If the vehicle is not moving and there was recent far motion but no current far motion and yet an object is present within the mid range sensing area ("yes" exit to decision block 442), this indicates that an animate or otherwise recently moved object (e.g., possibly a person who missed being detected by block 438, or perhaps an animal such as a pet, or another vehicle such as a car or bicycle) has recently been moved into the danger area and is now standing stiU. In this case, a "stop" or "alert” alarm is generated since this is an event that the vehicle operator ought to know about.
  • the exemplary embodiment appUes a velocity threshold test (block 422) to the prior far motion to determine whether a stop or alert alarm should be generated (block 422).
  • a "detect” or “alert” alarm may be generated depending upon whether there is closing motion of greater than a predetermined velocity. Generally, it is undesirable to disturb the vehicle operator by generating alarms for potential obstacles that are a long way off that the vehicle will have tittle risk of striking. On the other hand, in the exemplary embodiment, we do generate a lower priority "detect” alarm under certain circumstances if a far-away detected object is moving (block 444) closer relative to the vehicle (block 446).
  • the exemplary routine 400 wiU generate an "alert" alarm event (block 424) if the closing motion is greater than a predetermined velocity ("yes" exit to decision block 448), and will - generate a "detect” (i.e., lower priority) alarm event (block 452) if the closing motion has a velocity less than the predetermined threshold ("no" exit to decision block 448).
  • Figure 5 A shows an exemplary routine performed in response to a
  • FIG. 5 A routine is similar Figure 5B routine in the exemplary embodiment except that decision block 456 in this case determines whether the alarm is already in the two exemplary higher priority alarm modes ("stop" or "alert").
  • main loop 416 has some additional tests it continuaUy performs when obstacle presence is not detected ("no" exit to decision block 402). For example, if no obstacle presence has been detected (“no" exit to decision block 402), the exemplary main routine 400 determines whether motion has been detected (decision block 450). If motion has been detected ("yes" exit to decision block 450) but no presence has been detected (“no" exit to decision block 402), this may be an indication that presence detection sensor 106 has failed. For example, if the presence detection sensor 106 has failed, it will no longer detect the presence of an obstacle but the motion detector 108 will still produce a detection result.
  • the exemplary embodiment takes advantage of redundant and overlapping aspects of the sensitivity of dual detectors 106, 108 to provide a "fail safe" mechanism that will generate an alarm even if one of the detectors has failed.
  • Self test and component failure adjustments can be used in the event that one or both sensor subsystems are deemed to not be working. This is a factor within the first two detection ranges especially since within the STOP zone each technology has detection capabiUties that are unique which the system is relying on. In the alert zone this capabiUty is also used since it takes an identification by both technologies in order to process an alarm condition.
  • the exemplary embodiment Before generating an alarm under such a condition, however, the exemplary embodiment first performs a test to determine whether environment conditions may be responsible for setting off one or the other of the sensors (block 464).
  • environment conditions may be responsible for setting off one or the other of the sensors (block 464).
  • exemplary system 100 can determine whether environmental conditions may be responsible — one way being for example to use a moisture or humidity detector, another being to analyze the return wave signature, and stiU another way being to use an additional transducer of some other type. If there is no environmental condition that might be responsible (“no" exit to decision block 464), then exemplary routine 400 assumes that the presence detector 106 has failed and wiU perform the testing algorithm described above based on motion detection in order to generate an alarm ("no" exit to decision block 464). Weather detection/obstacle detection routines may also possibly alter the logic appUed within each zone in order to increase reUabiUty.
  • main routine main loop 416 determines whether output counters have expired indicating that an alarm has been generated for a sufficient amount of time after an event was triggered (and no longer exists) and can now be silenced or turned off (decision block 466, block 468).
  • the exemplary routine 400 has the capabiUty to place system
  • the controUer 124 can be enabled to process tracked targets starting from the outer detection zones through the "alert” and finally to the "stop” zones somewhat differently than instantaneous targets.
  • the exemplary sensor system 100 will preferably be configured to identify firstly that the vehicle is reversing towards a stationary object and then to identify various patterns of reversing that are typically employed on the vehicle. For instance, if the operator of the vehicle 102 tends to reverse in a nervous manner where the vehicle is continually stopped and started before reaching its resting place, then the sensor will start to adjust its counters and employ a TIMEOUT counter so that as the operator brakes and comes to a stop, the sensor does not loose lock on the object that it is tracking.
  • the algoriuWis then"' able to reinitiate detection of the object quickly if the reversing resumes from a stopped condition before the ⁇ MEOUT counter expires. This is accompUshed by not resetting the averaging counters that are used to help prevent falsing if movement ceases until the ⁇ MEOUT counter expires.
  • steering information is available to the sensor system 100 then while a vehicle 102 turns into parking spaces or turns while reversing the sensor system 100 can apply compensation routines to the zones so that the zones reduce the maximum range that they will detect objects within. This will prevent detection of adjacent vehicles and wall surfaces that do not actuaUy present any possibiUty for colUsion. This wiU also be enhanced by looking at the ranging information from sensors on either half of the rear of the vehicle and altering the detection ranges for each half.

Abstract

A vehicle obstacle warning system (100) uses at least two sensor technologies in order to accurately and robustly detect and qualify obstacles within a predetermined detection zone. By using a combination of motion and presence sensors (104, 106) coupled with ranging information, the unique detection and accuracy traits of each detection technology can be appropriately applied, to provide effective obstacle detection while minimizing false alarms. The sensor technologies are complementary in their detection capabilities so that each has a unique capability that allows it to detect characteristics about the intended objects that the other technology cannot.

Description

TITLE OF THE INVENTION
MULTI-TECHNOLOGY OBJECT DETECTION SYSTEM AND METHOD
CROSS-REFERENCES TO RELATED APPLICATIONS [0001] This application claims the benefit of U.S. Provisional Application No.
60/326,428, filed October 3, 2001 (Attorney Docket 2776-14); and is related to commonly-assigned copending U.S. Patent Application Serial No. 09/906,790, entitied "Programmable Microwave Back-Up Warning System and Method" filed July 18, 2001 (Attorney Docket 2776-15), the entire contents of which are hereby incorporated herein by reference.
STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT [0002] Not applicable.
FIELD OF THE INVENTION [0003] The invention relates to object detection systems including but not limited to vehicular warning systems, and more particularly, to devices mountable on a vehicle for warning the vehicle operator of obstacles. Still more particularly, this invention relates to an improved programmable warning system and method using a plurality of different sensing technologies (e.g., microwave and ultrasonic detection) to detect objects.
BACKGROUND AND SUMMARY OF THE INVENTION [0004] There has long been a need to cost effectively and more reliably warn vehicle operators of obstacles that pose threats for collision so that action can be taken to avoid them. The need for an effective and reliable back-up system is evident when one considers the amount of damage low-speed backing collisions cause each year. Such collisions translate into major repair bills, countless injuries and even worse, fatalities. Early warning can ehminate or reduce the potential for damage to the vehicle and/or obstacle in addition to preventing physical harm or even loss of life for human obstacles, human occupants within obstacles, human occupants of the vehicle and/or innocent bystanders. For example, vehicle back-up obstacle detection/collision warning systems are useful in preventing such accidents and injuries.
[0005] A system called GUARDIAN ALERT ™ that has been marketed and manufactured by Sense Technologies, Inc. is capable of warning a driver of the existence of objects within a defined area behind the vehicle when the vehicle is engaged in reverse gear. The GUARDIAN ALERT ™ Backup System is a back-up obstacle detection/collision warning system. This system is capable of warning a driver in the event that there is an obstacle in a predefined zone directly behind a rearwardly moving vehicle that could present an accident threatening circumstance. Drivers of motor vehicles have always had a certain degree of difficulty backing up due to their inability to see all obstacles while operating their vehicles in reverse.
[0006] The GUARDIAN ALERT ™ Backup System is comprised of a transceiver, an antenna, an intermediate frequency amplifier, a data acquisition and analysis circuit and an audio-visual alarm. The components are mounted in an environmentally sealed, high impact, plastic housing. The unit is adapted for mounting at the rearward end of the vehicle in order to direct its wave output rearwardly. The internal working components of the GUARDIAN ALERT ™ Backup System are smaller than a pack of cigarettes. The unit does not materially alter the exterior design of the vehicle and can, in fact, mount inconspicuously within the hidden, low-impact crash foam found behind the cosmetic bumpers of today's typical motor vehicles. The circuit of the device is adapted for electrical connection to the back-up light circuit of a vehicle for activation only when the vehicle transmission is engaged in reverse gear.
[0007] The GUARDIAN ALERT ™ Backup System employs microwave radar technology that applies the Doppler Shift Principle to detect the presence of a target within a certain predefined range that is moving with respect to the vehicle. Electromagnetic wave signals are projected by the transceiver rearwardly and return wave signals reflected from any object within the transceiver's detection range behind the vehicle which are supplied back to the transceiver through an antenna connection. An object that moves closer to the unit, due either to the movement of the object itself or the rearward motion of the vehicle, sends back a signal that is shifted slightly in frequency which is referred to as the Doppler Shift. The resultant Doppler Shift signal is amplified and detected by the data acquisition and analysis circuit which ultimately triggers the system's visual and audio alarm. A visual and audio display provides an indication as to the relative distance from the detected object, and most likely would be located inside the vehicle. The effective range of the system is adjustable to accommodate the needs of different-sized vehicles for blind area coverage.
[0008] The GUARDIAN ALERT ™ Backup System immediately sounds an alarm when any moving or stationary object is detected within the transceiver range behind a rearwardly moving vehicle on which it is installed. Even when the vehicle is stationary, but with back-up lights activated, the device will signal the presence of any moving object within the transceiver range. Because the system operates by microwave radar, it is unaffected by sound, noise, light, or weather conditions such as snow, rain, heat, fog or cold. See, for example, U.S. Patent No. RE34,773; U.S. Patent No. 4,797,673; U.S. Patent No. 4,864,298; and U.S. Patent No. 5,028,920.
[0009] While the GUARDIAN ALERT ™ product employs microwave radar sensing, others have used ultrasonic or infrared sensing technologies to sense obstacles from vehicles. These different sensing technologies have respective advantages and disadvantages.
[0010] Generally, the infrared systems can be both active or passive sensors where passive systems do not employ their own transmission source for infrared energy. Active infrared sensors project infrared beams and detect reflected infrared light from objects in the scanned zone behind the vehicle, which activates an audio or visual alarm. Infrared systems may not be particularly reliable for detecting objects in the rear blind area. Bright sunlight and reflections from bright objects could trigger false alarms and their range depends upon the reflectivity of the detected object. Also, the typically narrow beam pattern usually necessitates the use of arrays of sensors to scan the desired zone, increasing system cost and complexity and decreasing reliability.
[0011] Ultrasonic systems work on the principle of emitting an ultrasonic sound burst and detecting a reflected energy wave returned to the source by contact with an object in the detection zone, again activating an alarm. Ultrasonic sensors can be extremely cost-effective and offer presence sensing capabilities (the ability to detect objects whether they are moving or stationary) with range information. Ultrasonic sensors generally employ a round trip time of flight detection system in order to detect obstacles and the range to the obstacles. The detection range is more limited than microwave radar due to speed of sound issues, attenuation of sound energy issues and overall accuracy issues. The accuracy of the ultrasonic system can be easily compromised by the humidity in the environment. Also, ultrasonic noise such as tire rubbing can false trip these sensors as well. Other areas where ultrasonic technology has some problems includes build-up of dust, dirt, leaves, and various environmental phenomena such as snow over the transducers which prevents the system from receiving reflected ultrasonic energy.
[0012] Because any reflection of ultrasonic energy constitutes an object, ultrasonic sensors typically must be aligned to not see the ground in order to work effectively. However, this leaves them vulnerable to missing objects such as bricks and cement parking blocks along with children that have fallen or are hiding near the ground surface. Additionally, ultrasonic sensing does not inherently qualify moving vs. stationary targets since it is a presence sensing technology. Additionally, the round trip time for ultrasonic target information tends to limit processing time in low cost processors and thereby the maximum vehicle/object speed at which the solution can work.
[0013] Furthermore, both ultrasonic and infrared sensing technologies are to some degree affected by adverse weather conditions such as rain, fog, snow or wind.
[0014] Our past experience has demonstrated that if one were to choose a single sensing technology for at least certain types of vehicular warning systems (notably rearward-ranging backup obstacle detection systems), microwave-based Doppler radar detection is the optimal sensing technology to use. For example, microwave technology is an electromagnetic media which is fairly impervious to the environmental conditions and is readily available in fairly low cost forms such as Doppler motion sensors used for security intrusion alarms and other commercial applications. Generally, Doppler is best at identifying relative object movement towards or away from the sensor's radiating orientation. It is also capable of distinguishing direction of movement in addition to relative speed.
[0015] However, as noted above, microwave technology is not perfect for every sort of obstacle detection process. For example, microwave radar can sometimes have difficulty detecting objects with low dielectric constant and radar signature Uke plastic pipes, chain link fences with link spacings larger than the transmitted wavelength and shrubs. Microwave-based systems can also sometimes have difficulty detecting smaller or lower dielectric material objects in front of large metallic objects like chain link fences in front of a metallic warehouse wall. Furthermore, microwave-based systems can at times detect obstacles within its detection range that are not obstacles for the vehicle. For instance, microwave systems can detect items such as sewer pipes that are laying beneath the pavement surface which clearly would not pose a threat for collision to the vehicle. Therefore, sensing technologies such as ultrasonic may work under some limited circumstances where microwave sensing is not completely effective.
[0016] To overcome these potential sensing reliability issues while exploring ways to make the GUARDIAN ALERT ™ system still more reliable, we have explored the possibility of integrating a plurality of different sensing technologies into the same to achieve greater detection reliability while providing an appropriate tradeoff between cost, complexity and mean time between failures. For example, we have developed a preferred embodiment sensor system that mounts to a vehicle which employs multiple technologies so that the signals received from each, whether they possess overlapping or non-overlapping detection patterns, can more completely and more reliably cover the detection area for both moving and stationary human and non- human obstacles. Among other things, preferred embodiments of the invention could be used for vehicular backup warning/ reverse aid/parking aid sensors, vehicular forward looking collision warning sensors and vehicular blind spot sensors. Other applications areas may include, but are not limited to, toys, security sensors, occupancy sensors and other areas where non-contact object detection capabilities are required both indoors and outside.
[0017] One aspect of the invention preferably uses at least two sensor technologies and at least one sensor transducer for each technology to cost effectively and robustly detect objects within a predefined detection pattern. The sensor technologies are complementary in their detection capabilities so that each has a unique capability that allows it to detect characteristics about the intended objects that the other technology cannot. However, the sensor technologies or a portion of the technologies also preferably have some cross-over in their capabilities over some aspects of the detection pattern so that the detection of objects by one technology can be verified by the other technology in order to minimize falsely identifying objects within the detection pattern. In order to determine how the signals from the different technologies are considered, another aspect of the invention relies on at least one of the sensor technologies to have ranging information on objects that are detected. The sensor system utilizes the ranging information to help determine based on the distance between the object and the sensor how the various sensor signals will be applied to determine the likelihood that an object has been detected, the urgency of the situation, whether or not to activate an alert based on the situation and the characteristics of that alert.
BRIEF DESCRIPTION OF DRAWINGS [0018] These and other features and advantages will be better and more completely understood by referring to the following detailed description of presently preferred illustrative embodiments in conjunction with the drawings, of which:
[0019] Figures 1 A-1C show exemplary relationships of individual sensor element detection patterns to the Stop, Alert and further detection zones. [0020] Figure 2 shows an example illustrative dual-sensor technology sensing system wherein four ultrasonic sensor elements and two microwave sensor elements are arranged across the rear bumper on a vehicle.
[0021] Figure 3 shows an example functional block diagram applying dual- sensor technologies;
[0022] Figure 3A shows an example illustrative schematic circuit diagram; and
[0023] Figures 4 and 5A-5C are flowcharts of example software controlled processes performed by the illustrative embodiment.
DETAILED DESCRIPTION OF PRESENTLY PREFERRED EXAMPLE
EMBODIMENTS [0024] Figures 1 and 2 show an example dual-technology vehicle warning system 100 provided by a preferred embodiment of this invention. System 100 is mounted within the rear bumper 102 of a vehicle 104 and automatically activates when reverse gear is selected. System 100 includes two visible ultrasonic sensors 106 and one "bidden" next-generation Doppler Microwave sensor 108 mounted behind the vehicle's rear bumper 102 face.
[0025] The ultrasonic sensors 106 are preferably mounted in the bumper 102 corners 110 and provide detection in the example embodiment to 2 meters (near field) of any object, including stationary and moving. The radar sensor 108 in the illustrative embodiment is configured to detect any object the distance to which is diminishing, because either the vehicle 104 is moving toward the object, the object is moving toward the vehicle or both. Sensor 108 in the illustrative embodiment detects out to 6 meters (far field) depending on vehicle speed for safety and collision avoidance.
[0026] The example embodiment integrated system 100 reports any object, stationary or moving, microwave and/or ultrasonic, within 2 meters, so the driver does not begin to reverse the vehicle 104 without the system's notification. This detection range will minimize the false detection algorithms in order to improve response time for driver reaction due to the shorter stopping distance. For stationary targets detected here while the vehicle is in reverse with the brake applied may/may not squelch the audible alarm but keep the visual alarm in order to minimize annoyance to the driver.
[0027] In the particular illustrat γe examples shown, Figure IB shows both micrpw„aye a^μltr sqpic Retϊ,' and Figure 1C shows both microwave and ultrasonic
* <S* &*,!S . ϊ&Ssf ^/fΔZ-y^ y^
-ANBed. Additionally, the illustrated distances that are shown may apply better toward commercial vehicles than passenger vehicles in addition to also being perhaps more subjective to speed and time-to-collision.
[0028] Any object outside of the near field of 2 meters, but within the far field of 4 meters (low speed) to 6 meters (high speed), will be reported by illustrative and preferred system 100 to the driver only if the distance to the object is diminishing and is detected by both the ultrasonic and microwave sensor technologies. This results in fewer notifications to the driver, fewer "false alarms" or "unnecessary" notifications, and greater system reliability on behalf of the driver.
[0029] The system 100 also changes its notification to the driver as the object comes nearer the vehicle, giving the driver additional alerting assistance. The exemplary system 100 is designed to supplement other safety practices by enhancing the driver's awareness of difficult-to-see areas. The illustrative system 100 does not take any automatic action to prevent accidents. Responsibility for the safe operation of the vehicle remains with the driver. To this extent it is ideally important for the display to mount so that the driver does not alter their reversing habits in order to view its output. Thus natural mounting opportunities present themselves in or about the rear view and side view mirrors, and also in various positions around the rear window of the vehicle if that is where the driver looks while reversing.
[0030] The three examples shown in Figures 1 A-1C demonstrate that, for one illustrative and exemplary implementation of preferred embodiment dual-sensing system 100:
1. when reversing above 4 mph the time to collision is short enough that the Doppler radar extends its coverage (width and length) to give adequate warning time 2. when reversing at normal speed the Doppler radar gives warnings to 4-meters
3. when parking at "crawling" speed or, parked at zero speed the ultrasonic presence sensor provides coverage out to 2-meters
[0031] The illustrative system 100 offers several benefits including for example:
• audible and visual warnings of unseen obstacles and parking range;
• all weather protection down to ground level;
• down to zero speed parking assistance and high-speed obstacle warning alerts; and
• self-test, auto calibration and future integration capability.
[0032] System 100 preferably uses at least two sensor technologies and at least one sensor transducer for each technology to cost effectively and robustly detect objects within a predefined detection pattern. The sensor technologies are complementary in their detection capabilities so that each has a unique capability that allows it to detect characteristics about the intended objects that the other technology cannot. However, the sensor technologies or a portion of the technologies also preferably have some cross-over in their capabilities over some aspects of the detection pattern so that the detection of objects by one technology can be verified by the other technology in order to minimize falsely identifying objects within the detection pattern. In order to determine how the signals from the different technologies are considered, illustrative system 100 relies on at least one of the sensor technologies to have ranging information on objects that are detected. The sensor system 100 utilizes the ranging information to help determine based on the distance between the object and the sensor how the various sensor signals will be applied to determine the likelihood that an object has been detected, the urgency of the situation, whether or not to activate an alert based on the situation and the characteristics of that alert.
[0033] The number of sensor elements for each sensing technology is not explicitly limited and generally will be governed more by application detection pattern and cost. Additionally, the distances provided in the detection pattern are merely given distances for illustration. In preferred implementations, the distances will be tailored to the application. In addition, the detection range distances are not necessarily static and may correlate more to an urgency to detect that takes into account more than just the distance to the object.
[0034] Illustrative system 100 thus has at least two sensor technologies with their included transducers and antennas for providing an electrical signal that describes each technology's view of the environment and for focusing each technology's view of the environment into a specific area, respectively. The actual number of sensor elements can vary. The primary intention of the diagram is to illustrate that each sensor technology can have more than one sensor element and to show one example illustrative configuration.
[0035] As shown in the Figure 3 block diagram, illustrative system 100 further includes signal conditioning circuitry 120, 122 for each transducer that conditions the signals coming from the receivers within the transducers for analysis by a controller 124. A controller 124 accepts the conditioned signals from each sensor technology, applies predefined logic to determine how the information is used to not only detect objects within the detection area but also assess whether the objects should be identified, the nature of any identification via a display or alarm, and outputs the signal required to drive the display or alarm appropriately. A display or alarm 126 accepts the signal from the controller 124 and activates the appropriate indication. The indication can be one or a combination of several methods typical to the industry including audible indication (i.e. buzzers, beepers and speakers), visual indication (i.e., LEDs, LCDs and lamps) and vibrational methods. A power supply and conditioning circuit (not shown) transforms and conditions the power for activating the circuitry identified for each of the functions above.
[0036] In the example and illustrative embodiment, connection and conditioning circuitry (not shown) is preferably used to help determine the status of the environment to which the sensing system 100 is attached. For example, if the sensing system 100 is attached to a vehicle 102, status information based on brake activation, transmission status (Park, Reverse, Neutral, Forward - 1/2/3/OD), vehicle speed and/or steering orientation can be used to alter operational modes of the system in order to power or enable the output on the system, alter detection pattern width and range, activate and even to help discriminate object characteristics. Under another application example, a security sensing device mounted within a building where the status of the building is brought to the sensor such as occupied, day vs. night, time of day, HVAC status, security panel status (alarm/standby).
[0037] It is anticipated that the sensor technologies 106, 108 will mount on or about the rear of the vehicle 102 so that the detection patterns cover the area that the vehicle would typically reverse over. The display/alarm 126 will be expected to mount so that the operator of the vehicle 102 can monitor its alert activity without changing his or her driving habits. In this fashion, this sensing system 100 will act as a driver's aid. The display 126 may or may not be placed within the passenger area of the vehicle. The rest of the electronics can be configured in any number of topologies ranging from centralized (everything in one place) to distributed (each sensing technology operates as an independent sub-system whereby its electronics and output status is fed to a central controller) to a hybrid approach depending on the application.
[0038] Use of two different sensor technologies in the same illustrative system
100 provides certain advantages — especially when there are multiple ranges of detection created that can be broken down into primary technology responsibilities and where/how the technologies are combined. For example:
MICROWAVE RADAR:
• Microwave technology is an electromagnetic media which is fairly impervious to the environmental conditions and is readily available in fairly low cost forms such as Doppler motion sensors used for security intrusion alarms and other commercial applications.
• Fundamentally, Doppler is best at identifying relative object movement towards or away from the sensor's radiating orientation. It is capable of distinguishing direction of movement in addition to relative speed. • Microwave technology can have difficulty detecting objects with low dielectric constant and radar signature like plastic pipes, chain link fences with link spacings larger than the transmitted wavelength and shrubs. It has difficulty detecting smaller or lower dielectric material objects in front of large metallic objects like warehouse walls.
• An alternative technology should be chosen that helps detect these types of objects especially when the position of these objects is critical.
• Due to various processing opportunities including direction of motion detection and quahfying the amount of movement in wavelengths in a given direction the ground can be within the detection pattern and not be detected as an obstacle.
• A microwave Doppler FSK Ranging transducer will provide cost effective discrimination of target range rather than relying on signal level which changes with the radar signature of targets (size, shape and composition). This ranging information will be complemented with the ultrasonic system and its range measurement capabilities.
ULTRASONIC SENSOR:
• Ultrasonic sensors are extremely cost-effective and offer Presence Sensing capabilities with range information.
• Ultrasonic sensors employ a round trip time of flight detection system in order to detect obstacles and the range to the obstacles. The detection range is more limited than Microwave Radar and will therefore be used for closer ranges. In addition, the accuracy of the ultrasonic system can be more easily compromised by the humidity in the environment and thus by limiting the application range the sensor system hmits the tolerance on the measurement information.
• Ultrasonic noise such as tire rubbing can false trip these sensors as well. Other areas where ultrasonic technology has some problems includes buildup of dust, dirt, leaves, and various environmental phenomena such as snow over the transducers which prevents the system from receiving reflected ultrasonic energy.
• Because any reflection of ultrasonic energy constitutes an object, ultrasonic sensors typically must be aligned to not see the ground in order to work effectively. However, this leaves them vulnerable to missing objects such as bricks and cement parking blocks along with children that have fallen or are hiding. Microwave Doppler will be applied to help minimize this vulnerability.
• Ultrasonic sensing does not inherently qualify moving vs. stationary targets. It is a Presence Sensing Technology. Although processing routines could potentially be used to detect changes in distance in order to qualify movement on targets, a motion sensing system should be employed that can see to the ground and therefore a separate motion detection sub-system is required for alignment or detection pattern reasons. Therefore, the opportunity is there to utilize a second technology (microwave) for motion sensing to complement the technological strengths of the ultrasonic technology.
• Additionally, the round trip time for ultrasonic target information tends to limit processing time in low cost processors and thereby the maximum vehicle/object speed at which the solution can work. Here again a second technology such as microwave Doppler can bring about a complementary capability.
[0039] Thus, the different sensor technologies 106, 108 used by the illustrative embodiment described herein have niches of capabilities and yet also have overlapping capabiUties as well. Illustrative sensing system 100 employs the two sensor technologies 106, 108 as foUows.
[0040] Sensing system 100 in the example segments the detection zone into at least two priorities based on range from the vehicle while providing the opportunity for having many more levels of differentiation at farther ranges as appUcations require. The two closest detection zones in order of increasing distance from the vehicle wiU be referred to as the Stop Zone and the Alert Zone. The size of these detection zones will vary by appUcation to various types of vehicles. For instance, the Stop Zone for an economy passenger vehicle will most likely be considerably shorter than that for large earth moving dump trucks and may typicaUy faU in the 3 foot range.
[0041] The exemplary sensor sub-systems are configured to completely cover the width of the vehicle independently so that their detection zones roughly overlap close to the vehicle 102. It is expected that the range for the ultrasonic sensing system 106 wiU be shorter than that of the microwave sensing system 108. However, it is desired that the ultrasonic sensing system 106 at least cover through the Alert Zone. Meanwhile, the microwave sensing system 108 can reasonably extend farther in range beyond the Alert Zone to accommodate more detection ranges that might be required for larger commercial trucking or earth moving equipment appUcations, but these ranges are considered to be less critical in the need for the driver's attention due to the availability of stopping distance.
[0042] The sensing system may or may not be activated at all times on the vehicle 102. However, the display/alarm 126 will not be enabled until the vehicle transmission is placed into reverse in one illustrative embodiment.
[0043] The system 100 is preferably configured so that the display/alarm 126 always indicates the most pressing or highest priority alarm if more than one target is detected. Priority can be estabUshed in many ways by the controUer 124 but should relate to the need for driver attention: e.g. object range, object range + closing rate and object type (i.e., human/non-human). The priority actually changes based on the range to the object or zone that the object faUs within. Objects falUng within the Stop Zone always represent the highest priority which is a range priority. Objects detected in the Alert Zone but having extreme relative closing rates or having been identified as a human target can become associated with the Stop Zone and thereby become the highest priority.
[0044] In the example embodiment, the range that is closest to the vehicle (the stop zone) will apply both microwave Doppler and ultrasonic presence detection algorithms in an OR'd fashion so that an alert is activated by the controller 124 if either the ultrasonic or microwave system detects objects within this range. Therefore, combining microwave and ultrasonic detection provides the vehicle 102 with a warning on stationary targets before the vehicle starts to move using primarily the ultrasonic technology. As the vehicle 102 begins to reverse or if an object moves within this range before the vehicle 102 reverses, the microwave sensor sub-system wiU detect objects that appear within range even if they are close to the ground while the ultrasonic sensor sub-system wiU complement the microwave signals on objects within this range that are considerably higher than the ground. A detection made by the ultrasonic sub-system while the vehicle is in reverse and stationary wiU minimize the averaging routines appUed to the microwave system for objects detected within the same vicinity as the vehicle or object starts to move without fear of false detecting. This will aUow the ultrasonic alert to further quicken the response of the microwave system.
[0045] Additionally, if the system 100 is activated even when the vehicle is not in reverse AND if a vehicle has the transmission go from a forward gear to a park or neutral gear, then the sensor system 100 will automatically save into memory the range information found on objects detected by both the microwave and ultrasonic system regardless of direction of movement even though the alarm is not enabled as long as the sensor is powered/activated during these times. If the vehicle 102 is subsequently placed into reverse, the sensing system wiU reload the information on these targets and for objects detected within these ranges the averaging counters will be reduced in order to minimize the response time to alert.
[0046] The sensing system 100 may or may not be configured to quatify objects by their direction of motion with respect to the sensor. Typically, it would be safer within the Stop Zone if there were no direction of motion (DOM) quaUfication required as opposed to only signaUng an alarm if there were a closing distance between the object and sensor.
[0047] The detection range that is second closest to the vehicle 102 (the Alert
Zone) will also employ both microwave and ultrasonic sensing technologies. However, within this zone the detection made by both technologies 106, 108 is AND'ed together in the example embodiment in order to minimize false alarms. For example, before the system 100 enables an alarm for this range, the controller 124 must have confirmation of the target by both the microwave and ultrasonic sensor subsystems 106, 108. Thus, the system 100 does not alarm if the vehicle is in Reverse but stationary while a stationary object presents itself within this zone.
[0048] Although the illustrative sensor system 100 wiU not alert on stationary targets within the Alert Zone while the vehicle is in Reverse and stationary, the controller 124 will apply this information to the microwave detection system 108 and once again minimize the averaging counter required to acknowledge a target because the fear of false detection has been reduced by the redundancy of the ultrasonic sensor sub-system verifying the object.
[0049] Additionally, the microwave Doppler signals will not trigger an alarm by themselves in the exemplary system within this zone which will enable the system 100 to minimize the number of false detects on items buried within the ground structure such as sewer mains, pipes and steel support rods. Environmental elements wiU also be further discriminated against by both systems as the lack of complementary microwave detections will eUminate falsing in high wind conditions and on other sources of ultrasonic noise typically "seen" by ultrasonic sensors, while rain and other hydrometeors can be eUminated from the higher frequency microwave technology signals as the ultrasonic system 106does not duplicate detection of these items.
[0050] The sensing system 100 may or may not be configured within this zone to alert on moving objects only if there is a closing distance between the vehicle and object. Typically, since the Alert Zone should be trying to minimize falsing it would make sense to apply DOM quaUfication that requires a closing distance.
[0051] Within the Alert Zone, a closing rate assessment is also made on detected objects. If this is greater than a certain speed, then the object is considered to be in the Stop Zone because the time to collision is expected to be approaching the expected stopping time for the vehicle and/or the object. [0052] In detection ranges that are estabUshed beyond the Alert Zone, typically only microwave Doppler would be employed due to Umitations on the ultrasonic subsystem in range. These ranges would only be considered to be warnings to the vehicle operator. Most likely direction of motion (DOM) quaUfication would be employed here once again in order to πiinimize nuisance alarms. In addition a closing rate assessment may also be activated in order to determine if the object should be considered within the Alert or even Stop Zones.
EXAMPLE DETAILED SCHEMATIC DIAGRAM
[0053] Figure 3 A is a circuit diagram of an example illustrative circuit diagram.
The Figure 3A circuit diagram provides an additional input to accommodate the signal output from an ultrasonic sensing system 106 as well as an input from a motion detector 108 such as for example a microwave sensor using the Doppler shift principle. In the example embodiment, software is used to combine these two signals together to provide a correct response to the display 126. While it may be possible to integrate the individual sensing systems further, this illustrative and exemplary embodiment lets each sensing system 106, 108 operate independently and then combines the two outputs together using an algorithm to be described below.
[0054] Thus, the exemplary circuit diagram shown in Figure 3A is similar to that disclosed in U.S. Patent appUcation serial number 09/906,790 entitled "Programmable Microwave Backup Warning System and Method" filed July 18, 2001 (incorporated herein by reference) but has an additional input 500 for receiving the output of a ultrasonic presence detector 106. This input 500 is processed using a schottky inverter 502 in order to provide a degree of hysteresis in order to reduce system response to transient inputs. A microprocessor 124 is also capable of driving the input/output connector 500 as necessary.
EXAMPLE SOFTWARE IMPLEMENTATION
[0055] Figures 4 and 5A-5C show illustrative routines which identify whether an object is stationary or moving and operate by considering the vehicles transmission mode, the braking status, vehicle velocity information (if available), presence detection versus motion detection signals and the nature of the Doppler information from the microwave sensor sub-system. This information will help to identify special driving patterns as detailed further below, and also human recognition routines also further detailed below.
[0056] Figure 4 shows a flowchart of an iUustrative "main" software routine performed by controller 124 to sense the presence of objects such as obstacles. In this particular example, system 100 is configured to be powered on whenever power is appUed to a vehicle's reverse Ughts. A self-test is performed at power-up in the example embodiment but could also be considered to be employed periodically while the sensor is operating. System 100 in this particular example is configured to be direction-of-motion conscious, with presence detection being performed by the ultrasonic sensor 106 and motion detection being performed by the microwave sensor 108 in this example. In this example, powering by the reverse Ughts loses the opportunity to save the last target ranges in forward, neutral or park transmission settings so that detection counters cannot be minimized for ranges of the previous targets. However, other embodiments could perform such information saving techniques if desired.
[0057] The exemplary and iUustrative "main" routine shown in Figure 4 begins by testing whether a presence detect sensor 106 (e.g., an ultrasonic type sensor in the preferred exemplary embodiment) has detected the existence of an object (decision block 402). If the presence of an object has been detected by the presence detector 106 ("yes" exit to decision block 402), the exemplary and illustrative routine 400 uses additional information from the presence detector 106 (and possibly also the motion detector 108, such as, for example, a microwave sensor using the Doppler shift principle), to determine whether the obstacle that has been detected is at a short range (decision block 404), a mid range (decision block 406), or a far range (decision block 408). If a short range is detected ("yes" exit to decision block 404), this indicates that the vehicle is very close to an obstacle and a "stop" alarm event is generated (block 410). [0058] The "stop" alarm generation routine (the highest priority of three different priority alarms in the exemplary iUustrative embodiment) is shown in more detail in Figure 5C. In this particular example, generation of the "stop" alarm (block 410) involves enabling the stop alarm, resetting the stop output counter, and resetting the driver mode counter (block 412), and then powering the alarm (block 414) to actuaUy generate an audible and/or visual alarm. Control then returns to the main loop 416 of main routine 400 to continue the sensing and monitoring process.
[0059] Referring again to Figure 4, if the presence of a mid range object is detected ("yes" exit to decision block 406), then in the exemplary embodiment, routine 400 determines whether motion has also been detected and logged (decision block 418; see also decision block 450). If both motion and presence have been detected ("yes" exit to decision block 418), the exemplary routine 400 determines whether the motion detector 108 indicates that the object is closing with the vehicle (i.e., is moving closer relative to the vehicle) (decision block 420). If decision block 420 reveals that the object is closing, then an additional test performed by the exemplary and illustrative routine 400 determines whether the closing velocity is greater than a certain predetermined velocity threshold (decision block 422).
[0060] If the closing velocity is greater than a certain predetermined threshold
("yes" exit to decision block 422), then the "stop" alarm event is generated (block 410) to inform the vehicle operator that a colUsion is imminent. Otherwise, if the closing velocity is less than a predetermined threshold velocity ("no" exit to decision block 422), then a lower priority "alert" alarm event is generated instead (block 424). An illustrative "alert" alarm routine in the exemplary embodiment is shown in Figure 5B. In this example, the system 100 determines whether an alarm is already enabled (decision block 426). If the alarm is already enabled ("yes" exit to decision block 426), then the exemplary system 100 determines whether the alarm event that is being generated corresponds to "stop" (decision block 428). If a "stop" alarm event is already being generated ("yes" exit to decision block 428), then the exemplary embodiment resets the alert output counter (block 430) and returns (block 432). Otherwise, if the alarm is not already enabled or is not already in a "stop" type alarm mode ("no" exits to decision blocks 426, 428, respectively), the exemplary system enables the "alert" alarm, resets the alert output counter, and resets the driver mode counter (block 434) and then powers the alarm to provide a corresponding alert indication (block 436).
[0061] Referring again to Figure 4, if decision block 420 detects no closing motion (i.e., the object is in a mid range area relative to the vehicle and, while moving, is moving away relative to the vehicle ("no" exit to decision block 420), then in the exemplary and illustrative embodiment, routine 400 performs certain checks to determine whether the obstacle may be a human being (block 438). By taking into account whether the vehicle is moving, and by looking at the variabiUty in DOM information and consistency of Doppler velocity, an estimate can be made as to whether a potential object is human or not. The reason for doing so is quite simply due to the fact that in most cases the microwave sensor 108 may be mounted so that it is looking about knee height on a human. As such, the gate of the typical human stride does not always provide a consistent enough DOM indication to estabUsh detection on an object. Therefore, if a microwave Doppler signal has been picked up within the alert zone but is avoiding detection the controller 124 will apply an analysis routine to see if the sensor is picking up a human that is crossing the detection pattern or a human that has a particularly labored gate. If the object is determined to be a human within the alert zone, then DOM quaUfication wiU be removed from consideration and the controUer will activate the Display/Alarm. (A PIR sensor could be combined as weU, if desired.) If the object is a person ("yes" exit to decision block 438), then in the exemplary embodiment, the routine 400 essentially ignores or discounts the fact that the object appears to be moving away relative to the vehicle and nevertheless performs the velocity threshold check of decision block 422 anyway in order to generate a "stop" or an "alert" alarm. Thus, in the exemplary embodiment, routine 400 will generate some sort of an alarm whenever a person is detected within the mid range irrespective of the relative direction and velocity of motion. This is appropriate to minimize injury and fataUties to people, and also reflects the fact that human beings (unlike inanimate objects) will generally set off a motion detector even if they are standing very still because there is nearly always some movement of body position.
[0062] In the example embodiment, if decision block 418 determines that there is no motion ("no" exit to decision block 418), then the preferred exemplary and iUustrative routine 400 determines whether there was prior far motion detected (decision block 440). If there was no prior far motion detected ("no" exit to decision block 440), then control preferably returns to main loop 416. On the other hand, if prior far motion was detected ("yes" exit to decision block 440), then in the exemplary embodiment, routine 400 determines whether the vehicle is stationary ~ meaning that the object is moving as opposed to the vehicle (decision block 442). In the exemplary embodiment, determining whether the prior far motion was based on vehicle movement or obstacle movement can be determined by sensing, for example, whether the vehicle brake is being appUed. If the vehicle is not moving and there was recent far motion but no current far motion and yet an object is present within the mid range sensing area ("yes" exit to decision block 442), this indicates that an animate or otherwise recently moved object (e.g., possibly a person who missed being detected by block 438, or perhaps an animal such as a pet, or another vehicle such as a car or bicycle) has recently been moved into the danger area and is now standing stiU. In this case, a "stop" or "alert" alarm is generated since this is an event that the vehicle operator ought to know about. The exemplary embodiment appUes a velocity threshold test (block 422) to the prior far motion to determine whether a stop or alert alarm should be generated (block 422).
[0063] In the example embodiment, if an object has been detected within a
"far" range (decision block 408), then a "detect" or "alert" alarm may be generated depending upon whether there is closing motion of greater than a predetermined velocity. Generally, it is undesirable to disturb the vehicle operator by generating alarms for potential obstacles that are a long way off that the vehicle will have tittle risk of striking. On the other hand, in the exemplary embodiment, we do generate a lower priority "detect" alarm under certain circumstances if a far-away detected object is moving (block 444) closer relative to the vehicle (block 446). The exemplary routine 400 wiU generate an "alert" alarm event (block 424) if the closing motion is greater than a predetermined velocity ("yes" exit to decision block 448), and will - generate a "detect" (i.e., lower priority) alarm event (block 452) if the closing motion has a velocity less than the predetermined threshold ("no" exit to decision block 448).
[0064] Figure 5 A shows an exemplary routine performed in response to a
"detect" alarm event. The Figure 5 A routine is similar Figure 5B routine in the exemplary embodiment except that decision block 456 in this case determines whether the alarm is already in the two exemplary higher priority alarm modes ("stop" or "alert").
[0065] Referring again to Figure 4, main loop 416 has some additional tests it continuaUy performs when obstacle presence is not detected ("no" exit to decision block 402). For example, if no obstacle presence has been detected ("no" exit to decision block 402), the exemplary main routine 400 determines whether motion has been detected (decision block 450). If motion has been detected ("yes" exit to decision block 450) but no presence has been detected ("no" exit to decision block 402), this may be an indication that presence detection sensor 106 has failed. For example, if the presence detection sensor 106 has failed, it will no longer detect the presence of an obstacle but the motion detector 108 will still produce a detection result. The exemplary embodiment takes advantage of redundant and overlapping aspects of the sensitivity of dual detectors 106, 108 to provide a "fail safe" mechanism that will generate an alarm even if one of the detectors has failed. Self test and component failure adjustments can be used in the event that one or both sensor subsystems are deemed to not be working. This is a factor within the first two detection ranges especially since within the STOP zone each technology has detection capabiUties that are unique which the system is relying on. In the alert zone this capabiUty is also used since it takes an identification by both technologies in order to process an alarm condition.
[0066] Before generating an alarm under such a condition, however, the exemplary embodiment first performs a test to determine whether environment conditions may be responsible for setting off one or the other of the sensors (block 464). There are various ways that exemplary system 100 can determine whether environmental conditions may be responsible — one way being for example to use a moisture or humidity detector, another being to analyze the return wave signature, and stiU another way being to use an additional transducer of some other type. If there is no environmental condition that might be responsible ("no" exit to decision block 464), then exemplary routine 400 assumes that the presence detector 106 has failed and wiU perform the testing algorithm described above based on motion detection in order to generate an alarm ("no" exit to decision block 464). Weather detection/obstacle detection routines may also possibly alter the logic appUed within each zone in order to increase reUabiUty.
[0067] If there is neither presence detection nor motion detection ("no" exit to decision block 450) (or if motion detection can be ascribed to environmental conditions, "yes" exit to decision block 464), then main routine main loop 416 determines whether output counters have expired indicating that an alarm has been generated for a sufficient amount of time after an event was triggered (and no longer exists) and can now be silenced or turned off (decision block 466, block 468).
[0068] In addition, the exemplary routine 400 has the capabiUty to place system
100 into a "driver" mode indicative of stop and go driving to avoid annoying alarms from being generated in the bumper-to-bumper traffic type situations that we aU occasionaUy find ourselves in (block 470, block 474). Such "driver" mode is detected based upon a past history of recent events indicative of stop and" go traffic.
[0069] As discussed above, the controUer 124 can be enabled to process tracked targets starting from the outer detection zones through the "alert" and finally to the "stop" zones somewhat differently than instantaneous targets. The exemplary sensor system 100 will preferably be configured to identify firstly that the vehicle is reversing towards a stationary object and then to identify various patterns of reversing that are typically employed on the vehicle. For instance, if the operator of the vehicle 102 tends to reverse in a nervous manner where the vehicle is continually stopped and started before reaching its resting place, then the sensor will start to adjust its counters and employ a TIMEOUT counter so that as the operator brakes and comes to a stop, the sensor does not loose lock on the object that it is tracking. The algoriuWis then"' able to reinitiate detection of the object quickly if the reversing resumes from a stopped condition before the ΗMEOUT counter expires. This is accompUshed by not resetting the averaging counters that are used to help prevent falsing if movement ceases until the ΗMEOUT counter expires.
[0070] If steering information is available to the sensor system 100 then while a vehicle 102 turns into parking spaces or turns while reversing the sensor system 100 can apply compensation routines to the zones so that the zones reduce the maximum range that they will detect objects within. This will prevent detection of adjacent vehicles and wall surfaces that do not actuaUy present any possibiUty for colUsion. This wiU also be enhanced by looking at the ranging information from sensors on either half of the rear of the vehicle and altering the detection ranges for each half.
[0071] While the invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not to be Umited to the disclosed embodiment. While the invention has been described as it pertains to specific appUcations (i.e., primarily backup alarm/reverse aid sensing systems), the invention can be appUed to most any object detection application such as, but not Umited to: security sensors, occupancy sensors, door openers and toys. In addition, for the purposes of this discussion we have described two sensor technologies, i.e., ultrasonic and Doppler ranging microwave. However, more than two technologies could be appUed and/or other technological choices could also be made as long as at least one of the technologies possesses the capabiUty to determine the range to detected objects. SpecificaUy, this disclosure addresses employing these capabiUties with a backup warning/parking aid sensor for vehicles. However, the technology could be employed with many other appUcations where object detection requires a hierarchy of characteristics and priorities based on the range to the detected object. Thus, the invention is intended to cover various modifications and equivalent arrangements included within the scope of the appended claims.

Claims

1. A vehicular-based warning system for detecting an obstacle, comprising: an arrangement for mounting first and second sensors onto said vehicle, said first and second sensors using different sensing technologies; a processor that is operatively coupled to said first and second sensors, said processor combining said first sensor output with said second sensor output to produce a composite result, wherein said processor applies range gating to only one of said first sensor output and said second sensor output.
2. A vehicular-based warning system for detecting an obstacle, comprising: an arrangement for mounting first and second sensors onto said vehicle, said first and second sensors using different sensing technologies; a processor that is operatively coupled to said first and second sensors, said processor combining said first sensor output with said second sensor output to produce a composite result, wherein said processor ANDs said first sensor output with said second sensor output under predetermined conditions.
3. A vehicular-based warning system for detecting an obstacle, comprising: an arrangement for mounting first and second sensors onto said vehicle, said first and second sensors using different sensing technologies; a processor that is operatively coupled to said first and second sensors, said processor combining said first sensor output with said second sensor output to produce a composite result, wherein said processor detects closing motion based on at least one of said first sensor output and said second sensor output.
4. A vehicular-based warning system for detecting an obstacle, comprising: an arrangement for mounting first and second sensors onto said vehicle, said first and second sensors using different sensing technologies; a processor that is operatively coupled to said first and second sensors, said processor combining said first sensor output with said second sensor output to produce a composite result, wherein said processor ORs said first sensor output with said second sensor output under certain predeteπnined conditions.
5. A vehicular-based warning system for detecting an obstacle, comprising: an arrangement for mounting first and second sensors onto said vehicle, said first and second sensors using different sensing technologies; a processor that is operatively coupled to said first and second sensors, said processor combining said first sensor output with said second sensor output to produce a composite result, wherein said processor conditions the generation of a warning on determination of prior far motion and absence of vehicular motion.
6. A vehicular-based warning system for detecting an obstacle, comprising: an arrangement for mounting first and second sensors onto said vehicle, said first and second sensors using different sensing technologies; a processor that is operatively coupled to said first and second sensors, said processor combining said first sensor output with said second sensor output to produce a composite result, wherein said processor conditions generation of a warning on motion velocity threshold detection.
7. A vehicular-based warning system for detecting an obstacle, comprising: an arrangement for mounting first and second sensors onto said vehicle, said first and second sensors using different sensing technologies; a processor that is operatively coupled to said first and second sensors, said processor combining said first sensor output with said second sensor output to produce a composite result, wherein said processor conditions generation of a warning on detection of a human presence.
PCT/US2002/031213 2001-10-03 2002-10-02 Multi-technology object detection system and method WO2003029045A2 (en)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US32642801P 2001-10-03 2001-10-03
US60/326,428 2001-10-03
US13473702A 2002-04-30 2002-04-30
US10/134,737 2002-04-30

Publications (2)

Publication Number Publication Date
WO2003029045A2 true WO2003029045A2 (en) 2003-04-10
WO2003029045A3 WO2003029045A3 (en) 2003-07-10

Family

ID=26832629

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2002/031213 WO2003029045A2 (en) 2001-10-03 2002-10-02 Multi-technology object detection system and method

Country Status (1)

Country Link
WO (1) WO2003029045A2 (en)

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1510402A2 (en) * 2003-08-26 2005-03-02 Rf Monolithics, Inc. Vehicle back-up alarm system
DE10336681A1 (en) * 2003-08-09 2005-03-03 Audi Ag motor vehicle
GB2431306A (en) * 2005-10-11 2007-04-18 Zhitao Li Parking aid for a reversing vehicle providing a collision warning system when the vehicle is parked
EP2328780A1 (en) * 2008-09-25 2011-06-08 Binar Aktiebolag Warning system
WO2012177846A1 (en) * 2011-06-22 2012-12-27 Robert Bosch Gmbh Improved driver assistance systems using radar and video
GB2513929A (en) * 2013-05-09 2014-11-12 Silware Corp Distance Measuring Alert Sensor for Vehicle
EP2818889A3 (en) * 2013-06-29 2015-04-29 MAN Truck & Bus AG Motor vehicle with at least one driver assistance system using the data of a radar sensor
CN108603763A (en) * 2016-03-30 2018-09-28 三菱电机株式会社 Traveling plan generating means, traveling scheduling method and traveling plan generate program
CN111812650A (en) * 2020-06-30 2020-10-23 中国第一汽车股份有限公司 Parking radar system, control method, electronic device and storage medium
CN113022759A (en) * 2019-12-06 2021-06-25 格科特有限公司 Collision warning system and method for micro-mobile vehicles
US11173899B2 (en) * 2018-07-03 2021-11-16 Hyundai Mobis Co., Ltd. Control device and method for forward collision avoidance in vehicle

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4549181A (en) * 1981-10-31 1985-10-22 Nissan Motor Company, Limited Moving obstacle detection system for a vehicle
US5598164A (en) * 1992-08-10 1997-01-28 Reppas; George S. Vehicle obstacle avoidance system
US5754123A (en) * 1996-05-06 1998-05-19 Ford Motor Company Hybrid ultrasonic and radar based backup aid
US5977884A (en) * 1998-07-01 1999-11-02 Ultradata Systems, Inc. Radar detector responsive to vehicle speed
US6151539A (en) * 1997-11-03 2000-11-21 Volkswagen Ag Autonomous vehicle arrangement and method for controlling an autonomous vehicle
US6191686B1 (en) * 1999-09-27 2001-02-20 Edwin Z. Gabriel Collision avoidance system for cars
US6281786B1 (en) * 1998-04-20 2001-08-28 Matsushita Electric Works, Ltd. Obstacle detection system for a vehicle
US6433679B1 (en) * 1998-09-23 2002-08-13 Robert Bosch Gmbh Warning device for an automobile

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4549181A (en) * 1981-10-31 1985-10-22 Nissan Motor Company, Limited Moving obstacle detection system for a vehicle
US5598164A (en) * 1992-08-10 1997-01-28 Reppas; George S. Vehicle obstacle avoidance system
US5754123A (en) * 1996-05-06 1998-05-19 Ford Motor Company Hybrid ultrasonic and radar based backup aid
US6151539A (en) * 1997-11-03 2000-11-21 Volkswagen Ag Autonomous vehicle arrangement and method for controlling an autonomous vehicle
US6281786B1 (en) * 1998-04-20 2001-08-28 Matsushita Electric Works, Ltd. Obstacle detection system for a vehicle
US5977884A (en) * 1998-07-01 1999-11-02 Ultradata Systems, Inc. Radar detector responsive to vehicle speed
US6433679B1 (en) * 1998-09-23 2002-08-13 Robert Bosch Gmbh Warning device for an automobile
US6191686B1 (en) * 1999-09-27 2001-02-20 Edwin Z. Gabriel Collision avoidance system for cars

Cited By (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE10336681A1 (en) * 2003-08-09 2005-03-03 Audi Ag motor vehicle
DE10336681B4 (en) * 2003-08-09 2005-07-07 Audi Ag motor vehicle
EP1510402A2 (en) * 2003-08-26 2005-03-02 Rf Monolithics, Inc. Vehicle back-up alarm system
EP1510402A3 (en) * 2003-08-26 2006-04-19 Rf Monolithics, Inc. Vehicle back-up alarm system
US7148794B2 (en) 2003-08-26 2006-12-12 Rf Monolithics, Inc. Vehicle back-up alarm system, vehicle, transmitter module, and method
GB2431306A (en) * 2005-10-11 2007-04-18 Zhitao Li Parking aid for a reversing vehicle providing a collision warning system when the vehicle is parked
EP2328780A1 (en) * 2008-09-25 2011-06-08 Binar Aktiebolag Warning system
EP2328780A4 (en) * 2008-09-25 2014-11-19 Binar Aktiebolag Warning system
WO2012177846A1 (en) * 2011-06-22 2012-12-27 Robert Bosch Gmbh Improved driver assistance systems using radar and video
US8718899B2 (en) 2011-06-22 2014-05-06 Robert Bosch Gmbh Driver assistance systems using radar and video
CN104139730A (en) * 2013-05-09 2014-11-12 银剑无线电话股份有限公司 Vehicle ranging warning device
GB2513929A (en) * 2013-05-09 2014-11-12 Silware Corp Distance Measuring Alert Sensor for Vehicle
GB2513929B (en) * 2013-05-09 2015-04-29 Silware Corp Distance Measuring Alert Sensor for Vehicle
EP2818889A3 (en) * 2013-06-29 2015-04-29 MAN Truck & Bus AG Motor vehicle with at least one driver assistance system using the data of a radar sensor
CN108603763A (en) * 2016-03-30 2018-09-28 三菱电机株式会社 Traveling plan generating means, traveling scheduling method and traveling plan generate program
CN108603763B (en) * 2016-03-30 2022-06-24 三菱电机株式会社 Travel plan generating device, travel plan generating method, and computer-readable recording medium
US11173899B2 (en) * 2018-07-03 2021-11-16 Hyundai Mobis Co., Ltd. Control device and method for forward collision avoidance in vehicle
US11661054B2 (en) 2018-07-03 2023-05-30 Hyundai Mobis Co., Ltd. Control device and method for forward collision avoidance in vehicle
CN113022759A (en) * 2019-12-06 2021-06-25 格科特有限公司 Collision warning system and method for micro-mobile vehicles
US11623707B2 (en) 2019-12-06 2023-04-11 GEKOT Inc. Collision alert systems and methods for micromobility vehicles
US11878761B2 (en) 2019-12-06 2024-01-23 Gekot, Inc. Collision alert systems and methods for micromobility vehicles
CN111812650A (en) * 2020-06-30 2020-10-23 中国第一汽车股份有限公司 Parking radar system, control method, electronic device and storage medium

Also Published As

Publication number Publication date
WO2003029045A3 (en) 2003-07-10

Similar Documents

Publication Publication Date Title
US8552848B2 (en) System and method for combined blind spot detection and rear crossing path collision warning
US6744353B2 (en) Blind spot detector
US6268803B1 (en) System and method of avoiding collisions
US6680689B1 (en) Method for determining object classification from side-looking sensor data
US7403101B2 (en) Collision avoidance of unattended vehicles
US6452533B1 (en) Vehicle periphery monitoring device
JP4967840B2 (en) Collision mitigation device
CN107826106B (en) A kind of vehicle blind zone detection method of controlling security and control system
US6169478B1 (en) Vehicle control system having obstacle detector
JP3046186B2 (en) Vehicle obstacle handling device
US5341344A (en) Obstacle-detection system
US20040090318A1 (en) Turn signal assembly with tactile feedback
KR20170114054A (en) Collision preventing apparatus and collision preventing method
KR101328016B1 (en) Collision avoidance apparatus for car based on laser sensor and ultrasonic sensor and collision avoidance apparatus thereof
WO2003029045A2 (en) Multi-technology object detection system and method
US20090244741A1 (en) System, apparatus and method for active mirrors with blind spot detection
US5714927A (en) Method of improving zone of coverage response of automotive radar
US20090167514A1 (en) Combined Radar Backup Assist and Blindspot Detector and Method
KR20070012887A (en) Rear side view warning mirror module and control method
JP3800624B2 (en) Sensor system for judging the surrounding environment of a car
US6832137B2 (en) Leaky cable based method and system for automotive parking aid, reversing aid, and pre-collision sensing
JP2789437B2 (en) Inter-vehicle distance alarm
JPH07237521A (en) Side air bag controller for vehicle
JPH09287915A (en) Distance measuring apparatus and safety running system
CN112026697A (en) Automatic emergency braking system manufactured by utilizing far infrared thermal imaging sensor

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A2

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BY BZ CA CH CN CO CR CU CZ DE DM DZ EC EE ES FI GB GD GE GH HR HU ID IL IN IS JP KE KG KP KR LC LK LR LS LT LU LV MA MD MG MN MW MX MZ NO NZ OM PH PL PT RU SD SE SG SI SK SL TJ TM TN TR TZ UA UG UZ VN YU ZA ZM

AL Designated countries for regional patents

Kind code of ref document: A2

Designated state(s): GH GM KE LS MW MZ SD SL SZ UG ZM ZW AM AZ BY KG KZ RU TJ TM AT BE BG CH CY CZ DK EE ES FI FR GB GR IE IT LU MC PT SE SK TR BF BJ CF CG CI GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
122 Ep: pct application non-entry in european phase
NENP Non-entry into the national phase

Ref country code: JP

WWW Wipo information: withdrawn in national office

Country of ref document: JP