WO2017039546A1 - System, method and apparatus for navigating one or more vehicles - Google Patents

System, method and apparatus for navigating one or more vehicles Download PDF

Info

Publication number
WO2017039546A1
WO2017039546A1 PCT/SG2016/050430 SG2016050430W WO2017039546A1 WO 2017039546 A1 WO2017039546 A1 WO 2017039546A1 SG 2016050430 W SG2016050430 W SG 2016050430W WO 2017039546 A1 WO2017039546 A1 WO 2017039546A1
Authority
WO
WIPO (PCT)
Prior art keywords
vehicle
image
marker
leader
wheelchair
Prior art date
Application number
PCT/SG2016/050430
Other languages
French (fr)
Inventor
Weng Wah Khong
Hideharu Yanagi
Chin Keong SEE
Original Assignee
Cyclect Electrical Engineering Pte Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Cyclect Electrical Engineering Pte Ltd filed Critical Cyclect Electrical Engineering Pte Ltd
Publication of WO2017039546A1 publication Critical patent/WO2017039546A1/en

Links

Classifications

    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61GTRANSPORT, PERSONAL CONVEYANCES, OR ACCOMMODATION SPECIALLY ADAPTED FOR PATIENTS OR DISABLED PERSONS; OPERATING TABLES OR CHAIRS; CHAIRS FOR DENTISTRY; FUNERAL DEVICES
    • A61G5/00Chairs or personal conveyances specially adapted for patients or disabled persons, e.g. wheelchairs
    • A61G5/10Parts, details or accessories
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05DSYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02Control of position or course in two dimensions
    • G05D1/021Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0231Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
    • G05D1/0234Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using optical markers or beacons
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05DSYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02Control of position or course in two dimensions
    • G05D1/021Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0287Control of position or course in two dimensions specially adapted to land vehicles involving a plurality of land vehicles, e.g. fleet or convoy travelling
    • G05D1/0291Fleet control
    • G05D1/0293Convoy travelling
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61GTRANSPORT, PERSONAL CONVEYANCES, OR ACCOMMODATION SPECIALLY ADAPTED FOR PATIENTS OR DISABLED PERSONS; OPERATING TABLES OR CHAIRS; CHAIRS FOR DENTISTRY; FUNERAL DEVICES
    • A61G2203/00General characteristics of devices
    • A61G2203/10General characteristics of devices characterised by specific control means, e.g. for adjustment or steering
    • A61G2203/22General characteristics of devices characterised by specific control means, e.g. for adjustment or steering for automatically guiding movable devices, e.g. stretchers or wheelchairs in a hospital
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61GTRANSPORT, PERSONAL CONVEYANCES, OR ACCOMMODATION SPECIALLY ADAPTED FOR PATIENTS OR DISABLED PERSONS; OPERATING TABLES OR CHAIRS; CHAIRS FOR DENTISTRY; FUNERAL DEVICES
    • A61G5/00Chairs or personal conveyances specially adapted for patients or disabled persons, e.g. wheelchairs
    • A61G5/10Parts, details or accessories
    • A61G5/1051Arrangements for steering

Definitions

  • the present invention relates to a system, method and apparatus for navigating one or more vehicles.
  • the system, method and apparatus are suitable for convoying a plurality of wheelchairs.
  • Power-driven or motorized wheelchair is advantageous for navigating across terrains that would be physically fatiguing in a manual wheelchair.
  • One example would be at the airport, where a disabled or elderly passenger in a wheelchair would have to travel a long distance with carry-on luggage from the plane and passenger loading bridge to the transit area or immigration counter. In this tiring situation, having access to power-driven wheelchairs would be preferable.
  • power-driven wheelchairs are significantly more expensive and the use of manual wheelchairs is more widespread among passengers. It is generally understood that as part of the flying experience, a passenger requiring a wheelchair must be provided with one in a timely manner at the airport and in some countries, this is mandatory under law.
  • a further problem that passengers with wheelchairs usually encounter in navigating across the airport is unfamiliarity with the route. Consequently, manpower is necessary to provide adequate guidance in navigating through the airport.
  • Yet another problem associated with navigating a group of wheelchairs in the airport is the lack of coordination, resulting in disorderly or disorganized movement of a group of wheelchairs that may be a safety hazard to other pedestrians.
  • This invention seeks to autonomously navigate a vehicle following a leader.
  • a system for navigating a first vehicle to follow a leader comprising:
  • a first image capturing device attached to the first vehicle, the first image capturing device operable to capture a first image of the first marker
  • a first processor operable to process the first image captured by the first image capturing device to determine a first property of the first marker in the first image and send a first command signal to move the first vehicle based on at least the property of the first marker in the first image.
  • the first image capturing device is a silicon camera with infra-red filter removed.
  • the first marker is an infra-red LED array.
  • the first image is captured by using a band-pass filter and the band-pass filter is operable to selectively transmit radiation of a specified wavelength.
  • the first marker is an array emitting according to the specified wavelength of the band-pass filter.
  • the first property is any one of the following: the size, degree of rotation, intensity and location of the marker in the first image.
  • the first command signal is sent to a first electric motor of the first vehicle.
  • the first electric motor controls the speed of the first vehicle and/or direction of the first vehicle based on differential steering.
  • the first processor is further operable to determine a first blinking frequency of the first marker by way of the first image and a second image and send a second command signal to move the first vehicle at a speed based on at least the first blinking frequency.
  • the first vehicle is always maintained at a first positional coordinate that is inclined at an angle relative to a direction of navigation of the leader.
  • the first marker comprises a first circular shaped pattern and a second circular shaped pattern wherein the first and second circular shaped patterns are arranged laterally beside each other.
  • the first circular shaped pattern has a first and a second circular shaped pattern wherein the first and second circular shaped patterns are arranged laterally beside each other.
  • the first circular shaped pattern and the second circular shaped pattern are LED arrays with the same emission wavelength.
  • the first vehicle is a wheelchair and the leader is either a wheelchair or a human or a robot.
  • the system further comprises a first controller that is operable to stop the first image capturing device, and is further operable to send a third command signal by way of the first processor to move the first vehicle based on at least a first input into the first controller.
  • a first controller that is operable to stop the first image capturing device, and is further operable to send a third command signal by way of the first processor to move the first vehicle based on at least a first input into the first controller.
  • the first controller communicates with the first processor by way of ZigBee communication protocol.
  • the first processor is further operable to communicate with a light detection and ranging system or an ultrasonic system to determine a movement speed of the leader and move the vehicle at the movement speed.
  • the system is further for used navigating a second vehicle to follow the leader further comprising: a second marker that is attached to the first vehicle; a second image capturing device attached to the second vehicle operable to capture a second image with the second marker; and a second processor operable to process the second image captured by the second image capturing device to determine a second property of the second marker in the second image and send a fourth command signal to move the second vehicle based on at least the property of the second marker in the second image.
  • a method for navigating a first vehicle to follow a leader comprising the steps of: determining, by the first vehicle, a direction of movement of the leader, a movement speed of the leader and a position of the leader relative to the first vehicle; and sending a first command signal to move the first vehicle based on at least the direction of movement of the leader, the movement speed of the leader and the position of the leader relative to the vehicle.
  • the step of determining the direction of movement of the leader, the movement speed of the leader and the position of the leader relative to the first vehicle is based on any one of the following: processing a first image to determine a first property of a first marker in the first image or using a light detection and ranging system or using an ultrasonic system.
  • the first image is captured by a first image capturing device with infra-red filter removed.
  • the first marker is an infra-red LED array attached to the leader.
  • the first image is captured with a band-pass filter, and the band-pass filter is operable to selectively transmit radiation of a specified wavelength.
  • the first marker is due to an LED array emitting according to the specified wavelength of the band-pass filter.
  • the first property of the first marker in the first image is one of the following: the size, degree of rotation, intensity and location of the marker in the image.
  • the first command signal is sent to a first electric motor of the first vehicle.
  • the first electric motor control the speed of the first vehicle and/or the direction of the first vehicle based on differential steering.
  • the method further comprising the step of determine a first blinking frequency of a first marker and sending the first command signal to move the first vehicle at a speed based on at least the first blinking frequency of the first marker.
  • the method further comprising the step of maintaining the first vehicle at a first positional coordinate relative to the leader.
  • the method further comprising the steps of: stopping the operation of the first image capturing device; and sending a third command signal to move the first vehicle based on at least a first input into a first controller.
  • first controller communicates by way of ZigBee communication protocol.
  • the first vehicle is a wheelchair and the leader is either a wheelchair or a human.
  • the first marker comprises a first circular shaped pattern and a second circular shaped pattern wherein the first and second circular shaped patterns are arranged laterally beside each other.
  • the first circular shaped pattern has a first and a second circular shaped pattern wherein the first and second circular shaped patterns are arranged laterally beside each other.
  • the first circular shaped pattern and the second circular shaped pattern are LED arrays with the same emission wavelength.
  • the method further used for navigating a second vehicle to follow the leader further comprising the steps of: determining, by the second vehicle, a direction of movement of the first vehicle, a movement speed of the first vehicle and a position of the first vehicle relative to the second vehicle; and sending a second command signal to move the second vehicle based on at least the direction of movement of the first vehicle, the movement speed of the first vehicle and the position of the first vehicle relative to the second vehicle.
  • the method further comprising the step of:
  • the method further comprising the steps of:
  • an apparatus for navigating a first vehicle to follow a leader comprising:
  • a latching device for attaching the apparatus to a coupling frame attached to the first vehicle
  • an image capturing device operable to capture an image with a marker, wherein the marker is attached to the leader;
  • a processor operable to process the image captured by the image capturing device to determine a property of the marker in the image and send a command signal to the motor based on at least the property of the marker in the image to drive the wheel.
  • the latching device comprises of an insertion pin that is magnetically or mechanically actuated.
  • the insertion pin is magnetically or mechanically actuated to slot into a pin hole located in the coupling frame of the first vehicle.
  • the apparatus further comprising a contact sensor for triggering the magnetically or mechanically actuated insertion pin when contact on the coupling frame is detected.
  • the latching device comprises of a chamfer that is magnetically or mechanically actuated.
  • the chamfer is magnetically or mechanically actuated into a circlip located in the coupling frame when contact on the coupling frame is detected.
  • the apparatus comprises a top portion releasably attachable to a bottom portion, wherein the top portion is detached from the bottom portion and the bottom portion is releasably attachable to a steering unit for conversion to an electric scooter.
  • the steering unit comprises either an actuator for steering the electric scooter.
  • the actuator may be in the form of a push button or joystick.
  • Fig. 1 shows an apparatus in accordance with an embodiment of the invention attached to a wheelchair.
  • Fig. 2 shows the apparatus in various perspective views: (a) front view, (b) isometric view, (c) side view and (d) enlarged or zoomed in isometric view.
  • Fig. 3 shows the side views of the apparatus in various states; (a) when the latching mechanism is not activated; (b) when the latching mechanism is activated; and (c) when the latching mechanism is not activated and the apparatus is separable into two portions.
  • Fig. 4 shows the coupling frame attached to the wheelchair in various views; (a) front view with the wheelchair and coupling frame folded (a first state); (b) front view with the wheelchair and coupling frame un-folded (a second state); and (c) side view with the wheelchair and coupling frame un-folded.
  • Fig. 5 shows a close up view of the coupling frame; (a) front cross-section view of the coupling frame attached to the wheelchair and (b) side view of the coupling frame attached to the side rod bar wheelchair.
  • Fig. 6 shows the coupling between the wheelchair and the apparatus via the latching mechanism on the apparatus and the coupling frame attached to the wheelchair.
  • Fig. 7 shows a system for remotely navigating a wheelchair, wherein the apparatus interacts or communicates with a human companion via either a controller and/or an IR LED array pattern.
  • Fig. 8 shows an embodiment of the invention in which the IR LED array pattern is located at the back of the wheelchair in front.
  • Fig. 9 shows an embodiment of the invention in which a convoy of wheelchairs escorted by a human companion.
  • Fig. 10 illustrates two possible configurations in which one or more wheelchairs may navigate behind the human companion: (a) top-view of one or more wheelchairs navigate at an angle with respect to the human companion 704 and (b) top view of one or more wheelchairs navigate directly behind the human companion.
  • Fig. 1 1 shows an alternative pattern for the IR LED array comprising of two circular shaped patterns with the same emission wavelength (the two patterns may also have different emission wavelengths) arranged at opposite sides of an imaginary middle axis 1 106 on the back of the battery-powered vest 706.
  • Fig. 12 shows schematic drawings of (a) the apparatus with the mast removed and a joystick control that can communicate with the apparatus to effect navigation under the local control mode and (b) the mounting of the apparatus and the joystick on a wheelchair to move or transport the wheelchair by local control mode.
  • Fig. 13 shows schematic drawings of (a) steering unit (an interchangeable top base with steering rod) and a modified version of the apparatus with the top part removed and (b) the mounting of the interchangeable top base to the modified apparatus to form an electric scooter. Description of Embodiments of the Invention
  • the "X" direction is defined as the direction that is substantially perpendicular to the axle of the wheelchair (parallel to the direction of motion of the wheelchair) and the “Y” direction is defined to be substantially parallel to the axle of the wheelchair.
  • the X- and Y- axes are substantially parallel to the horizontal plane. The "right” and “left” sides are construed accordingly with respect to a person sitting on the wheelchair facing the front; the "right” side would correspond to the right side of the person and the "left” side corresponds to the left side of the person. "Up” and “down” will be construed with respect to the vertical axis.
  • the apparatus 102 may be attachable to the wheelchair 104 via a securing means.
  • the securing means may be a coupling mechanism that comprises a first part that is disposed or affixed to a portion of the apparatus 102 and a second part that is disposed or affixed to a portion of the wheelchair 104.
  • the first part of the coupling mechanism may be a latching device 302 as shown in Fig. 3 and the second part of the coupling mechanism may be a coupling frame 402 as shown in Fig. 4 and Fig. 5.
  • the latching device 302 and coupling frame 402 shall be described sequentially below.
  • the latching device 302 may comprise a permanent magnet 306 and an insertion pin 308.
  • the permanent magnet 306 may be attached near to the distal end (or the end further away from the apparatus 102) of the insertion pin 308.
  • the latching device 302 may have two states of operation. In the first or un-activated state as shown in Fig. 3 (a), the permanent magnet 306 and the insertion pin 308 are retracted or withdrawn into a recess 310 under a cover 304.
  • the recess 310 may be shaped according to the outline or contour of the composite body comprising of the permanent magnet 306 and the insertion pin 308 as shown in Fig. 3 (a).
  • the latching device 302 may only comprise of the insertion pin 308 without the permanent magnet 306.
  • the cover 304 may be operable to act as a trigger for the latching device 302.
  • the cover 304 is lifted as shown in Fig. 3 (b)
  • the cover 304 is in communication with a contact sensor (not shown) such that when the cover 304 is contacted by the second part of the coupling mechanism (coupling frame 402), the contact is sensed by the contact sensor and the latching device 302 may be triggered into the second or activated state.
  • a contact sensor not shown
  • the insertion pin 308 may be magnetically actuated upwards along its longitudinal axis, wherein the magnetic actuation may be achieved at least by way of the permanent magnet 306 and/or an electromagnet (not shown) that is located near the recess 310.
  • the electromagnet may be a solenoid.
  • a control signal may be sent to the electromagnet to produce or generate a magnetic field to repel the permanent magnet 306 when the contact sensor detects or senses any contact with the cover 304.
  • the repel force due to the interaction between the magnetic field of the permanent magnet 306 and the magnetic field of the electromagnet, may actuate the insertion pin 308 upwards from the recess 310 into the second state.
  • the insertion pin 308 may be magnetised and the magnetic actuation may be due to the interaction between the magnetic field of the magnetised insertion pin 308 and the magnetic field of the electromagnet. In various embodiments, the insertion pin 308 may also be mechanically actuated upwards. In various embodiments, the insertion pin 308, when actuated upwards, may be slotted into a pin hole 508 located on the second part of the coupling mechanism or coupling frame 402 as shown partly in Fig. 6. In various embodiments, the permanent magnet 306 may also operable to be attracted or engaged to the coupling frame 402.
  • the latching device 302 may comprise a chamfer (not shown) that is operable to couple with a circlip located on the coupling frame 402.
  • the chamfer may be mechanically or magnetically actuated upwards when the cover 304 is contacted by the second part of the coupling frame 402. Thereafter, the mechanically or magnetically actuated chamfer may be coupled with a circlip located on the coupling frame 402 for securing the apparatus 102 to the wheelchair 104.
  • the coupling frame 402 may comprise two flat bars 502 that are attached to each other at their respective proximal ends via a hinge joint 504.
  • the hinge joint 504 allows the coupling frame 402 to be folded when the wheelchair 104 is folded as shown in Fig. 4 (a).
  • the respective distal ends of the flat bars 502 may be attached to a securing means, wherein the securing means is for attachment to the corresponding wheelchair side rod bars 404.
  • the securing means may be a sleeve clamp 510 that is operable to be coupled onto the circumference or outer surface of the wheelchair side rod bar 404 as shown in Fig. 5.
  • a thumb-screw 512 and nut 514 may be used to secure the sleeve clamp 510 on the wheelchair side rod bar 404 by screwing the thumb-screw 512 against the nut 514 to effect clamping.
  • the installation of the coupling frame 402 via the wheelchair side rode bar 404 would not substantially affect the basic functionality of the wheelchair 04. Further, minimal modification is required in the installation of the coupling frame 402.
  • the sleeve clamp 510 may be loosen through the thumb- screw 512 and nut 514 to allow the sleeve clamp 510 rotate about the wheelchair side rod bar 404 when the wheelchair 104 is being folded or unfolded.
  • the distal ends of the flat bars 502 may be rotatably attached to the sleeve clamp 510 such that the sleeve clamp 510 does not need to be loosen when the wheelchair 104 is being folded or unfolded.
  • the apparatus 102 may further comprise at least one front wheel 106 and two rear wheels 108.
  • the two rear wheels 108 of the apparatus may be normal wheels and the front wheel 106 of the apparatus 102 may be a double-row transwheel.
  • the normal wheels may be bi-directional and transwheel may be multi-directional.
  • the rear wheels 108 may be driven by at least one electric motor (not shown). In various embodiments, there may be more than two rear wheels.
  • each rear wheel 108 may be independently driven by an electric motor so as to steer the wheelchair 104 using differential steering in which one of the rear wheels 108 is rotating faster than the other to effect steering.
  • the front wheel 106 which may be a double-row transwheel may be operable to accommodate movement in different directions as determined by differential steering.
  • the apparatus 102 may further comprise a mast 1 10, wherein the mast 1 10 may be a vertical rod or pole as illustrated in Fig. 2.
  • the mast 1 10 may be releasably attached to the apparatus 102 such that the longitudinal axis of the mast 1 10 is substantially aligned with the vertical direction.
  • the mast 1 10 may contain an image capturing device or camera 1 12 and a warning light 114 which are attached near the top of the mast 1 10.
  • the camera 1 12 may be a NoIR camera (i.e. a conventional camera with an infra-red (IR) filter removed) that is more sensitive to IR radiation.
  • the camera 112 may also be omni-directional.
  • the camera 1 12 is operable to capture images real-time related to the surrounding with increased or enhanced sensitivity to IR radiation within its field of view.
  • a system 700 for convoying or platooning a wheelchair 104 to follow a human companion 704 comprising the apparatus 102 which may be in communication with a master remote controller 708 (a first remote controller) and a client remote controller 710 (a second remote controller).
  • the system 700 may be useful for remotely navigating a wheelchair 104 at an airport to assist a passenger in wheelchair get from one place to another.
  • the master remote controller 708 may be hand-held by a human companion 704 (also known as the leader) and the client remote control 710 may either be hand-held by the user on the wheelchair 104 or installed on the wheelchair 104.
  • the communication between the master remote controller 708, the client remote controller 710 and the apparatus 102 may be achieved by Bluetooth or low energy Bluetooth or ZigBee communication protocol.
  • the master remote controller 708 and/or the client remote controller 710 may be used to select the type of operation mode. Depending on the selected operation mode, the apparatus 102 may be able navigate the wheelchair 104 either remotely or locally.
  • the six available operation modes that will be described in the later part of the description are namely 1 ) self-navigation or autonomous mode, 2) convoy mode, 3) manual mode, 4) remote control mode, 5) local control mode and 6) electric scooter mode.
  • the system 700 may also comprise an infra-red (IR) light emitting diode (LED) array 702 that is located on the back of a human companion 704 by way of a battery-powered vest 706 that is worn by the human companion 704.
  • the camera 112 may then capture real-time image containing at least the IR LED pattern 701 that may be due to IR radiation emitted from an IR LED array 702.
  • the processor may then communicate, in real-time, the captured realtime image to a processor (not shown).
  • the captured real-time image may then be processed upon receipt to detect or recognize for the IR LED pattern 701 .
  • the IR LED array 702 is arranged substantially similar to the IR LED pattern 701 using discrete IR LEDs.
  • the detected or recognized IR LED pattern 701 provides a visual marker or a dedicated external reference point to effect self-navigation or convoying which may enable one or more wheelchairs to be led or escorted by a human companion 704 wearing the battery- powered vest 706 with the IR LED pattern 701 .
  • Providing or attaching the IR LED array 702 that has a pre-determined pattern to the human companion 704 is advantageous in a crowded area because the pre-determined pattern acts as a marker to allow the human companion 704 be easily distinguished from the crowd as compared to existing technology relying solely on laser range sensors to detect the change in position of the human companion 704.
  • An inherent drawback of the foregoing is that it would be challenging for a laser range sensor to distinguish between two or more humans that are present within the field of view of the laser range sensor.
  • the drawback associated with the prior art is further compounded by physical objects that are present within the field of view of range of the laser range sensors.
  • IR is invisible to the human eye and hence will not be a distraction to other pedestrians; and 2) IR LED array 702 may also be discriminated or distinguished from visible light through the use of a filter to enhance the signal to noise ratio of the captured image which would advantageously allow the IR LED array to be low-powered. It may be appreciated that street lamps/lightings and typical signboards emit strongly in the visible spectrum for practical reasons, making the discrimination or recognition of a visible LED array more challenging.
  • the processor may be further operable to translate the recognized real-time IR LED pattern 701 into a set of real-time positional information of the human companion 704 relative to the wheelchair 104 using either the image recognition software or a separate software.
  • the foregoing is achieved by determining at least one property of the IR LED pattern 701.
  • the at least one property may be, but not limited to, any one of the following: the size, degree of rotation, intensity and location of the captured IR LED pattern 701 in the captured image frame.
  • the relative distance between the wheelchair 104 and the human companion 704 may be calculated or determined using at least the focal length of the camera lens, the real height of the IR LED pattern 701 , the pixel height or size of the captured image, the pixel height of the IR LED pattern 701 and the real height or vertical dimension of the sensor in the camera.
  • the degree of rotation of the IR LED pattern 701 about the vertical axis in the captured image may also translate to the degree of rotation of the human companion 704 about his/her vertical axis (or turning angle of the human companion 704) as the IR LED array 702 may lie substantially flat on the back of the human companion 704.
  • the degree of rotation about the vertical axis may then provide information relating to the direction in which the human companion 704 is heading.
  • the gradient in light intensity of the captured IR LED pattern 701 may also provide information relating to the degree of rotation of the IR LED pattern 701 as the part of the IR LED pattern 701 that is located further away would have relatively lower light intensity and the gradient in light intensity across the IR LED pattern 701 may be translated to the degree of rotation of the human companion 704.
  • the location or vertical and horizontal positions of the IR LED pattern 701 in the captured real-time image frame may also provide positional information of the human companion 704. If the IR LED pattern 701 is detected or recognized at a location substantially towards the right of the image, then the processor or software would be able to determine that the human companion 704 is located towards the right side of the wheelchair 104 and calculate the relative distance or displacement of the human companion 704 accordingly.
  • the determination of the real-time positional information such as distance of the human companion 704 relative to the wheelchair 104 may allow the processor to track the movement direction and/or speed of the human companion 704 by processing a series of consecutively captured real-time images. For example, the increase in relative distance between the human companion 704 and the wheelchair 104 may indicate that the human companion 704 is moving faster, hence the speed of the wheelchair has to increase for maintain a fixed relative distance between the human companion 704 and the wheelchair 104.
  • the processor may respond to follow the change in direction and/or speed of the human companion 704 by sending an electrical command or control signals to one or more electric motors of each rear wheel 108. In response, the electrical command or control signal may control the speed of wheelchair 104 and steer the wheelchair 104 in different directions through differential steering of the rear wheels 108 as discussed above.
  • a Light Detection and Ranging (LIDAR) or an ultrasonic sensor system may also be used independently or in combination of the camera 1 12 to determine the relative distance between the wheelchair 104 and the human companion 704 and/or to perform three dimensional mapping and localization.
  • the movement speed of the human companion 704 may also be determined directly or indirectly (by processing time series of the relative distance between the wheelchair 104 and the human companion 704) by the LIDAR or an ultrasonic system.
  • the LIDAR or ultrasonic system may be attached on the mast 1 10 near to the camera 112 and may communicate with the processor.
  • stopping of the wheelchair 104 may be achieved when it is determined that there is no change in the real-time position of the human companion 704, i.e. the human companion 704 has stopped moving and is remaining at the same location.
  • the processor may be operable to stop the one or more electric motors of each rear wheel 108.
  • stopping of the wheelchair 104 may also be achieved when the human companion 704 turns off the IR LED array 702.
  • the processor fails to detect any IR LED pattern 701 in the captured image, it is operable to stop the one or more electric motors of each rear wheel 108.
  • the processor may slow down the one or more electric motors gradually before coming to a complete stop.
  • the one or more electric motors may come to a complete stop quickly. The foregoing feature may be useful to effect emergency stop of the wheelchair 104 during autonomous operation.
  • the flashing or blinking modes of the IR LED array 702 may also be used to move the wheelchairs 104 according to the speed of the human companion 704.
  • the movement speed of the human companion 704 may correspond to a frequency of flashing or blinking of the IR LED array 702.
  • the processor on the apparatus 102 may be operable to determine the frequency of flashing or blinking of the IR LED array 702 from a succession of captured images acquired or captured with a known camera frame-rate and send the appropriate control signal to the one or more electric motors accordingly.
  • the camera 1 2 may also be a video camera or may operate in video camera mode for detecting dynamic changes (i.e. blinking frequencies) of the IR LED array 702.
  • the IR LED array 702 may be connected to a speed sensor or detector (not shown) that is attached to the battery-powered vest 706 wherein the speed sensor is operable to detect the movement speed of the human companion 704.
  • the speed sensor may detect movement speed of the human companion 704 using Base Transceiver Station (BTS) cellular triangulation or global positioning system (GPS) based methods.
  • BTS Base Transceiver Station
  • GPS global positioning system
  • the movement speed of the human companion 704 may also be estimated by way of a pedometer. Thereafter, the movement speed of the human companion 704 as determined or measured by the speed sensor may then then be translated into a flashing or blinking frequency of the IR LED array 702 for guiding or escorting the wheelchair 104.
  • the IR LED array 702 may also be located at the back of another wheelchair as shown in Fig. 8 such that the back or follower wheelchair may rely on the IR LED array 702 located at the back of the front or leader wheelchair for navigation rather than the IR LED array 702 located at the back of the human companion 704.
  • the foregoing feature would be useful in coordinating the simultaneous movement of two or more wheelchairs.
  • the two or more wheelchairs may also move as a convoy behind a human companion 704 as shown in Fig. 9.
  • a first wheelchair 902 may be programmed via the apparatus 102 to track and follow the front human companion 704 while maintaining a safe distance.
  • the second wheelchair 904 may be programmed via its apparatus 102 to follow the first wheelchair by tracking the IR LED array 702 located at the back of the first wheelchair 902 while maintaining a safe distance.
  • the third wheelchair 906 may be programmed to follow the second wheelchair 904 by way of the IR LED array 702 located at the back of the second wheelchair 904.
  • a system 900 comprising wheelchairs 902, 904 and 906 that may be programmed to travel in a substantially straight line (i.e. single file formation) directly behind the human companion 704.
  • the straight line arrangement or single file formation is advantageous in parts of the airports where space is limited especially at the gate or the security.
  • the following wheelchairs 902, 904 and 906 may also be programmed to travel in a path without being directly behind the human companion 704.
  • the foregoing configuration or convoy formation would be advantageous to avoid collision between the first wheelchair 902 and the human companion 704 if the first wheelchair 902 cannot stop in time.
  • the apparatus 102 may comprise three units: a control unit 312 (a first portion), a battery unit 3 4 (a second portion) and the mast 110 (a third portion).
  • the three units may be connected or assembled together via the respective electrical interfaces (not shown).
  • the control unit 312 may comprise the processor.
  • the battery unit 314 may comprise a battery and at least one motor for driving the wheels.
  • the electrical interfaces allows electrical signal to be exchanged between the three units.
  • the apparatus 102 When on-demand power driving and/or autonomous navigation capability is to be provided to a wheelchair 104, the apparatus 102 as described above is rolled underneath the wheelchair 104 from the back of the wheelchair 104 as shown in Fig. 6.
  • the latching device 302 When the flat bars 502 of the coupling frame 402 contacts or touches the cover 304 as the apparatus 102 is rolled underneath the wheelchair 104, the latching device 302 is activated to the second state and the insertion pins 308 are magnetically or mechanically actuated upwards along its longitudinal axis and are slotted into the pin hole 508 located in the flat bars 502 of the coupling frame 402.
  • the permanent magnet 306 may also be attracted to the flat bars 502 to secure the coupling between the coupling frame 402 and the latching device 302.
  • a circlip and chamfer combination may also be used to secure the coupling frame 402 and the latching device 302.
  • the human companion 704 may select the operation mode using the master remote controller 708.
  • There are at least six available operation modes are: 1 ) self-navigation mode, 2) convoy mode, 3) manual mode, 4) remote control mode, 5) local control mode and 6) electric scooter mode. However, only the first five operation modes (excluding the electric scooter mode) may be selected when the apparatus 102 is attached to the wheelchair 104.
  • one wheelchair is able to move autonomously by being guided or escorted by a human companion 704 who is wearing the battery-powered vest 706 with the IR LED array 702.
  • the NoIR camera 1 12 of the apparatus 102 begins to capture real-time images.
  • the processor with the help of a pre-loaded software, is operable to recognize and track the IR LED pattern 701 in the image that is emitted from an IR LED array 702 from the back of the human companion 704.
  • the detected IR LED pattern 701 may then be translated into positional information of the human companion 704 relative to the wheelchair 104 based on at least the size, degree of rotation, intensity and location of the captured IR LED pattern 701.
  • the positional information of the human companion 704 enables the apparatus 102 to initialize the wheelchair 104 to an initial position behind the human companion 704.
  • the apparatus 102 may try to align the wheelchair 104 substantially directly behind the human companion 704.
  • the processor detects that the IR LED pattern 701 is located substantially towards the right of the captured image
  • electrical command or control signal may be sent from the processor to the rear wheels 108 to steer the wheelchair 104 towards the right or rotate the wheelchair 104 clockwise such that the IR LED pattern 701 in subsequent captured images becomes substantially centered within the frame of the images.
  • the apparatus 102 may further move the wheelchair 104 forward or backward to a pre-determined safe distance based on the positional information of the human companion 704 that is determined from the captured real-time images.
  • the relative distance between the human companion 704 and the wheelchair 104 may be determined using any one of the following: processing the captured image to determine a property of the IR LED pattern 701 in the captured image or using a light detection and ranging system or using an ultrasonic system.
  • the time period is inversely proportional to the frame rate or frame frequency of the camera 1 12.
  • the apparatus 102 may determine the change in the positional information of the human companion 704 over a time period based on a blinking or flashing frequency of the IR LED pattern 701 as described above. In various embodiments, the apparatus 102 may determine the change in the positional information of the human companion 704 over a time period using either a LIDAR or ultrasonic system.
  • the processor is further operable to respond to this positional change of the human companion 704 during this time period by sending electrical command or control signal to the rear wheels 108 of the apparatus 102 to move and steer the wheelchair 104 according to the direction of movement, speed of the human companion 704 and position of the human companion 704 relative to the wheelchair 104 while still maintaining a safe distance behind the human companion 704.
  • the safe distance may translate to a physical positional coordinate relative to the human companion 704 and the wheelchair should be located at this positional coordinate to prevent collision with the human companion 704. For example and as illustrated in Fig.
  • the safe positional coordinate may be defined in Cartesian form (X, Y) where the X- axis is substantially perpendicular to the longitudinal axis of the axle of the wheelchair and Y-axis is substantially parallel with the longitudinal axis of the axle of the wheelchair and both the X- and Y-axes are parallel to the horizontal plane and the origin (0, 0) is the position of the human companion 704; a positional coordinate of (-1 , 0) as shown in Fig. 10 (b) indicates that the wheelchair 104 should always self-navigate or move autonomously at a distance of one meter substantially directly behind the human companion 704.
  • the wheelchair 104 may also initialize and navigate at an angle with respect to the human companion 704.
  • the positional coordinate of the wheelchair 104 may be (-1 , -1 ) whereby the wheelchair 104 would self-navigate at a distance of one meter behind and one meter to the left side (at 45 degrees angle) of the human companion 704.
  • the wheelchair 104 is maintained at a positional coordinate that is inclined at an angle relative to the direction of navigation or movement of the human companion 704, collision between the wheelchair 104 and the human companion 704 may be avoided if the wheelchair is unable to stop in time when the human companion 704 has stopped moving.
  • the positional coordinate may be defined according to polar coordinates (r, theta).
  • the time period for detecting change in positional information of the human companion 704 relative to the wheelchair 104 may affect the response time of the apparatus 102 and hence may be changed according to the requirements of the user.
  • reducing the time period or increasing the frame rate shortens the response time as the apparatus 102 may be able to react faster as the positional information of the human companion 704 is updated more frequently, resulting in higher self-navigation accuracy.
  • increasing the time period or reducing the frame rate may increase the response time of the apparatus 102, leading to the need to maintain the wheelchair 104 at a longer safety distance away from the human companion 704 to avoid collision.
  • time period may also be constrained by the exposure time required by the camera 1 12 to achieve an image with sufficient quality for image recognition and processing. Consequently, the use of brighter IR LED array 702 may help reduce the exposure time required and allows images to be taken in quicker succession (reduced time period and reduced response time).
  • Convoy mode In the convoy mode as shown in Fig. 9, two or more wheelchairs are guided and escorted by a human companion 704 who is wearing the battery-powered vest 706 with the IR LED array 702. To navigate in the convoy mode, the users of the two or more wheelchairs may first simultaneously select the convoy mode option via the respective client remote controllers 710. In various embodiments, the human companion 704 may also group the two or more wheelchairs into a convoy using the master remote controller 708.
  • the wheelchairs may move to its respective uniquely assigned positional coordinates behind either the human companion 704 or another wheelchair in a similar manner as described in the self-navigation mode.
  • the unique positional coordinates help prevent collision from occurring between the wheelchairs in the convoy and may be assigned in at least two ways as described below.
  • the respective apparatus 102 of each wheelchair may be programmed to self-navigate at a pre-determined unique positional coordinate behind the human companion 704. This would be similar to the self-navigation of one wheelchair as discussed above except that each wheelchair in the convoy has a unique positional coordinate with respect to the human companion 704.
  • a possible drawback of this method of operation is that the wheelchairs further away from the human companion 704 may have a challenge capturing a reliable image of the IR LED pattern 701 due to the IR LED array 702 located at the back of the human companion 704.
  • a first wheelchair 902 may be assigned within the convoy to track and follow the front human companion 704 while maintaining a safe distance or at a positional coordinate behind the human companion 704.
  • the second wheelchair 904 may be assigned to follow the first wheelchair 902 by tracking the IR LED pattern 701 located at the back of the first wheelchair 902 while maintaining a safe distance or at a positional coordinate behind (or with respect to) the first wheelchair 902.
  • the third wheelchair 906 may be assigned to follow the second wheelchair 904 by way of the IR LED pattern 701 located at the back of the second follower wheelchair 904.
  • the wheelchairs During the operation phase when the human companion 704 starts moving, the wheelchairs would maintain in its unique positional coordinate with respect to either the human companion 704 or the wheelchair in front based on the embodiments described above.
  • the first wheelchair 902, second wheelchair 904 and third wheelchair 906 may move respectively at a coordinates of (-1 ,-1 ), (-2, -1 ) and (-3, -1 ) with respect to the human companion 704.
  • this configuration may avoid collision between the first wheelchair 902 and the human companion 704 if the wheelchair 902 is unable to stop in time when the human companion 704 has stopped moving.
  • the apparatus 102 may also operate in manual mode wherein the apparatus 102 is disengaged from the wheels of the wheelchair without detaching it from the wheelchair 104. In another possible manual mode, the apparatus 102 may still be engaged to the wheelchair 104, and the human companion 704 is required to push the wheelchair manually.
  • the manual mode is advantageous when self-navigation is not advisable such as in a lift or moving on a sky-bridge.
  • the apparatus 102 may also be controlled remotely using a master remote controller 708.
  • the apparatus 102 may also be controlled locally by the user on the wheelchair 104 through a client remote controller 710 as shown in Fig. 7 or a joystick control 1202 as shown in Fig. 12.
  • Commands may be communicated from either the master remote controller 708 or the client remote controller 710 or the joystick control 1202 to the apparatus 102 by wireless or wired means.
  • established wireless communication protocols such as Bluetooth or Zigbee communication protocol may be used.
  • the processor is operable to first stop the operation of the camera 112.
  • the processor then processes the received commands and send an electrical command or control signal to the rear wheels 108 to navigate the wheelchair according to the received commands.
  • the mast 1 10, together with the camera 1 12 may be removed as they are not required to effect the local control mode.
  • the apparatus 102 may also be converted into an electric scooter 1314 as shown in Fig. 13.
  • the mast 1 10 and a top part or top portion of the control unit 312 and a top part or top portion of the battery unit 314 may be removed from the apparatus 102 as shown in Fig. 13 (a), leaving the bottom parts or bottom portions of the control unit 312 and battery unit 314 to create a modified apparatus 1301 .
  • a steering unit 1302 (otherwise may also be referred to as an interchangeable top base with steering rod) comprising a steering handle 1304, a standing platform 1306 and a steering rod 1308 may be mounted or secured on the modified apparatus 1301 to create an electric scooter 1314.
  • the steering handle 1304 is mounted on a steering rod 1308 and the steering rod 1308 extends upwardly from the standing platform 1306.
  • the length of the vertical steering rod 1308 may be adjustable to cater to passengers with different heights.
  • the steering handle 1304 may comprise a left steering handle 1310 and a right steering handle 1312, wherein either one or both of the steering handles may have a steering control unit or actuator for controlling the speed and direction of the electric scooter 1314.
  • the steering control unit may enable the speed of the electric scooter 1314 to be controlled within the range of 0 to 10 km/h.
  • the steering control unit may be operable to allow the speed to be controlled in three discrete stages or levels - slow, medium and fast.
  • the steering control unit or actuator may be a left push button 1316 located on the left steering handle 1310 and a right push button 1318 located on the right steering handle 1312.
  • the push buttons are operable to detect a left pushing force and right pushing force of the corresponding hand for manoeuvring. For example, a larger force on the right push button 1318 would steer the electric scooter 1314 towards the right and vice versa. Pushing both buttons with equal force will navigate the electric scooter 1314 forward at a speed corresponding to the force on both the push buttons.
  • the push button may comprise a force or pressure sensor operable to detect the amount force applied by the passenger and thereafter, the force may be correlated to the speed of the corresponding rear wheel via a first transfer function.
  • the push button may also be operable to determine the amount of compressive force or displacement of the button from its original position and correlate the amount of displacement of the push button to the speed of the corresponding rear wheel via a second transfer function (i.e. left push button 1316 to left rear wheel and right push button 1318 to right rear wheel).
  • the push buttons may be biased by a spring such that the push button returns to its original position once the pushing force is removed so that the electric scooter 1314 automatically slows down when the passenger cease to apply the force on the push buttons.
  • the force or pressure sensor in either of the push buttons may be a piezo-resistive or piezo-electric sensor.
  • the said piezo-resistive or piezo-electric sensor may induce a voltage or potential difference which may be used as a control signal.
  • the piezo-resistive sensor may comprise of a piezo-resistive element connected to a Wheatstone bridge. The piezo-resistive element and the Wheatstone bridge may be attached or placed on a diaphragm.
  • the Wheatstone bridge may be used for more accurate determination of the change in resistance of the piezo-resistive element when the said piezo-resistive element is strained.
  • a strain or voltage amplifier may be used to amplify the signal from the piezo-resistive or piezo-electric sensor
  • control signals (amount of force or amount of displacement) from the force or pressure sensor in each of the push buttons may be transmitted to a processor for further processing. Thereafter, the processor may calculate the speed of each rear wheel through at least either the first or second transfer function and further transmit further control signals to control the rear wheels via differential steering.
  • an H-bridge circuit may be used to enable reversing of the control signals from the push buttons. The H-bridge circuit would be useful to allow the passenger to reverse the electric scooter 1314 using the push buttons. In various embodiments, the rear wheels of the electric scooter 1314 may automatically slow down and gradually come to a stop within a pre-determined period of time once the passenger disengages the push buttons.
  • the steering control unit or actuator may be a joystick mounted on either the left steering handle 310 or the right steering handle 1312.
  • the joystick may enable forward, backward, left and right control and may comprise of built in switches and potentiometer which are operable to give a stable voltage output to a Pulse Width Modulation (PWM) or analog speed control circuit.
  • PWM Pulse Width Modulation
  • a micro-controller such as an chicken UnoTM control board may be in data communication with the actuator which may be in the form of a joystick.
  • the microcontroller is further operable to receive digital electrical signals from the joystick for generating a PWM output control signal to simulate a variable direct current (DC) supply voltage to control the electric motor of the electric scooter 1314.
  • DC direct current
  • the microcontroller may operate to receive an analog signal from the potentiometer in the joystick and generate a PWM output control signal to control the speed of the electric motor.
  • the microcontroller may be programmed to output the PWM control signal by modifying the duty cycle of the output PWM control signal according to the magnitude of the analog input, for example a high analog input will result in a higher duty cycle.
  • the duty cycle of the PWM control signal will directly affect the effective voltage output to the DC electric motor which will in turn affect the speed of the electric motor.
  • an electronic switch such as, but not limited to an electronic transistor may also be used in combination with the microcontroller for switching the electric motor.
  • the steering unit 1302 may be mounted on the modified apparatus 1301 through the standing platform 1306 via securing means such as nuts and bolts or other types of commonly known securing means.
  • the standing platform 1306 may be as close to the surface where the electric scooter 1314 is travelling on as possible (so as to maintain a low center of gravity for stability, especially at higher speeds).
  • wheel fenders (not shown) may be installed at the rear wheels of the modified apparatus to prevent injury to the passenger.
  • the area of the standing platform 1306 may be greater than the top surface area of the modified apparatus 301 to provide the passenger or rider with more surface or room to stand on.
  • the human companion 704 with the IR marker or IR LED array 702 may ride on the electric scooter 1314 when escorting a convoy of wheelchairs 102.
  • the electric scooter 1314 has the advantage of being able to maintain the moving speed of the human companion 704 at a specific or predetermined speed for the ease of convoying the wheelchairs 102.
  • the modularity of the control unit 312, the battery unit 314, the mast 110 and the steering unit 1302 enables the apparatus 102 be modified or adapted to suit different needs or modes of operation.
  • the IR LED array 702 may be arranged in other patterns instead of the pattern "A" that is described in the above embodiments.
  • the IR LED array 702 may have either a symmetrical or asymmetrical pattern or design.
  • An asymmetrical pattern which does not possess symmetry about the vertical axis may allow the direction that the human companion 704 is facing be determined without using other properties of the image such as gradient in light intensity.
  • the IR LED array 1 00 at the back of the battery-powered vest 706 may comprise two circular shaped patterns (a first circular shaped pattern 1102 and a second circular shaped pattern 1104).
  • the first and second circular shaped pattern may have the same emission wavelength in IR (for e.g. emission wavelength in the range of 850 nm to 2000 nm).
  • the two circular shaped patterns may be arranged on the opposite sides of an imaginary middle axis 1106 of the vest 706 (i.e. the two circular shaped patterns are arranged laterally beside one another). Further, the two circular shaped patterns may also be located at an equal lateral distance from the middle axis of the vest 706.
  • the circular shaped pattern that is further away will become sharper (more oval-like) and the circular shaped pattern that is nearer one will become larger.
  • the turning angle or the degree of rotation about the vertical axis may be thus be determined based on the change in the relative change in shape and/or size of the two circular shaped patterns and/or intensity in the captured image.
  • the two circular shaped patterns may also have different emission wavelengths.
  • the different emission wavelengths may be in visible spectrum (for example red LED emitting at around 600 nm and green LED emitting at around 565 nm). It may be appreciated that a different detector apart from one which is primarily sensitive in the N1R region may have to be used to detect emission in the visible spectrum, i.e. red and/or green emission. In various embodiments, this may include combining InGaAs or any other detectors that is sensitive mostly in the IR spectrum with a silicon detector that is sensitive mostly in the visible spectrum if the first emission wavelength is in the IR and the second emission wavelength is in the visible. The two circular shaped pattern with different emission wavelengths may be used to determine the turning angle.
  • the relative distance of each of the two circular shaped pattern from the apparatus 102 may be determined by comparing the intensity of the emission and/or the relative shape of the two circular shaped patterns in the captured image and/or the relative size of the two circular shaped patterns in the captured image.
  • the IR LED array 702 for two or more wheelchairs may each have a unique design to prevent confusion since the respective leaders would each have a unique marker.
  • the IR LED array 702 may also be substituted by a visible LED array or any other radiation sources emitting at a pre-determined or specified wavelength. Accordingly, the camera 112 should be sensitive to the pre-determined or specified wavelength of radiation that is emitted by either the LED array or any other radiation sources.
  • the IR LED array 702 may comprise of LED stripes instead of discrete LED units/lamps.
  • the IR LED array 702 may also be substituted by a reflective tape.
  • a band-pass filter that selectively transmits radiation of the pre-determined or specified wavelength may be attached to the lens of the camera 112 to improve the signal-to-noise ratio of the captured image. This is because only radiation of the pre-determined or specified wavelength is selectively captured by the sensor of the camera 112 and background radiation or radiative emissions are substantially suppressed.
  • the use of a band-pass filter may reduce the exposure time required for the camera 112 to capture an image of sufficient quality.
  • the band pass filter transmits 850 nm emission that can be detected by the camera 112 which is sensitive at near-IR.
  • a camera 112 with a sensor that is sensitive to the IR may also be used.
  • a non-exclusive example is Indium-Gallium-Arsenide (InGaAs) sensor.
  • the leader When operating in convoy mode, the leader may be a robot rather than the human companion 704. In various embodiments, the robot may also be remotely control.
  • the leader and at least one follower may be operable to use wireless positioning technologies such as WIFI triangulation, Base Transceiver Station (BTS) cellular triangulation or GPS to determine its own positional information. Thereafter, the leader is operable to broadcast or communicate its positional information to the followers and the at least one followers is operable to respond accordingly to positional change of the leader by sending electrical command or control signal to the rear wheels 08. In this case, the leader may broadcast or communicate its positional information by way of any known wireless technologies such as Bluetooth, WIFI or cellular.
  • the apparatus 102 may be equipped with further sensors for environmental perception to negotiate the surrounding environment and prevent collisions without human intervention.
  • the apparatus 102 may be equipped with a flashing warning light for alerting people nearby.
  • the apparatus 102 may also have an emergency stop and hand brake to over-ride the autonomous mode.
  • the system 700 may be applied to a convoy of vehicles, where a lead vehicle or person (i.e. the leader) is provided with a marker to effect remote navigation of the following vehicles, each following vehicle provided with an image capturing device.
  • a lead vehicle or person i.e. the leader
  • each following vehicle may be provided with their unique marker(s).
  • Non-limiting examples of the vehicle may be a baggage cart or trolley, an automobile, a train or any transportation means.
  • the force or pressure sensor for detecting compressive load may also be known as a compression load cell.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • General Physics & Mathematics (AREA)
  • Automation & Control Theory (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Animal Behavior & Ethology (AREA)
  • Electromagnetism (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Lighting Device Outwards From Vehicle And Optical Signal (AREA)

Abstract

The present invention relates to a system, method and apparatus for navigating one or more vehicles. In particular, but not exclusively, the system, method and apparatus are suitable for convoying a plurality of wheelchairs. The system for navigating a first vehicle to follow a leader comprising a first marker that is attached to the leader, a first image capturing device attached to the first vehicle operable to capture a first image with the first marker, and a first processor operable to process the first image captured by the first image capturing device to determine a first property of the first marker in the first image and send a first command signal to move the first vehicle based on at least the property of the first marker in the first image.

Description

SYSTEM, METHOD AND APPARTUS FOR NAVIGATING ONE OR MORE
VEHICLES
Field of Invention
The present invention relates to a system, method and apparatus for navigating one or more vehicles. In particular, but not exclusively, the system, method and apparatus are suitable for convoying a plurality of wheelchairs.
Background Art
The following discussion of the background to the invention is intended to facilitate an understanding of the present invention. However, it should be appreciated that the discussion is not an acknowledgment or admission that any of the material referred to was published, known or part of the common general knowledge in any jurisdiction as at the priority date of the application.
Power-driven or motorized wheelchair is advantageous for navigating across terrains that would be physically fatiguing in a manual wheelchair. One example would be at the airport, where a disabled or elderly passenger in a wheelchair would have to travel a long distance with carry-on luggage from the plane and passenger loading bridge to the transit area or immigration counter. In this tiring situation, having access to power-driven wheelchairs would be preferable. Unfortunately, power-driven wheelchairs are significantly more expensive and the use of manual wheelchairs is more widespread among passengers. It is generally understood that as part of the flying experience, a passenger requiring a wheelchair must be provided with one in a timely manner at the airport and in some countries, this is mandatory under law. In some countries, a prevalent solution at the airport for passengers in manual wheelchairs is the assignment of caretakers to help push them through the airport. However, as many as 150 wheelchairs and caretakers are usually requested by passengers in a typical landing or taking-off flight. This results in an increase in the demand for manpower at the airport leading to a higher labour cost.
A further problem that passengers with wheelchairs usually encounter in navigating across the airport is unfamiliarity with the route. Consequently, manpower is necessary to provide adequate guidance in navigating through the airport. Yet another problem associated with navigating a group of wheelchairs in the airport is the lack of coordination, resulting in disorderly or disorganized movement of a group of wheelchairs that may be a safety hazard to other pedestrians.
There are also situations other than those used with wheelchairs wherein the above- stated problems are also encountered. For example, in moving a broken down car back to the workshop and in moving hospital beds from one location of the hospital to another without relying on extra caretakers
In view of the above, there exists a need for a better solution to ameliorate the aforementioned problems at least through providing autonomous navigation and/or convoying wheelchairs or vehicles. Summary of the Invention
This invention seeks to autonomously navigate a vehicle following a leader. In accordance with a first aspect of the invention there is a system for navigating a first vehicle to follow a leader comprising:
a first marker that is attached to the leader;
a first image capturing device attached to the first vehicle, the first image capturing device operable to capture a first image of the first marker; and
a first processor operable to process the first image captured by the first image capturing device to determine a first property of the first marker in the first image and send a first command signal to move the first vehicle based on at least the property of the first marker in the first image.
Preferably, the first image capturing device is a silicon camera with infra-red filter removed.
Preferably, the first marker is an infra-red LED array.
Preferably, the first image is captured by using a band-pass filter and the band-pass filter is operable to selectively transmit radiation of a specified wavelength.
Preferably, the first marker is an array emitting according to the specified wavelength of the band-pass filter.
Preferably, the first property is any one of the following: the size, degree of rotation, intensity and location of the marker in the first image. Preferably, the first command signal is sent to a first electric motor of the first vehicle.
Preferably, the first electric motor controls the speed of the first vehicle and/or direction of the first vehicle based on differential steering.
Preferably, the first processor is further operable to determine a first blinking frequency of the first marker by way of the first image and a second image and send a second command signal to move the first vehicle at a speed based on at least the first blinking frequency.
Preferably, the first vehicle is always maintained at a first positional coordinate that is inclined at an angle relative to a direction of navigation of the leader.
Preferably, the first marker comprises a first circular shaped pattern and a second circular shaped pattern wherein the first and second circular shaped patterns are arranged laterally beside each other.
Preferably, the first circular shaped pattern has a first and a second circular shaped pattern wherein the first and second circular shaped patterns are arranged laterally beside each other.
Preferably, the first circular shaped pattern and the second circular shaped pattern are LED arrays with the same emission wavelength.
Preferably, the first vehicle is a wheelchair and the leader is either a wheelchair or a human or a robot.
Preferably, the system further comprises a first controller that is operable to stop the first image capturing device, and is further operable to send a third command signal by way of the first processor to move the first vehicle based on at least a first input into the first controller.
Preferably, the first controller communicates with the first processor by way of ZigBee communication protocol.
Preferably, the first processor is further operable to communicate with a light detection and ranging system or an ultrasonic system to determine a movement speed of the leader and move the vehicle at the movement speed.
Preferably, the system is further for used navigating a second vehicle to follow the leader further comprising: a second marker that is attached to the first vehicle; a second image capturing device attached to the second vehicle operable to capture a second image with the second marker; and a second processor operable to process the second image captured by the second image capturing device to determine a second property of the second marker in the second image and send a fourth command signal to move the second vehicle based on at least the property of the second marker in the second image.
In accordance with a second aspect of the invention, there is a method for navigating a first vehicle to follow a leader comprising the steps of: determining, by the first vehicle, a direction of movement of the leader, a movement speed of the leader and a position of the leader relative to the first vehicle; and sending a first command signal to move the first vehicle based on at least the direction of movement of the leader, the movement speed of the leader and the position of the leader relative to the vehicle. Preferably, the step of determining the direction of movement of the leader, the movement speed of the leader and the position of the leader relative to the first vehicle is based on any one of the following: processing a first image to determine a first property of a first marker in the first image or using a light detection and ranging system or using an ultrasonic system. Preferably, the first image is captured by a first image capturing device with infra-red filter removed.
Preferably, the first marker is an infra-red LED array attached to the leader.
Preferably, the first image is captured with a band-pass filter, and the band-pass filter is operable to selectively transmit radiation of a specified wavelength. Preferably, the first marker is due to an LED array emitting according to the specified wavelength of the band-pass filter.
Preferably, the first property of the first marker in the first image is one of the following: the size, degree of rotation, intensity and location of the marker in the image. Preferably, the first command signal is sent to a first electric motor of the first vehicle. Preferably, the first electric motor control the speed of the first vehicle and/or the direction of the first vehicle based on differential steering.
Preferably, the method further comprising the step of determine a first blinking frequency of a first marker and sending the first command signal to move the first vehicle at a speed based on at least the first blinking frequency of the first marker.
Preferably, the method further comprising the step of maintaining the first vehicle at a first positional coordinate relative to the leader.
Preferably, the method further comprising the steps of: stopping the operation of the first image capturing device; and sending a third command signal to move the first vehicle based on at least a first input into a first controller.
Preferably, first controller communicates by way of ZigBee communication protocol.
Preferably, the first vehicle is a wheelchair and the leader is either a wheelchair or a human. Preferably, the first marker comprises a first circular shaped pattern and a second circular shaped pattern wherein the first and second circular shaped patterns are arranged laterally beside each other.
Preferably, the first circular shaped pattern has a first and a second circular shaped pattern wherein the first and second circular shaped patterns are arranged laterally beside each other.
Preferably, the first circular shaped pattern and the second circular shaped pattern are LED arrays with the same emission wavelength.
Preferably, the method further used for navigating a second vehicle to follow the leader further comprising the steps of: determining, by the second vehicle, a direction of movement of the first vehicle, a movement speed of the first vehicle and a position of the first vehicle relative to the second vehicle; and sending a second command signal to move the second vehicle based on at least the direction of movement of the first vehicle, the movement speed of the first vehicle and the position of the first vehicle relative to the second vehicle.
Preferably, the method further comprising the step of:
maintaining the first vehicle at a first positional coordinate that is inclined at an angle relative to a direction of navigation of the leader.
Preferably, the method further comprising the steps of:
stopping the operation of the second image capturing device; and
sending a fourth command signal to move the second vehicle based on at least a second input into a second controller.
In accordance with a third aspect of the invention, there is an apparatus for navigating a first vehicle to follow a leader comprising:
a latching device for attaching the apparatus to a coupling frame attached to the first vehicle;
a motor for driving a wheel;
an image capturing device operable to capture an image with a marker, wherein the marker is attached to the leader; and
a processor operable to process the image captured by the image capturing device to determine a property of the marker in the image and send a command signal to the motor based on at least the property of the marker in the image to drive the wheel.
Preferably, the latching device comprises of an insertion pin that is magnetically or mechanically actuated.
Preferably, the insertion pin is magnetically or mechanically actuated to slot into a pin hole located in the coupling frame of the first vehicle.
Preferably, the apparatus further comprising a contact sensor for triggering the magnetically or mechanically actuated insertion pin when contact on the coupling frame is detected. Preferably, the latching device comprises of a chamfer that is magnetically or mechanically actuated.
Preferably, the chamfer is magnetically or mechanically actuated into a circlip located in the coupling frame when contact on the coupling frame is detected. Preferably, the apparatus comprises a top portion releasably attachable to a bottom portion, wherein the top portion is detached from the bottom portion and the bottom portion is releasably attachable to a steering unit for conversion to an electric scooter.
Preferably, the steering unit comprises either an actuator for steering the electric scooter. The actuator may be in the form of a push button or joystick.
Other aspects of the invention will become apparent to those of ordinary skilled in the art upon review of the following description of specific embodiments of the invention in conjunction with the accompanying figures.
Brief Description of the Drawings The present invention will now be described, by way of example only, with reference to the accompanying drawings, in which:
Fig. 1 shows an apparatus in accordance with an embodiment of the invention attached to a wheelchair.
Fig. 2 shows the apparatus in various perspective views: (a) front view, (b) isometric view, (c) side view and (d) enlarged or zoomed in isometric view.
Fig. 3 shows the side views of the apparatus in various states; (a) when the latching mechanism is not activated; (b) when the latching mechanism is activated; and (c) when the latching mechanism is not activated and the apparatus is separable into two portions. Fig. 4 shows the coupling frame attached to the wheelchair in various views; (a) front view with the wheelchair and coupling frame folded (a first state); (b) front view with the wheelchair and coupling frame un-folded (a second state); and (c) side view with the wheelchair and coupling frame un-folded. Fig. 5 shows a close up view of the coupling frame; (a) front cross-section view of the coupling frame attached to the wheelchair and (b) side view of the coupling frame attached to the side rod bar wheelchair.
Fig. 6 shows the coupling between the wheelchair and the apparatus via the latching mechanism on the apparatus and the coupling frame attached to the wheelchair.
Fig. 7 shows a system for remotely navigating a wheelchair, wherein the apparatus interacts or communicates with a human companion via either a controller and/or an IR LED array pattern.
Fig. 8 shows an embodiment of the invention in which the IR LED array pattern is located at the back of the wheelchair in front.
Fig. 9 shows an embodiment of the invention in which a convoy of wheelchairs escorted by a human companion.
Fig. 10 illustrates two possible configurations in which one or more wheelchairs may navigate behind the human companion: (a) top-view of one or more wheelchairs navigate at an angle with respect to the human companion 704 and (b) top view of one or more wheelchairs navigate directly behind the human companion.
Fig. 1 1 shows an alternative pattern for the IR LED array comprising of two circular shaped patterns with the same emission wavelength (the two patterns may also have different emission wavelengths) arranged at opposite sides of an imaginary middle axis 1 106 on the back of the battery-powered vest 706.
Fig. 12 shows schematic drawings of (a) the apparatus with the mast removed and a joystick control that can communicate with the apparatus to effect navigation under the local control mode and (b) the mounting of the apparatus and the joystick on a wheelchair to move or transport the wheelchair by local control mode. Fig. 13 shows schematic drawings of (a) steering unit (an interchangeable top base with steering rod) and a modified version of the apparatus with the top part removed and (b) the mounting of the interchangeable top base to the modified apparatus to form an electric scooter. Description of Embodiments of the Invention
Throughout the specification, unless the context requires otherwise, the word "comprise" or variations such as "comprises" or "comprising, will be understood to imply the inclusion of a stated integer or group of integers but not the exclusion of any other integer or group of integers.
Furthermore, throughout the specification, unless the context requires otherwise, the word "include" or variations such as "includes" or "including" will be understood to imply the inclusion of a stated integer or group of integers but not the exclusion of any other integer or group of integers.
Additionally, the "X" direction is defined as the direction that is substantially perpendicular to the axle of the wheelchair (parallel to the direction of motion of the wheelchair) and the "Y" direction is defined to be substantially parallel to the axle of the wheelchair. Further, the X- and Y- axes are substantially parallel to the horizontal plane. The "right" and "left" sides are construed accordingly with respect to a person sitting on the wheelchair facing the front; the "right" side would correspond to the right side of the person and the "left" side corresponds to the left side of the person. "Up" and "down" will be construed with respect to the vertical axis.
In accordance with various embodiments of the invention as shown in Fig. 1 and Fig. 2, there comprises an apparatus 102 for moving a wheelchair 104. The apparatus 102 may be attachable to the wheelchair 104 via a securing means. The securing means may be a coupling mechanism that comprises a first part that is disposed or affixed to a portion of the apparatus 102 and a second part that is disposed or affixed to a portion of the wheelchair 104. The first part of the coupling mechanism may be a latching device 302 as shown in Fig. 3 and the second part of the coupling mechanism may be a coupling frame 402 as shown in Fig. 4 and Fig. 5. The latching device 302 and coupling frame 402 shall be described sequentially below.
As shown in Fig. 3, the latching device 302 may comprise a permanent magnet 306 and an insertion pin 308. In various embodiments, the permanent magnet 306 may be attached near to the distal end (or the end further away from the apparatus 102) of the insertion pin 308. The latching device 302 may have two states of operation. In the first or un-activated state as shown in Fig. 3 (a), the permanent magnet 306 and the insertion pin 308 are retracted or withdrawn into a recess 310 under a cover 304. In various embodiments, the recess 310 may be shaped according to the outline or contour of the composite body comprising of the permanent magnet 306 and the insertion pin 308 as shown in Fig. 3 (a). In various embodiments, the latching device 302 may only comprise of the insertion pin 308 without the permanent magnet 306.
In various embodiments, the cover 304 may be operable to act as a trigger for the latching device 302. When the cover 304 is lifted as shown in Fig. 3 (b), the cover 304 is in communication with a contact sensor (not shown) such that when the cover 304 is contacted by the second part of the coupling mechanism (coupling frame 402), the contact is sensed by the contact sensor and the latching device 302 may be triggered into the second or activated state. In the second or activated state as shown in Fig. 3 (b), the insertion pin 308 may be magnetically actuated upwards along its longitudinal axis, wherein the magnetic actuation may be achieved at least by way of the permanent magnet 306 and/or an electromagnet (not shown) that is located near the recess 310. In various embodiments, the electromagnet may be a solenoid. In this case, a control signal may be sent to the electromagnet to produce or generate a magnetic field to repel the permanent magnet 306 when the contact sensor detects or senses any contact with the cover 304. The repel force, due to the interaction between the magnetic field of the permanent magnet 306 and the magnetic field of the electromagnet, may actuate the insertion pin 308 upwards from the recess 310 into the second state. In various embodiments, the insertion pin 308 may be magnetised and the magnetic actuation may be due to the interaction between the magnetic field of the magnetised insertion pin 308 and the magnetic field of the electromagnet. In various embodiments, the insertion pin 308 may also be mechanically actuated upwards. In various embodiments, the insertion pin 308, when actuated upwards, may be slotted into a pin hole 508 located on the second part of the coupling mechanism or coupling frame 402 as shown partly in Fig. 6. In various embodiments, the permanent magnet 306 may also operable to be attracted or engaged to the coupling frame 402. In various embodiments, the latching device 302 may comprise a chamfer (not shown) that is operable to couple with a circlip located on the coupling frame 402. In this case, the chamfer may be mechanically or magnetically actuated upwards when the cover 304 is contacted by the second part of the coupling frame 402. Thereafter, the mechanically or magnetically actuated chamfer may be coupled with a circlip located on the coupling frame 402 for securing the apparatus 102 to the wheelchair 104.
In various embodiments as shown in Fig. 5 (a), the coupling frame 402 may comprise two flat bars 502 that are attached to each other at their respective proximal ends via a hinge joint 504. The hinge joint 504 allows the coupling frame 402 to be folded when the wheelchair 104 is folded as shown in Fig. 4 (a). In various embodiments as shown in Fig. 5, the respective distal ends of the flat bars 502 may be attached to a securing means, wherein the securing means is for attachment to the corresponding wheelchair side rod bars 404. The securing means may be a sleeve clamp 510 that is operable to be coupled onto the circumference or outer surface of the wheelchair side rod bar 404 as shown in Fig. 5. Through a receiving means 511 of the sleeve clamp 510, a thumb-screw 512 and nut 514 may be used to secure the sleeve clamp 510 on the wheelchair side rod bar 404 by screwing the thumb-screw 512 against the nut 514 to effect clamping. Advantageously, the installation of the coupling frame 402 via the wheelchair side rode bar 404 would not substantially affect the basic functionality of the wheelchair 04. Further, minimal modification is required in the installation of the coupling frame 402.
In various embodiments, the sleeve clamp 510 may be loosen through the thumb- screw 512 and nut 514 to allow the sleeve clamp 510 rotate about the wheelchair side rod bar 404 when the wheelchair 104 is being folded or unfolded. In various embodiments, the distal ends of the flat bars 502 may be rotatably attached to the sleeve clamp 510 such that the sleeve clamp 510 does not need to be loosen when the wheelchair 104 is being folded or unfolded. Referring back to the apparatus 102 as shown in Fig. 2, the apparatus 102 may further comprise at least one front wheel 106 and two rear wheels 108. In various embodiments, the two rear wheels 108 of the apparatus may be normal wheels and the front wheel 106 of the apparatus 102 may be a double-row transwheel. Herein, the normal wheels may be bi-directional and transwheel may be multi-directional. The rear wheels 108 may be driven by at least one electric motor (not shown). In various embodiments, there may be more than two rear wheels. In various embodiments, each rear wheel 108 may be independently driven by an electric motor so as to steer the wheelchair 104 using differential steering in which one of the rear wheels 108 is rotating faster than the other to effect steering. Accordingly, the front wheel 106 which may be a double-row transwheel may be operable to accommodate movement in different directions as determined by differential steering.
In various embodiments, the apparatus 102 may further comprise a mast 1 10, wherein the mast 1 10 may be a vertical rod or pole as illustrated in Fig. 2. The mast 1 10 may be releasably attached to the apparatus 102 such that the longitudinal axis of the mast 1 10 is substantially aligned with the vertical direction. In various embodiments, the mast 1 10 may contain an image capturing device or camera 1 12 and a warning light 114 which are attached near the top of the mast 1 10. Preferably, the camera 1 12 may be a NoIR camera (i.e. a conventional camera with an infra-red (IR) filter removed) that is more sensitive to IR radiation. In various embodiments, the camera 112 may also be omni-directional. The camera 1 12 is operable to capture images real-time related to the surrounding with increased or enhanced sensitivity to IR radiation within its field of view.
In various embodiments as shown in Fig. 7, there is a system 700 for convoying or platooning a wheelchair 104 to follow a human companion 704 comprising the apparatus 102 which may be in communication with a master remote controller 708 (a first remote controller) and a client remote controller 710 (a second remote controller). The system 700 may be useful for remotely navigating a wheelchair 104 at an airport to assist a passenger in wheelchair get from one place to another. As shown in Fig. 7, the master remote controller 708 may be hand-held by a human companion 704 (also known as the leader) and the client remote control 710 may either be hand-held by the user on the wheelchair 104 or installed on the wheelchair 104. In various embodiments, the communication between the master remote controller 708, the client remote controller 710 and the apparatus 102 may be achieved by Bluetooth or low energy Bluetooth or ZigBee communication protocol. In various embodiments of the system 700, the master remote controller 708 and/or the client remote controller 710 may be used to select the type of operation mode. Depending on the selected operation mode, the apparatus 102 may be able navigate the wheelchair 104 either remotely or locally. The six available operation modes that will be described in the later part of the description are namely 1 ) self-navigation or autonomous mode, 2) convoy mode, 3) manual mode, 4) remote control mode, 5) local control mode and 6) electric scooter mode. In various embodiments, the system 700 may also comprise an infra-red (IR) light emitting diode (LED) array 702 that is located on the back of a human companion 704 by way of a battery-powered vest 706 that is worn by the human companion 704. The camera 112 may then capture real-time image containing at least the IR LED pattern 701 that may be due to IR radiation emitted from an IR LED array 702. Thereafter, the processor may then communicate, in real-time, the captured realtime image to a processor (not shown). Using an image recognition software that may be pre-loaded in the processor, the captured real-time image may then be processed upon receipt to detect or recognize for the IR LED pattern 701 . In this case, the IR LED array 702 is arranged substantially similar to the IR LED pattern 701 using discrete IR LEDs. As will be further described below, the detected or recognized IR LED pattern 701 provides a visual marker or a dedicated external reference point to effect self-navigation or convoying which may enable one or more wheelchairs to be led or escorted by a human companion 704 wearing the battery- powered vest 706 with the IR LED pattern 701 . Providing or attaching the IR LED array 702 that has a pre-determined pattern to the human companion 704 is advantageous in a crowded area because the pre-determined pattern acts as a marker to allow the human companion 704 be easily distinguished from the crowd as compared to existing technology relying solely on laser range sensors to detect the change in position of the human companion 704. An inherent drawback of the foregoing is that it would be challenging for a laser range sensor to distinguish between two or more humans that are present within the field of view of the laser range sensor. The drawback associated with the prior art is further compounded by physical objects that are present within the field of view of range of the laser range sensors. Furthermore, the use of IR instead of visible light for the LED array 702 has at least two further technical advantages: 1 ) IR is invisible to the human eye and hence will not be a distraction to other pedestrians; and 2) IR LED array 702 may also be discriminated or distinguished from visible light through the use of a filter to enhance the signal to noise ratio of the captured image which would advantageously allow the IR LED array to be low-powered. It may be appreciated that street lamps/lightings and typical signboards emit strongly in the visible spectrum for practical reasons, making the discrimination or recognition of a visible LED array more challenging. Upon detecting or recognizing the IR LED pattern 701 in the captured real-time image, the processor may be further operable to translate the recognized real-time IR LED pattern 701 into a set of real-time positional information of the human companion 704 relative to the wheelchair 104 using either the image recognition software or a separate software. The foregoing is achieved by determining at least one property of the IR LED pattern 701. The at least one property may be, but not limited to, any one of the following: the size, degree of rotation, intensity and location of the captured IR LED pattern 701 in the captured image frame. In various embodiments, the relative distance between the wheelchair 104 and the human companion 704 may be calculated or determined using at least the focal length of the camera lens, the real height of the IR LED pattern 701 , the pixel height or size of the captured image, the pixel height of the IR LED pattern 701 and the real height or vertical dimension of the sensor in the camera.
In various embodiments, the degree of rotation of the IR LED pattern 701 about the vertical axis in the captured image may also translate to the degree of rotation of the human companion 704 about his/her vertical axis (or turning angle of the human companion 704) as the IR LED array 702 may lie substantially flat on the back of the human companion 704. The degree of rotation about the vertical axis may then provide information relating to the direction in which the human companion 704 is heading. In addition, the gradient in light intensity of the captured IR LED pattern 701 may also provide information relating to the degree of rotation of the IR LED pattern 701 as the part of the IR LED pattern 701 that is located further away would have relatively lower light intensity and the gradient in light intensity across the IR LED pattern 701 may be translated to the degree of rotation of the human companion 704.
In various embodiments, the location or vertical and horizontal positions of the IR LED pattern 701 in the captured real-time image frame may also provide positional information of the human companion 704. If the IR LED pattern 701 is detected or recognized at a location substantially towards the right of the image, then the processor or software would be able to determine that the human companion 704 is located towards the right side of the wheelchair 104 and calculate the relative distance or displacement of the human companion 704 accordingly.
In various embodiments, the determination of the real-time positional information such as distance of the human companion 704 relative to the wheelchair 104 may allow the processor to track the movement direction and/or speed of the human companion 704 by processing a series of consecutively captured real-time images. For example, the increase in relative distance between the human companion 704 and the wheelchair 104 may indicate that the human companion 704 is moving faster, hence the speed of the wheelchair has to increase for maintain a fixed relative distance between the human companion 704 and the wheelchair 104. The processor may respond to follow the change in direction and/or speed of the human companion 704 by sending an electrical command or control signals to one or more electric motors of each rear wheel 108. In response, the electrical command or control signal may control the speed of wheelchair 104 and steer the wheelchair 104 in different directions through differential steering of the rear wheels 108 as discussed above.
In various embodiments, a Light Detection and Ranging (LIDAR) or an ultrasonic sensor system may also be used independently or in combination of the camera 1 12 to determine the relative distance between the wheelchair 104 and the human companion 704 and/or to perform three dimensional mapping and localization. The movement speed of the human companion 704 may also be determined directly or indirectly (by processing time series of the relative distance between the wheelchair 104 and the human companion 704) by the LIDAR or an ultrasonic system. In various embodiments, the LIDAR or ultrasonic system may be attached on the mast 1 10 near to the camera 112 and may communicate with the processor.
In various embodiments, stopping of the wheelchair 104 may be achieved when it is determined that there is no change in the real-time position of the human companion 704, i.e. the human companion 704 has stopped moving and is remaining at the same location. In this case, the processor may be operable to stop the one or more electric motors of each rear wheel 108. In various embodiments, stopping of the wheelchair 104 may also be achieved when the human companion 704 turns off the IR LED array 702. When the processor fails to detect any IR LED pattern 701 in the captured image, it is operable to stop the one or more electric motors of each rear wheel 108. In various embodiments, the processor may slow down the one or more electric motors gradually before coming to a complete stop. In various embodiments, the one or more electric motors may come to a complete stop quickly. The foregoing feature may be useful to effect emergency stop of the wheelchair 104 during autonomous operation.
In various embodiments, the flashing or blinking modes of the IR LED array 702 may also be used to move the wheelchairs 104 according to the speed of the human companion 704. For example, the movement speed of the human companion 704 may correspond to a frequency of flashing or blinking of the IR LED array 702. In this case, the processor on the apparatus 102 may be operable to determine the frequency of flashing or blinking of the IR LED array 702 from a succession of captured images acquired or captured with a known camera frame-rate and send the appropriate control signal to the one or more electric motors accordingly. In various embodiments, the camera 1 2 may also be a video camera or may operate in video camera mode for detecting dynamic changes (i.e. blinking frequencies) of the IR LED array 702. In various embodiments, the IR LED array 702 may be connected to a speed sensor or detector (not shown) that is attached to the battery-powered vest 706 wherein the speed sensor is operable to detect the movement speed of the human companion 704. In various embodiments, the speed sensor may detect movement speed of the human companion 704 using Base Transceiver Station (BTS) cellular triangulation or global positioning system (GPS) based methods. In various embodiments, the movement speed of the human companion 704 may also be estimated by way of a pedometer. Thereafter, the movement speed of the human companion 704 as determined or measured by the speed sensor may then then be translated into a flashing or blinking frequency of the IR LED array 702 for guiding or escorting the wheelchair 104.
In various embodiments, the IR LED array 702 may also be located at the back of another wheelchair as shown in Fig. 8 such that the back or follower wheelchair may rely on the IR LED array 702 located at the back of the front or leader wheelchair for navigation rather than the IR LED array 702 located at the back of the human companion 704. The foregoing feature would be useful in coordinating the simultaneous movement of two or more wheelchairs. In various embodiments, the two or more wheelchairs may also move as a convoy behind a human companion 704 as shown in Fig. 9. In various embodiments, a first wheelchair 902 may be programmed via the apparatus 102 to track and follow the front human companion 704 while maintaining a safe distance. The second wheelchair 904 may be programmed via its apparatus 102 to follow the first wheelchair by tracking the IR LED array 702 located at the back of the first wheelchair 902 while maintaining a safe distance. Similarly, the third wheelchair 906 may be programmed to follow the second wheelchair 904 by way of the IR LED array 702 located at the back of the second wheelchair 904.
In various embodiments as shown in Fig. 9 and Fig. 10 (b), there is a system 900 comprising wheelchairs 902, 904 and 906 that may be programmed to travel in a substantially straight line (i.e. single file formation) directly behind the human companion 704. The straight line arrangement or single file formation is advantageous in parts of the airports where space is limited especially at the gate or the security. In various embodiments as illustrated in Fig. 10 (a), the following wheelchairs 902, 904 and 906 may also be programmed to travel in a path without being directly behind the human companion 704. The foregoing configuration or convoy formation would be advantageous to avoid collision between the first wheelchair 902 and the human companion 704 if the first wheelchair 902 cannot stop in time. In various embodiments as shown in Fig. 3 (c), the apparatus 102 may comprise three units: a control unit 312 (a first portion), a battery unit 3 4 (a second portion) and the mast 110 (a third portion). The three units may be connected or assembled together via the respective electrical interfaces (not shown). In various embodiments, the control unit 312 may comprise the processor. In various embodiments, the battery unit 314 may comprise a battery and at least one motor for driving the wheels. The electrical interfaces allows electrical signal to be exchanged between the three units. The key technical advantage of a modular design is to allow the apparatus 102 to be modified to cater to different needs which will be described in greater detail below.
The present invention will now be described in relation to its methods of operation or operation modes. When on-demand power driving and/or autonomous navigation capability is to be provided to a wheelchair 104, the apparatus 102 as described above is rolled underneath the wheelchair 104 from the back of the wheelchair 104 as shown in Fig. 6. When the flat bars 502 of the coupling frame 402 contacts or touches the cover 304 as the apparatus 102 is rolled underneath the wheelchair 104, the latching device 302 is activated to the second state and the insertion pins 308 are magnetically or mechanically actuated upwards along its longitudinal axis and are slotted into the pin hole 508 located in the flat bars 502 of the coupling frame 402. The permanent magnet 306 may also be attracted to the flat bars 502 to secure the coupling between the coupling frame 402 and the latching device 302. Alternatively, a circlip and chamfer combination may also be used to secure the coupling frame 402 and the latching device 302.
Thereafter, the human companion 704 may select the operation mode using the master remote controller 708. There are at least six available operation modes are: 1 ) self-navigation mode, 2) convoy mode, 3) manual mode, 4) remote control mode, 5) local control mode and 6) electric scooter mode. However, only the first five operation modes (excluding the electric scooter mode) may be selected when the apparatus 102 is attached to the wheelchair 104.
1) Self-navigation or autonomous mode:
In the self-navigation or autonomous mode, one wheelchair is able to move autonomously by being guided or escorted by a human companion 704 who is wearing the battery-powered vest 706 with the IR LED array 702. Once the self- navigation or autonomous mode is activated, the NoIR camera 1 12 of the apparatus 102 begins to capture real-time images. The processor, with the help of a pre-loaded software, is operable to recognize and track the IR LED pattern 701 in the image that is emitted from an IR LED array 702 from the back of the human companion 704. In various embodiments, the detected IR LED pattern 701 may then be translated into positional information of the human companion 704 relative to the wheelchair 104 based on at least the size, degree of rotation, intensity and location of the captured IR LED pattern 701. The positional information of the human companion 704 enables the apparatus 102 to initialize the wheelchair 104 to an initial position behind the human companion 704.
For example during the initialization phase, the apparatus 102 may try to align the wheelchair 104 substantially directly behind the human companion 704. In various embodiments, if the processor detects that the IR LED pattern 701 is located substantially towards the right of the captured image, electrical command or control signal may be sent from the processor to the rear wheels 108 to steer the wheelchair 104 towards the right or rotate the wheelchair 104 clockwise such that the IR LED pattern 701 in subsequent captured images becomes substantially centered within the frame of the images. After the wheelchair 104 is substantially aligned directly behind the human companion 704, the apparatus 102 may further move the wheelchair 104 forward or backward to a pre-determined safe distance based on the positional information of the human companion 704 that is determined from the captured real-time images. In various embodiments, the relative distance between the human companion 704 and the wheelchair 104 may be determined using any one of the following: processing the captured image to determine a property of the IR LED pattern 701 in the captured image or using a light detection and ranging system or using an ultrasonic system. During the operation phase, the human companion 704 starts to move and the processor in the apparatus 102 begins to determine the change in the positional information of the human companion 704 over a time period based on the captured image (for example from a first image captured at t=0 seconds and a second image captured at t=1 seconds). The time period is inversely proportional to the frame rate or frame frequency of the camera 1 12. In various embodiments, the apparatus 102 may determine the change in the positional information of the human companion 704 over a time period based on a blinking or flashing frequency of the IR LED pattern 701 as described above. In various embodiments, the apparatus 102 may determine the change in the positional information of the human companion 704 over a time period using either a LIDAR or ultrasonic system.
The processor is further operable to respond to this positional change of the human companion 704 during this time period by sending electrical command or control signal to the rear wheels 108 of the apparatus 102 to move and steer the wheelchair 104 according to the direction of movement, speed of the human companion 704 and position of the human companion 704 relative to the wheelchair 104 while still maintaining a safe distance behind the human companion 704. The safe distance may translate to a physical positional coordinate relative to the human companion 704 and the wheelchair should be located at this positional coordinate to prevent collision with the human companion 704. For example and as illustrated in Fig. 10, the safe positional coordinate may be defined in Cartesian form (X, Y) where the X- axis is substantially perpendicular to the longitudinal axis of the axle of the wheelchair and Y-axis is substantially parallel with the longitudinal axis of the axle of the wheelchair and both the X- and Y-axes are parallel to the horizontal plane and the origin (0, 0) is the position of the human companion 704; a positional coordinate of (-1 , 0) as shown in Fig. 10 (b) indicates that the wheelchair 104 should always self-navigate or move autonomously at a distance of one meter substantially directly behind the human companion 704.
In various embodiments as illustrated in Fig. 10 (a), the wheelchair 104 may also initialize and navigate at an angle with respect to the human companion 704. For example, the positional coordinate of the wheelchair 104 may be (-1 , -1 ) whereby the wheelchair 104 would self-navigate at a distance of one meter behind and one meter to the left side (at 45 degrees angle) of the human companion 704. As the wheelchair 104 is maintained at a positional coordinate that is inclined at an angle relative to the direction of navigation or movement of the human companion 704, collision between the wheelchair 104 and the human companion 704 may be avoided if the wheelchair is unable to stop in time when the human companion 704 has stopped moving. In various embodiments, the positional coordinate may be defined according to polar coordinates (r, theta).
In various embodiments, the time period for detecting change in positional information of the human companion 704 relative to the wheelchair 104 may affect the response time of the apparatus 102 and hence may be changed according to the requirements of the user. In various embodiments, reducing the time period or increasing the frame rate shortens the response time as the apparatus 102 may be able to react faster as the positional information of the human companion 704 is updated more frequently, resulting in higher self-navigation accuracy. On the contrary, increasing the time period or reducing the frame rate may increase the response time of the apparatus 102, leading to the need to maintain the wheelchair 104 at a longer safety distance away from the human companion 704 to avoid collision. In various embodiments time period may also be constrained by the exposure time required by the camera 1 12 to achieve an image with sufficient quality for image recognition and processing. Consequently, the use of brighter IR LED array 702 may help reduce the exposure time required and allows images to be taken in quicker succession (reduced time period and reduced response time).
2) Convoy mode: In the convoy mode as shown in Fig. 9, two or more wheelchairs are guided and escorted by a human companion 704 who is wearing the battery-powered vest 706 with the IR LED array 702. To navigate in the convoy mode, the users of the two or more wheelchairs may first simultaneously select the convoy mode option via the respective client remote controllers 710. In various embodiments, the human companion 704 may also group the two or more wheelchairs into a convoy using the master remote controller 708.
During the initialization phase, the wheelchairs may move to its respective uniquely assigned positional coordinates behind either the human companion 704 or another wheelchair in a similar manner as described in the self-navigation mode. The unique positional coordinates help prevent collision from occurring between the wheelchairs in the convoy and may be assigned in at least two ways as described below.
In various embodiment, the respective apparatus 102 of each wheelchair may be programmed to self-navigate at a pre-determined unique positional coordinate behind the human companion 704. This would be similar to the self-navigation of one wheelchair as discussed above except that each wheelchair in the convoy has a unique positional coordinate with respect to the human companion 704. A possible drawback of this method of operation is that the wheelchairs further away from the human companion 704 may have a challenge capturing a reliable image of the IR LED pattern 701 due to the IR LED array 702 located at the back of the human companion 704.
In various embodiment as shown in Fig. 9, a first wheelchair 902 may be assigned within the convoy to track and follow the front human companion 704 while maintaining a safe distance or at a positional coordinate behind the human companion 704. The second wheelchair 904 may be assigned to follow the first wheelchair 902 by tracking the IR LED pattern 701 located at the back of the first wheelchair 902 while maintaining a safe distance or at a positional coordinate behind (or with respect to) the first wheelchair 902. Similarly, the third wheelchair 906 may be assigned to follow the second wheelchair 904 by way of the IR LED pattern 701 located at the back of the second follower wheelchair 904.
During the operation phase when the human companion 704 starts moving, the wheelchairs would maintain in its unique positional coordinate with respect to either the human companion 704 or the wheelchair in front based on the embodiments described above. In various embodiments during the operation as illustrated in Fig. 10 (a), the first wheelchair 902, second wheelchair 904 and third wheelchair 906 may move respectively at a coordinates of (-1 ,-1 ), (-2, -1 ) and (-3, -1 ) with respect to the human companion 704. As highlighted above, this configuration may avoid collision between the first wheelchair 902 and the human companion 704 if the wheelchair 902 is unable to stop in time when the human companion 704 has stopped moving.
3) Manual mode:
The apparatus 102 may also operate in manual mode wherein the apparatus 102 is disengaged from the wheels of the wheelchair without detaching it from the wheelchair 104. In another possible manual mode, the apparatus 102 may still be engaged to the wheelchair 104, and the human companion 704 is required to push the wheelchair manually. The manual mode is advantageous when self-navigation is not advisable such as in a lift or moving on a sky-bridge.
4) Remote control mode and 5) Local control mode: The apparatus 102 may also be controlled remotely using a master remote controller 708. In various embodiments, the apparatus 102 may also be controlled locally by the user on the wheelchair 104 through a client remote controller 710 as shown in Fig. 7 or a joystick control 1202 as shown in Fig. 12. Commands may be communicated from either the master remote controller 708 or the client remote controller 710 or the joystick control 1202 to the apparatus 102 by wireless or wired means. In the use of wireless communication, established wireless communication protocols such as Bluetooth or Zigbee communication protocol may be used. The processor is operable to first stop the operation of the camera 112. Thereafter, the processor then processes the received commands and send an electrical command or control signal to the rear wheels 108 to navigate the wheelchair according to the received commands. In various embodiments as shown in Fig. 12, the mast 1 10, together with the camera 1 12, may be removed as they are not required to effect the local control mode.
6) Electric scooter mode:
Apart from the five operation modes described above, the apparatus 102 may also be converted into an electric scooter 1314 as shown in Fig. 13. In various embodiments, the mast 1 10 and a top part or top portion of the control unit 312 and a top part or top portion of the battery unit 314 may be removed from the apparatus 102 as shown in Fig. 13 (a), leaving the bottom parts or bottom portions of the control unit 312 and battery unit 314 to create a modified apparatus 1301 . Thereafter, a steering unit 1302 (otherwise may also be referred to as an interchangeable top base with steering rod) comprising a steering handle 1304, a standing platform 1306 and a steering rod 1308 may be mounted or secured on the modified apparatus 1301 to create an electric scooter 1314. In various embodiments, the steering handle 1304 is mounted on a steering rod 1308 and the steering rod 1308 extends upwardly from the standing platform 1306. In various embodiments, the length of the vertical steering rod 1308 may be adjustable to cater to passengers with different heights. In various embodiments, the steering handle 1304 may comprise a left steering handle 1310 and a right steering handle 1312, wherein either one or both of the steering handles may have a steering control unit or actuator for controlling the speed and direction of the electric scooter 1314. In various embodiments, the steering control unit may enable the speed of the electric scooter 1314 to be controlled within the range of 0 to 10 km/h. In various embodiments, the steering control unit may be operable to allow the speed to be controlled in three discrete stages or levels - slow, medium and fast.
In various embodiments and as shown in the inset of Fig. 13 (b), the steering control unit or actuator may be a left push button 1316 located on the left steering handle 1310 and a right push button 1318 located on the right steering handle 1312. The push buttons are operable to detect a left pushing force and right pushing force of the corresponding hand for manoeuvring. For example, a larger force on the right push button 1318 would steer the electric scooter 1314 towards the right and vice versa. Pushing both buttons with equal force will navigate the electric scooter 1314 forward at a speed corresponding to the force on both the push buttons. In various embodiments, the push button may comprise a force or pressure sensor operable to detect the amount force applied by the passenger and thereafter, the force may be correlated to the speed of the corresponding rear wheel via a first transfer function. In various embodiments, the push button may also be operable to determine the amount of compressive force or displacement of the button from its original position and correlate the amount of displacement of the push button to the speed of the corresponding rear wheel via a second transfer function (i.e. left push button 1316 to left rear wheel and right push button 1318 to right rear wheel). In various embodiments, the push buttons may be biased by a spring such that the push button returns to its original position once the pushing force is removed so that the electric scooter 1314 automatically slows down when the passenger cease to apply the force on the push buttons.
In various embodiments, the force or pressure sensor in either of the push buttons may be a piezo-resistive or piezo-electric sensor. When subjected to a strain due to a compressive force, the said piezo-resistive or piezo-electric sensor may induce a voltage or potential difference which may be used as a control signal. In various embodiments, the piezo-resistive sensor may comprise of a piezo-resistive element connected to a Wheatstone bridge. The piezo-resistive element and the Wheatstone bridge may be attached or placed on a diaphragm. As the change in resistance of the piezo-resistive sensor may be small, the Wheatstone bridge may be used for more accurate determination of the change in resistance of the piezo-resistive element when the said piezo-resistive element is strained. Further, a strain or voltage amplifier may be used to amplify the signal from the piezo-resistive or piezo-electric sensor
In this case, the control signals (amount of force or amount of displacement) from the force or pressure sensor in each of the push buttons may be transmitted to a processor for further processing. Thereafter, the processor may calculate the speed of each rear wheel through at least either the first or second transfer function and further transmit further control signals to control the rear wheels via differential steering. In various embodiments, an H-bridge circuit may be used to enable reversing of the control signals from the push buttons. The H-bridge circuit would be useful to allow the passenger to reverse the electric scooter 1314 using the push buttons. In various embodiments, the rear wheels of the electric scooter 1314 may automatically slow down and gradually come to a stop within a pre-determined period of time once the passenger disengages the push buttons.
In various embodiments, the steering control unit or actuator may be a joystick mounted on either the left steering handle 310 or the right steering handle 1312. The joystick may enable forward, backward, left and right control and may comprise of built in switches and potentiometer which are operable to give a stable voltage output to a Pulse Width Modulation (PWM) or analog speed control circuit. In various embodiments, a micro-controller, such as an Arduino Uno™ control board may be in data communication with the actuator which may be in the form of a joystick. The microcontroller is further operable to receive digital electrical signals from the joystick for generating a PWM output control signal to simulate a variable direct current (DC) supply voltage to control the electric motor of the electric scooter 1314. In various embodiments, the microcontroller may operate to receive an analog signal from the potentiometer in the joystick and generate a PWM output control signal to control the speed of the electric motor. In various embodiments, the microcontroller may be programmed to output the PWM control signal by modifying the duty cycle of the output PWM control signal according to the magnitude of the analog input, for example a high analog input will result in a higher duty cycle. The duty cycle of the PWM control signal will directly affect the effective voltage output to the DC electric motor which will in turn affect the speed of the electric motor. In various embodiments, an electronic switch such as, but not limited to an electronic transistor may also be used in combination with the microcontroller for switching the electric motor.
In various embodiments, the steering unit 1302 may be mounted on the modified apparatus 1301 through the standing platform 1306 via securing means such as nuts and bolts or other types of commonly known securing means. In various embodiments, the standing platform 1306 may be as close to the surface where the electric scooter 1314 is travelling on as possible (so as to maintain a low center of gravity for stability, especially at higher speeds). In various embodiments wheel fenders (not shown) may be installed at the rear wheels of the modified apparatus to prevent injury to the passenger. In various embodiments, the area of the standing platform 1306 may be greater than the top surface area of the modified apparatus 301 to provide the passenger or rider with more surface or room to stand on.
In various embodiments of operation, the human companion 704 with the IR marker or IR LED array 702 may ride on the electric scooter 1314 when escorting a convoy of wheelchairs 102. The electric scooter 1314 has the advantage of being able to maintain the moving speed of the human companion 704 at a specific or predetermined speed for the ease of convoying the wheelchairs 102.
As it may be appreciated from the examples provided above, the modularity of the control unit 312, the battery unit 314, the mast 110 and the steering unit 1302 enables the apparatus 102 be modified or adapted to suit different needs or modes of operation.
It should be further appreciated by the person skilled in the art that variations and combinations of features described above, not being alternatives or substitutes, may be combined to form yet further embodiments falling within the intended scope of the invention. In particular,
• The IR LED array 702 may be arranged in other patterns instead of the pattern "A" that is described in the above embodiments. In various embodiments, the IR LED array 702 may have either a symmetrical or asymmetrical pattern or design. An asymmetrical pattern which does not possess symmetry about the vertical axis may allow the direction that the human companion 704 is facing be determined without using other properties of the image such as gradient in light intensity.
In various embodiments as shown in Fig. 11 , the IR LED array 1 00 at the back of the battery-powered vest 706 may comprise two circular shaped patterns (a first circular shaped pattern 1102 and a second circular shaped pattern 1104). The first and second circular shaped pattern may have the same emission wavelength in IR (for e.g. emission wavelength in the range of 850 nm to 2000 nm). The two circular shaped patterns may be arranged on the opposite sides of an imaginary middle axis 1106 of the vest 706 (i.e. the two circular shaped patterns are arranged laterally beside one another). Further, the two circular shaped patterns may also be located at an equal lateral distance from the middle axis of the vest 706. As the human companion 704 makes a turn, the circular shaped pattern that is further away will become sharper (more oval-like) and the circular shaped pattern that is nearer one will become larger. The turning angle or the degree of rotation about the vertical axis may be thus be determined based on the change in the relative change in shape and/or size of the two circular shaped patterns and/or intensity in the captured image.
In various embodiments, the two circular shaped patterns may also have different emission wavelengths. In various embodiments, the different emission wavelengths may be in visible spectrum (for example red LED emitting at around 600 nm and green LED emitting at around 565 nm). It may be appreciated that a different detector apart from one which is primarily sensitive in the N1R region may have to be used to detect emission in the visible spectrum, i.e. red and/or green emission. In various embodiments, this may include combining InGaAs or any other detectors that is sensitive mostly in the IR spectrum with a silicon detector that is sensitive mostly in the visible spectrum if the first emission wavelength is in the IR and the second emission wavelength is in the visible. The two circular shaped pattern with different emission wavelengths may be used to determine the turning angle. For example, if a first emission wavelength which corresponds to the first circular shaped pattern 1102 is detected to be further away as compared relatively to a second emission wavelength which corresponds to the second circular shaped pattern 1104, it may be determined that the human companion 704 has made a right turn (i.e. more facing the right) and vice versa. In various embodiments, the relative distance of each of the two circular shaped pattern from the apparatus 102 may be determined by comparing the intensity of the emission and/or the relative shape of the two circular shaped patterns in the captured image and/or the relative size of the two circular shaped patterns in the captured image.
Further, the IR LED array 702 for two or more wheelchairs may each have a unique design to prevent confusion since the respective leaders would each have a unique marker. The IR LED array 702 may also be substituted by a visible LED array or any other radiation sources emitting at a pre-determined or specified wavelength. Accordingly, the camera 112 should be sensitive to the pre-determined or specified wavelength of radiation that is emitted by either the LED array or any other radiation sources.
The IR LED array 702 may comprise of LED stripes instead of discrete LED units/lamps. The IR LED array 702 may also be substituted by a reflective tape.
A band-pass filter that selectively transmits radiation of the pre-determined or specified wavelength may be attached to the lens of the camera 112 to improve the signal-to-noise ratio of the captured image. This is because only radiation of the pre-determined or specified wavelength is selectively captured by the sensor of the camera 112 and background radiation or radiative emissions are substantially suppressed. The use of a band-pass filter may reduce the exposure time required for the camera 112 to capture an image of sufficient quality. In various embodiments, the band pass filter transmits 850 nm emission that can be detected by the camera 112 which is sensitive at near-IR.
Although the various embodiments above are described in the context of a NoIR or silicon camera 12, it is appreciated that a camera 112 with a sensor that is sensitive to the IR may also be used. A non-exclusive example is Indium-Gallium-Arsenide (InGaAs) sensor.
When operating in convoy mode, the leader may be a robot rather than the human companion 704. In various embodiments, the robot may also be remotely control.
Instead of using the IR LED array 702 as a marker to effect self-navigation or convoy mode, the leader and at least one follower may be operable to use wireless positioning technologies such as WIFI triangulation, Base Transceiver Station (BTS) cellular triangulation or GPS to determine its own positional information. Thereafter, the leader is operable to broadcast or communicate its positional information to the followers and the at least one followers is operable to respond accordingly to positional change of the leader by sending electrical command or control signal to the rear wheels 08. In this case, the leader may broadcast or communicate its positional information by way of any known wireless technologies such as Bluetooth, WIFI or cellular. The apparatus 102 may be equipped with further sensors for environmental perception to negotiate the surrounding environment and prevent collisions without human intervention.
The apparatus 102 may be equipped with a flashing warning light for alerting people nearby. The apparatus 102 may also have an emergency stop and hand brake to over-ride the autonomous mode.
The system 700 may be applied to a convoy of vehicles, where a lead vehicle or person (i.e. the leader) is provided with a marker to effect remote navigation of the following vehicles, each following vehicle provided with an image capturing device. In some embodiments where order of convey is necessary, each following vehicle may be provided with their unique marker(s).
Non-limiting examples of the vehicle may be a baggage cart or trolley, an automobile, a train or any transportation means.
It may be appreciated that the force or pressure sensor for detecting compressive load may also be known as a compression load cell.

Claims

Claims:
1. A system for navigating a first vehicle to follow a leader comprising:
a first marker that is attached to the leader;
a first image capturing device attached to the first vehicle, the first image capturing device operable to capture a first image of the first marker; and a first processor operable to process the first image captured by the first image capturing device to determine a first property of the first marker in the first image and send a first command signal to move the first vehicle based on at least the property of the first marker in the first image.
2. The system according to claim 2, wherein the first image capturing device is a silicon camera with infra-red filter removed.
3. The system according to any one of claims 1 or 2, wherein the first marker is an infra-red LED array.
4. The system according to any one of claims 1 to 3, wherein the first image is captured by using a band-pass filter and the band-pass filter is operable to selectively transmit radiation of a specified wavelength.
5. The system according to claim 4, wherein the first marker is an array emitting according to the specified wavelength of the band-pass filter.
6. The system according to any one of claims 1 to 5, wherein the first property is any one of the following: the size, degree of rotation, intensity and location of the marker in the first image.
7. The system according to any one of claims 1 to 6 wherein the first command signal is sent to a first electric motor of the first vehicle.
8. The system according to claim 7, wherein the first electric motor controls the speed of the first vehicle and/or direction of the first vehicle based on differential steering.
9. The system according to any one of claims 1 to 8, wherein the first processor is further operable to determine a first blinking frequency of the first marker by way of the first image and a second image and send a second command signal to move the first vehicle at a speed based on at least the first blinking frequency.
10. The system according to any one of claims 1 to 9, wherein the first vehicle is always maintained at a first positional coordinate that is inclined at an angle relative to a direction of navigation of the leader.
1 1. The system according to claim 1 , wherein the first marker comprises a first circular shaped pattern and a second circular shaped pattern.
12. The system according to claim 1 1 , wherein the first and second circular shaped patterns are arranged laterally beside each other.
13. The system according to claim 12, wherein the first circular shaped pattern and the second circular shaped pattern are LED arrays with the same emission wavelength.
14. The system according to any one of claims 1 to 13, wherein the first vehicle is a wheelchair and the leader is either a wheelchair or a human or a robot.
15. The system according to any one of claims 1 to 14 further comprising a first controller that is operable to stop the first image capturing device, and is further operable to send a third command signal by way of the first processor to move the first vehicle based on at least a first input into the first controller.
16. The system according to claim 15, wherein the first controller communicates with the first processor by way of ZigBee communication protocol.
17. The system according to any one of claims 1 to 16, wherein the first processor is further operable to communicate with a light detection and ranging system or an ultrasonic system to determine a movement speed of the leader and move the vehicle at the movement speed.
18. The system according to any one of claims 1 to 17 further used for navigating a second vehicle to follow the leader comprising:
a second marker that is attached to the first vehicle;
a second image capturing device attached to the second vehicle operable to capture a second image with the second marker; and
a second processor operable to process the second image captured by the second image capturing device to determine a second property of the second marker in the second image and send a fourth command signal to move the second vehicle based on at least the property of the second marker in the second image.
19. A method for navigating a first vehicle to follow a leader comprising the steps of: determining, by the first vehicle, a direction of movement of the leader, a movement speed of the leader and a position of the leader relative to the first vehicle; and
sending a first command signal to move the first vehicle based on at least the direction of movement of the leader, the movement speed of the leader and the position of the leader relative to the vehicle.
20. The method according to claim 9, wherein the step of determining the direction of movement of the leader, the movement speed of the leader and the position of the leader relative to the first vehicle is based on any one of the following: processing a first image to determine a first property of a first marker in the first image or using a light detection and ranging system or using an ultrasonic system.
21. The method according to claim 20, wherein the first image is captured by a first image capturing device with infra-red filter removed.
22. The method according to any one of claim 20 or 21 , wherein the first marker is an infra-red LED array attached to the leader.
23. The method according to claim 22, wherein the first image is captured with a band-pass filter, and the band-pass filter is operable to selectively transmit radiation of a specified wavelength.
24. The method according to claim 23, wherein the first marker is due to an LED array emitting according to the specified wavelength of the band-pass filter.
25. The method according to any one of claims 20 to 24, wherein the first property of the first marker in the first image is one of the following: the size, degree of rotation, intensity and location of the marker in the image.
26. The method according to any one of claims 19 to 25, wherein the first command signal is sent to a first electric motor of the first vehicle.
27. The method according to claim 26, wherein the first electric motor control the speed of the first vehicle and/or the direction of the first vehicle based on differential steering.
28. The method according to any one of claims 20 to 27 further comprising the step of determine a first blinking frequency of a first marker and sending the first command signal to move the first vehicle at a speed based on at least the first blinking frequency of the first marker.
29. The method according to any one of claims 19 to 28 further comprising the step of maintaining the first vehicle at a first positional coordinate relative to the leader.
30. The method according to any one of claims 20 to 29 further comprising the steps of:
stopping the operation of the first image capturing device; and
sending a third command signal to move the first vehicle based on at least a first input into a first controller.
31. The method according to claim 30, wherein first controller communicates by way of ZigBee communication protocol.
32. The method according to any one of claims 19 to 31 , wherein the first vehicle is a wheelchair and the leader is either a wheelchair or a human.
33. The method according to claim 20, wherein the first marker comprises a first circular shaped pattern and a second circular shaped pattern.
34. The system according to claim 33, wherein the first and second circular shaped patterns are arranged laterally beside each other.
35. The system according to claim 34, wherein the first circular shaped pattern and the second circular shaped pattern are LED arrays with the same emission wavelength.
36. The method of according to any one of claim 9 to 35 further used for navigating a second vehicle to follow the leader comprising the steps of:
determining, by the second vehicle, a direction of movement of the first vehicle, a movement speed of the first vehicle and a position of the first vehicle relative to the second vehicle; and
sending a second command signal to move the second vehicle based on at least the direction of movement of the first vehicle, the movement speed of the first vehicle and the position of the first vehicle relative to the second vehicle.
37. The method according to claim 36 further comprising the step of:
maintaining the first vehicle at a first positional coordinate that is inclined at an angle relative to a direction of navigation of the leader.
38. The method according to claim 37 further comprising the steps of:
stopping the operation of the second image capturing device; and sending a fourth command signal to move the second vehicle based on at least a second input into a second controller.
39. An apparatus for navigating a first vehicle to follow a leader comprising:
a latching device for attaching the apparatus to a coupling frame attached to the first vehicle;
a motor for driving a wheel;
an image capturing device operable to capture an image with a marker, wherein the marker is attached to the leader; and
a processor operable to process the image captured by the image capturing device to determine a property of the marker in the image and send a command signal to the motor based on at least the property of the marker in the image to drive the wheel.
40. The apparatus according to claim 39, wherein the latching device comprises of an insertion pin that is magnetically or mechanically actuated.
41 . The apparatus according to claim 40, wherein the insertion pin is magnetically or mechanically actuated to slot into a pin hole located in the coupling frame of the first vehicle.
42. The apparatus according to any one of claims 39 to 41 further comprising a contact sensor for triggering the magnetically or mechanically actuated insertion pin when contact on the coupling frame is detected.
43. The apparatus according claim 39, wherein the latching device comprises of a chamfer that is magnetically or mechanically actuated.
44. The apparatus according to claim 43, and the chamfer is magnetically or mechanically actuated into a circlip located in the coupling frame when contact on the coupling frame is detected.
45. The apparatus according to claim 39, comprising a top portion releasably attachable to a bottom portion, wherein the top portion is detached from the bottom portion and the bottom portion is releasably attachable to a steering unit for conversion to an electric scooter.
46. The apparatus according to claim 45, wherein the steering unit comprises an actuator.
47. The apparatus according to claim 46, wherein the actuator is either a push button or joystick.
PCT/SG2016/050430 2015-09-01 2016-09-01 System, method and apparatus for navigating one or more vehicles WO2017039546A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
SG10201506928R 2015-09-01
SG10201506928R 2015-09-01

Publications (1)

Publication Number Publication Date
WO2017039546A1 true WO2017039546A1 (en) 2017-03-09

Family

ID=58188159

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/SG2016/050430 WO2017039546A1 (en) 2015-09-01 2016-09-01 System, method and apparatus for navigating one or more vehicles

Country Status (1)

Country Link
WO (1) WO2017039546A1 (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180143640A1 (en) * 2015-08-26 2018-05-24 Hanwha Land Systems Co., Ltd. Following system for mobile robot to follow moving object
WO2018175391A1 (en) * 2017-03-21 2018-09-27 HealthierStep, Inc. Ultraviolet light-sterlized and illuminating caster wheel sets
WO2019240664A1 (en) * 2018-06-12 2019-12-19 St Engineering Land Systems Ltd. Convoying system based on fusion of data from vision sensors and lidar
EP3647896A1 (en) * 2018-10-29 2020-05-06 Fq Ip Ab Virtual coupling
US10932964B2 (en) 2018-03-27 2021-03-02 Toyota Motor Engineering & Manufacturing North America, Inc. Modular wheelchair system
US12018949B2 (en) 2018-11-07 2024-06-25 Google Llc Providing navigation instructions to one device in view of another device

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5835880A (en) * 1995-07-19 1998-11-10 Vi & T Group, Inc. Apparatus and method for vehicle following with dynamic feature recognition
US20090012666A1 (en) * 2007-07-06 2009-01-08 Simpson Rich C Powered vehicle convoying systems and methods of convoying powered vehicles
US20110172850A1 (en) * 2009-09-14 2011-07-14 Israel Aerospace Industries Ltd. Infantry robotic porter system and methods useful in conjunction therewith
US20140100771A1 (en) * 2012-10-04 2014-04-10 Frank Edughom Ekpar Method and apparatus for synchronized navigation by mobile agents
CN104049633A (en) * 2014-06-13 2014-09-17 深圳市宇恒互动科技开发有限公司 Servo control method, servo device and servo system

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5835880A (en) * 1995-07-19 1998-11-10 Vi & T Group, Inc. Apparatus and method for vehicle following with dynamic feature recognition
US20090012666A1 (en) * 2007-07-06 2009-01-08 Simpson Rich C Powered vehicle convoying systems and methods of convoying powered vehicles
US20110172850A1 (en) * 2009-09-14 2011-07-14 Israel Aerospace Industries Ltd. Infantry robotic porter system and methods useful in conjunction therewith
US20140100771A1 (en) * 2012-10-04 2014-04-10 Frank Edughom Ekpar Method and apparatus for synchronized navigation by mobile agents
CN104049633A (en) * 2014-06-13 2014-09-17 深圳市宇恒互动科技开发有限公司 Servo control method, servo device and servo system

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
"Development of a Vision-Based Robotic Follower Vehicle", DEFENCE RESEARCH AND RECHERCHE ET DÉVELOPPEMENTDEVELOPMENT CANADA, 28 February 2009 (2009-02-28), XP055373094, Retrieved from the Internet <URL:www.dtic.mil/cgi-bin/GetTRDoc?AD=ADA512953> *
BENHIMANE S. ET AL.: "Vision-based Control for Car Platooning using Homography Decomposition.", PROCEEDINGS OF THE 2005 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION, 22 April 2005 (2005-04-22), pages 2161 - 2166, XP010872029 *
MIO M. ET AL.: "PLATOON SYSTEM BASED ON OPTICAL INTER-VEHICLE COMMUNICATION.", INTELLIGENT TRANSPORT SYSTEMS WORLD CONGRESS, vol. 3, 11 November 1995 (1995-11-11), pages 1272 - 1277 *
MIRO J. V. ET AL.: "Low-cost Visual Tracking with an Intelligent Wheelchair for Innovative Assistive Care.", 2012 12TH INTERNATIONAL CONFERENCE ON CONTROL AUTOMATION ROBOTICS & VISION, 7 December 2012 (2012-12-07), pages 1540 - 1545, XP032347357 *

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180143640A1 (en) * 2015-08-26 2018-05-24 Hanwha Land Systems Co., Ltd. Following system for mobile robot to follow moving object
US10775794B2 (en) * 2015-08-26 2020-09-15 Hanwha Defense Co., Ltd. Following system for mobile robot to follow moving object
WO2018175391A1 (en) * 2017-03-21 2018-09-27 HealthierStep, Inc. Ultraviolet light-sterlized and illuminating caster wheel sets
US10383965B2 (en) 2017-03-21 2019-08-20 Harbor Innovations, LLC Ultraviolet light-sterilized and illuminating caster wheel sets
EP3620147A1 (en) * 2017-03-21 2020-03-11 Healthierstep, Inc. Ultraviolet light-sterilized and illuminating caster wheel sets
US10932964B2 (en) 2018-03-27 2021-03-02 Toyota Motor Engineering & Manufacturing North America, Inc. Modular wheelchair system
WO2019240664A1 (en) * 2018-06-12 2019-12-19 St Engineering Land Systems Ltd. Convoying system based on fusion of data from vision sensors and lidar
EP3647896A1 (en) * 2018-10-29 2020-05-06 Fq Ip Ab Virtual coupling
US12018949B2 (en) 2018-11-07 2024-06-25 Google Llc Providing navigation instructions to one device in view of another device

Similar Documents

Publication Publication Date Title
WO2017039546A1 (en) System, method and apparatus for navigating one or more vehicles
US10150505B2 (en) Automated hitching assist system
WO2017219751A1 (en) Mobile suitcase having automatic following and obstacle avoidance functions, and using method therefor
CN107636546B (en) Autonomous moving system
CN111436189B (en) Self-driving system
CA2620062C (en) Systems and methods for locating and controlling powered vehicles
JP7169593B2 (en) Smart self-driving system with lateral following and obstacle avoidance
CN205540279U (en) Independently car is followed to human body
US10471904B2 (en) Systems and methods for adjusting the position of sensors of an automated vehicle
KR101792112B1 (en) Crosswalk walking assistance system and controlling method thereof
US20190092109A1 (en) Automated Trailer Hitching Using Image Coordinates
US20140074352A1 (en) Park assist system
JP2019503022A (en) Autonomous vehicle traction system and method
JP2018067183A (en) Mobile body tracking controller
KR20130085235A (en) System for driving manless of car
NL1039154C2 (en) Target-following vehicle.
WO2017056334A1 (en) Object following movement system
US20190086927A1 (en) Automated Trailer Hitching Using GPS
WO2017038883A1 (en) Autonomous moving body and signal control system
JP2010282568A (en) Autonomous mobile body and mobile system using the same
CN203898662U (en) Guide walking stick system for blind person
CN107933557A (en) A kind of vehicle right-angled bend control system and its control method based on multi-cam
EP3532179B1 (en) A vehicle comprising an electric motor and a method of starting an electric motor for propulsion of a vehicle
CN113044095B (en) Automatic following baby carriage based on UWB positioning and working method
KR102345800B1 (en) System for controlling driving in cargo robot

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16842437

Country of ref document: EP

Kind code of ref document: A1

DPE1 Request for preliminary examination filed after expiration of 19th month from priority date (pct application filed from 20040101)
WWE Wipo information: entry into national phase

Ref document number: 11201801671T

Country of ref document: SG

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16842437

Country of ref document: EP

Kind code of ref document: A1