WO2019127261A1

WO2019127261A1 - Method for automatic driving of smart wheelchair, system and computer readable medium

Info

Publication number: WO2019127261A1
Application number: PCT/CN2017/119542
Authority: WO
Inventors: 刘伟荣; 李家鑫; 焦寅; 闫励
Original assignee: 四川金瑞麒智能科学技术有限公司
Priority date: 2017-12-28
Filing date: 2017-12-28
Publication date: 2019-07-04
Also published as: CN111542295A

Abstract

A method (600) for automatic driving of a smart wheelchair, a system (100), and a computer readable medium, said method (600) comprising acquiring destination information of a user (602), and determining current location information of the wheelchair (604); determining a traveling path of the smart wheelchair according to the destination information of the user and the current location information of the wheelchair (606); and detecting obstacle information in the traveling path of the wheelchair (608), and adjust the movement of the wheelchair according to the detected obstacle information (610).

Description

Intelligent wheelchair automatic driving method, system and computer readable medium

Technical field

The present application relates to the field of automatic driving, and in particular to an automatic driving method and system for an intelligent wheelchair.

Background technique

As a means of transportation, wheelchairs provide great convenience for people with physical disabilities and people with reduced mobility. Existing wheelchairs are usually driven manually, so when the travel distance is long, the user needs a lot of physical effort and time. When encountering obstacles during the trip, it is not easy to avoid.

Summary of the invention

One aspect of the present application relates to an automated driving method for a smart wheelchair. The method includes obtaining destination information; determining a current location of the wheelchair; determining a travel path of the wheelchair based on the current location and destination information; detecting obstacle information in a travel path of the wheelchair; and detecting an obstacle based on the obstacle Information adjusts the movement of the wheelchair.

In some embodiments, the obtaining of the destination information includes any one of user manual input, voice input, and system recommendation input.

In some embodiments, the determining the travel path of the wheelchair according to the current location and destination information comprises: calculating a plurality of candidate paths according to the current location and destination information; and scoring the plurality of candidate paths; The plurality of candidate paths are sorted according to the level of the score, and the sorted candidate paths are presented to the user for selection by the user; and the driving path of the wheelchair is determined according to the selection result of the user.

In some embodiments, the user is allowed to set the travel speed of the wheelchair after determining the travel path of the wheelchair.

In some embodiments, the obstacle information includes the type, location, size, and state of motion of the obstacle.

In some embodiments, the adjusting the movement of the wheelchair includes adjusting a speed of the wheelchair, controlling wheelchair steering, and controlling wheelchair braking.

In some embodiments, the automated driving method of the smart wheelchair further includes providing the user with something of interest and automatically adjusting the travel route when the user confirms that they are interested in the event.

Yet another aspect of the present application relates to a system for controlling automatic driving of a wheelchair. The system includes: an acquisition module, a processing module, a detection module, and a control module, wherein: the acquisition module is configured to acquire destination information; the processing module is configured to determine a current location of the wheelchair, and according to the current location and destination information Determining a travel path of the wheelchair; the detection module is configured to detect obstacle information in the travel path of the wheelchair; and the control module is configured to adjust the movement of the wheelchair according to the detected obstacle information.

Another aspect of the present application is directed to a non-transitory computer readable medium having a computer program product. The computer program product can include instructions configured to cause a computing device to perform the method.

The invention has the following technical effects:

First, based on the user-entered destination information, combined with automatic positioning technology, plan a reasonable route for the intelligent wheelchair, and improve the user's travel experience.

Second, through a variety of obstacle detection means to accurately identify obstacle information in real time, and based on the obstacle information planning a reasonable circumvention plan to improve travel safety.

Third, through a variety of input methods to improve the efficiency and accuracy of information input, to provide convenience for users.

Fourth, the candidate path is scored to provide customers with more reasonable travel routes.

5. Providing the user with the prompt function of interest, and adjusting the path according to the user's choice, so that it does not miss the interest point on the driving path and improve the user experience.

Some additional features of this application can be described in the following description. Some additional features of the present application will be apparent to those skilled in the art from a review of the following description and the accompanying drawings. The features of the present disclosure can be realized and attained by the practice or use of the methods and methods and combinations of various aspects of the specific embodiments described below.

DRAWINGS

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings used in the description of the embodiments will be briefly described below. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can apply the present application to other similarities according to these drawings without any creative work. scene. Unless otherwise apparent from the language environment or otherwise stated, the same reference numerals in the drawings represent the same structure and operation.

1 is a schematic illustration of an exemplary intelligent wheelchair automated driving system, in accordance with some embodiments of the present application;

2 is a schematic diagram of exemplary hardware and/or software components of an exemplary computing device that can implement a processing engine, in accordance with some embodiments of the present application;

3 is a schematic diagram of hardware and/or software components of an exemplary mobile device, shown in accordance with some embodiments of the present application;

4 is a block diagram of an exemplary processing engine shown in accordance with some embodiments of the present application;

FIG. 5 is a block diagram showing an exemplary processing module shown in accordance with some embodiments of the present application; FIG.

6 is an exemplary flow chart of intelligent wheelchair automatic driving, in accordance with some embodiments of the present application;

7 is an exemplary flow chart for determining a travel path of a wheelchair, as shown in some embodiments of the present application;

8 is a schematic diagram of an intelligent wheelchair autopilot shown in accordance with some embodiments of the present application.

Detailed ways

In the following detailed description, specific details of the embodiments are set forth However, it will be apparent to those skilled in the art that the present invention may be practiced without the specific details. In other instances, well known methods, procedures, systems, components, and/or circuits have been described in a high level (not in detail) in order to avoid unnecessarily obscuring aspects of the present application. Various modifications to the embodiments of the present application will be apparent to those skilled in the art, and the general principles defined in the present application can be applied to other embodiments and application scenarios without departing from the spirit and scope of the application. Therefore, the application is not limited to the embodiments shown, but is in the broadest scope.

The terminology used herein is for the purpose of describing the particular embodiments embodiments As used herein, the singular forms "a", "the" It should be further understood that the terms "comprising" and "comprising", when used in the specification, are meant to mean the presence of the recited features, integers, steps, operations, elements and/or components, but does not exclude the presence or addition of a Or a plurality of other features, shaping constants, steps, operations, elements, components, and/or combinations thereof.

It should be understood that the terms "system," "engine," "unit," "module," and/or "block" as used herein are used to distinguish different components, components, components, or components in ascending order. The level method of the component. However, if other expressions achieve the same purpose, these terms may be replaced by other expressions.

Generally, a "module," "unit," or "block" as used herein refers to logic embodied in a collection of hardware or firmware or software instructions. The modules, units or blocks described herein may be executed in software and/or hardware and may be stored in any type of non-transitory computer readable medium or other storage device. In some embodiments, software modules, units, blocks may be compiled and linked into an executable program. It should be understood that a software module can be called from other modules, units, blocks or themselves and/or can be invoked in response to a detected event or interrupt. Software modules/units/blocks (eg, processing engine 120 as shown in FIG. 1) configured for execution on a computing device can be provided on a computer readable medium such as an optical disk, a digital video disk, a flash drive, a disk Or any other tangible medium or as a digital download (and may be originally stored in a compressed or installable format, which needs to be installed, decompressed or decrypted prior to execution). The software code may be stored, in part or in whole, on a storage device executing the computing device for execution by the computing device. Software instructions can be embedded in the firmware, such as EPROM. It should be understood that hardware modules, units or blocks may be included in the connected logical components, such as gates and flip-flops and/or may be included in a programmable unit such as a programmable gate array or processor. The modules, units, blocks or computing device functions described herein may be implemented as software modules/units/blocks, but may be represented by hardware or firmware. Generally, the modules, units, and blocks described herein refer to logical modules, units, and blocks that can be combined with other modules, units, blocks, or divided into sub-modules, sub-units, sub-blocks, although their physical organization or storage. . The description can be applied to a system, an engine, or a portion thereof.

It will be understood that when a unit, engine, module or block is referred to as "on", "connected" or "coupled" to another unit, engine, module or block, it can be directly Units, engines, modules, or blocks communicate, or units, engines, modules, or blocks may exist, unless the context clearly indicates an exception. The term "and/or" as used herein includes any and all combinations of one or more of the associated listed items.

These and other features of the present application, as well as the related structural elements, and the components of the invention, and the method of operation and function of the combination of the embodiments of the present invention will become more apparent, and are all part of the present application. It is to be expressly understood, however, that the claims It should be understood that the drawings are not to scale.

FIG. 1 is a schematic illustration of an exemplary intelligent wheelchair automated driving system 100, in accordance with some embodiments of the present application. As shown, the intelligent wheelchair automated driving system 100 can include a smart wheelchair 110, a processing engine 120, a memory 130, one or more terminals 140, and a network 150. In some embodiments, smart wheelchair 110, processing engine 120, memory 130, and/or terminal 140 can be connected and/or communicated to each other through a wireless connection (eg, network 150), a wired connection, or a combination thereof. The connections between components in the intelligent wheelchair automated driving system 100 can vary. For example only, the smart wheelchair 110 may be coupled to the processing engine 120 via the network 150 as shown in FIG. As another example, the smart wheelchair 110 can be directly coupled to the processing engine 120. As another example, the memory 130 can be coupled to the processing engine 120 via the network 150 as shown in FIG. 1, or directly to the processing engine 120.

In some embodiments, the smart wheelchair 110 can transmit travel data to the processing engine 120, the memory 130, and/or the terminal 140 over the network 150. For example, the travel data may be sent to the processing engine 120 for further processing or may be stored in the memory 130.

Processing engine 120 may process data and/or information obtained from smart wheelchair 110, memory 130, and/or terminal 140. For example, the processing engine 120 can generate control commands based on the travel data collected by the smart wheelchair 110. In some embodiments, processing engine 120 can be a single server or a server group. Server groups can be centralized or distributed. In some embodiments, processing engine 120 can be local or remote. For example, processing engine 120 may access information and/or data of smart wheelchair 110, memory 130, and/or terminal 140 over network 150. As another example, processing engine 120 can be directly coupled to smart wheelchair 110, terminal 140, and/or memory 130 to access information and/or data. In some embodiments, the processing engine 120 can be implemented on a cloud platform. For example, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an inter-cloud, a cloudy, etc., or a combination thereof. In some embodiments, the processing engine 120 can be implemented by a computing device 200 that includes one or more components as described in FIG.

Memory 130 can store data, instructions, and/or any other information. In some embodiments, the memory 130 can store data acquired from the processing engine 120, the terminal 140. In some embodiments, memory 130 may store data and/or instructions that processing engine 120 may execute or use to perform the example methods described in this application. In some embodiments, memory 130 can include mass storage, removable memory, volatile read/write memory, read only memory (ROM), and the like, or any combination thereof. Exemplary mass storage devices can include magnetic disks, optical disks, solid state drives, and the like. Exemplary removable storage may include flash drives, floppy disks, optical disks, memory cards, compact disks, magnetic tape, and the like. An exemplary volatile read/write memory can include random access memory (RAM). Exemplary RAMs may include dynamic RAM (DRAM), double data rate synchronous dynamic RAM (DDR SDRAM), static RAM (SRAM), thyristor RAM (T-RAM), and zero-capacity RAM (Z-RAM). Exemplary ROMs may include mask ROM (MROM), programmable ROM (PROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), compact disk ROM (CD-ROM), and digital general purpose Disk ROM, etc. In some embodiments, memory 130 can be implemented on a cloud platform as described elsewhere herein.

In some embodiments, memory 130 may be coupled to network 150 to communicate with one or more other components (eg, processing engine 120, terminal 140, etc.) in intelligent wheelchair automated driving system 100. One or more components of the intelligent wheelchair automated driving system 100 can access data or instructions stored in the memory 130 via the network 150. In some embodiments, memory 130 can be part of processing engine 120.

Terminal 140 can be coupled to and/or in communication with smart wheelchair 110, processing engine 120, and/or memory 130. For example, terminal 140 may obtain a processed travel data (eg, travel speed) from processing engine 120. As another example, terminal 140 may obtain image data acquired by smart wheelchair 110 (eg, a street scene image acquired by a camera) and transmit the image data to processing engine 120 for processing. In some embodiments, terminal 140 can include mobile device 140-1, tablet 140-2, a portable computer 140-3, and the like, or any combination thereof. For example, mobile device 140-1 can include a mobile phone, a personal digital assistant (PDA), a gaming device, a navigation device, a point of sale (POS) device, a laptop, a tablet, a desktop computer, and the like, or any combination thereof. In some embodiments, terminal 140 can include an input device, an output device, and the like. Input devices may include alphanumeric keys and other keys that may be input through a keyboard, touch screen (eg, with tactile or tactile feedback), voice input, line-of-sight tracking input, brain monitoring system, or any other similar input mechanism. Input information received by the input device can be transmitted to the processing engine 120 via, for example, a bus for further processing. Other types of input devices may include cursor control devices such as a mouse, trackball or cursor direction keys, and the like. The output device can include a display, a speaker, a printer, and the like, or a combination thereof. In some embodiments, terminal 140 can be part of processing engine 120.

Network 150 may include any suitable network that may facilitate the exchange of information and/or data for intelligent wheelchair automated driving system 100. In some embodiments, one or more components of the intelligent wheelchair automated driving system 100 (eg, smart wheelchair 110, processing engine 120, memory 130, terminal 140, etc.) may be through network 150 with one or more other of imaging system 100 The component communicates information and/or data. For example, processing engine 120 may obtain image data from scanner 110 over network 150. As another example, the processing engine 120 can obtain user instructions from the terminal 140 over the network 150. Network 150 may be and/or include a public network (eg, the Internet), a private network (eg, a local area network (LAN), a wide area network (WAN), etc.), a wired network (eg, Ethernet), a wireless network (eg, an 802.11 network, Wi-Fi networks, etc.), cellular networks (eg, Long Term Evolution (LTE) networks), Frame Relay networks, Virtual Private Networks (VPNs), satellite networks, telephone networks, routers, hubs, switches, server computers, and/or Any combination thereof. For example, network 150 may include a cable network, a wired network, a fiber network, telecommunications network, internal network, a wireless local area network (WLAN), metropolitan area network (MAN), a public switched telephone network (PSTN), Bluetooth ^TM network, ZigBee ^TM network, Near field communication (NFC) network, etc., or any combination thereof. In some embodiments, network 150 can include one or more network access points. For example, network 150 can include wired and/or wireless network access points, such as base stations and/or Internet switching points, through which one or more components of intelligent wheelchair automated driving system 100 can be connected to network 150 Exchange data and/or information.

The above description is for illustrative purposes only and does not limit the scope of protection of the application. Many alternatives, modifications and variations will be apparent to those skilled in the art. The features, structures, methods, and other features of the exemplary embodiments described herein may be combined in various ways to obtain additional and/or alternative exemplary embodiments. For example, the memory 130 may be a data storage including a cloud computing platform, which may be a public cloud, a private cloud, a community cloud, a hybrid cloud, or the like. However, variations and modifications do not depart from the scope of the present application.

2 is a schematic diagram of exemplary hardware and/or software components of an exemplary computing device 200 that can implement processing engine 120, in accordance with some embodiments of the present application. As shown in FIG. 2, computing device 200 can include a processor 210, a memory 220, an input/output (I/O) 230, and a communication port 240.

Processor 210 can execute computer instructions (eg, program code) and perform the functions of processing engine 120 in accordance with the techniques described herein. Computer instructions may include, for example, routines, programs, objects, components, data structures, operations, modules, and functions that perform the specific functions described herein. For example, processor 210 can process data acquired from smart wheelchair 110, terminal 140, memory 130, and/or any other component of intelligent wheelchair automated driving system 100. In some embodiments, processor 210 may include one or more hardware processors, such as a microcontroller, a microprocessor, a reduced instruction computer (RISC), an application specific integrated circuit (ASIC), a special application instruction set processor ( ASIP), central processing unit (CPU), graphics processing unit (GPU), physical processor (PPU), microcontroller unit, digital signal processor (DSP), field programmable gate array (FPGA), advanced reduced instruction system A computer (ARM), a programmable logic device (PLD), any circuit or processor capable of performing one or more functions, or the like, or any combination thereof.

For purposes of illustration only, only one processor in computing device 200 is described. However, it should be noted that the computing device 200 in this application may also include multiple processors, and thus the operations and/or method steps performed by one processor described herein may also be processed by multiple processors, either collectively or independently. Execution. For example, if, in the present application, the processor of computing device 200 performs process A and process B, it should be understood that process A and process B may also be shared or independently by two or more different processors in computing device 200. Execution (eg, the first processor executes process A, the second processor performs process B; or the first processor and the second processor collectively perform processes A and B).

Memory 220 may store data/information obtained from smart wheelchair 110, terminal 140, memory 130, and/or any other component of intelligent wheelchair automated driving system 100. In some embodiments, memory 220 can include mass storage, removable memory, volatile read/write memory, read only memory (ROM), and the like, or any combination thereof. For example, mass storage can include magnetic disks, optical disks, solid state drives, and the like. The removable storage may include a flash drive, a floppy disk, an optical disk, a memory card, a compact disk, a magnetic tape, and the like. Volatile read/write memory may include random access memory (RAM). The RAM may include dynamic RAM (DRAM), double data rate synchronous dynamic RAM (DDR SDRAM), static RAM (SRAM), thyristor RAM (T-RAM), and zero capacitor RAM (Z-RAM). The ROM may include a mask ROM (MROM), a programmable ROM (PROM), an erase programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a compact disk ROM (CD-ROM), a digital versatile disk ROM, and the like. In some embodiments, memory 220 may store one or more programs and/or instructions to perform the example methods described herein. For example, memory 220 can store a program that can cause processing engine 120 to determine a regular item.

I/O 230 can input and/or output signals, data, information, and the like. In some embodiments, I/O 230 can implement user interaction with processing engine 120. In some embodiments, I/O 230 can include an input device and an output device. Examples of the input device may include a keyboard, a mouse, a touch screen, a microphone, and the like, or a combination thereof. Examples of the output device may include a display device, a speaker, a printer, a projector, and the like, or a combination thereof. Examples of the display device may include a liquid crystal display (LCD), a light emitting diode (LED) based display, a flat panel display, a curved screen, a television device, a cathode ray tube (CRT), a touch screen, and the like, or a combination thereof.

Communication port 240 can be connected to a network (e.g., network 150) to facilitate data communication. Communication port 240 may establish a connection between processing engine 120 and smart wheelchair 110, terminal 140, and/or memory 130. The connection may be a wired connection, a wireless connection, any other communication connection that enables data transmission and/or reception, and/or any combination of these connections. Wired connections may include, for example, cables, fiber optic cables, telephone lines, and the like, or any combination thereof. The wireless connection may comprise, for example, Bluetooth ^TM connection, Wi-Fi ^TM connections, WiMax ^TM communication, WLAN connection, ZigBee connection, a mobile network connection (e.g., 3G, 4G, 5G, etc.) and the like, or combinations thereof. In some embodiments, communication port 240 can be (or include) a standardized communication port, such as RS232, RS485, and the like. In some embodiments, communication port 240 can be a specially designed communication port. For example, communication port 240 can be designed in accordance with an autonomous driving communication protocol.

3 is a schematic diagram of exemplary hardware and/or software components of an exemplary mobile device 300, which may be implemented on a mobile device 300, in accordance with some embodiments of the present application. As shown in FIG. 3, mobile device 300 can include communication platform 310, display 320, graphics processing unit (GPU) 330, central processing unit (CPU) 340, I/O 350, memory 360, and memory 390. In some embodiments, mobile device 300 may also include any other suitable components including, but not limited to, a system bus or controller (not shown). In some embodiments, operating system 370 ^{^{(e.g., iOS TM, Android TM, Windows}} Phone TM , etc.) and one or more application programs 380 may be loaded from memory 390 to the memory 360, thereby performing by the CPU 340. Application 380 can include a browser or any other suitable mobile application for receiving and presenting information regarding image processing or other information from processing engine 120. Information flow interaction with the user may be implemented by I/O 350 and provided to the processing engine 120 and/or other components of the intelligent wheelchair automated driving system 100 via the network 150.

To implement the various modules, units, and functions thereof described in this application, a computer hardware platform can be utilized as a hardware platform for one or more of the elements described in this application. A computer with user interface elements can be used to implement a personal computer (PC) or any other type of workstation or external device. The computer can also be used as a server if properly programmed.

FIG. 4 is a schematic diagram of an exemplary processing engine 120, shown in accordance with some embodiments of the present application. The processing engine 120 can include an acquisition module 402, a processing module 404, a detection module 406, a control module 408, and a storage module 410. At least a portion of the processing engine 120 can be implemented on a computing device as shown in FIG. 2 or a mobile device as shown in FIG.

The acquisition module 402 can acquire data. In some embodiments, the acquisition module 402 can obtain data related to the smart wheelchair 110 from the smart wheelchair 110, the memory 130, the terminal 140, and/or an external data source (not shown). In some embodiments, the data may include raw data (eg, travel data), instructions, and the like, or a combination thereof. For example, the travel speed of the smart wheelchair can be generated based on the rolling speed of the smart wheelchair wheel. The instructions may be executed by a processor of processing engine 120 to implement the example methods described herein. In some embodiments, the acquired data can be transmitted to the storage module 410 for storage.

Processing module 404 can process the data. Processing module 404 can process information provided by multiple modules of processing engine 120. Processing module 404 can process data that has been acquired by acquisition module 402, data detected by detection module 406, data obtained from storage module 410 and/or memory 130, and the like. For example, the processing module 404 can process the GPS positioning data acquired by the acquisition module 402 to determine the real-time location of the smart wheelchair 110. For another example, the processing module 404 can process the destination information acquired by the acquisition module 402 and determine the travel path of the smart wheelchair 110 according to the real-time location of the smart wheelchair 110. In some embodiments, the user can change the destination in real time, and the processing module 404 can adjust the travel path of the smart wheelchair 110 in real time according to the user's needs. For example, during the automatic driving process, when the user expresses interest in a certain thing (for example, supermarket promotion, street performance, etc.), the original destination information may be modified in real time, and the processing module 404 may perform the purpose of real-time modification. Ground information, adjust the driving path. In some embodiments, the detection module 406 can detect related data of obstacles on the travel path by detecting devices (eg, cameras, radars, distance sensors, etc.) mounted on the smart wheelchair 110, and the processing module 404 can correlate the obstacles. The data is processed. For example, the smart wheelchair 110 may be equipped with a camera, the camera may acquire image data of the obstacle, and the processing module 404 may determine the type of the obstacle according to a preset prediction model.

The detection module 406 can detect the data of the obstacle. The relevant data of the obstacle includes information on the type, location, and size of the obstacle. In some embodiments, the obstacle may be stationary. For example, a vehicle parked on the side of a road. The detecting module 406 can acquire the picture data of the vehicle through the camera, and can detect the movement information of the vehicle by using a radar. Further, these vehicle related information may be sent to the storage module 410 for subsequent operations (eg, sample information as a training model), or may be sent to the processing module 404 for processing to generate related operational instructions. For example, the processing module 404 can generate a corresponding obstacle avoidance instruction according to the position and size information of the vehicle to control the movement of the smart wheelchair 110 (eg, deceleration, steering, etc.). In some embodiments, the obstacle may be an object that moves in real time (eg, a vehicle or pedestrian moving in real time). At this time, the detecting module 404 can detect the moving speed information of the object, and the processing module 406 can predict the moving trajectory of the object based on the speed information of the object, and determine whether it will collide with the smart wheelchair 110. If the result of the determination is yes, the smart wheelchair 110 can automatically adjust the movement state (such as braking, steering, etc.) for effective and reasonable avoidance. Further, if the detected moving object is a pedestrian, the smart wheelchair 110 may identify the pedestrian and determine whether the pedestrian is a person recognized by the user, and if the judgment result is yes, issue a reminder. The identifying the identity of the pedestrian may identify the walking posture of the pedestrian by detecting the picture information acquired by the device. The manner of issuing the reminder may include a display prompt, a voice prompt, a vibration prompt, and the like.

The control module 408 can accept the control instruction information sent by the processing module 406, stored in the storage module 410, or manually input by the user, to control the wheelchair. The control instructions may include a movement control instruction, a function control instruction, and the like. The movement control commands include deceleration, braking, steering, etc. of the wheelchair. The function control command includes a temperature adjustment control command, a voice/video open command, and a seat adjustment command. In some embodiments, control module 408 can receive real-time instructions from a user or predetermined instructions provided by a user to control one or more operations of smart wheelchair 110, acquisition module 402, and/or processing module 406. For example, when an obstacle is detected, the processing module 404 issues an obstacle avoidance command, and the control module can perform braking and steering control on the smart wheelchair 110 according to the instruction. For another example, when receiving the seat adjustment command of the user, the control module can automatically adjust the seat tilt angle of the smart wheelchair. For another example, the smart wheelchair 110 can automatically turn on the voice/video call function when receiving the user's voice/video open command. In some embodiments, control module 408 can communicate with one or more other modules of processing engine 120 for information and/or data exchange.

The storage module 410 can store image data, detection data, control parameters, processed image data, and the like, or a combination thereof. In some embodiments, storage module 410 can store one or more programs and/or instructions that can be executed by a processor of processing engine 120 to implement the example methods described herein. For example, storage module 410 can store programs and/or instructions that can be executed by the processor of processing engine 120 to acquire obstacle information, generate obstacle based data, and/or display any intermediate results.

In some embodiments, one or more of the modules shown in FIG. 4 can be implemented in at least a portion of the imaging system shown in FIG. For example, the acquisition module 402, the control module 408, the storage module 410, and/or the processing module 404 can be integrated into a console (not shown). Through the console, the user can set the destination of the trip, select the route to travel, set the speed of the trip, change the route of travel, and the like. In some embodiments, the console can be implemented by the processing engine 120 and/or the terminal 140.

FIG. 5 is a block diagram of an exemplary processing module 406, shown in accordance with some embodiments of the present application. The processing module 404 can include a candidate path determining unit 502, a path scoring unit 504, a sorting unit 506, and a travel path determining unit 508. Processing module 406 can be implemented on multiple components (e.g., processor 210 of computing device 200 as shown in FIG. 2). For example, at least a portion of the processing module 406 can be implemented on a computing device as shown in FIG. 2 or a mobile device as shown in FIG.

The candidate path determining unit 502 can determine a plurality of candidate paths. In some embodiments, the plurality of paths have the same starting point and the same destination. The starting point is the current location of the smart wheelchair. The current location of the smart wheelchair can be obtained by existing positioning techniques. The positioning techniques used in this application may include Global Positioning System (GPS), Global Navigation Satellite System (GLONASS), Beidou Navigation System (COMPASS), Galileo Positioning System, Quasi-Zenith Satellite System (QZSS), Wireless Fidelity (WiFi). Positioning technology, picture recognition positioning, etc. or any combination of the above examples. One or more of the above positioning techniques may be used interchangeably in this application. The endpoint can be selected or entered by the user. For example, the system can provide the user with a plurality of recent destination information for the user to select based on historical travel records. When the system provides a destination that does not contain user requirements, the user can enter a new destination. The input method includes manual input, voice input, and the like.

After determining the starting point of the smart wheelchair 110 travel and the destination of the user, the candidate path determining unit 502 can determine a plurality of candidate routes accordingly. In some embodiments, the plurality of candidate paths may have different path characteristics. For example, having different lengths, the length of the trunk route in the route is different, whether it passes through a certain area, and the like. In some embodiments, the different path characteristics and the corresponding multiple candidate paths may be provided to the user together to provide a reference for the user's selection. As shown in FIG. 8 , the route 1 has the shortest route length; the route of the trunk route has the highest proportion of the route and the route of the route 3 can pass the user's region of interest C.

The path scoring unit 504 can separately score the plurality of candidate paths. In some embodiments, the rating is determined based on the path characteristics. Each path feature corresponds to a weight, and the score of each candidate path takes into account the path characteristics. For example: consider the length of the path, the number of traffic lights, the number of turns required, the degree of road congestion, and so on. Each path feature can be scored, and the weighted value of each path feature is multiplied to obtain a comprehensive score for the path. The scores of the above path features corresponding to a candidate route may be 10, 10, 8, and 6 points, respectively, and the corresponding weight values may be 0.3, 0.1, 0.2, and 0.4, respectively. Thus, the comprehensive score of the last modified candidate route can be obtained as 8 points. It should be noted that the score can be displayed with a specific score, or by level, star rating, and the like.

The sorting unit 506 can sort the plurality of candidate routes according to the comprehensive score of each candidate path, and present the sorting result to the client for selection. The travel route determining unit 508 can determine the final travel route based on the result of the customer's selection. In some embodiments, the final travel path of the wheelchair may be any of the candidate paths or a user-defined travel path.

It should be noted that the above description of the processing module 406 is for illustrative purposes only and is not intended to limit the scope of the application. Many variations or modifications can be made by those skilled in the art in light of the teachings herein. However, such variations and modifications do not depart from the scope of the present application. For example, sorting unit 506 can be integrated into path score 504 and/or travel path determination unit 508. As another example, candidate path unit 502 can be omitted or integrated into path scoring unit 504.

6 is an exemplary 600 flow diagram of intelligent wheelchair automated driving, in accordance with some embodiments of the present application. In some embodiments, the process 600 can include: obtaining destination information 602, determining a current location 604 of the wheelchair, determining a travel path 606 of the wheelchair based on the destination information and the current location, detecting obstacle information 608 in the travel path of the wheelchair, and The movement 610 of the wheelchair is adjusted based on the detected obstacle information. In some embodiments, one or more operations for automated wheelchair driving of flow 600 shown in FIG. 6 may be implemented in imaging system 100 shown in FIG. For example, the process 600 shown in FIG. 6 may be stored in the memory 130 in the form of instructions, and by the processing engine 120 (eg, the processor 210 of the computing device 200 shown in FIG. 2, the CPU of the mobile device 300 shown in FIG. 340) Call and/or execute.

In operation 602, destination information of the user may be acquired. Operation 602 is performed by acquisition module 402. In some embodiments, the destination information can be obtained from the smart wheelchair 110, the memory 130, the terminal 140, or an external data source. As used herein, the destination information may include the name of the destination, the latitude and longitude coordinates of the destination, and the like. The user can directly enter the name of the destination, enter the latitude and longitude coordinates of the destination, and specify the destination via an electronic map. In some embodiments, the user may provide the user with a selection by selecting the destination of the most recent travel in the historical travel record. In some embodiments, when a destination is provided to a client, the probability of the user selecting the corresponding destination may be predicted by a recommendation model, machine learning, or artificial neural network, and the destination with high probability may be prioritized or presented in a conspicuous manner. To the user.

In operation 604, the current location of the wheelchair can be determined. Operation 604 is performed by processing module 404. In some embodiments, the current location of the wheelchair can be determined based on one or more positioning techniques mentioned in this application. For example, operation 604 can be implemented based on GPS positioning techniques. In some embodiments, the positioning technology may further include a global satellite navigation system (GLONASS), a Beidou navigation system (COMPASS), a Galileo positioning system, a quasi-zenith satellite system (QZSS), a wireless fidelity (WiFi) positioning technology, etc., or the like. Any combination of examples. One or more of the above positioning techniques may be used interchangeably in this application.

In operation 606, a travel path of the wheelchair may be determined based on the destination information and the current location. Operation 606 is performed by processing module 404. In some embodiments, the travel path of the wheelchair may be system recommended or user input. For example, after obtaining destination information and current location information for the wheelchair, the system can determine a plurality of candidate paths for the user to select. The user can select any of these candidate routes as the driving path of the wheelchair. In some embodiments, the user can customize a route as a driving route for the wheelchair. For example, when the system determines that the current wheelchair position is "home" and then the user input destination is "supermarket". The system can recommend three candidate routes for the user. These three candidate routes have the shortest distance, the least traffic lights and the least bends. If the user feels that they need to go to another location (such as a breakfast shop) at this time, the three recommended routes may not meet the user's needs. At this point, the user can customize the driving path of the wheelchair to meet the actual needs of the user. In some embodiments, the user can change the travel path of the wheelchair in real time. For example, when the user receives a reminder of the transaction of interest and confirms interest in it, the system can modify the driving path of the wheelchair in real time to meet the real-time needs of the user.

In operation 608, obstacle information in the travel path of the wheelchair can be detected. Operation 608 is performed by detection module 406. In some embodiments, multiple detection devices (eg, cameras, radar, distance sensors, etc.) can be mounted on the smart wheelchair 110. With these detection devices, the system can detect conditions around the smart wheelchair 110. For example, obstacle information on the route of travel. The obstacle may be stationary, such as a car parked on the side of the road. It can also be mobile, such as pedestrians walking on the road. In some embodiments, the moving object may not be on the travel route at the beginning, ie, does not interfere with the travel of the wheelchair, but during its subsequent movement, may move to the path of the wheelchair and travel to the wheelchair Such moving objects are also considered to be obstacles when the normal driving causes interference.

In some embodiments, operation 608 can obtain information of an obstacle. The obstacle information may include the type, position, size, and motion state of the obstacle. For example, a picture of an obstacle can be acquired by using a camera, and the type of the obstacle can be determined by a picture recognition method according to the picture of the obstacle. The method for picture recognition includes: machine learning, deep learning, artificial neural network, and the like. For another example, the relative position information of the obstacle and the wheelchair, the actual size and speed information of the obstacle, and the like can be obtained by using a radar and/or a distance sensor. The acquired obstacle information may provide a reference for the system to plan an obstacle avoidance scheme.

In operation 610, the movement of the wheelchair may be adjusted based on the detected obstacle information. Operation 610 is performed by control module 408. In some embodiments, adjusting the movement of the wheelchair includes adjusting the direction of movement of the wheelchair (to the left or to the right) and adjusting the speed (acceleration or deceleration) of the wheelchair. For example, during the travel of the smart wheelchair 110, when it is detected that a car is parked on the right side of the roadside, the system will adjust the smart wheelchair 110 to move to the left to avoid the car. For another example, during the traveling of the smart wheelchair 110, when it is detected that there is a pedestrian in front of the smart wheelchair, and the user confirms that the pedestrian recognizes it, the system can adjust the moving speed (acceleration) of the smart wheelchair to catch up with the The pedestrian in front of the smart wheelchair. Also for example, during the travel of the smart wheelchair 110, when it is detected that there is an intersection ahead and there is a car traveling in other directions, the system will adjust the speed of the smart wheelchair 110 to the moving speed (deceleration) to prevent traveling with the other directions. The car collided.

It is to be noted that the above description is for illustrative purposes only and is not intended to limit the scope of the application. Many variations or modifications can be made by those skilled in the art in light of the teachings herein. However, such variations and modifications do not depart from the scope of the present application. For example, the process 600 can include an operation for pre-processing obstacle related information prior to 610.

FIG. 7 is an exemplary flow diagram 700 of determining a travel path for a wheelchair, in accordance with some embodiments of the present application. The process 700 can be performed by the processing module 406. In some embodiments, operation 606 shown in FIG. 6 can be performed in accordance with process 700. In some embodiments, one or more operations of determining the travel path of the wheelchair of the process 700 illustrated in FIG. 7 may be implemented in the imaging system 100 illustrated in FIG. For example, the process 700 shown in FIG. 7 may be stored in the memory 130 in the form of instructions and by the processing engine 120 (eg, the processor 210 of the computing device 200 shown in FIG. 2, the CPU of the mobile device 300 shown in FIG. 340) Call and/or execute.

In operation 702, a plurality of candidate paths may be calculated based on the current location and destination information. Operation 702 can be performed by candidate path determination unit 502. In some embodiments, after obtaining the destination information of the user and determining the current location of the user, the system may determine a plurality of candidate travel paths for the customer to select according to the acquired destination and the current location. In some embodiments, the plurality of candidate paths may have different path characteristics. For example, having different lengths, the length of the trunk route in the route is different, whether it passes through a certain area, and the like. In some embodiments, the different path characteristics and corresponding multiple candidate paths may be provided to the user together to provide a reference for the user's selection.

In operation 704, the plurality of candidate paths may be scored. Operation 704 can be performed by path scoring unit 504. In some embodiments, the score is related to the path characteristics of the candidate path. Each path feature corresponds to a weight, and different path characteristics may have the same weight. The score of each candidate path needs to take into account a plurality of the characteristics of the path, such as: considering the path length, the number of traffic lights, the number of corners to be turned, and the degree of road congestion. In some embodiments, the system can score each path characteristic and multiply the weight value of each path feature to obtain a composite score for the path. The rating may be a numerical score (eg, 8.0 points, 9.5 points, etc.) or a rating (eg, 3 stars, 5 stars, etc.).

In operation 706, the plurality of candidate paths may be sorted according to the level of the score, and the sorted plurality of candidate paths are presented to the user for selection by the user. Operation 706 can be performed by sequencing unit 506. In some embodiments, the system can rank multiple candidate routes from highest to lowest scores and present the highest ranked priority to the user. For example, the candidate track with the highest score can be marked as the recommended route, and the other candidate paths are marked as the alternate route. For another example, when presenting a candidate path to the user, the candidate path with the highest score is placed at the top or most conspicuous position. For another example, when the candidate path is presented in the electronic map, the candidate path with the highest score can be highlighted (such as setting a color different from other candidate paths, or setting the highest transparency, etc.).

In operation 708, the travel path of the wheelchair may be determined based on the result of the user's selection. Operation 708 can be performed by travel path determination unit 508. In some embodiments, the user may select any one of the plurality of candidate paths as the driving path of the wheelchair, such as the user may directly select the candidate route with the highest rating as the driving path of the wheelchair. In some embodiments, the user can customize the driving route of the wheelchair according to the actual situation. For example, in an actual situation, the user may need to pass through a different location than the destination, and the driving path of the wheelchair may be set by an input device (manual input device, voice input device, or other input device having similar functions). To meet user needs.

It is to be noted that the above description is for illustrative purposes only and is not intended to limit the scope of the application. Many variations or modifications can be made by those skilled in the art in light of the teachings herein. However, such variations and modifications do not depart from the scope of the present application. For example, the process 700 can include an operation for screening candidate paths prior to 704 to delete portions of the inappropriate candidate routes.

8 is a schematic diagram of an intelligent wheelchair autopilot shown in accordance with some embodiments of the present application. As shown in Fig. 8, A is the current position of the wheelchair, B is the destination of the user, and C is the thing of interest to the user. After determining the user's destination B and the current location A of the wheelchair, the system has planned three candidate routes for the user (represented by 1, 2, 3, respectively) for selection. Route 1 has the shortest distance, Route 2 has the highest proportion of trunk roads, and Route 3 passes the point of interest C.

Different aspects of the method outlined above and/or methods of implementing other steps by the program. Program portions of the technology may be considered to be "products" or "articles of manufacture" in the form of executable code and/or related data, which are embodied or implemented by a computer readable medium. Tangible, permanent storage media includes the memory or memory used by any computer, processor, or similar device or associated module. For example, various semiconductor memories, tape drives, disk drives, or the like that can provide storage functions for software at any time.

All software or parts of it may sometimes communicate over a network, such as the Internet or other communication networks. Such communication can load software from one computer device or processor to another. For example, a system that loads from a management server or host computer of an on-demand service system to a computer environment, or other computer environment that implements the system, or a similar function associated with the information needed to provide on-demand services. Therefore, another medium capable of transmitting software elements can also be used as a physical connection between local devices, such as light waves, electric waves, electromagnetic waves, etc., through cable, fiber optic cable or air. Physical media used for carrier waves such as cables, wireless connections, or fiber optic cables can also be considered as media for carrying software. Usage herein Unless the tangible "storage" medium is limited, other terms referring to a computer or machine "readable medium" mean a medium that participates in the execution of any instruction by the processor.

Thus, a computer readable medium can take many forms, including but not limited to, a tangible storage medium, carrier medium or physical transmission medium. Stable storage media include: optical or magnetic disks, as well as storage systems used in other computers or similar devices that enable the implementation of the system components described in the figures. Unstable storage media include dynamic memory, such as the main memory of a computer platform. Tangible transmission media include coaxial cables, copper cables, and fiber optics, including the circuitry that forms the bus within the computer system. The carrier transmission medium can transmit electrical signals, electromagnetic signals, acoustic signals or optical signals, which can be generated by radio frequency or infrared data communication methods. Typical computer readable media include hard disks, floppy disks, magnetic tape, any other magnetic media; CD-ROM, DVD, DVD-ROM, any other optical media; perforated cards, any other physical storage media containing aperture patterns; RAM, PROM , EPROM, FLASH-EPROM, any other memory slice or tape; a carrier, cable or carrier for transmitting data or instructions, any other program code and/or data that can be read by a computer. Many of these forms of computer readable media appear in the process of the processor executing instructions, passing one or more results.

The computer program code required for the operation of various parts of the application can be written in any one or more programming languages, including object oriented programming languages such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB.NET, Python. Etc., regular programming languages such as C, Visual Basic, Fortran 2003, Perl, COBOL 2002, PHP, ABAP, dynamic programming languages such as Python, Ruby and Groovy, or other programming languages. The program code can run entirely on the user's computer, or run as a stand-alone software package on the user's computer, or partially on the user's computer, partly on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer via any network, such as a local area network (LAN) or wide area network (WAN), or connected to an external computer (eg, via the Internet), or in a cloud computing environment, or as Service usage such as Software as a Service (SaaS).

Those skilled in the art will appreciate that many variations and modifications can be made in the present disclosure. For example, the various system components described above are implemented by hardware devices, but may also be implemented only through software solutions. For example: install a system on an existing server. Moreover, the provision of location information disclosed herein may be accomplished by a combination of firmware, firmware/software, firmware/hardware, or hardware/firmware/software.

The above describes the present application and/or some other examples. According to the above, the present application can also make various modifications. The subject matter disclosed herein can be implemented in various forms and examples, and the application can be applied to a large number of applications. All applications, modifications and variations as claimed in the following claims are intended to be within the scope of the application.

Claims

A method for controlling automatic driving of a wheelchair, comprising:

Obtain destination information;

Determining a current location of the wheelchair;

Determining a driving path of the wheelchair according to the current location and destination information;

Detecting obstacle information in the driving path of the wheelchair;

The movement of the wheelchair is adjusted based on the detected obstacle information.
The method according to claim 1, wherein the obtaining of the destination information comprises any one of a user manual input, a voice input, and a system recommendation input.
The method according to claim 1, wherein the determining the driving path of the wheelchair based on the current location and destination information comprises:

Calculating a plurality of candidate paths according to the current location and destination information;

Scoring the plurality of candidate paths;

Sorting a plurality of candidate paths according to the level of the score, and presenting the sorted candidate paths to the user for selection by the user;

The driving path of the wheelchair is determined based on the result of the user's selection.
The method according to claim 1, wherein said user is allowed to set a travel speed of the wheelchair after determining the travel path of the wheelchair.
The method according to claim 1, wherein the obstacle information includes a type, a position, a size, and a motion state of the obstacle.
The method of claim 1 wherein said adjusting said movement of said wheelchair comprises adjusting a speed of the wheelchair, controlling wheelchair steering, and controlling wheelchair braking.
The method of claim 1 further comprising providing the user with the item of interest and automatically adjusting the route of travel when the user confirms that he is interested in the item.
A system for controlling automatic driving of a wheelchair, comprising: an obtaining module, a processing module, a detecting module and a control module;

The obtaining module is configured to acquire destination information;

The processing module is configured to determine a current location of the wheelchair, and determine a driving path of the wheelchair according to the current location and destination information;

The detecting module is configured to detect obstacle information in a driving path of the wheelchair; and

The control module is configured to adjust movement of the wheelchair according to the detected obstacle information.
The system according to claim 8, wherein said processing module comprises a candidate path determining unit, a path scoring unit, a sorting unit, and a travel path determining unit,

The candidate path determining unit calculates a plurality of candidate paths according to the current location and destination information;

The path scoring unit scores the plurality of candidate paths;

The sorting unit sorts the plurality of candidate paths according to the level of the score, and presents the sorted plurality of candidate paths to the user for the user to select;

The travel route determining unit determines a travel route of the wheelchair based on a result of the user's selection.
A computer readable storage medium, characterized in that the storage medium stores a computer program that, when executed, performs the following methods:

Obtaining the destination information of the wheelchair and the location information of the current wheelchair, and determining the driving route of the wheelchair according to the destination information and the current location information;

The obstacle information in the travel path of the wheelchair is detected, and the movement of the wheelchair is adjusted based on the detected obstacle information.