CN113576780A

CN113576780A - Intelligent wheelchair based on semantic vision SLAM

Info

Publication number: CN113576780A
Application number: CN202110889170.XA
Authority: CN
Inventors: 袁亮; 陈渊博
Original assignee: Beijing University of Chemical Technology
Current assignee: Beijing University of Chemical Technology
Priority date: 2021-08-04
Filing date: 2021-08-04
Publication date: 2021-11-02

Abstract

The invention discloses an intelligent wheelchair based on semantic vision SLAM, which mainly comprises a wheelchair main body and an intelligent motion control mechanism, wherein the wheelchair main body comprises a chassis, a seat, a backrest, a leg guard and other mechanisms, and the intelligent motion control mechanism comprises a binocular camera, an intelligent operation panel, an integrated intelligent control module and the like. The intelligent wheelchair acquires images by means of a binocular camera, semantic road signs are detected by combining with deep learning, an environment map is constructed by fusing semantics and geometric information, and autonomous navigation is completed on the constructed semantic map. The intelligent wheelchair uses ultrasonic waves and an infrared distance measuring sensor to automatically avoid obstacles during autonomous operation, and meanwhile, the LED illuminating lamp helps the intelligent wheelchair to reliably operate at night or in an environment with insufficient illumination. The intelligent wheelchair has the functions of key touch screen, voice input, stable positioning, autonomous navigation, real-time obstacle avoidance and the like, helps people with inconvenient actions to go out independently, effectively relieves the pressure of society and nursing staff, and has a great application prospect.

Description

Intelligent wheelchair based on semantic vision SLAM

Technical Field

The invention relates to the technical field of intelligent wheelchairs and control thereof, in particular to an intelligent wheelchair based on semantic vision SLAM.

Background

With the aging of the social population and the increase of the disabled, providing a suitable transportation tool for the elderly and the disabled with inconvenient actions becomes one of the hot problems of social attention. The progress of society and the increase in the level of technology make the combination of wheelchairs with modern high and new technologies more urgent, with the aim of improving their quality and freedom of life. Traditional electronic wheelchair compares with manual wheelchair and has increased power supply and the control unit, still needs the user to control velocity of motion and direction always, also can not be according to task demand autopilot, and is high to user's operation requirement, and intelligence and human-computer interaction degree are low.

The demand of society for wheelchairs is increasing, and the requirement for the intelligence of wheelchairs is increasing. The autonomous navigation capability of the intelligent wheelchair is an important mark for measuring the intelligence of the intelligent wheelchair, and accurate navigation depends on accurate positioning and map construction. Simultaneous localization and mapping (SLAM) technology helps the wheelchair to localize in an unknown environment and construct an environmental map by collecting sensor data. The intelligent wheelchair based on the traditional visual SLAM is limited in application in a complex scene based on simple geometric characteristics, the computer vision technology based on deep learning is rapidly developed, and the accuracy and the stability of wheelchair positioning can be improved by combining semantic information and geometric information.

The intelligent wheelchair has wide market prospect, can help the old and the disabled to independently live, alleviates social and family's nursing pressure. Therefore, the development of the intelligent wheelchair which can realize stable positioning, autonomous navigation, real-time obstacle avoidance and simple operation has important social significance and application value.

Disclosure of Invention

The invention aims to overcome the defects of the prior art and provide an intelligent wheelchair based on semantic vision SLAM, wherein a binocular camera is used as a main information acquisition tool, a touch panel interface can be adopted to issue tasks in a one-key mode, the voice task issuing is supported, the reliable positioning and navigation of the intelligent wheelchair can be realized, and the intelligent wheelchair is suitable for different users and application environments.

In order to achieve the purpose, the invention provides the following technical scheme: the invention provides an intelligent wheelchair based on semantic vision SLAM, which comprises: comprises a chassis mechanism, a seat mechanism, a backrest mechanism, a leg protection mechanism and an intelligent motion control mechanism.

The chassis mechanism comprises two wheel driving motors arranged on the bottom surface of a chassis supporting frame, wheelchair rear wheels respectively connected with the driving motors and two front wheels connected through a front wheel connecting frame. The front wheel and the front wheel connecting frame are connected through the double-seal-face bearing and the thickening rivet to form a movable universal front wheel, and the movable universal front wheel can flexibly move while the load is increased. The front wheel and the rear wheel are made of rubber or plastic materials, the front wheel is of a solid tire structure, and the rear wheel is of an inflation-free explosion-proof tire structure, so that the bearing weight of the wheelchair can be enhanced; the wheel tire adopts horizontal and vertical mixed patterns, so that the ground contact area can be increased, and meanwhile, better flexibility is kept.

The seat mechanism is arranged on the chassis supporting frame and comprises a seat cushion block connected with the wheelchair cushion connecting frame, side guardrail handrails respectively connected to the left side and the right side, and a cushion arranged on the seat cushion block. The side guardrail handrail comprises structure support frame, baffle and extension support handrail, and the extension support handrail has can dismantle wear-resisting non-slip property, the cushion uses the ventilative memory cotton pad that kick-backs slowly of honeycomb, can fully alleviate buttock and waist pressure, increases wheelchair comfort in use.

The backrest mechanism comprises an upper backrest cushion plate and a lower backrest cushion plate which are respectively arranged at the upper end and the lower end of the backrest connecting frame, a back cushion connected with the upper backrest cushion plate and backrest armrests connected with the lower backrest cushion plate. The angle between the backrest mechanism and the horizontal ground surface can be adjusted within the range of 90 degrees to 150 degrees, and the backrest armrests support manual pushing and can be used for daily walking exercise.

Leg guard mechanism includes two leg guard backing plates of being connected with the leg guard link, installs leg guard and callus on the sole on the leg guard backing plate, the callus on the sole structure comprises two callus on the sole cushion blocks and callus on the sole link. Leg guard mechanism and horizontal ground angle can be adjusted at 90 degrees to 150 degrees within ranges, have multiunit callus on the sole link mounting hole on leg guard backing plate and the leg guard, can adjust callus on the sole mounted position according to different needs.

The intelligent control mechanism comprises a binocular camera, an intelligent operation panel, an emergency stop switch and an intelligent integrated control module. The binocular camera is horizontally arranged on the intelligent operation panel, and the binocular camera and the intelligent operation panel are both arranged at one end of the right armrest of the wheelchair; the emergency stop switch is arranged at one end of a left side armrest of the wheelchair, so that the power supply can be quickly cut off.

Furthermore, a motion control board of the intelligent controller module is connected with a driving motor, and the motion of the two rear wheels is controlled by differential speed, so that the functions of speed conversion and direction rotation are realized.

Furthermore, the chassis mechanism and the seat mechanism are provided with front and rear damping springs during installation, so that the whole frame is damped, road jolts can be effectively filtered, stable operation of the wheelchair is guaranteed, and a user can comfortably use the wheelchair under different road conditions. The outside of the side guardrail handrail that the LED light was installed, accessible intelligent operation panel controls help intelligent wheelchair work in the environment that the illumination is not enough.

Furthermore, the intelligent operation panel is connected with the intelligent controller module, environment information is obtained through a binocular camera, semantic objects are identified by combining a deep learning method, a semantic map of the environment is constructed, and the wheelchair is helped to realize independent positioning and navigation in different environments.

Furthermore, ultrasonic sensors and infrared distance measuring sensors are installed in four directions of the intelligent wheelchair, when the wheelchair encounters an obstacle in the autonomous operation process, automatic obstacle avoidance is carried out, and a path is re-planned until a navigation task is finished.

The invention has the following beneficial effects: the intelligent wheelchair provided by the invention combines the semantic vision SLAM technology, utilizes the binocular camera to collect image information, combines the semantic detection algorithm to construct the semantic map of the environment, and can realize autonomous positioning and navigation according to the semantic map. In the process of autonomous navigation, obstacles are automatically avoided by matching ultrasonic sensors and infrared distance measuring sensors arranged around the wheelchair, and meanwhile, an emergency stop switch is matched to deal with the sudden situation, so that the safety of autonomous operation is improved. On the other hand, the intelligent wheelchair can help the intelligent wheelchair to reliably work in a dark environment by combining the LED illuminating lamp, is suitable for outdoor and indoor environments, has outstanding adaptivity, and has wide market prospect and good economic benefit.

Drawings

FIG. 1 is a schematic overall structure diagram of the intelligent wheelchair;

FIG. 2 is a schematic front view of the intelligent wheelchair;

FIG. 3 is a schematic left side view of the present intelligent wheelchair;

FIG. 4 is a schematic top view in elevation of the present intelligent wheelchair;

FIG. 5 is a schematic structural view of a wheelchair main body of the intelligent wheelchair;

FIG. 6 is a schematic structural view of the intelligent control mechanism of the intelligent wheelchair

FIG. 7 is an overall workflow diagram of the present intelligent wheelchair;

fig. 8 is a flow chart of the whole semantic vision SLAM process of the intelligent wheelchair combined with a binocular camera.

In the figure: 1-chassis support frame, 2-front wheel link, 3-front wheel, 4-rear wheel, 5-wheelchair cushion link, 6-backrest link, 7-side guardrail armrest, 8-backrest armrest, 9-seat cushion, 10-cushion, 11-lower backrest cushion, 12-upper backrest cushion, 13-back cushion, 14-right armrest support, 15-operating panel link, 16-intelligent operating panel, 17-camera link, 18-binocular camera, 19-driving motor, 20-intelligent control module, 21-damping spring, 22-leg guard link, 23-left hand slider, 24-leg guard cushion, 25-foot pad, 26-foot rest link, 27-leg guard, 28-emergency stop switch, 29-LED lighting lamp.

Detailed Description

In order to make the technical solution and the achievement effect of the present invention easy to understand, the present invention will be further explained with reference to the accompanying drawings.

Examples

The embodiment discloses a semantic vision SLAM-based intelligent wheelchair, which comprises a wheelchair mechanical body and an intelligent motion control module, wherein the wheelchair mechanical body mainly comprises a chassis, a seat, a backrest, legs and the like, as shown in figures 1-6.

The chassis mechanism comprises a chassis supporting frame 1, a driving motor 19, wheelchair rear wheels 4, a front wheel connecting frame 2 and front wheels 3; the front wheel and the front wheel connecting frame are connected through a double-seal bearing and a thickening rivet to form a movable universal front wheel, the two driving motors are respectively connected with a left rear wheel and a right rear wheel of the wheelchair, and the movement of the wheelchair is controlled in a differential driving mode. The seat mechanism comprises a wheelchair cushion connecting frame 5, a seat cushion block 9, a cushion 10 and side guardrail handrails 7 on the left side and the right side; the middle positions of the left side and the right side of the cushion connecting frame are respectively installed on the side guardrail handrails, so that the intelligent operation panel is convenient to support and operate by hands. The backrest mechanism comprises a backrest connecting frame 6, a lower backrest cushion plate 11, an upper backrest cushion plate 12, a back cushion 13 and backrest armrests 8; the cushion 10 and the back cushion 13 both use honeycomb-structure breathable slow-rebound memory cotton cushions, so that the comfort of a user is improved; the leg protection mechanism comprises a leg protection connecting frame 22, a leg protection cushion plate 24, a leg protection 27, a foot pad connecting frame 26 and a foot pad cushion block 25; the leg protection cushion plate 24 and the leg protection 27 are provided with a plurality of groups of foot pad connecting frame 26 mounting holes, and the height of the foot pads can be adjusted according to different requirements. The intelligent motion control section includes a binocular camera 18, an intelligent operating panel 16, an integrated intelligent control module 20, and an emergency stop switch 28. The binocular camera 18 is horizontally installed on the intelligent operation panel 16 through the camera connecting support 17, the intelligent operation panel 16 is connected with the operation panel connecting support 15 through fastening screws, the operation panel connecting support 15 is concentrically connected with the right armrest support 14 fixed on the right side of the wheelchair, and the adjustment of the left and right ranges of 270 degrees can be realized. The binocular camera 18 and the intelligent operation panel 16 are both installed on the right side of the wheelchair, the intelligent integrated control module 20 mainly comprises a processor module and a motion control module, image information acquired by the binocular camera is processed by the processor module, rotating speed information of a rear wheel of the wheelchair is obtained according to a navigation task and the wheelchair model, and the rotating speeds of the two driving motors are respectively controlled through instructions issued by the motion control module, so that the advancing and steering functions of the intelligent wheelchair are realized.

Preferably, in order to better implement the present invention, the following configuration is adopted by further optimizing on the basis of the above embodiment: as shown in fig. 1 to 6, the LED lighting lamp 29 is installed outside the side rail handrail, and can be turned on or off by a switch button on the intelligent operation panel, so as to help the intelligent wheelchair to work in an environment with insufficient illumination. The emergency stop switch is arranged at the front end of a left side armrest of the wheelchair and can be quickly pressed down in case of emergency.

Preferably, in order to better implement the present invention, the following configuration is adopted by further optimizing on the basis of the above embodiment: as shown in fig. 7, the workflow of the intelligent wheelchair based on semantic vision SLAM is the first work to create a map of the surroundings. Starting a wheelchair power supply, supplying power to a device needing power supply by a battery, and initializing parameters of each device; the user passes through the passable road motion of intelligent operation panel control wheelchair in the environment, and the map is built to the wheelchair before the motion, and the wheelchair will be fixed a position and semantic map is built according to the information that binocular camera acquireed, will save automatically also can manually after map is built, follow-up can call when automatic navigation.

Preferably, in order to better implement the present invention, the following configuration is adopted by further optimizing on the basis of the above embodiment: as shown in fig. 8, the binocular semantic SLAM project is used, which mainly includes four threads: a tracking thread, an optimization thread, a loop detection thread and a map building thread. Firstly, geometric feature detection and semantic landmark detection are carried out on a current image obtained by a binocular camera, key points and descriptors for describing surrounding areas are extracted through the geometric feature detection, feature matching is carried out through the obtained ORB feature descriptors, and PnP solution is carried out on successfully matched point pairs to estimate the motion of the camera. Meanwhile, semantic road signs in the image are detected through a deep learning network, and the semantic road signs are associated by using a nonparametric Bayes method to construct semantic road sign distribution in the environment. In the binocular semantic SLAM project, loop detection judges whether the intelligent wheelchair arrives at a position which is reached before, when loop detection is carried out, the rear-end optimization processes the whole pose and track, accumulated errors are eliminated, and an accurate map is constructed.

Preferably, in order to better implement the present invention, the following configuration is adopted by further optimizing on the basis of the above embodiment: the autonomous positioning and navigation are realized by calling an operation map on the basis of the existing constructed map, manually planning a path or automatically planning a motion path according to a target task point, and opening the intelligent wheelchair with the autonomous motion function to perform autonomous positioning and navigation. When a target task is input, the intelligent wheelchair supports button touch screen operation and voice operation aiming at different crowds, the intelligent wheelchair matches the wheelchair position according to the environment information collected by the camera during autonomous operation, the track and the positioning change of the intelligent wheelchair are displayed on the intelligent operation panel in real time, meanwhile, the change of road signs in the environment can be updated, and the updated map is stored after the current task is finished. In the navigation process, when an obstacle is detected on a passing path, the intelligent wheelchair decelerates or stops, and the intelligent wheelchair plans the path again through detecting the surrounding environment and continues to execute the navigation task until the intelligent wheelchair runs to a destination and stops at a specified position.

The present invention is described in detail with reference to the embodiments of the drawings, which are only preferred embodiments of the present invention and are not intended to limit the present invention. Certain details of the examples should not be construed as limitations of the invention, which are to be interpreted as having a scope defined by the appended claims.

Claims

1. An intelligent wheelchair based on semantic vision SLAM is characterized in that: the intelligent wheelchair comprises a chassis mechanism, a seat mechanism, a backrest mechanism, a leg protection mechanism and an intelligent motion control mechanism, wherein the chassis mechanism comprises two wheel driving motors (19) arranged on the bottom surface of a chassis supporting frame (1), wheelchair rear wheels (4) respectively connected with the driving motors and two front wheels (6) connected through a front wheel connecting frame (2); the seat mechanism is arranged on the chassis support frame (1) and comprises a seat cushion block (9) connected with the wheelchair cushion connecting frame (5), side guardrail handrails (7) respectively arranged on the left side and the right side, and a cushion (10) arranged on the seat cushion block (9); the backrest mechanism comprises an upper backrest cushion plate (12) arranged at the upper part of the backrest connecting frame (6), a lower backrest cushion plate (11) arranged at the lower part of the backrest connecting frame, a back cushion (13) connected with the upper backrest cushion plate (12) and backrest armrests (8) connected with the lower backrest cushion plate (11); the leg protection mechanism comprises two leg protection base plates (24) connected with a leg protection connecting frame (22), a leg protection (27) connected with the upper ends of the leg protection base plates (24) and a foot pad structure; the foot pad structure consists of two foot pad cushion blocks (25) and a foot pad connecting frame (26); the intelligent motion control mechanism comprises a binocular camera (18), an intelligent operation panel (16), an emergency stop switch (28) and an integrated intelligent control module (20).

2. The semantic vision SLAM-based smart wheelchair of claim 1, wherein: the front wheel (3) and the rear wheel (4) are made of rubber or plastic materials, the front wheel (3) adopts a solid tire structure, and the rear wheel (4) adopts an inflation-free anti-explosion tire structure, so that the bearing weight of the wheelchair can be enhanced; the wheel tire adopts horizontal and vertical mixed patterns, so that the ground contact area can be increased, and meanwhile, better flexibility is kept.

3. The semantic vision SLAM-based smart wheelchair of claim 1, wherein: the motion control board of the integrated intelligent motion control module (20) is connected with a driving motor (19) to differentially control the motion of the two rear wheels (4) so as to realize the functions of speed conversion and steering.

4. The semantic vision SLAM-based smart wheelchair of claim 1, wherein: the binocular camera (18) is horizontally arranged on the intelligent operation panel (16), and the binocular camera (18) and the intelligent operation panel (16) are both arranged at one end of a right armrest of the wheelchair; the emergency stop switch (28) is installed at one end of a left side armrest of the wheelchair, so that the power supply can be quickly cut off.

5. The semantic vision SLAM-based smart wheelchair of claim 1, wherein: the intelligent operation panel (16) is connected with the integrated intelligent control module (20), environment information is obtained through a binocular camera, semantic objects are identified by combining a deep learning method, a semantic map of the environment is constructed, and the wheelchair is helped to realize independent positioning and navigation in different environments.