CN112869968B - Autonomous operation method and device based on electric wheelchair - Google Patents
Autonomous operation method and device based on electric wheelchair Download PDFInfo
- Publication number
- CN112869968B CN112869968B CN202110045788.8A CN202110045788A CN112869968B CN 112869968 B CN112869968 B CN 112869968B CN 202110045788 A CN202110045788 A CN 202110045788A CN 112869968 B CN112869968 B CN 112869968B
- Authority
- CN
- China
- Prior art keywords
- electric wheelchair
- point cloud
- reference object
- cloud data
- target carrier
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61G—TRANSPORT, PERSONAL CONVEYANCES, OR ACCOMMODATION SPECIALLY ADAPTED FOR PATIENTS OR DISABLED PERSONS; OPERATING TABLES OR CHAIRS; CHAIRS FOR DENTISTRY; FUNERAL DEVICES
- A61G5/00—Chairs or personal conveyances specially adapted for patients or disabled persons, e.g. wheelchairs
- A61G5/04—Chairs or personal conveyances specially adapted for patients or disabled persons, e.g. wheelchairs motor-driven
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61G—TRANSPORT, PERSONAL CONVEYANCES, OR ACCOMMODATION SPECIALLY ADAPTED FOR PATIENTS OR DISABLED PERSONS; OPERATING TABLES OR CHAIRS; CHAIRS FOR DENTISTRY; FUNERAL DEVICES
- A61G5/00—Chairs or personal conveyances specially adapted for patients or disabled persons, e.g. wheelchairs
- A61G5/10—Parts, details or accessories
- A61G5/1051—Arrangements for steering
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61G—TRANSPORT, PERSONAL CONVEYANCES, OR ACCOMMODATION SPECIALLY ADAPTED FOR PATIENTS OR DISABLED PERSONS; OPERATING TABLES OR CHAIRS; CHAIRS FOR DENTISTRY; FUNERAL DEVICES
- A61G2203/00—General characteristics of devices
- A61G2203/10—General characteristics of devices characterised by specific control means, e.g. for adjustment or steering
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61G—TRANSPORT, PERSONAL CONVEYANCES, OR ACCOMMODATION SPECIALLY ADAPTED FOR PATIENTS OR DISABLED PERSONS; OPERATING TABLES OR CHAIRS; CHAIRS FOR DENTISTRY; FUNERAL DEVICES
- A61G2203/00—General characteristics of devices
- A61G2203/10—General characteristics of devices characterised by specific control means, e.g. for adjustment or steering
- A61G2203/12—Remote controls
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
Abstract
The application discloses an autonomous operation method and device based on an electric wheelchair. And processing the three-dimensional point cloud data of the reference object acquired by the point cloud data acquisition equipment and the visual sensor and performing decision fusion to obtain the spatial position and attitude information of the reference object. The position and orientation of the electric wheelchair body required by the user to effect autonomous transfer between the electric wheelchair and the target carrier is then calculated. And the electric wheelchair is adjusted according to the calculation result. The method can enable a user to independently and conveniently realize the autonomous transfer between the electric wheelchair and the carriers such as the sofa, the bed and the like without the assistance of other people, and the electric wheelchair can also autonomously move between the standby space and the target carrier. The electric wheelchair is more humanized and intelligent, and the physiological and psychological comfort level of a user is improved.
Description
Technical Field
The application relates to the technical field of electric wheelchairs, in particular to an automatic operation method and device of an electric wheelchair.
Background
The wheelchair brings great convenience for people with inconvenient mobility, the electric wheelchair saves manpower, but the traditional electric wheelchair focuses on a series of function upgrading in the driving process and neglects the user transfer function in a home environment, namely, a user is difficult to independently and conveniently realize independent transfer between the electric wheelchair and carriers such as a sofa and a bed, the electric wheelchair still needs assistance of other people or consumes more time and physical power, and the user easily feels discomfort in physiology and psychology in the process.
In addition, after the user transfers to the carrier, the electric wheelchair stays beside the carrier continuously, which brings inconvenience to the user for using the wheelchair or others, and meanwhile, if the electric wheelchair is not put back in time, the electric wheelchair occupies space and blocks sight.
Disclosure of Invention
Object of the application
Based on this, in order to enable a user to independently and conveniently realize the autonomous transfer between the electric wheelchair and a carrier such as a sofa or a bed without the assistance of other people and simultaneously realize the autonomous movement of the electric wheelchair between a standby space and a target carrier, the following technical scheme is disclosed in the present application.
(II) technical scheme
As a first aspect of the present application, the present application discloses a method for autonomous operation based on an electric wheelchair, comprising:
the electric wheelchair moves to the vicinity of a target carrier by using an autonomous navigation function and automatically determines a reference object;
acquiring three-dimensional point cloud data and an image of a reference object, and processing the three-dimensional point cloud data and the image to obtain spatial position and attitude information of the reference object;
and calculating the position adjustment amount and/or the orientation adjustment amount of the electric wheelchair body according to the spatial position posture information of the reference object, and adjusting the position and/or the orientation of the electric wheelchair body according to the calculation result until the position and/or the orientation meet the requirement that a user transfers between the electric wheelchair and a target carrier.
In one possible embodiment, moving to the vicinity of the object carrier with the autonomous navigation function comprises:
after receiving the moving instruction, the electric wheelchair plans a moving path according to a pre-established three-dimensional indoor map;
the electric wheelchair moves to the vicinity of the target carrier according to the planned moving path;
the electric wheelchair obtains an environment point cloud of the surrounding environment by using a point cloud data acquisition device in the moving process, detects an obstacle based on the environment point cloud, replans a moving path according to the position of the obstacle and moves according to the newly planned moving path.
In one possible implementation, the step of creating a three-dimensional indoor map comprises:
acquiring indoor three-dimensional point cloud data through point cloud data acquisition equipment of the electric wheelchair;
filtering, registering and dividing the three-dimensional point cloud data to obtain a three-dimensional point cloud map;
and converting the obtained three-dimensional point cloud map into a grid type three-dimensional indoor map.
In one possible implementation, the processing the three-dimensional point cloud data and the image to obtain the spatial position and orientation information of the reference object includes:
point cloud data acquisition equipment and a visual sensor perform combined calibration and calculation of coordinate conversion parameters;
modeling is carried out after the obtained three-dimensional point cloud data are processed, and the three-dimensional point cloud data of a reference object to be detected are screened out;
calculating an area corresponding to the three-dimensional point cloud data of the reference object to be detected in the image by using the coordinate conversion parameters;
and performing decision fusion on the obtained three-dimensional point cloud data and the image, and comprehensively judging the spatial position and attitude information of the reference object to be detected.
In one possible embodiment, when the user is transferred from the destination carrier to the electric wheelchair, the destination carrier and the user himself are taken as references;
when the user transfers the target carrier from the electric wheelchair, the target carrier is used as a reference object.
As a second aspect of the present application, the present application discloses a system for autonomous operation based on an electric wheelchair, comprising:
the autonomous navigation module is used for realizing autonomous movement of the electric wheelchair to the vicinity of a target carrier and automatically determining a reference object;
the point cloud data acquisition equipment is used for acquiring three-dimensional point cloud data and images of a reference object;
the reference object detection module is used for processing the three-dimensional point cloud data and the image to obtain spatial position and attitude information of a reference object;
the calculation and adjustment module is used for calculating the position adjustment amount and/or the orientation adjustment amount of the electric wheelchair body according to the spatial position posture information of the reference object, and adjusting the position and/or the orientation of the electric wheelchair body according to the calculation result until the position and/or the orientation meet the requirement that a user transfers between the electric wheelchair and the target carrier.
In one possible embodiment, the autonomous navigation module comprises:
the path planning unit is used for planning a moving path according to a pre-established three-dimensional indoor map after receiving the moving instruction;
the control unit is used for sending corresponding driving signals to the driving unit according to the planned moving path;
the driving unit is used for controlling the motor to operate according to the driving signal output by the control unit and driving the electric wheelchair body to move to the position near the target carrier through the motor;
the electric wheelchair obtains an environment point cloud of the surrounding environment by using a point cloud data acquisition device in the moving process, detects an obstacle based on the environment point cloud, replans a moving path according to the position of the obstacle and moves according to the newly planned moving path.
In one possible embodiment, the system further comprises a map creation module, the map creation module comprising:
the data acquisition unit is used for acquiring indoor three-dimensional point cloud data through point cloud data acquisition equipment of the electric wheelchair;
the data processing unit is used for filtering, registering and dividing the three-dimensional point cloud data collected by the data collecting unit to obtain a three-dimensional point cloud map;
the map conversion unit is used for converting the obtained three-dimensional point cloud map into a grid type three-dimensional indoor map.
In one possible embodiment, the system further comprises a map creation module, the map creation module comprising:
the joint calibration unit is used for carrying out joint calibration on the point cloud data acquisition equipment and the visual sensor to calculate coordinate conversion parameters;
the data screening unit is used for processing the acquired three-dimensional point cloud data and then modeling to screen out the three-dimensional point cloud data of the reference object to be detected;
the information conversion unit is used for calculating an area corresponding to the three-dimensional point cloud data of the reference object to be detected in the image by using the coordinate conversion parameter;
and the comprehensive judgment unit is used for performing decision fusion on the obtained three-dimensional point cloud data and the image and comprehensively judging the spatial position and posture information of the reference object to be detected.
In one possible embodiment, when the user is transferred from the destination carrier to the electric wheelchair, the destination carrier and the user himself are taken as references;
when the user transfers the target carrier from the electric wheelchair, the target carrier is used as a reference object.
(III) advantageous effects
According to the method and the device for autonomous operation of the electric wheelchair, the relative position between the reference object and the wheelchair is judged by detecting the position and the posture of the wheelchair, so that the position and the posture of the wheelchair required by autonomous transfer between the reference object and the wheelchair can be calculated by a user, the position and the posture of the wheelchair can be adjusted according to the position and the posture, the user can independently and conveniently realize autonomous transfer between the electric wheelchair and carriers such as a sofa and a bed without assistance of other people, and meanwhile, the electric wheelchair can also realize autonomous movement between a standby space and a target carrier, so that the electric wheelchair is more humanized and intelligent, and the physiological and psychological comfort level of the user is improved.
Drawings
The embodiments described below with reference to the drawings are exemplary and intended to be used for explaining and illustrating the present application and should not be construed as limiting the scope of protection of the present application.
Fig. 1 is a flowchart of an embodiment of an autonomous operation method based on an electric wheelchair disclosed in the present application.
Fig. 2 is a block diagram of an embodiment of an autonomous operating system based on an electric wheelchair according to the present disclosure.
Detailed Description
In order to make the implementation objects, technical solutions and advantages of the present application clearer, the technical solutions in the embodiments of the present application will be described in more detail below with reference to the drawings in the embodiments of the present application.
Referring to fig. 1, a method for autonomous operation based on an electric wheelchair disclosed in the present application is described in detail, and as shown in fig. 1, the method disclosed in the present embodiment mainly includes the following steps 100 to 300.
And step 100, the electric wheelchair moves to the vicinity of the target carrier by using the autonomous navigation function, and automatically determines a reference object.
The object carrier refers to a carrier in a home space, such as a sofa, a bed, a chair, etc., which can carry the body of a user. If the user wants to realize the autonomous transfer between the electric wheelchair and the target carrier, the electric wheelchair should move to the vicinity of the target carrier first, and then the user performs the transfer action.
The reference object is a reference datum set for a user to calculate the position and the orientation of the electric wheelchair to be adjusted in the process of autonomous transfer between the electric wheelchair and the target carrier, and comprises the target carrier and the user.
When a user wants to transfer the target carrier from the electric wheelchair, the user firstly takes the electric wheelchair to autonomously operate to the vicinity of the target carrier, and the electric wheelchair automatically determines the target carrier which the user wants to transfer as a reference object.
If the user is located in the target carrier and wants to transfer to the electric wheelchair, the electric wheelchair automatically runs to the position near the target carrier, and the user and the target carrier are automatically determined to be reference objects.
And 200, acquiring three-dimensional point cloud data and an image of the reference object, and processing the three-dimensional point cloud data and the image to obtain spatial position and attitude information of the reference object.
After the electric wheelchair automatically determines the reference object, in order to obtain comprehensive and accurate spatial position and attitude information of the reference object, a point cloud data acquisition device and a visual sensor are adopted for joint detection, and firstly, two devices respectively obtain the region information containing the reference object. The point cloud data acquisition equipment acquires three-dimensional point cloud data information of an area containing a reference object, wherein the point cloud data is a set of vectors in a three-dimensional coordinate system. The scan data is recorded in the form of dots, each dot containing three-dimensional coordinates, some of which may contain color information or reflection intensity information. The vision sensor acquires image information of a region including a reference object. In order to better fuse the information acquired by the point cloud data acquisition equipment and the visual sensor, the point cloud data acquisition equipment and the visual sensor are firstly subjected to combined calibration and then the acquired information is processed, and finally the spatial position and attitude information of the reference object is finally determined by integrating two detection results. After the spatial position attitude information of the reference object is determined, the electric wheelchair can calculate the position and/or orientation of the electric wheelchair to be adjusted according to the spatial position attitude information of the reference object and then adjust according to a calculation result, so that a user can realize the autonomous transfer between the electric wheelchair and a target carrier. The spatial position and orientation information of the reference object includes coordinates, shape, height, angle, orientation, and the like.
And 300, calculating the position adjustment quantity and/or the orientation adjustment quantity of the electric wheelchair body according to the spatial position posture information of the reference object, and adjusting the position and/or the orientation of the electric wheelchair body according to the calculation result until the position and/or the orientation meet the requirement that the user is transferred between the electric wheelchair and the target carrier.
After the spatial position attitude information of the reference object is determined, in order to realize the autonomous transfer of the user between the electric wheelchair and the target carrier, the electric wheelchair needs to be adjusted, and calculation needs to be carried out before the adjustment. Firstly, calculating the orientation and the adjustment angle to be adjusted, and firstly, determining the regional center point of the electric wheelchair according to the regional three-dimensional point cloud data information and the regional image information which contain the reference object and are obtained in the step 200; determining the center point of the reference object according to the three-dimensional point cloud data information and the image information of the reference object finally determined in the step 200, and calculating the direction and the angle of the electric wheelchair to be adjusted when the area center point of the reference object is coincident with the reference object center point, namely when the reference object is located in the center of the area; after the orientation and the adjusting angle of the electric wheelchair to be adjusted are determined, the distance to be adjusted is calculated, the adjusting distance is calculated according to the adjusting coefficient, and the optimal adjusting coefficient is the area ratio of the reference object to the region where the reference object is located when the user can realize the autonomous transfer between the electric wheelchair and the target carrier. And calculating the distance required to be adjusted by the electric wheelchair when the area ratio of the reference object to the area where the reference object is located reaches the optimal adjustment coefficient, and adjusting the orientation, the angle and the distance of the electric wheelchair according to the calculation result. After the primary calculation and adjustment are finished, the electric wheelchair judges whether the expected effect of the autonomous transfer of the user between the electric wheelchair and the target carrier can be realized, and if the expected effect can be realized, the adjustment is finished; if not, the calculation and adjustment actions are repeated until the user is able to achieve the desired effect of autonomous transfer between the powered wheelchair and the target carrier, and the adjustment is complete.
In at least one embodiment, the calculation and adjustment actions of the electric wheelchair may be performed once or may be performed a plurality of times in a cycle, that is: and when the expected effect is not achieved after the calculation, adjustment and judgment steps are carried out once, the calculation, adjustment and judgment steps are continued until the judgment result of the electric wheelchair reaches the expected effect, namely, the user can realize the autonomous transfer between the electric wheelchair and the target carrier. Step 300 ends.
Illustratively, when a user wants to effect a transfer from an electric wheelchair to a sofa, the user travels in the vicinity of the sofa in the electric wheelchair. Acquire regional three-dimensional point cloud data information including the sofa through point cloud data acquisition equipment, acquire regional image information including the sofa through visual sensor, integrate regional three-dimensional point cloud data information and regional image information, confirm the whole regional image including the sofa jointly, and then confirm the geometric centre point of whole regional image, it is regional to screen out the sofa after handling this whole regional image, and then confirm the regional geometric centre point of sofa, orientation and the rotatory angle of orientation that the orientation that electronic wheelchair required to adjust when electronic wheelchair calculates the geometric centre point of whole regional image and the regional geometric centre point coincidence of sofa of geometric centre point. And then, calculating the distance, and calculating the adjustment distance according to the adjustment coefficient, wherein the optimal adjustment coefficient can achieve that the gap between the body of the electric wheelchair and the edge of the sofa is not generated, the height of the wheelchair seat and the height of the sofa are positioned on the same plane, and a user can move to the sofa through translation.
When the user accomplished the transfer action from carriers such as electronic wheelchair to sofa, bed, electronic wheelchair can independently plan the route, returns the standby space to prevent placing at will and occupy the indoor space or shelter from the sight, and can independently switch on in the standby space when the electric quantity is not enough, can also realize the electric quantity and be full of the back and independently cut off the power supply.
In at least one embodiment, the step 100 of operating in proximity to the object carrier using the autonomous navigation function further comprises steps 110-120.
And 110, after receiving the transfer instruction, planning a moving path according to a pre-established three-dimensional indoor map by the electric wheelchair.
When the electric wheelchair is located indoors where a three-dimensional indoor map is created in advance, a user can send an instruction for moving to a target carrier to the electric wheelchair through voice calling or Bluetooth and the like, after the electric wheelchair receives the instruction, modeling is firstly carried out on the surrounding environment of the three-dimensional indoor map created in advance, all feasible routes are planned according to the three-dimensional indoor map, and a shortest route which can be used for allowing the electric wheelchair to freely pass is determined based on a D-x algorithm. The electric wheelchair is located in a room where a three-dimensional indoor map is created in advance, and the electric wheelchair is located in the room where the three-dimensional indoor map is created in advance when a user takes the electric wheelchair and is located in the room where the three-dimensional indoor map is created in advance when the electric wheelchair is in an empty state.
And step 120, the electric wheelchair moves to the vicinity of the target carrier according to the planned moving path.
After a movement path is planned by the electric wheelchair, the electric wheelchair controller sends a driving signal to the driving device according to the planned path, the driving device drives the motor to drive the electric wheelchair body to move according to the set movement path, after the electric wheelchair starts to move, the point cloud data acquisition equipment is started to acquire environmental point cloud of the surrounding environment, obstacle detection is carried out on the basis of the environmental point cloud, and if no obstacle exists, the electric wheelchair moves to the vicinity of a target carrier according to the planned path; and if the obstacle is encountered, replanning the moving path according to the position of the obstacle and moving according to the newly planned moving path until the obstacle reaches the vicinity of the target carrier.
In at least one embodiment, the electric wheelchair needs to be used for the first time before a map is not created in a room, and a three-dimensional indoor map is created, wherein the creation process of the three-dimensional indoor map comprises steps 111 to 113.
And step 111, acquiring indoor three-dimensional point cloud data through point cloud data acquisition equipment of the electric wheelchair.
The electric wheelchair moves from any position in an indoor environment of unknown environment, a 360-degree panoramic camera of a vision sensor is started, and the current position, the passed area and the not-passed area are identified at any time. The vision sensor can help the electric wheelchair to effectively grasp the surrounding environment. Meanwhile, the electric wheelchair point cloud data acquisition equipment acquires the environmental point cloud of the surrounding environment, and when the point cloud data is collected, obstacle detection is carried out on the basis of the environmental point cloud to avoid obstacles, so that indoor comprehensive movement in an unknown environment and collection of environmental three-dimensional point cloud data are smoothly completed.
The point cloud acquisition equipment can be built by combining two-dimensional point cloud data acquisition equipment and a rotating platform, and can also be directly three-dimensional point cloud acquisition equipment.
And 112, filtering, registering and segmenting the three-dimensional point cloud data to obtain a three-dimensional point cloud map.
Firstly, filtering all three-dimensional point cloud data acquired in the step 111, specifically filtering sparse outliers in the three-dimensional point cloud data, secondly, registering the point cloud data, and connecting effective point clouds of different positions subjected to filtering processing into a complete three-dimensional point cloud map by taking a geometric structure as a reference; the complete three-dimensional point cloud map comprises all indoor information such as articles, walls, passages and the like, the walls, the ground and the ceiling are separated by point cloud segmentation, and then the indoor articles are separated one by one. And obtaining a complete and clear three-dimensional indoor point cloud map.
In at least one embodiment, an effective area is set, distant outliers outside the area are deleted, the outliers in the area are deleted based on statistical analysis, point cloud registration is carried out by using an ICP (inductively coupled plasma) algorithm after filtering is finished, a complete three-dimensional point cloud map is obtained, the wall, the ground and the ceiling are separated by using a RANSAC (RANdom SAmple Consensus) algorithm after the complete three-dimensional point cloud map is obtained, a better visual effect is obtained, and indoor articles are separated by European clustering.
And 113, converting the obtained three-dimensional point cloud map into a grid type three-dimensional indoor map.
The obtained three-dimensional point cloud map is converted into a three-dimensional grid map by using the OctMap, the size of the grid can be adjusted according to the size of the indoor space, and the obtained three-dimensional grid map can be used for planning a moving path more easily.
In at least one embodiment, the processing the three-dimensional point cloud data and the image in step 200 to obtain the spatial position and orientation information of the reference object further includes steps 210 to 240.
And 210, carrying out combined calibration on the point cloud data acquisition equipment and the visual sensor to calculate coordinate conversion parameters.
Firstly, determining a point cloud data acquisition device and a calibration method of a visual sensor, establishing a mathematical model based on a plane calibration object, calculating transformation matrix parameters between the point cloud data acquisition device and coordinates of the visual sensor by using a least square method, carrying out error analysis after calculation, and adjusting after analysis. And finally determining coordinate conversion parameters of the point cloud data acquisition equipment and the visual sensor.
And step 220, modeling is carried out after the obtained three-dimensional point cloud data are processed, and the three-dimensional point cloud data of the reference object to be detected are screened out.
The method comprises the steps of carrying out a series of processing such as filtering, registering and dividing on the acquired point cloud data to obtain regional three-dimensional point cloud data including a reference object, carrying out feature extraction according to the geometric shape of the reference object, screening out a suspected reference object point cloud region in the regional point cloud data including the reference object by utilizing the geometric shape feature, removing the residual point cloud data, and carrying out accuracy detection on a screening result by using a classifier trained by an SVM mathematical model according to the geometric point cloud feature of the reference object.
And 230, calculating an area corresponding to the three-dimensional point cloud data of the reference object to be detected in the image by using the coordinate conversion parameters.
According to the three-dimensional point cloud area of the reference object determined in step 220, the corresponding reference object area in the image can be obtained by direct projection by using the coordinate conversion parameter between the point cloud data acquisition device and the visual sensor obtained by joint calibration in step 210.
And 240, performing decision fusion on the obtained three-dimensional point cloud data and the image, and comprehensively judging the spatial position and attitude information of the reference object to be detected.
In order to fully utilize the three-dimensional point cloud data and the image information, in the embodiment, different classifiers are obtained by respectively training the three-dimensional point cloud data and the image to be detected independently, then decision fusion is performed on the results of the independent detection, comparison is performed, and when the detection results are inconsistent, the final detection result is judged according to a specific judgment criterion.
The specific judgment criterion is based on Bayes theory, firstly, a probability matrix is used for outputting and combining detection results, image information is divided into a reference object area (A) and a non-reference object area (B), three-dimensional point cloud data is divided into reference object data (C) and non-reference object data (D), and four situations can occur: (1) a + C; (2) a + D; (3) b + C; (4) b + D; secondly, calculating the probability of the possible occurrence of the four situations according to a joint density function; secondly, training a classifier by using the characteristics of the three-dimensional point cloud data and detecting the target of the image information to obtain a target confidence coefficient, normalizing the obtained confidence coefficient, determining the prior probability of the two sensors, and determining a loss value with an incorrect judgment result by using a defined loss function; and finally determining a final judgment result according to a Bayes decision theory.
In at least one embodiment, step 300 is followed by step 310 after the adjustment of the electric wheelchair body.
And step 310, adjusting the seat of the electric wheelchair.
After the position and/or orientation of the electric wheelchair is calculated, the height of the electric wheelchair seat to be adjusted is calculated according to the height of the reference object,
if the user wants to transfer the electric wheelchair to the target carrier, the height of the electric wheelchair seat to be adjusted is calculated by taking the hip contact height of the user in the target carrier as a reference.
If a user wants to transfer the target carrier to the electric wheelchair, the reference object is the user and the target carrier, firstly, the target carrier where the user is located is determined, and when the target carrier is a soft surface, the difference between the height of the lowest point of the depression of the reference object and the height of the seat of the electric wheelchair is the adjustment quantity of the seat of the electric wheelchair; when the target carrier is a hard surface, the height of the position of the target carrier contacted by the hip of the user in the reference object is the adjustment amount of the electric wheelchair seat. The electric wheelchair seat can be inclined by a small degree to assist a user to realize autonomous transfer.
The following is an example of an autonomous operation method of an electric wheelchair when a user transfers the electric wheelchair from the electric wheelchair to a target carrier.
A user wants to transfer to a sofa by taking the electric wheelchair, firstly, an instruction for transferring to the sofa is sent to the electric wheelchair, the electric wheelchair plans an optimal route according to a pre-established three-dimensional indoor map after receiving the instruction, and moves according to the planned optimal route to autonomously avoid obstacles; the method comprises the steps of determining a sofa as a reference object after the sofa is moved to the position near the sofa, acquiring information such as the height and the position of the sofa through point cloud data acquisition equipment and a visual sensor, calculating the direction, the angle and the distance of the electric wheelchair body, which are required by a user to transfer from the electric wheelchair to the sofa, needing to be adjusted, of the user through the electric wheelchair, calculating the height of the seat needing to be adjusted, adjusting the electric wheelchair according to a calculation result after the calculation is finished, and enabling the user to realize the autonomous transfer from the electric wheelchair to the sofa. After the user accomplishes the transfer action from carriers such as electronic wheelchair to sofa, bed, electronic wheelchair can independently plan the route, returns the standby space to prevent placing at will and occupy the indoor space or shelter from the sight, and can independently switch on in the standby space when the electric quantity is not enough, can also realize the electric quantity and be full of the back and independently cut off the power supply.
The following is an example of an autonomous operation method of an electric wheelchair when a user transfers from a target carrier to the electric wheelchair.
The method comprises the following steps that a user wants to transfer the electric wheelchair from a sofa to the electric wheelchair, firstly, an instruction for transferring the electric wheelchair to the sofa is sent out, the electric wheelchair plans an optimal route according to a pre-established three-dimensional indoor map after receiving the instruction, and the electric wheelchair moves according to the planned optimal route and autonomously avoids obstacles; the method comprises the steps of determining the sofa and a user to be reference objects after the sofa and the user move to the position near the sofa, acquiring information such as the height, the position and the posture of the sofa and the user through point cloud data acquisition equipment and a visual sensor, calculating the orientation, the angle and the distance, required to be adjusted, of the electric wheelchair body required by the user to be transferred from the electric wheelchair to the sofa, calculating the height, required to be adjusted, of the seat, adjusting the height, required to be adjusted, of the seat according to a calculation result after calculation is finished, and enabling the user to achieve automatic transfer from the electric wheelchair to the sofa.
Referring to fig. 2, a system for autonomous operation based on an electric wheelchair disclosed in the present application will be described in detail, and as shown in fig. 2, the system disclosed in the present embodiment includes:
the autonomous navigation module is used for realizing autonomous movement of the electric wheelchair to the vicinity of a target carrier and automatically determining a reference object;
the point cloud data acquisition equipment is used for acquiring three-dimensional point cloud data and images of a reference object;
the reference object detection module is used for processing the three-dimensional point cloud data and the image to obtain spatial position and attitude information of a reference object;
the calculation and adjustment module is used for calculating the position adjustment amount and/or the orientation adjustment amount of the electric wheelchair body according to the spatial position posture information of the reference object, and adjusting the position and/or the orientation of the electric wheelchair body according to the calculation result until the position and/or the orientation meet the requirement that a user transfers between the electric wheelchair and the target carrier.
In at least one embodiment, the autonomous navigation module comprises:
the path planning unit is used for planning a moving path according to a pre-established three-dimensional indoor map after receiving the moving instruction;
the control unit is used for sending a corresponding driving signal to the driving unit according to the planned moving path;
the driving unit is used for controlling the motor to operate according to the driving signal output by the control unit and driving the electric wheelchair body to move to the position near the target carrier through the motor;
the electric wheelchair obtains an environment point cloud of the surrounding environment by using a point cloud data acquisition device in the moving process, detects an obstacle based on the environment point cloud, replans a moving path according to the position of the obstacle and moves according to the newly planned moving path.
In at least one embodiment, the system further comprises a map creation module comprising:
the data acquisition unit is used for acquiring indoor three-dimensional point cloud data through point cloud data acquisition equipment of the electric wheelchair;
the data processing unit is used for filtering, registering and dividing the three-dimensional point cloud data collected by the data collecting unit to obtain a three-dimensional point cloud map;
the map conversion unit is used for converting the obtained three-dimensional point cloud map into a grid type three-dimensional indoor map.
In at least one embodiment, the system further comprises a map creation module comprising:
the joint calibration unit is used for carrying out joint calibration on the point cloud data acquisition equipment and the visual sensor to calculate coordinate conversion parameters;
the data screening unit is used for modeling the obtained three-dimensional point cloud data after processing the three-dimensional point cloud data, and screening out the three-dimensional point cloud data of the reference object to be detected;
the information conversion unit is used for calculating an area corresponding to the three-dimensional point cloud data of the reference object to be detected in the image by using the coordinate conversion parameter;
and the comprehensive judgment unit is used for performing decision fusion on the obtained three-dimensional point cloud data and the image and comprehensively judging the spatial position and attitude information of the reference object to be detected.
In at least one embodiment, the target carrier and the user themselves are used as references when the user is transferred from the target carrier to the electric wheelchair;
when the user transfers the target carrier from the electric wheelchair, the target carrier is used as a reference object.
The division of the modules and units herein is only one division of logical functions, and other divisions may be possible in actual implementation, for example, a plurality of modules and/or units may be combined or integrated in another system. The modules and units described as separate parts may be physically separated or not. The components displayed as the cells may or may not be physical cells, may be located in a specific place, or may be distributed in grid cells. Therefore, some or all of the units can be selected according to actual needs to implement the scheme of the embodiment.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present application should be covered within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (6)
1. An autonomous operation method based on an electric wheelchair, comprising:
the electric wheelchair moves to the vicinity of a target carrier by using an autonomous navigation function and automatically determines a reference object;
acquiring three-dimensional point cloud data and an image of the reference object, and processing the three-dimensional point cloud data and the image to obtain spatial position and attitude information of the reference object;
calculating the position adjustment amount and/or the orientation adjustment amount of the electric wheelchair body according to the spatial position posture information of the reference object, and adjusting the position and/or the orientation of the electric wheelchair body according to the calculation result until the position and/or the orientation meet the requirement that a user transfers between the electric wheelchair and a target carrier;
the processing of the three-dimensional point cloud data and the image to obtain the spatial position and attitude information of the reference object comprises the following steps:
point cloud data acquisition equipment and a visual sensor perform combined calibration and calculation of coordinate conversion parameters;
processing the obtained three-dimensional point cloud data, modeling, and screening out the three-dimensional point cloud data of the reference object to be detected;
calculating an area corresponding to the three-dimensional point cloud data of the reference object to be detected in the image by using the coordinate conversion parameters;
performing decision fusion on the obtained three-dimensional point cloud data and the image, and comprehensively judging the spatial position and attitude information of the reference object to be detected;
when the user transfers from the target carrier to the electric wheelchair, the target carrier and the user are taken as reference objects; when the user transfers the target carrier from the electric wheelchair, the target carrier is used as a reference object; the reference object is a reference datum set for a user to calculate the adjusted position and orientation of the wheelchair during autonomous movement between the electric wheelchair and the target carrier;
the method for adjusting the position and/or orientation of the electric wheelchair body according to the spatial position and posture information of the reference object includes the following steps:
determining the central point of the area including the reference object;
determining a center point of the reference object;
calculating the orientation adjustment quantity of the electric wheelchair body based on the central point of the area and the central point of the reference object;
calculating the position adjustment quantity of the electric wheelchair body according to the optimal adjustment coefficient; the optimal adjustment coefficient is the area ratio of a reference object to an area where the reference object is located when a user can realize autonomous transfer between the electric wheelchair and the target carrier;
adjusting the position and/or the orientation of the electric wheelchair body according to the orientation adjustment amount and the position adjustment amount;
when the electric wheelchair cannot achieve the expected effect after the calculation, adjustment and judgment steps are carried out for one time, the calculation, adjustment and judgment steps are continued until the electric wheelchair achieves the expected effect; the expected effect is to enable autonomous transfer of the user between the electric wheelchair and the target carrier.
2. The method of claim 1, wherein said moving to the vicinity of the object carrier using autonomous navigation functionality comprises:
after receiving the moving instruction, the electric wheelchair plans a moving path according to a pre-established three-dimensional indoor map;
the electric wheelchair moves to the vicinity of the target carrier according to the planned moving path;
the electric wheelchair obtains an environment point cloud of the surrounding environment by using a point cloud data acquisition device in the moving process, detects an obstacle based on the environment point cloud, replans a moving path according to the position of the obstacle and moves according to the newly planned moving path.
3. The method of claim 2, wherein the creating of the three-dimensional indoor map comprises:
acquiring indoor three-dimensional point cloud data through point cloud data acquisition equipment of the electric wheelchair;
filtering, registering and dividing the three-dimensional point cloud data to obtain a three-dimensional point cloud map;
and converting the obtained three-dimensional point cloud map into a grid type three-dimensional indoor map.
4. A system for autonomous operation of an electric wheelchair, comprising:
the autonomous navigation module is used for realizing that the electric wheelchair autonomously moves to the vicinity of a target carrier and automatically determines a reference object;
the point cloud data acquisition equipment is used for acquiring three-dimensional point cloud data and images of the reference object;
the reference object detection module is used for processing the three-dimensional point cloud data and the image to obtain spatial position and attitude information of a reference object;
the calculation and adjustment module is used for calculating the position adjustment quantity and/or the orientation adjustment quantity of the electric wheelchair body according to the spatial position posture information of the reference object, and adjusting the position and/or the orientation of the electric wheelchair body according to the calculation result until the position and/or the orientation meet the requirement that a user transfers between the electric wheelchair and a target carrier;
the reference object detection module includes:
the joint calibration unit is used for carrying out joint calibration on the point cloud data acquisition equipment and the visual sensor to calculate coordinate conversion parameters;
the data screening unit is used for modeling the obtained three-dimensional point cloud data after processing and screening out the three-dimensional point cloud data of the reference object to be detected;
the information conversion unit is used for calculating an area corresponding to the three-dimensional point cloud data of the reference object to be detected in the image by using the coordinate conversion parameter;
the comprehensive judgment unit is used for carrying out decision fusion on the obtained three-dimensional point cloud data and the image and comprehensively judging the spatial position and posture information of the reference object to be detected;
when the user transfers from the target carrier to the electric wheelchair, the target carrier and the user are taken as reference objects; when the user transfers the target carrier from the electric wheelchair, the target carrier is used as a reference object; the reference object is a reference datum set for calculating the adjusted position and orientation of the wheelchair during the autonomous movement of the user between the electric wheelchair and the target carrier;
the calculation adjusting module specifically comprises:
determining the central point of the area containing the reference object;
determining a center point of the reference object;
calculating the orientation adjustment quantity of the electric wheelchair body based on the central point of the area and the central point of the reference object;
calculating the position adjustment quantity of the electric wheelchair body according to the optimal adjustment coefficient; the optimal adjustment coefficient is the area ratio of a reference object to an area where the reference object is located when a user can realize autonomous transfer between the electric wheelchair and the target carrier;
adjusting the position and/or orientation of the electric wheelchair body according to the calculation result;
when the electric wheelchair cannot achieve the expected effect after the calculation, adjustment and judgment steps are carried out for one time, the calculation, adjustment and judgment steps are continued until the electric wheelchair achieves the expected effect; the expected effect is to enable autonomous transfer of the user between the electric wheelchair and the target carrier.
5. The system of claim 4, wherein the autonomous navigation module comprises:
the path planning unit is used for planning a moving path according to a pre-established three-dimensional indoor map after receiving the moving instruction;
the control unit is used for sending corresponding driving signals to the driving unit according to the planned moving path;
the driving unit is used for controlling the motor to operate according to the driving signal output by the control unit and driving the electric wheelchair body to move to the position near the target carrier through the motor;
the electric wheelchair obtains the environmental point cloud of the surrounding environment by using the point cloud data acquisition equipment in the moving process, detects the obstacle based on the environmental point cloud, replans the moving path according to the position of the obstacle and moves according to the newly planned moving path.
6. The system of claim 5, further comprising a map creation module, the map creation module comprising:
the data acquisition unit is used for acquiring indoor three-dimensional point cloud data through point cloud data acquisition equipment of the electric wheelchair;
the data processing unit is used for filtering, registering and dividing the three-dimensional point cloud data collected by the data collection unit to obtain a three-dimensional point cloud map;
and the map conversion unit is used for converting the obtained three-dimensional point cloud map into a grid-type three-dimensional indoor map.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110045788.8A CN112869968B (en) | 2021-01-14 | 2021-01-14 | Autonomous operation method and device based on electric wheelchair |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110045788.8A CN112869968B (en) | 2021-01-14 | 2021-01-14 | Autonomous operation method and device based on electric wheelchair |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112869968A CN112869968A (en) | 2021-06-01 |
| CN112869968B true CN112869968B (en) | 2023-01-17 |
Family
ID=76045837
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110045788.8A Active CN112869968B (en) | 2021-01-14 | 2021-01-14 | Autonomous operation method and device based on electric wheelchair |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112869968B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TWI822096B (en) * | 2022-06-08 | 2023-11-11 | 和碩聯合科技股份有限公司 | Control system and control method for controlling electric walking aid device |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2020004182A (en) * | 2018-06-29 | 2020-01-09 | 株式会社国際電気通信基礎技術研究所 | Robot, robot control program and robot control method |
| US10660806B1 (en) * | 2020-01-15 | 2020-05-26 | Blanche Michelle Nelson-Herron | Wheelchair safety systems and related methods |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140018994A1 (en) * | 2012-07-13 | 2014-01-16 | Thomas A. Panzarella | Drive-Control Systems for Vehicles Such as Personal-Transportation Vehicles |
| CN104390645B (en) * | 2014-12-09 | 2017-11-07 | 重庆邮电大学 | A kind of intelligent wheel chair indoor navigation method of view-based access control model information |
| CN107450556A (en) * | 2017-09-11 | 2017-12-08 | 河北农业大学 | A kind of independent navigation intelligent wheel chair based on ROS |
| CN209154263U (en) * | 2018-07-09 | 2019-07-26 | 江西斯麦高医疗器械科技有限公司 | A kind of automatic obstacle avoiding intelligent wheel chair based on binocular vision |
| CN108897328B (en) * | 2018-08-27 | 2024-05-10 | 桂林电子科技大学 | Wheelchair robot and indoor autonomous navigation method thereof |
| CN110471422A (en) * | 2019-08-29 | 2019-11-19 | 南京理工大学 | The detection of obstacles and automatic obstacle avoiding method of intelligent wheel chair |
| CN111708368B (en) * | 2020-07-07 | 2023-03-10 | 上海工程技术大学 | Intelligent wheelchair based on fusion of laser and visual SLAM |
-
2021
- 2021-01-14 CN CN202110045788.8A patent/CN112869968B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2020004182A (en) * | 2018-06-29 | 2020-01-09 | 株式会社国際電気通信基礎技術研究所 | Robot, robot control program and robot control method |
| US10660806B1 (en) * | 2020-01-15 | 2020-05-26 | Blanche Michelle Nelson-Herron | Wheelchair safety systems and related methods |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112869968A (en) | 2021-06-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110059608B (en) | Object detection method and device, electronic equipment and storage medium | |
| US7865267B2 (en) | Environment recognizing device, environment recognizing method, route planning device, route planning method and robot | |
| CN108290294B (en) | Mobile robot and control method thereof | |
| Chen | Kalman filter for robot vision: a survey | |
| CN112025729B (en) | Multifunctional intelligent medical service robot system based on ROS | |
| US11020294B2 (en) | Mobility and mobility system | |
| CN207488823U (en) | A kind of mobile electronic device | |
| CN108073167A (en) | A kind of positioning and air navigation aid based on depth camera and laser radar | |
| CN109528089B (en) | Method, device and chip for continuously walking trapped cleaning robot | |
| KR20180070932A (en) | A movable object and a method for controlling the same | |
| US11378693B2 (en) | Floor surveying system | |
| CN106774318B (en) | Multi-agent interactive environment perception and path planning motion system | |
| US20220382282A1 (en) | Mobility aid robot navigating method and mobility aid robot using the same | |
| CN112869968B (en) | Autonomous operation method and device based on electric wheelchair | |
| CN114167866B (en) | Intelligent logistics robot and control method | |
| CN111168685A (en) | Robot control method, robot, and readable storage medium | |
| CN113081525A (en) | Intelligent walking aid equipment and control method thereof | |
| CN112182122A (en) | Method and device for acquiring navigation map of working environment of mobile robot | |
| Tiozzo Fasiolo et al. | Combining LiDAR SLAM and deep learning-based people detection for autonomous indoor mapping in a crowded environment | |
| Atsuzawa et al. | Robot navigation in outdoor environments using odometry and convolutional neural network | |
| US11837006B2 (en) | Human posture determination method and mobile machine using the same | |
| Chen et al. | ARMSAINTS: An AR-based Real-time Mobile System for Assistive Indoor Navigation with Target Segmentation | |
| Zhu et al. | Wheelchair automatic docking method for body-separated nursing bed based on grid map | |
| JP7354528B2 (en) | Autonomous mobile device, method and program for detecting dirt on lenses of autonomous mobile device | |
| CN116385489B (en) | RGBD camera-based target following method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 100089 building 01, South 30, dongxindu east station, Qinghe Xisanqi, Haidian District, Beijing Applicant after: Beijing Wujiang Technology Co.,Ltd. Address before: 100089 building 01, South 30, dongxindu east station, Qinghe Xisanqi, Haidian District, Beijing Applicant before: BEIJING DELTA-ROBOTICS TECHNOLOGY Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20220208 Address after: 239400 No. 116, qicang Road, Mingguang Industrial Park, Chuzhou City, Anhui Province Applicant after: Anhui jinbaihe Medical Equipment Co.,Ltd. Address before: 100089 building 01, South 30, dongxindu east station, Qinghe Xisanqi, Haidian District, Beijing Applicant before: Beijing Wujiang Technology Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |