WO2023124572A1

WO2023124572A1 - Driving assistance system and method applied to scooter for elderly person, and storage medium

Info

Publication number: WO2023124572A1
Application number: PCT/CN2022/131216
Authority: WO
Inventors: 李建国; 刘伯锋; 程晨航; 易明亮; 彭同新; 廖香成
Original assignee: 上海邦邦机器人有限公司
Priority date: 2021-12-31
Filing date: 2022-11-10
Publication date: 2023-07-06
Also published as: CN116409312A

Abstract

A driving assistance system applied to a scooter for an elderly person. The system comprises: a control module (10), a sensor module (20), and a display module (30). The sensor module (20) comprises a binocular camera group (21), a wide-angle camera group (22), and an ultrasonic module (23), wherein the binocular camera group (21) may collect and process environmental data in front of a scooter; the wide-angle camera group (22) may capture multi-directional scenes; and the ultrasonic module (23) may collect and process environmental data on the side and rear of the scooter. The control module (10) may determine obstacle information according to the multi-directional scenes and the environmental data, execute a corresponding obstacle avoidance strategy, generate a holographic scene according to the multi-directional scenes, and send the holographic scene to the display module (30) for display. Therefore, a driver can be assisted in effectively avoiding an obstacle, and the driver can also know surrounding environmental information by means of a holographic scene, thereby improving the driving safety of the driver. Further provided are a driving assistance method applied to a scooter for an elderly person, and a computer-readable storage medium.

Description

Assisted driving system, method and storage medium applied to elderly scooters

This application claims the priority of the Chinese patent application with the application number 202111668156.3 and the application name "Auxiliary driving system, method and storage medium applied to elderly scooters" submitted to the China Patent Office on December 31, 2021, the entire content of which Incorporated in this application by reference.

technical field

The present application relates to the technical field of traffic safety, in particular to an auxiliary driving system, method and storage medium applied to elderly scooters.

Background technique

At present, our country has gradually entered an aging society, and the travel problem of the elderly has become a common concern of the society. With the growth of age, the normal physiological functions of the elderly gradually decline, and the mobility is not very convenient. Therefore, the elderly scooter has become An optional way for seniors to travel.

The user group of the elderly scooter is the elderly and the disabled with limited mobility. Because the external road conditions and environment are usually complicated, and the reaction and thinking of the elderly or the disabled are often slow, once they are used away from their guardians, it is very easy to Dangerous accidents such as vehicle overturning or collision are not conducive to the safety of the driver and surrounding pedestrians. Therefore, there are certain potential safety hazards in the existing elderly scooters.

The above information disclosed in this Background section is only for enhancement of understanding of the background of the application and therefore it may contain information that does not form the prior art that is already known in the art to a person of ordinary skill in the art.

Contents of the invention

The present application provides an auxiliary driving system, method and storage medium applied to elderly scooters, so as to solve the problems existing in the prior art.

In the first aspect, the present application provides an assisted driving system applied to elderly scooters, including: a control module, a sensor module, and a display module, and the control module is connected to the sensor module and the display module in communication;

Among them, the sensor module includes a binocular camera group, a wide-angle camera group and an ultrasonic module;

The binocular camera group is set to face the front of the scooter, and is used to collect and process the environmental data in front of the scooter, and send the obtained first environmental data to the control module;

The wide-angle camera group is used to shoot multi-directional scenes and send them to the control module;

The ultrasonic module is set towards the side and rear of the scooter, and is used to collect and process environmental data from the side and rear of the scooter, and send the obtained second environmental data to the control module;

The control module is used to determine obstacle information according to the first environmental data, the multi-directional scene and the second environmental data, execute an obstacle avoidance strategy according to the obstacle information, and generate a holographic scene according to the multi-directional scene, and send it to the display module for display .

In some embodiments, the binocular camera group includes at least one of a first binocular camera and a second binocular camera;

Wherein, the first binocular camera is arranged on the left front of the scooter, and is used for collecting and processing the environmental data of the left front of the scooter, so as to obtain the left front environmental data;

The second binocular camera is arranged on the right front of the scooter, and is used for collecting and processing the environmental data of the right front of the scooter, so as to obtain the right front environmental data;

The first environmental data includes at least one item of left front environmental data and right front environmental data.

In some embodiments, the wide-angle camera group includes at least one of a first fisheye camera, a second fisheye camera, a third fisheye camera, a fourth fisheye camera, and a fifth fisheye camera;

Wherein, the first fisheye camera is set on the left front of the scooter, and is used to capture the scene in the left front position of the scooter;

The second fisheye camera is set on the right front of the scooter, and is used to photograph the scene in the right front direction of the scooter;

The third fisheye camera is set on the left side of the scooter, and is used to shoot the scene on the left side of the scooter;

The fourth fisheye camera is set on the right side of the scooter, and is used to shoot the scene on the right side of the scooter;

The fifth fisheye camera is set at the rear of the scooter, and is used to photograph the scene at the rear of the scooter;

The multi-directional scene includes at least one of the scene in the left front direction of the scooter, the scene in the right front direction of the scooter, the scene in the left direction of the scooter, the scene in the right direction of the scooter, and the scene in the rear direction of the scooter.

In some embodiments, the ultrasound module includes at least one of a first ultrasound probe, a second ultrasound probe, a third ultrasound probe, a fourth ultrasound probe, a fifth ultrasound probe, and a sixth ultrasound probe;

Wherein, the first ultrasonic probe and the second ultrasonic probe are arranged on the left side of the scooter, and are arranged sequentially along the front and rear directions of the left side, and are used to collect and process the environmental data on the left side of the scooter to obtain the left side environmental data;

The third ultrasonic probe and the fourth ultrasonic probe are arranged on the right side of the scooter, and are arranged sequentially along the front and rear directions of the right side, and are used to collect and process the environmental data on the right side of the scooter to obtain the environmental data on the right side;

The fifth ultrasonic probe and the sixth ultrasonic probe are arranged at the rear of the scooter, and are arranged in sequence along the left and right directions on the left side, and are used to collect and process environmental data behind the scooter to obtain rear environmental data;

The second environmental data includes at least one item of left environmental data, right environmental data and rear environmental data.

In some embodiments, the sensor module further includes: a level sensor;

The horizontal sensor is used to obtain the horizontal posture information of the scooter, and sends the horizontal posture information to the control module;

The control module is also used to judge whether the scooter is in a horizontal state according to the horizontal attitude information, and output alarm information when the scooter is not in a horizontal state.

In some embodiments, the control module is specifically used to determine the current distance between the obstacle and the scooter according to the first environmental data, the multi-directional scene and the second environmental data, and according to the corresponding relationship between different distances and different control strategies of the scooter, Determine the scooter control strategy corresponding to the current distance, and execute the scooter control strategy for obstacle avoidance processing.

In some embodiments, the corresponding relationship between different distances and the control strategy of the scooter includes:

When the distance between the obstacle and the scooter is less than or equal to the first preset distance, control the scooter to stop;

When the distance between the obstacle and the scooter is greater than the first preset distance and less than or equal to the second preset distance, the scooter is controlled to decelerate;

When the distance between the obstacle and the scooter is greater than the second preset distance and less than or equal to the third preset distance, an alarm prompt message is output.

In the second aspect, the present application provides an assisted driving method applied to an elderly scooter, which is applied to the above-mentioned assisted driving system, and the method includes:

Obtain the first environmental data obtained by the binocular camera group in the sensor module by collecting and processing the environmental data in front of the scooter;

Obtain the multi-directional scene captured by the wide-angle camera group in the sensor module;

Obtain the second environmental data obtained by the ultrasonic module in the sensor module by collecting and processing the environmental data on the sides and rear of the scooter;

Obstacle information is determined according to the first environmental data, the multi-directional scene and the second environmental data, and an obstacle avoidance strategy is executed according to the obstacle information, and a holographic scene is generated according to the multi-directional scene and sent to the display module for display.

In some embodiments, also include:

Obtain the horizontal attitude information of the scooter obtained by the detection of the horizontal sensor in the sensor module;

Judging whether the scooter is in a horizontal state according to the horizontal attitude information, and outputting an alarm message when the scooter is in a non-horizontal state.

In a third aspect, the present application provides a computer-readable storage medium, where computer-executable instructions are stored in the computer-readable storage medium, and when the computer-executable instructions are executed by a processor, they are used to implement the above-mentioned scooters applied to the elderly assisted driving methods.

The assisted driving system, method and storage medium applied to elderly scooters provided by this application include a control module, a sensor module and a display module, wherein the sensor module includes a binocular camera group, a wide-angle camera group and an ultrasonic module, and the binocular camera group It can collect and process environmental data in front of the scooter. The wide-angle camera group can capture multi-directional scenes. The ultrasonic module can collect and process environmental data from the side and rear of the scooter. The control module can determine obstacle information based on multi-directional scenes and environmental data. , and execute a corresponding obstacle avoidance strategy, and generate a holographic scene according to the multi-directional scene, and send it to the display module for display. Therefore, it can not only assist the driver to effectively avoid obstacles, but also allow the driver to know the surrounding environmental information through the holographic scene, thereby improving the driving safety of the driver.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description serve to explain the principles of the disclosure.

FIG. 1 is a schematic diagram of a driving assistance system provided in an embodiment of the present application;

FIG. 2 is a schematic diagram of the binocular camera group 21 in the assisted driving system provided by the embodiment of the present application;

FIG. 3 is a schematic diagram of the wide-angle camera group 22 in the assisted driving system provided by the embodiment of the present application;

FIG. 4 is a schematic diagram of the ultrasonic module 23 in the assisted driving system provided by the embodiment of the present application;

Fig. 5 is another schematic diagram of the assisted driving system provided by the embodiment of the present application;

Fig. 6 is a schematic diagram of an assisted driving method applied to an elderly scooter provided by an embodiment of the present application.

Explanation of reference numbers:

10. Control module; 20. Sensor module; 30. Display module;

21. Binocular camera group; 22. Wide-angle camera group; 23. Ultrasonic module; 24. Level sensor;

211. The first binocular camera; 212. The second binocular camera;

221. The first fisheye camera; 222. The second fisheye camera; 223. The third fisheye camera; 224. The fourth fisheye camera; 225. The fifth fisheye camera;

231. The first ultrasonic probe; 232. The second ultrasonic probe; 233. The third ultrasonic probe; 234. The fourth ultrasonic probe; 235. The fifth ultrasonic probe; 236. The sixth ultrasonic probe.

By means of the above-mentioned drawings, certain embodiments of the present disclosure have been shown and will be described in more detail hereinafter. These drawings and written description are not intended to limit the scope of the disclosed concept in any way, but to illustrate the disclosed concept for those skilled in the art by referring to specific embodiments.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments It is a part of embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a" and "the" used in the embodiments of the present application are also intended to include plural forms, unless the context clearly indicates otherwise.

It should also be noted that the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a good or system comprising a set of elements includes not only those elements but also includes items not expressly listed. other elements of the product, or elements inherent in the commodity or system. Without further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the article or system comprising said element.

The assisted driving system, method and storage medium applied to elderly scooters provided by this application aim to solve the above technical problems in the prior art.

The main idea of the application scheme is: this application proposes an assisted driving system applied to elderly scooters, including a control module, a sensor module and a display module, wherein the sensor module includes a binocular camera group, a wide-angle camera group and an ultrasonic module. The binocular camera group can collect and process environmental data in front of the scooter. The wide-angle camera group can capture multi-directional scenes. The ultrasonic module can collect and process environmental data from the side and rear of the scooter. Obstacle information is determined, a corresponding obstacle avoidance strategy is implemented, and a holographic scene is generated based on a multi-directional scene, and is sent to a display module for display. Therefore, it can not only assist the driver to effectively avoid obstacles, but also allow the driver to know the surrounding environmental information through the holographic scene, thereby improving the driving safety of the driver.

The technical solution of the present application and how the technical solution of the present application solves the above technical problems will be described in detail below with specific embodiments. The following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present application will be described below in conjunction with the accompanying drawings.

In some embodiments, an assisted driving system applied to elderly scooters is provided.

Fig. 1 is a schematic diagram of the assisted driving system provided by the embodiment of the present application. As shown in Fig. 1, the system includes: a control module 10, a sensor module 20 and a display module 30, and the control module 10 communicates with the sensor module 20 and the display module 30 respectively The connection may specifically be a wired communication connection or a wireless communication connection. Wherein, the control module 10 may specifically be a controller, such as a Microcontroller Unit (Microcontroller Unit, MCU) and the like.

Referring to FIG. 1 , the sensor module 20 includes a binocular camera group 21 , a wide-angle camera group 22 and an ultrasonic module 23 .

Wherein, the binocular camera group 21 includes a plurality of binocular cameras, as shown in the arrow direction in the figure, the binocular camera group 21 is set towards the front of the scooter, and is used to collect and process the environmental data in front of the scooter, and will get The first environmental data is sent to the control module 10 .

The wide-angle camera group 22 includes a plurality of wide-angle cameras, such as fisheye cameras, etc., as shown in the direction of the arrow in the figure, the wide-angle camera group 22 is used for shooting multi-directional scenes, and sends them to the control module 10; wherein, the multi-directional scenes are specifically Can include views from the front, rear, left, and right of the scooter.

The ultrasonic module 23 is set to the side and the rear of the scooter, as shown in the arrow direction in the figure, the ultrasonic module 23 is used to collect and process the environmental data of the side and rear of the scooter, and send the obtained second environmental data to control module 10.

In this embodiment, the fisheye camera + ultrasonic module 23 is used to detect obstacles on the side and rear of the user to realize the obstacle avoidance function. Use to avoid side and rear collisions.

The control module 10 is used to determine obstacle information according to the first environment data, the multi-directional scene and the second environment data, and execute an obstacle avoidance strategy according to the obstacle information, so that when the driver's reaction is relatively slow, the control module 10 Automatic control can help the driver to avoid obstacles in time, so as to ensure the safety of the driver.

In addition, the control module 10 is also configured to generate a holographic scene according to the multi-directional scene, and send it to the display module 30 for display. Therefore, the driver can know the surrounding environment information according to the holographic scene, thereby facilitating the driver to perform driving control.

For ease of explanation, the scooter shown in the figure is a four-wheel scooter. It can be understood that the technical solution of the present application can also be applied to a three-wheel scooter.

This embodiment proposes an assisted driving system applied to elderly scooters, including a control module 10, a sensor module 20 and a display module 30, wherein the sensor module 20 includes a binocular camera group 21, a wide-angle camera group 22 and an ultrasonic module 23, The binocular camera group 21 can collect and process environmental data in front of the scooter, the wide-angle camera group 22 can shoot multi-directional scenes, the ultrasonic module 23 can collect and process environmental data from the side and rear of the scooter, and the control module 10 can collect and process environmental data according to the multi-directional Obstacle information is determined from the scene and environment data, and a corresponding obstacle avoidance strategy is executed, and a holographic scene is generated according to the multi-directional scene, and is sent to the display module 30 for display. Therefore, it can not only assist the driver to effectively avoid obstacles, but also allow the driver to know the surrounding environmental information through the holographic scene, thereby improving the driving safety of the driver.

FIG. 2 is a schematic diagram of a binocular camera group 21 in the assisted driving system provided by the embodiment of the present application. As shown in FIG. 2 , the binocular camera group 21 includes at least one of a first binocular camera 211 and a second binocular camera 212 item;

Referring to Fig. 2, the first binocular camera 211 is arranged on the left front of the scooter, as shown in the direction of the arrow in the figure, the first binocular camera 211 is used to collect and process the environmental data of the left front of the scooter to obtain the left front environment data;

The second binocular camera 212 is arranged on the right front of the scooter, as shown in the direction of the arrow in the figure, the second binocular camera 212 is used to collect and process the environmental data of the right front of the scooter to obtain the right front environment data; The environmental data includes at least one item of left front environmental data and right front environmental data.

Since the driver's front is the main dangerous area, which is a high-incidence area of safety accidents, it is necessary to collect dense spatial information such as point clouds to ensure user safety. Therefore, two binocular cameras are selected to monitor all dangerous situations within a radius of 3 meters ahead.

Thereby, by arranging the first binocular camera 211 and the second binocular camera 212, the environmental data acquisition processing of the left front and the right front of the scooter can be carried out respectively, so that the environmental data acquisition process without dead angle can be carried out to the front of the scooter, ensuring driver safety.

3 is a schematic diagram of the wide-angle camera group 22 in the assisted driving system provided by the embodiment of the present application. As shown in FIG. 3 , the wide-angle camera group 22 includes a first fisheye camera 221, a second fisheye camera 222, and a third fisheye camera 223, at least one of the fourth fisheye camera 224 and the fifth fisheye camera 225;

Referring to Fig. 3, the first fisheye camera 221 is arranged on the left front of the scooter, as shown in the direction of the arrow in the figure, the first fisheye camera 221 is used to photograph the scene in the left front direction of the scooter;

The second fisheye camera 222 is arranged on the right front of the scooter, as shown in the direction of the arrow in the figure, the second fisheye camera 222 is used to photograph the scene in the right front direction of the scooter;

The third fisheye camera 223 is arranged on the left side of the scooter, as shown in the direction of the arrow in the figure, the third fisheye camera 223 is used to photograph the scene on the left side of the scooter;

The fourth fisheye camera 224 is arranged on the right side of the scooter, as shown in the direction of the arrow in the figure, the fourth fisheye camera 224 is used to photograph the scene on the right side of the scooter;

The fifth fisheye camera 225 is arranged at the rear of the scooter, as shown in the direction of the arrow in the figure, the fifth fisheye camera 225 is used to photograph the scene at the rear of the scooter;

Based on the above multiple fisheye cameras, the obtained multi-directional scene includes the scene of the left front of the scooter, the scene of the right front of the scooter, the scene of the left side of the scooter, the scene of the right side of the scooter and the rear position of the scooter Therefore, the control module 10 can accurately construct a holographic scene of the surrounding environment of the scooter according to the above-mentioned scenes in multiple directions, which is convenient for the driver to watch.

Specifically, the control module 10 can obtain the holographic image through the following panorama algorithm flow: calibrate the fisheye camera; obtain the de-distorted image according to various azimuth scenes; determine the relative position of the virtual bird's-eye camera relative to the four world coordinate systems; transform the de-distorted image into Bird's-eye view; using nonlinear optimization technology to obtain panoramic stitching images.

Figure 4 is a schematic diagram of the ultrasonic module 23 in the assisted driving system provided by the embodiment of the present application. As shown in Figure 4, the ultrasonic module 23 includes a first ultrasonic probe 231, a second ultrasonic probe 232, a third ultrasonic probe 233, a fourth ultrasonic probe At least one of the probe 234, the fifth ultrasonic probe 235 and the sixth ultrasonic probe 236;

With reference to Fig. 4, the first ultrasonic probe 231 and the second ultrasonic probe 232 are arranged on the left side of the scooter, and are arranged in sequence along the front and back direction of the left side, such as the first ultrasonic probe 231 is at the front position on the left side, and the second ultrasonic probe The probe 232 is at the rear position on the left side, as shown by the arrow in the figure, the first ultrasonic probe 231 and the second ultrasonic probe 232 are used to collect and process the environmental data on the left side of the scooter to obtain the left environmental data;

The third ultrasonic probe 233 and the fourth ultrasonic probe 234 are arranged on the right side of the scooter, and are arranged in sequence along the front and rear directions of the right side, such as the third ultrasonic probe 233 is at the front position on the right side, and the fourth ultrasonic probe 234 is at the right side. The position behind the side, as shown in the direction of the arrow in the figure, the third ultrasonic probe 233 and the fourth ultrasonic probe 234 are used to collect and process the environmental data on the right side of the scooter to obtain the environmental data on the right side;

The fifth ultrasonic probe 235 and the sixth ultrasonic probe 236 are arranged at the rear of the scooter, and are arranged in turn along the left and right directions of the left side. For example, the fifth ultrasonic probe 235 is at the rear to the left, and the sixth ultrasonic probe 236 is at the rear to the right. As shown in the direction of the arrow in the figure, the fifth ultrasonic probe 235 and the sixth ultrasonic probe 236 are used to collect and process environmental data behind the scooter to obtain rear environmental data;

Thereby, by arranging the first ultrasonic probe 231, the second ultrasonic probe 232, the third ultrasonic probe 233, the fourth ultrasonic probe 234, the fifth ultrasonic probe 235 and the sixth ultrasonic probe 236, the left side and the right side of the scooter can be carried out respectively. And the environmental data collection and processing of the rear, so that the environmental data collection and processing of the side and rear of the scooter without dead ends can be carried out to ensure the safety of the driver.

FIG. 5 is another schematic diagram of the assisted driving system provided by the embodiment of the present application. As shown in FIG. 5 , the sensor module 20 further includes: a level sensor 24;

Horizontal sensor 24 is used to obtain the horizontal posture information of scooter, and horizontal posture information is sent to control module 10; , an alarm message is output.

For example, the horizontal sensor 24 can use a gyroscope, by installing the gyroscope on the scooter and calibrating the horizontal attitude in advance, and judging whether the scooter has tilted, collided, dropped or other dangerous events by judging the sampling values of the gyroscope in real time.

Optionally, a networking module can be installed on the scooter and perform data communication with the background server. When the scooter detects that a corresponding dangerous event occurs, the control module 10 can send an alarm message to the server through an agreed protocol format, and the server communicates with the server. The guardian application also maintains communication, and the alarm information is pushed to the guardian through the server, so that the guardian can know the occurrence of dangerous events in time and help the driver in time.

In this embodiment, the level sensor 24 is set to detect the horizontal posture of the scooter. If the scooter is in a non-horizontal posture due to dangerous situations such as rollover, the control module 10 can give an alarm in time to ensure the safety of the driver.

In some embodiments, the control module 10 is specifically used to determine the current distance between the obstacle and the scooter according to the first environmental data, the multi-directional scene, and the second environmental data, and according to the correspondence between different distances and different control strategies of the scooter relationship, determine the control strategy of the scooter corresponding to the current distance, and execute the control strategy of the scooter for obstacle avoidance processing.

Specifically, the control module can obtain the obstacle information in front, rear, left and right sides of the scooter according to the multi-directional scene, the first environment data and the second environment data, and according to the above information, the control module 10 can determine The current distance between obstacles, and then, based on the preset correspondence between the distance and the control strategy, the control module 10 can determine the control strategy of the scooter that matches the current distance, so as to perform corresponding obstacle avoidance processing.

For example, the first preset distance can be set to 1.2m, the second preset distance can be set to 1.7m, and the third preset distance can be set to 3m, then when the distance between the obstacle and the scooter is less than Or when it is equal to 1.2m, the control module 10 controls the scooter to stop; when the distance between the obstacle and the scooter is greater than 1.2m, less than or equal to 1.7m, the control module 10 controls the scooter to decelerate; When the distance between the scooters is greater than 1.7m and less than or equal to 3m, the control module 10 outputs an alarm prompt message.

In some embodiments, an assisted driving method applied to an elderly scooter is provided, which is applied to the assisted driving system of each of the above embodiments, and specifically applied to a control module.

Fig. 6 is a schematic diagram of an assisted driving method applied to an elderly scooter provided by the embodiment of the present application. As shown in Fig. 6, the method mainly includes the following steps:

S100. Obtain the first environmental data obtained by the binocular camera group in the sensor module by collecting and processing the environmental data in front of the scooter;

Wherein, the first environmental data includes at least one item of left front environmental data and right front environmental data.

S200. Obtain the multi-directional scene captured by the wide-angle camera group in the sensor module;

Wherein, the multi-directional scene includes at least one of the scene in the left front direction of the scooter, the scene in the right front direction of the scooter, the scene in the left direction of the scooter, the scene in the right direction of the scooter, and the scene in the rear direction of the scooter.

S300. Obtain the second environmental data obtained by the ultrasonic module in the sensor module by collecting and processing the environmental data on the sides and rear of the scooter;

Wherein, the second environment data includes at least one item of left environment data, right environment data and rear environment data.

S400. Determine obstacle information according to the first environmental data, the multi-directional scene and the second environmental data, execute an obstacle avoidance strategy according to the obstacle information, and generate a holographic scene according to the multi-directional scene, and send it to the display module for display.

Optionally, S400 specifically includes determining the current distance between the obstacle and the scooter according to the first environmental data, the multi-directional scene, and the second environmental data, and determining the corresponding relationship between the current distance and the scooter according to the correspondence between different distances and different control strategies of the scooter. The scooter control strategy, and execute the scooter control strategy for obstacle avoidance.

Among them, the corresponding relationship between different distances and scooter control strategies includes:

This embodiment proposes an assisted driving method applied to elderly scooters. The control module can determine obstacle information based on environmental data, execute corresponding obstacle avoidance strategies, and generate holographic scenes based on multi-directional scenes, and send them to the display module to display. Therefore, it can not only assist the driver to effectively avoid obstacles, but also allow the driver to know the surrounding environmental information through the holographic scene, thereby improving the driving safety of the driver.

In some embodiments, the method also includes:

S500. Obtain the horizontal posture information of the scooter detected by the horizontal sensor in the sensor module; judge whether the scooter is in a horizontal state according to the horizontal posture information, and output an alarm message when the scooter is in a non-horizontal state.

It should be understood that although the steps in the flow charts in the above embodiments are shown sequentially as indicated by the arrows, these steps are not necessarily executed sequentially in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the figure may include multiple sub-steps or multiple stages, these sub-steps or stages are not necessarily executed at the same time, but may be executed at different times, and the execution order is not necessarily sequential Instead, it may be performed alternately or alternately with at least a part of other steps or sub-steps or stages of other steps.

In some embodiments, a computer-readable storage medium is provided, and computer-executable instructions are stored in the computer-readable storage medium, and the computer-executable instructions are used to implement the steps of the various method embodiments of the present application when executed by a processor.

In some embodiments, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps of the various method embodiments of the present application are implemented.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented through computer programs to instruct related hardware. The computer programs can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it may include the procedures of the embodiments of the above-mentioned methods. Wherein, any references to memory, storage, database or other media used in the various embodiments provided in the present application may include non-volatile and/or volatile memory. Nonvolatile memory can include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Chain Synchlink DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

Other embodiments of the disclosure will be readily apparent to those skilled in the art from consideration of the specification and practice of the applications disclosed herein. This application is intended to cover any modification, use or adaptation of the present disclosure, and these modifications, uses or adaptations follow the general principles of the present disclosure and include common knowledge or conventional technical means in the technical field not disclosed in the present disclosure . The specification and examples are to be considered exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It should be understood that the present disclosure is not limited to the precise constructions which have been described above and shown in the drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

An assisted driving system applied to elderly scooters, characterized in that it includes: a control module, a sensor module and a display module, and the control module is respectively connected to the sensor module and the display module in communication;

Wherein, the sensor module includes a binocular camera group, a wide-angle camera group and an ultrasonic module;

The binocular camera group is set towards the front of the scooter, and is used to collect and process the environmental data in front of the scooter, and send the obtained first environmental data to the control module;

The wide-angle camera group is used to shoot multi-directional scenes and send them to the control module;

The ultrasonic module is set towards the side and rear of the scooter, and is used to collect and process environmental data from the side and rear of the scooter, and send the obtained second environmental data to the control module;

The control module is configured to determine obstacle information according to the first environmental data, the multi-directional scene, and the second environmental data, and execute an obstacle avoidance strategy according to the obstacle information, and, according to the multi-directional The scene generates a holographic scene and sends it to the display module for display.
The system according to claim 1, wherein the binocular camera group includes at least one of a first binocular camera and a second binocular camera;

Wherein, the first binocular camera is set on the left front of the scooter, and is used to collect and process the environmental data of the left front of the scooter, so as to obtain the left front environmental data;

The second binocular camera is set at the right front of the scooter, and is used to collect and process the environmental data of the right front of the scooter, so as to obtain the right front environmental data;

The first environmental data includes at least one of the left front environmental data and the right front environmental data.
The system according to claim 1, wherein the wide-angle camera group includes one of the first fisheye camera, the second fisheye camera, the third fisheye camera, the fourth fisheye camera and the fifth fisheye camera. at least one;

Wherein, the first fisheye camera is set at the left front of the scooter, and is used to photograph the scene in the left front direction of the scooter;

The second fisheye camera is arranged on the right front of the scooter, and is used to photograph the scene in the right front direction of the scooter;

The third fisheye camera is arranged on the left side of the scooter, and is used to photograph the scene on the left side of the scooter;

The fourth fisheye camera is set on the right side of the scooter, and is used to photograph the scene on the right side of the scooter;

The fifth fisheye camera is arranged at the rear of the scooter, and is used to photograph the scene in the rear direction of the scooter;

The multi-directional scene includes the scene of the front left of the scooter, the scene of the front right of the scooter, the scene of the left side of the scooter, the scene of the right side of the scooter, and the scene of the scooter At least one item of the scene in the rear direction.
The system according to claim 1, wherein the ultrasonic module includes at least one of the first ultrasonic probe, the second ultrasonic probe, the third ultrasonic probe, the fourth ultrasonic probe, the fifth ultrasonic probe and the sixth ultrasonic probe one item;

Wherein, the first ultrasonic probe and the second ultrasonic probe are arranged on the left side of the scooter, and are arranged in sequence along the front and rear directions of the left side, and are used to collect and process environmental data on the left side of the scooter, so as to obtain Environmental data on the left;

The third ultrasonic probe and the fourth ultrasonic probe are arranged on the right side of the scooter, and are arranged in sequence along the front and rear directions of the right side, and are used to collect and process environmental data on the right side of the scooter to obtain the right side environmental data;

The fifth ultrasonic probe and the sixth ultrasonic probe are arranged at the rear of the scooter, and are arranged in sequence along the left and right directions on the left side, and are used to collect and process environmental data behind the scooter to obtain rear environmental data;

The second environmental data includes at least one of the left environmental data, the right environmental data, and the rear environmental data.
The system according to claim 1, wherein the sensor module further comprises: a level sensor;

The horizontal sensor is used to obtain the horizontal posture information of the scooter, and send the horizontal posture information to the control module;

The control module is also used for judging whether the scooter is in a horizontal state according to the horizontal attitude information, and outputting an alarm message when the scooter is not in a horizontal state.
The system according to any one of claims 1-5, wherein the control module is specifically configured to determine obstacles and walking alternatives according to the first environmental data, the multi-directional scene and the second environmental data. The current distance between vehicles, according to the corresponding relationship between different distances and different scooter control strategies, determine the scooter control strategy corresponding to the current distance, and execute the scooter control strategy to perform obstacle avoidance processing.
The system according to claim 6, wherein the corresponding relationship between the different distances and the control strategy of the scooter comprises:

When the distance between the obstacle and the scooter is less than or equal to a first preset distance, control the scooter to stop;

When the distance between the obstacle and the scooter is greater than a first preset distance and less than or equal to a second preset distance, control the scooter to decelerate;

When the distance between the obstacle and the scooter is greater than the second preset distance and less than or equal to the third preset distance, an alarm prompt message is output.
An assisted driving method applied to elderly scooters, applied to the assisted driving system according to any one of claims 1-7, characterized in that the method comprises:

Obtain the first environmental data obtained by the binocular camera group in the sensor module by collecting and processing the environmental data in front of the scooter;

Obtain the multi-directional scene captured by the wide-angle camera group in the sensor module;

Obtain the second environmental data obtained by the ultrasonic module in the sensor module by collecting and processing the environmental data on the sides and rear of the scooter;

Determining obstacle information according to the first environment data, the multi-directional scene, and the second environmental data, and executing an obstacle avoidance strategy according to the obstacle information, and generating a holographic scene according to the multi-directional scene, and sent to the display module for display.
The method according to claim 8, further comprising:

Obtain the horizontal attitude information of the scooter obtained by the detection of the horizontal sensor in the sensor module;

Judging whether the scooter is in a horizontal state according to the horizontal posture information, and outputting alarm information when the scooter is in a non-horizontal state.
A computer-readable storage medium, characterized in that, computer-executable instructions are stored in the computer-readable storage medium, and the computer-executable instructions are used to implement any one of claims 8-9 when executed by a processor The assisted driving method applied to the elderly scooter.