CN108062868B

CN108062868B - Bicycle detection system and method for vehicle and vehicle

Info

Publication number: CN108062868B
Application number: CN201610984676.8A
Authority: CN
Inventors: 唐帅; 吕尤
Original assignee: Audi AG
Current assignee: Audi AG
Priority date: 2016-11-09
Filing date: 2016-11-09
Publication date: 2021-06-29
Anticipated expiration: 2036-11-09
Also published as: CN108062868A

Abstract

The present disclosure discloses a bicycle detection system and method and a vehicle equipped with the system. The system comprises: an image acquisition unit configured to acquire an image of an environment around the vehicle; a processor configured to: identifying a self-propelled vehicle from the acquired images; detecting the treading state of the bicycle; and predicting a motion of the bicycle based at least in part on the pedaling state. This approach may allow for a more accurate prediction of the movement of the bicycle and may thereby reduce collisions with the bicycle.

Description

Bicycle detection system and method for vehicle and vehicle

Technical Field

The present disclosure relates to the field of vehicles, and more particularly, to a bicycle detection system and method for a vehicle, and a vehicle equipped with the system.

Background

There are already devices or systems on the vehicle that can be used to detect objects around the vehicle in order to facilitate operation during driving. Generally, detection is accomplished by processing data relating to the surroundings of the vehicle, which is collected by sensors (e.g., cameras, or ultrasonic sensors) mounted on the vehicle. Based on such detection, the movement of the object may be predicted, and thereby the driver may be warned of a possible collision, so that appropriate action may be taken to ensure safe driving.

One example of an object is a bicycle in the middle of riding, whose movement may be much more difficult to predict than other objects (e.g., vehicles or pedestrians).

Disclosure of Invention

Embodiments of the present disclosure are provided to address the problems discussed above. In fact, embodiments of the present disclosure present a bicycle detection system and method that can provide a better prediction of the movement of a bicycle. The following presents a simplified summary of one or more aspects of the disclosure in order to provide a basic understanding of such aspects.

In one exemplary embodiment of the present disclosure, there is provided a bicycle detection system for a vehicle, including: an image acquisition unit configured to acquire an image of an environment around the vehicle; a processor configured to: identifying a self-propelled vehicle from the acquired images; detecting a pedaling state of the bicycle; and predicting a motion of the bicycle based at least in part on the pedaling state.

Optionally, the processing is further configured to predict the motion of the bicycle based at least in part on a change in pedaling frequency of the bicycle.

Optionally, the processor is further configured to: identifying structural features of the bicycle from the image; determining a gear-based classification of the bicycle based on the identified structural features; and predicting the motion of the bicycle based on the change in pedaling frequency and the classification. Optionally, the processor is further configured to identify a structural feature associated with a gear, a flywheel, a brake assembly, or any combination thereof of the bicycle.

Optionally, the gear-based classification of bicycles includes single speed bicycles, variable speed bicycles, and dead-fly bicycles.

Optionally, the processor is further configured to: determining the bicycle as a single speed bicycle if a single rear gear is identified; determining the bicycle as a variable speed bicycle if a plurality of rear gears are identified; and identifying the bicycle as a dead-flying bicycle if the flywheel and/or brake assembly is not identified.

Optionally, the processor is further configured to: predicting a coasting state of the bicycle if the pedaling frequency of the bicycle is detected to drop to zero.

Optionally, the processor is further configured to: for a single-speed bicycle, predicting deceleration of the bicycle if it is detected that the pedaling frequency is decreasing, and predicting acceleration of the bicycle if it is detected that the pedaling frequency is increasing; for a variable speed bicycle, if a sudden drop in the pedaling frequency is detected, acceleration of the bicycle is predicted, and if a sudden increase in the pedaling frequency is detected, deceleration of the bicycle is predicted.

Optionally, the processor is further configured to detect the change in the pedaling frequency by calculating the pedaling frequency for a plurality of images within a time window.

Optionally, the processor is further configured to detect the change in the pedaling frequency by detecting a change in motion of pedals of the bicycle, or legs or feet of a rider of the bicycle.

Optionally, the system further comprises one or more sensors for acquiring data associated with the travel of said vehicle and/or said bicycle; the processor is further configured to assist driving of the vehicle based on the data and the prediction of the motion of the bicycle.

Optionally, the processor is further configured to estimate a likelihood of a potential collision based on the data and the prediction of the motion of the bicycle, and issue an alert based on the estimation.

In another exemplary embodiment, a vehicle is provided that is equipped with the system described above.

In yet another exemplary embodiment, there is provided a bicycle detection method of a user vehicle, including: identifying a self-propelled vehicle from the acquired image of the surroundings of the vehicle; detecting a pedaling state of the bicycle; and predicting a motion of the bicycle based at least in part on the pedaling state.

Optionally, detecting the pedaling state of the bicycle comprises detecting a change in pedaling frequency of the bicycle.

Optionally, the method further comprises: identifying structural features of the bicycle from the image; and determining a gear-based classification of the bicycle based on the identified structural features; the predicting of the movement of the bicycle further comprises: predicting the motion of the bicycle based on the change and classification of the pedaling frequency.

Optionally, identifying the mechanical feature of the bicycle comprises identifying a structural feature associated with a gear, a flywheel, a brake assembly, or any combination thereof of the bicycle.

In a further exemplary embodiment of the present disclosure, there is provided a bicycle detecting device, including: the identification unit is used for identifying the self-running vehicle from the acquired image of the surrounding environment of the vehicle; a detection unit for detecting a pedaling state of the bicycle; and a prediction unit for predicting a motion of the bicycle based at least in part on the pedaling state.

According to some embodiments of the present disclosure, a motion of a bicycle is predicted based on a recognized pedaling state of the bicycle, so that a more accurate prediction may be obtained and a collision with the bicycle may be reduced thereby.

Drawings

In order to clearly illustrate the technical solutions in the embodiments of the present disclosure, a brief description of drawings required in the description of the embodiments is given below. It is apparent that the drawings described below are some embodiments of the present disclosure, and based on these drawings, other drawings can be obtained by those of ordinary skill in the art without any inventive effort.

FIG. 1 is a schematic diagram illustrating an exemplary bicycle detection system in accordance with an embodiment of the present disclosure.

FIG. 2 is a schematic diagram illustrating an exemplary bicycle detection system in accordance with another embodiment of the present disclosure.

Fig. 3 is a schematic flow chart diagram illustrating a bicycle detection method according to an embodiment of the present disclosure.

FIG. 4 is a block diagram illustrating a bicycle detection device according to some embodiments of the present disclosure.

Detailed Description

The detailed description set forth below in connection with the appended drawings is intended as a description of various configurations and is not intended to represent the only configurations in which the concepts and features described herein may be practiced. The following description includes specific details for the purpose of providing a thorough understanding of various concepts. It will be apparent, however, to one skilled in the art that these concepts may be practiced without these specific details.

A Driving Assistance System (DAS) can recognize objects (e.g., pedestrians, other vehicles, and obstacles) located near a vehicle, for example, from an image of the surroundings of the vehicle captured by a camera. Further, with various sensors mounted on the vehicle, various data (e.g., position, size, and speed) associated with these objects can be acquired, and thus the motion of these objects can be predicted. Further, based on the prediction, an alarm or guidance may be provided to avoid an accident such as a collision or scratch. For objects such as vehicles, this approach may work well, but may present challenges when applied to the prediction of a bicycle in riding.

Generally, a vehicle travels on a certain lane on a road as required by traffic regulations, and its motion and behavior are regular and stable. Further, the vehicle may whistle or turn on the lights, if necessary, to give an indication of its next action. In conjunction with the images described above and various sensor data, the behavior of the vehicle is readily predictable. However, the behaviour of the bicycle cannot be determined so easily. For example, in riding, a rider may change his or her route or speed at will, sometimes even ignoring traffic regulations. Simple collection of data such as distance and position may not be sufficient to accurately predict bicycle behavior.

Embodiments are directed to providing a prediction of a motion of a bicycle, such prediction being based at least in part on a pedaling state of the bicycle.

The various concepts presented throughout this disclosure may be implemented in a variety of vehicles, including sport-utility vehicles (SUVs), passenger cars, utility vehicles (SUVs), Hybrid Electric Vehicles (HEVs), battery cars, trucks, and the like. However, those skilled in the art will appreciate that these are provided for illustrative purposes only, and that one or more aspects of the present disclosure may be implemented or included in one or more other types of vehicles.

FIG. 1 illustrates a bicycle detection system for a vehicle according to one embodiment of the present disclosure. The system may be part of a Driving Assistance System (DAS) on the vehicle. As shown in fig. 1, the system includes an image acquisition unit 11 and a processor 12 coupled together by an on-board network 13. Examples of the in-vehicle network 13 may include CAN or Flexray.

The image capturing unit 11 may be configured to capture an image of the surroundings of the vehicle. In some embodiments of the present disclosure, the image acquisition unit 11 may be a camera mounted on the vehicle, for example, an RGB or infrared camera.

The processor 12 may be a general-purpose processor such as a Central Processing Unit (CPU), Micro Control Unit (MCU), Digital Signal Processor (DSP), etc., configured to perform some or all of the functions described herein by executing program instructions stored in a data storage medium. Additionally or alternatively, the processor 12 may also include programmable hardware elements, such as Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs), and the like.

The data storage medium may be a memory 23 as will be shown in fig. 2. The memory 23 may be volatile memory, such as Random Access Memory (RAM), static RAM (sram), dynamic RAM (dram), or non-volatile memory, such as Read Only Memory (ROM), flash memory, magnetic, electro-optical storage, or the like, or some combination of the two. Memory 23 may be used to store program instructions capable of being executed by processor 12.

In an embodiment of the present disclosure, the processor 12 is configured to identify a bicycle with a rider from the image captured by the image capturing unit 11. Optionally, the processor 12 may also identify structural features associated with one or more components of the bicycle from the image. Identification of bicycles and their structural features may be accomplished through the use of computer vision or pattern recognition techniques, the processes of which are well known to those skilled in the art, and thus the details of such techniques are omitted herein in order not to unnecessarily obscure the present disclosure.

In embodiments of the present disclosure, the identified structural features may be used by the processor 12 to determine a gear-based classification of the bicycle.

Gear-based classifications of bicycles can include single speed, variable speed, and dead-flight. The following presents a simplified summary of the structural features of each category in order to provide a thorough understanding of the concepts of the present disclosure.

A dead-run bicycle (or fixed-wheel bicycle, also referred to as "fixed-gear" in some cases) is a bicycle with a drive train without a flywheel mechanism one perceived major attraction of dead-run bicycles is low weight.

Furthermore, dead-flying bicycles are usually not braked. In other words, there are no brake assemblies such as brakes, brake wires, or cables of various types on a dead-fly bicycle. As a result, a dead-flying bicycle may have a longer braking distance than other types of bicycles and may therefore be more dangerous.

A single speed bicycle is a type of bicycle that has a single gear ratio and does not have a transmission. In other words, the rider cannot change the gear ratio during riding. Generally, a single speed bicycle is mechanically simpler than a variable speed bicycle because there is no derailleur or other shifting system. In particular, single speed bicycles have only a single rear gear on the rear wheel of the bicycle.

A variable speed bicycle is a type of bicycle having multiple gear ratios. In riding, a rider may shift gear ratios through one or more transmissions. To enable shifting between different gear ratios, there are a plurality (e.g., 7 to 10) of rear gears and at least one transmission on the rear wheels. Typically, there may also be multiple (e.g., 3) front gears on the bicycle.

In addition, single and variable speed bicycles typically have a brake assembly and a flywheel thereon.

In general, each type has unique structural features that can distinguish it from the other types. Based on the identification of one or more of such structural features, the processor 12 may determine a gear-based classification of the bicycle.

In some embodiments, the processor 12 is configured to determine the classification of the bicycle based on structural features associated with the rear gear of the bicycle. A rear gear refers to one or more gears at the hub of the rear wheel of the bicycle. In particular, if a single rear gear is identified, the processor may determine the bicycle as a single speed bicycle, and if multiple rear gears are identified, the bicycle may be identified as a variable speed bicycle.

Alternatively or in addition, the processor may determine a classification of the bicycle based on a characteristic associated with the transmission. For example, if a derailleur is identified on a bicycle, the bicycle may be determined to be a shifting bicycle. In another example, if the bicycle does not have a derailleur, it may be determined to be a single speed bicycle or a dead-fly bicycle.

In some other embodiments, the processor 12 is configured to identify a dead-flight type of bicycle based on structural features associated with the flywheel. Specifically, if the flywheel is not identified on the bicycle, the processor 12 may determine the bicycle as a dead-flying bicycle.

Alternatively or in addition, the processor may determine a dead-flying bicycle based on structural features associated with the brake assembly. The brake assembly may include various components such as a disc brake/V-brake, brake lever, brake cable or cable, etc. For example, if none of these components are detected, the processor 12 may determine the bicycle as a dead-flying bicycle.

Furthermore, the various structural features may be employed in any combination. For example, a bicycle with a flywheel and no transmission may be identified by the processor 12 as a single speed bicycle, while a bicycle with neither a flywheel nor shifting gas would be identified as a dead-fly bicycle.

It should be understood that the various structural features described above are provided for purposes of illustration and not limitation. The classification of the bicycle may also be determined based on structural features associated with other components (e.g., front gears, handlebars, dials, or any combination thereof).

In some embodiments, the processor 12 may be configured to identify the pedaling state of the bicycle from the image captured by the image capturing unit 11.

The "stepping state" herein refers to a state of motion of the pedal. For example, while the pedal is normally stepped forward by the rider's foot during running, sometimes the pedal may be stepped backward or held stationary during a coasting state of the bicycle. In addition, the pedal state may also include some physical representation reflecting the movement of the pedal, such as the frequency or angular velocity of the pedal.

In an exemplary embodiment, the processor 12 is further configured to detect a change in the tread frequency from the image acquired by the image acquisition unit 11. The pedals are generally driven by the legs or feet of the rider, and therefore, a change in pedaling frequency can be reflected by a change in movement of the legs or feet.

In particular, the processor 12 may be further configured to detect a change in the pedaling frequency by calculating the pedaling frequency from a plurality of consecutive images within the time window.

The gear-based classification and the pedaling state of the bicycle have been identified according to the above description. In an exemplary embodiment, the processor 12 is further configured to predict the motion of the bicycle based on the classification and the tread state.

Consider, for example, a variable speed bicycle. When switching to a higher gear ratio, a sudden drop in the pedaling frequency may be detected from a plurality of consecutive images, which may reflect the rider's intention to accelerate. Conversely, when switching to a lower gear ratio, a sudden increase in the pedaling frequency may be detected, which may reflect the intention to decelerate. However, a gradual decrease or increase in frequency generally does not indicate a switch, but rather reflects the intent to slow down or speed up, respectively.

For a single speed bicycle or a dead-flying bicycle, since there is only one gear ratio, a decrease (or increase) in the pedaling frequency, whether abrupt or gradual, merely indicates that the rider is decelerating (accelerating).

In some embodiments, the processor 12 may be further configured to: for a single-speed bicycle, if it is detected that the pedaling frequency is decreasing, then the deceleration of the bicycle is predicted, and if it is detected that the pedaling frequency is increasing, then the acceleration of the bicycle is predicted; for a variable speed bicycle, if a sudden drop in pedaling frequency is detected, acceleration of the bicycle is predicted, and if a sudden increase in pedaling frequency is detected, deceleration of the bicycle is predicted.

For any classification, if it is detected within the time window that the pedals remain stationary, the bicycle will be in a coasting state. The processor 12 may thus be configured to predict a coasting state of the bicycle if it is detected that the pedal frequency is zero.

Referring now to FIG. 2, an exemplary bicycle detection system is illustrated in accordance with another embodiment of the present disclosure. As shown in fig. 2, system 200 includes an image acquisition unit 21, a processor 22, a memory 23, and one or more sensors.

The image capturing unit 21 is configured to capture images of the surroundings of the vehicle, and the processor 22 is configured to obtain a prediction of the movement of the bicycle according to the operations described with reference to the processor 12 in fig. 1. For the sake of brevity, details thereof are not repeated here.

The one or more sensors 24 may include sensors for acquiring various driving data of the vehicle (e.g., an acceleration sensor, a speed sensor, and/or a steering angle sensor mounted on a steering column). The processor 22 may be configured to determine a driving state or intent of the vehicle based on data acquired by these sensors.

Additionally or alternatively, the one or more sensors 24 may include sensors for acquiring motion data (e.g., location, distance, speed, etc.) associated with the travel of the bicycle. Such sensors may include one or more of a laser sensor, a radar sensor, or an acoustic sensor.

In this embodiment, the processor 22 may be further configured to assist driving of the vehicle based on the prediction of the motion of the bicycle, the driving intent of the vehicle, and the motion data associated with the bicycle. In further embodiments, the processor 22 may estimate the likelihood of a potential collision between the bicycle and the vehicle based on the predictions and various sensor data. In further embodiments, the processor 22 may issue a visual, audible, or tactile alert through a screen, audio, or other interactive device. Additionally or alternatively, in some examples, processor 22 may activate an autonomous driving operation of a Driving Assistance System (DAS) in response to the estimated likelihood of the collision.

According to some embodiments of the present disclosure, the motion of the bicycle is predicted based on a pedaling state of the bicycle. Such an approach may allow for a more accurate prediction of the movement of the bicycle and, thus, collisions with the bicycle may be reduced.

In another aspect of the present disclosure, a vehicle equipped with the

bicycle detection system

100 or 200 described above is provided.

Fig. 3 is a schematic flow chart diagram illustrating a bicycle detection method according to an embodiment of the present disclosure. The method 300 may be performed in the system 100 shown in fig. 1. The method 300 includes the following steps.

In step 301, a self-propelled vehicle is identified from an image of the surroundings of the vehicle.

In some embodiments, structural features from the bicycle may be identified from the image, and a gear-based classification of the bicycle may be determined based on the identified structural features.

In some embodiments, the structural features may be associated with one or more components of the bicycle, including, but not limited to, gears, flywheels, and brake assemblies.

In embodiments of the present disclosure, the gear-based classification may include single-speed bicycles, variable speed bicycles, or dead-flight bicycles.

In one implementation of this embodiment, the bicycle is determined to be a single speed bicycle if a single rear gear is identified. In another implementation, if multiple rear gears are identified, the bicycle is determined to be a variable speed bicycle. In yet another implementation, if no flywheel is identified, the bicycle is identified as a dead-flying bicycle.

In step 302, a pedaling state of the bicycle is detected.

In some embodiments, the tread state is detected by detecting a change in tread frequency from the image. In particular, the identification of the change in the pedaling frequency may be achieved by calculating the pedaling frequency from a plurality of images within a time window.

In step 303, a motion of the bicycle is predicted based at least on the pedaling state.

For example, if a gradual decrease in the pedaling frequency is detected, acceleration of the bicycle may be predicted, and if a gradual decrease is detected, deceleration of the bicycle may be predicted. In another example, when a pedaling frequency that drops to zero is detected, a coasting state of the bicycle may be predicted.

In some embodiments, to make the prediction more accurate, the motion of the bicycle is predicted in conjunction with the category of the bicycle based on the pedaling state. Specifically, for a single-speed bicycle, if it is detected that the pedaling frequency is decreasing, deceleration of the bicycle is predicted, and if it is detected that the pedaling frequency is increasing, acceleration of the bicycle is predicted; whereas for a variable speed bicycle, if a sudden drop in the pedaling frequency is detected, acceleration of the bicycle is predicted, and if a sudden increase in the pedaling frequency is detected, deceleration of the bicycle is predicted.

FIG. 4 is a block diagram illustrating a bicycle detection device according to some embodiments of the present disclosure. As shown in fig. 4, the apparatus 400 includes a recognition unit 41, a detection unit 42, and a prediction unit 43. The apparatus 400 may be a software entity, a hardware entity, or a combination thereof. Which may be included in the system 100 described with reference to fig. 1. Alternatively or in addition, the apparatus 400 may be an entity or application executed by the processor 12.

The recognition unit 41 may be adapted to recognizing the outgoing vehicle from the captured image of the surroundings of the vehicle.

In a further embodiment, the identification unit 41 may be adapted to identify a structural feature of the bicycle.

The detection unit 42 may be adapted to detect a pedaling state of the bicycle.

Specifically, as described above, the detection unit 42 may detect a change in the pedaling frequency of the bicycle.

In a further embodiment, the detection unit 42 may be further adapted to determine a gear-based classification of the bicycle based on the structural features identified by the identification unit 41.

The prediction unit 43 may be adapted to predict the movement of the bicycle at least partly on the basis of the pedaling conditions.

In a further embodiment, the prediction unit 43 may be adapted to predict the movement of the bicycle based on the pedaling state and the classification.

Those skilled in the art will appreciate that the elements of the devices disclosed herein may be distributed among the devices of the embodiments and may also be variably located in one or more devices different from those of the embodiments. The units of the above embodiments may be integrated into one unit or may be further divided into a plurality of sub-units.

While the disclosure has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood by those skilled in the art that such limitations are not to be limited to the disclosed embodiment, but is intended to cover various arrangements included without departing from the broadest interpretation so as to encompass all such modifications and equivalent arrangements.

Claims

1. A bicycle detection system for a vehicle, comprising:

an image acquisition unit configured to acquire an image of an environment around the vehicle;

a processor configured to:

identifying a self-propelled vehicle from the acquired images;

detecting a pedaling state of the bicycle, wherein the pedaling state comprises a change in pedaling frequency; and

predicting a motion of the bicycle based at least in part on the change in pedaling frequency.

2. The system of claim 1, wherein the processor is further configured to:

identifying structural features of the bicycle from the image;

determining a gear-based classification of the bicycle based on the identified structural features; and

predicting a motion of the bicycle based on the change in pedaling frequency and the classification.

3. The system of claim 2, wherein the processor is further configured to identify a structural feature associated with a gear, a flywheel, a brake assembly, or any combination thereof, of the bicycle.

4. The system of claim 2 or 3, wherein the gear-based classification of bicycles includes single speed bicycles, variable speed bicycles, and dead-fly bicycles.

5. The system of claim 4, wherein the processor is further configured to:

determining the bicycle as a single speed bicycle if a single rear gear is identified;

determining the bicycle as a variable speed bicycle if a plurality of rear gears are identified; and

if the flywheel and/or brake assembly is not identified, the bicycle is identified as a dead-flying bicycle.

6. The system of claim 1, wherein the processor is further configured to: predicting a coasting state of the bicycle if the pedaling frequency of the bicycle is detected to drop to zero.

7. The system of claim 2, wherein the processor is further configured to:

for a single-speed bicycle, predicting deceleration of the bicycle if it is detected that the pedaling frequency is decreasing, and predicting acceleration of the bicycle if it is detected that the pedaling frequency is increasing;

for a variable speed bicycle, if a sudden drop in the pedaling frequency is detected, acceleration of the bicycle is predicted, and if a sudden increase in the pedaling frequency is detected, deceleration of the bicycle is predicted.

8. The system of claim 1, wherein the processor is further configured to detect the change in the pedaling frequency by calculating the pedaling frequency for a plurality of images within a time window.

9. The system of claim 1 or 8, wherein the processor is further configured to detect the change in pedaling frequency by detecting a change in motion of pedals of the bicycle, or legs or feet of a rider of the bicycle.

10. The system of any of claims 1-3, 5-8, further comprising one or more sensors for acquiring data associated with travel of the vehicle and/or the bicycle;

the processor is further configured to assist driving of the vehicle based on the data and the prediction of the motion of the bicycle.

11. The system of claim 10, wherein the processor is further configured to estimate a likelihood of a potential collision based on the data and a prediction of motion of the bicycle, and issue an alert based on the estimation.

12. A vehicle equipped with a system according to any one of claims 1-11.

13. A bicycle detection method for a vehicle, comprising:

identifying a self-propelled vehicle from the acquired image of the surroundings of the vehicle;

14. The method of claim 13, further comprising:

identifying structural features of the bicycle from the image; and

determining a gear-based classification of the bicycle based on the identified structural features;

the predicting of the motion of the bicycle further comprises: predicting a motion of the bicycle based on the change in pedaling frequency and the classification.

15. The method of claim 14, wherein identifying a mechanical feature of the bicycle comprises identifying a structural feature associated with a gear, a flywheel, a brake assembly, or any combination thereof of the bicycle.

16. A bicycle detecting device, comprising:

the identification unit is used for identifying the self-running vehicle from the acquired image of the surrounding environment of the vehicle;

a detecting unit for detecting a pedaling state of the bicycle, wherein the pedaling state includes a variation in pedaling frequency; and

a prediction unit to predict a motion of the bicycle based at least in part on the change in pedaling frequency.