CN111766855A - Method, device and system for controlling vision sensor of vehicle - Google Patents
Method, device and system for controlling vision sensor of vehicle Download PDFInfo
- Publication number
- CN111766855A CN111766855A CN201910239422.7A CN201910239422A CN111766855A CN 111766855 A CN111766855 A CN 111766855A CN 201910239422 A CN201910239422 A CN 201910239422A CN 111766855 A CN111766855 A CN 111766855A
- Authority
- CN
- China
- Prior art keywords
- target area
- information
- component
- scene
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 33
- 230000004438 eyesight Effects 0.000 title claims description 55
- 230000001681 protective effect Effects 0.000 claims abstract description 65
- 230000000007 visual effect Effects 0.000 claims abstract description 58
- 238000004140 cleaning Methods 0.000 claims abstract description 47
- 230000008447 perception Effects 0.000 claims abstract description 31
- 238000005485 electric heating Methods 0.000 claims description 9
- 239000012459 cleaning agent Substances 0.000 claims description 7
- 238000007635 classification algorithm Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 4
- 230000016776 visual perception Effects 0.000 abstract description 2
- 230000007613 environmental effect Effects 0.000 description 13
- 238000004422 calculation algorithm Methods 0.000 description 11
- 239000011521 glass Substances 0.000 description 9
- 230000008569 process Effects 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 238000001514 detection method Methods 0.000 description 6
- 238000012549 training Methods 0.000 description 6
- 239000003599 detergent Substances 0.000 description 5
- 230000003213 activating effect Effects 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 2
- 230000010365 information processing Effects 0.000 description 2
- 230000001172 regenerating effect Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 239000000126 substance Substances 0.000 description 2
- 241000238631 Hexapoda Species 0.000 description 1
- 241000256626 Pterygota <winged insects> Species 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000010485 coping Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 238000011895 specific detection Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
- G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
- G05D1/0253—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means extracting relative motion information from a plurality of images taken successively, e.g. visual odometry, optical flow
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
- G05D1/0221—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory involving a learning process
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
- G05D1/0223—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory involving speed control of the vehicle
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0257—Control of position or course in two dimensions specially adapted to land vehicles using a radar
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0276—Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Aviation & Aerospace Engineering (AREA)
- Automation & Control Theory (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Electromagnetism (AREA)
- Investigating Or Analysing Materials By Optical Means (AREA)
Abstract
The embodiment of the application discloses a method, a device and a system for controlling a visual sensor of a vehicle, wherein a protective component is arranged outside hardware equipment of the visual sensor, a target area of the protective component has a transparent property, the target area corresponds to the sensing range of the visual sensor, and the protective component is also provided with a foreign matter cleaning component; the control system includes: a scene recognition module and a control module; the scene recognition module is used for recognizing a target scene of which a target area of the protection component is shielded by foreign matters according to an environment perception result of the visual sensor and generating a control instruction according to recognized target scene information; and the control module is used for controlling the foreign matter cleaning component to work according to the control instruction so as to clean the foreign matter on the target area. Through the embodiment of the application, the stability of visual perception information in the automatic driving system can be improved.
Description
Technical Field
The present application relates to the field of sensor control technologies, and in particular, to a method, an apparatus, and a system for controlling a vision sensor of a vehicle.
Background
Currently, most automatic driving systems rely on vision-related perception systems, such as vision-based object detection, target tracking, semantic segmentation, and the like. However, since it is necessary to sense the environment around the vehicle, the vision sensor is usually mounted on the roof or the like to sense the state of other vehicles around the vehicle, traffic participants such as pedestrians, traffic signals on the road, sign lines, and the like. In consideration of the aesthetic and waterproof and dustproof purposes, the existing data sensor hardware configuration scheme is to add a glass cover on the sensor. However, for simple glass cover protection, because ponding on the glass cover can not be cleared up in time, consequently for the data of vision sensor, such as camera can not normally be used in rainy day to autopilot vehicle. In addition, in some special cases, the glass cover may be shielded by other foreign objects, such as flying insects, and after shielding, the vision sensor may not work normally, and cannot output stable environment sensing information.
Therefore, how to improve the stability of the visual perception information in the automatic driving system becomes a technical problem to be solved by those skilled in the art.
Disclosure of Invention
The application provides a method, a device and a system for controlling a vision sensor of a vehicle, which can improve the stability of vision perception information in an automatic driving system.
The application provides the following scheme:
a vision sensor control system for a vehicle,
a protective component is arranged outside hardware equipment of the visual sensor, a target area of the protective component has a transparent property, the target area corresponds to the sensing range of the visual sensor, and the protective component is also provided with a foreign matter cleaning component;
the control system includes:
a scene recognition module and a control module;
the scene recognition module is used for recognizing a target scene of which a target area of the protection component is shielded by foreign matters according to an environment perception result of the visual sensor and generating a control instruction according to recognized target scene information;
and the control module is used for controlling the foreign matter cleaning component to work according to the control instruction so as to clean the foreign matter on the target area.
A vision sensor control method of a vehicle, comprising:
obtaining environment perception result information of a vision sensor;
according to an environment sensing result, a target scene of a target area of a protection component, which is shielded by foreign matters, is identified, wherein the protection component is arranged outside hardware equipment of the visual sensor, the target area of the protection component has a transparent property, the target area corresponds to the sensing range of the visual sensor, and the protection component is also provided with a foreign matter cleaning component;
and generating a control instruction according to the identified target scene information, so as to control the foreign matter cleaning component to work through a control module, so as to clean the foreign matter on the target area.
A vision sensor control apparatus of a vehicle, comprising:
an environment perception result information obtaining unit for obtaining environment perception result information of the vision sensor;
the scene recognition unit is used for recognizing a target scene of which a target area of a protection component is shielded by foreign matters according to an environment sensing result of a visual sensor, wherein the protection component is arranged outside hardware equipment of the visual sensor, the target area of the protection component has a transparent property, the target area corresponds to the sensing range of the visual sensor, and the protection component is also provided with a foreign matter cleaning component;
and the instruction control unit is used for generating a control instruction according to the identified target scene information so as to control the foreign matter cleaning component to work through a control module, so that the foreign matters on the target area are cleaned.
A computer system, comprising:
one or more processors; and
a memory associated with the one or more processors for storing program instructions that, when read and executed by the one or more processors, perform operations comprising:
obtaining environment perception result information of a vision sensor;
according to the environment sensing result of the visual sensor, a target scene of a target area of a protective component, which is shielded by foreign matters, is identified, wherein the protective component is arranged outside hardware equipment of the visual sensor, the target area of the protective component has a transparent property, the target area corresponds to the sensing range of the visual sensor, and the protective component is also provided with a foreign matter cleaning component;
and generating a control instruction according to the identified target scene information, so as to control the foreign matter cleaning component equipment to work through a control module, so as to clean the foreign matter on the target area.
According to the specific embodiments provided herein, the present application discloses the following technical effects:
through the embodiment of the application, the specific vision sensor can still be arranged outside the vehicle such as the top of the vehicle, so that the more complete perception of the surrounding environment is realized. Meanwhile, a protective component is arranged outside the visual sensor, for example, a glass cover and the like, wiper equipment is also arranged outside the protective component, in addition, a scene recognition module can be used for recognizing a target scene of the protective component, which is shielded by foreign matters, the recognition operation can be carried out based on an environment sensing result of the visual sensor instead of simply determining according to a rainfall sensor, and therefore, various different target scenes can be recognized. After the target scene is identified, the windscreen wiper device is started to enter the working state, so that the windscreen wiper device can be started as few as possible, and the influence of the windscreen wiper device on the accuracy of an environment sensing result in the working state is reduced.
Of course, it is not necessary for any product to achieve all of the above-described advantages at the same time for the practice of the present application.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
FIG. 1 is a schematic diagram of a system provided by an embodiment of the present application;
fig. 2 is a schematic diagram illustrating a sensing result provided by the embodiment of the application being blocked by a wiper blade;
FIG. 3 is a flow chart of a method provided by an embodiment of the present application;
FIG. 4 is a schematic view of an apparatus provided by an embodiment of the present application;
FIG. 5 is a schematic diagram of a computer system provided by an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments that can be derived from the embodiments given herein by a person of ordinary skill in the art are intended to be within the scope of the present disclosure.
First, it should be noted that, in order to solve the problem of instability and the like caused in the process of protecting a vision sensor by using a common glass cover, a conceivable solution is to install the vision sensor in a vehicle, use front and rear windshields of the vehicle as a protective cover of the sensor, and meanwhile, since the windshields of the vehicle are usually equipped with a wiper, even in rainy days, rainwater on the windshields can be removed in time, so that the environmental perception of the vision sensor is prevented from being influenced by the shielding of the rainwater.
However, the inventor of the present application has found in the process of implementing the present application that the above-mentioned manner has at least the following disadvantages:
1. in an automatic driving system, a wiper of a vehicle is usually used in rainy days by means of a rainfall sensor, and cannot be identified and processed for other types of foreign matter pollution such as non-rainwater; in the process of sensing the environment by the visual sensor, the visual sensor usually has high requirements on image acquisition results and the like, and the acquisition results are possibly invalid due to pollution of various foreign matters;
2. also, since the wiper of the vehicle is usually used in rainy weather by means of a rain sensor, and the sensitivity of the rain sensor is usually very high, the wiper is turned on for the first time as long as the rain sensor senses that there is a rain drop on the windshield. In the case of a visual sensor, where the glass cover is not covered by standing water or where the coverage is not very high, it is generally not necessary to scrape off raindrops on the protective cover. Since the wiper is usually within the visible range of the visual sensor, the collected environmental data is only affected by the wiper which continuously swings when the wiper is used too frequently;
3. because the vision sensor is mainly used for collecting information of the surrounding environment of the vehicle, although the windshield of the vehicle is transparent, the vision of the vision sensor is more or less blocked due to the existence of other non-transparent parts of the vehicle body, so that the collected data may not be comprehensive enough. In addition, since the vision sensor may be a combination of a plurality of different sensors, for example, a camera, a radar, etc., the volume of the hardware combined together may be relatively large, and in this case, if the sensor is installed in the vehicle, the occupied space is relatively large, and the sensor is also installed at the front and rear windshields of the vehicle, and in the case where the vehicle is turned to be driven manually by the driver, the view of the driver may be seriously blocked.
In summary, for various reasons, it is hardly feasible to adopt the above-described solution for coping with the influence of rain factors by installing the vision sensor in the vehicle interior, the front and rear windshields, etc., and by means of the wiper blades of the windshields.
For this reason, the embodiment of the application provides a corresponding solution. In this aspect, the vision sensor hardware device may still be mounted at a roof or the like outside the vehicle, and a protective member (e.g., a glass cover or the like) may also be added outside the vision sensor hardware device, but unlike the related art, a foreign substance cleaning member (e.g., a wiper blade, an electric heater or the like) may also be provided at the protective member. That is to say, in this application embodiment, can be equipped with special foreign matter clearance part for the protective part of vision sensor hardware equipment, can also set for certain algorithm in advance simultaneously, detect the condition that protective part is sheltered from by the foreign matter (including rainwater, winged insect etc.), if detect the condition of sheltering from that has influenced the vision acquisition result, just can this foreign matter clearance part of automatic start to in time clear away sheltering from of foreign matter, guarantee the degree of accuracy of vision perception result, can not too frequent use foreign matter clearance part again simultaneously. In addition, for the foreign matter cleaning component of the windscreen wiper and the like, the foreign matter cleaning component can be arranged on the outer side of the protection component, so that the foreign matter cleaning component is inevitably sensed by the visual sensor in the working process of the windscreen wiper, namely, the 'body shadow' of the windscreen wiper can appear in an image acquired by the visual sensor in the sensing process of the environment, and part of the actual environment is shielded by the windscreen wiper. In order to eliminate the influence, in an optional implementation manner, information such as a rotation angle of the wiper blade can be fed back in real time, then a sheltering area of the wiper blade on an image can be judged according to the rotation angle fed back by the wiper blade, and accordingly, the influence of the area can be eliminated by a perception algorithm, and a more stable perception result can be obtained.
The following describes in detail a specific technical solution provided in an embodiment of the present application.
Example one
Firstly, the first embodiment provides a vision sensor control system of a vehicle, a protective component is arranged outside hardware equipment of a vision sensor, a target area of the protective component has a transparent property, the target area corresponds to a sensing range of the vision sensor, and the protective component is also provided with a foreign matter cleaning component;
referring to fig. 1, the control system includes:
a scene recognition module 11 and a control module 12;
the scene recognition module 11 is configured to recognize, according to an environment sensing result of the visual sensor, a target scene in which a target area of the protection component is blocked by a foreign object, and generate a control instruction according to recognized target scene information;
and the control module 12 is configured to control the foreign matter cleaning component to operate according to the control instruction, so as to clean the foreign matter on the target area.
The scene recognition module and the control module can be operated in a central control computer of the vehicle automatic driving system or other computer systems in the vehicle.
In the embodiment of the application, the specific scene recognition module does not simply recognize the scene through the sensing result of the rainfall sensor, but recognizes the scene according to the environment sensing result actually acquired by the vision sensor. The environmental sensing result may specifically be acquired image data or the like. Because the protection component surrounds the vision sensor, the vision sensor needs to acquire images of the surrounding environment through the target area of the protection component, and the target area has the transparent property, so that if foreign matters exist on the protection component, the foreign matters can be reflected in an environment sensing result. The embodiment of the application realizes the identification of the target scene by using the principle.
In the specific implementation, some training samples may be used to perform algorithm training in advance, for example, a CNN convolutional neural network may be specifically used as an algorithm model, and the like. The specific training samples can include image samples acquired by the vision sensor in a state that the target area of the protective component is shielded by some foreign matters, the image samples are input into the convolutional neural network for learning, and the training process of the algorithm can be obtained after the algorithm is converged. Then, the environment sensing result actually acquired by the vision sensor can be predicted by using the algorithm, and whether the environment sensing result belongs to a certain target scene or not is judged.
Of course, in the embodiment of the present application, under the condition that the degree of shielding the shielding member is relatively slight, the accuracy of the environmental perception result may not be greatly affected, and on the contrary, if the foreign object cleaning member is started, the influence of the existence of the foreign object cleaning member on the environmental perception result may be greater than the influence of the foreign object itself. Therefore, in an optional implementation manner, the target scene that the protection component is shielded by the foreign matter and the accuracy of the environmental perception result is affected can be identified according to index data such as the definition and/or the integrity of the real-time environmental perception result of the visual sensor, and the foreign matter cleaning component can be started to work only after the scene is identified, so as to remove the influence of the foreign matter. In this case, the recognition may also be performed by way of an algorithm, and when the algorithm is trained, the selected training samples may also be mainly the environmental sensing results that have an influence on the accuracy of the environmental sensing results.
In addition, because in practical application, the foreign matter category that actually shelters from protective member may have a plurality ofly, and correspondingly, specific foreign matter clearance part also can have a plurality ofly, like this, to the foreign matter of different categories, can adopt different foreign matter clearance parts and clearance mode to clear up. For example, in a scene blocked by rain, the required clearing method may be to start the wiper, and the wiper needs to work continuously until the rain stops, or so small that the accuracy of the environmental perception result is not affected any more. However, since the stains may be relatively stubborn and may not be removed completely by simply using the wiper blade, the stains may be removed by activating the detergent sprayer in cooperation with the function of the wiper blade. For foreign matter such as frost, the electric heater may be activated to heat and deice.
Therefore, for the above situation, when a target scene of which the protection component is blocked by a foreign object is identified, the target scene and the category to which the target scene belongs may be specifically identified according to the real-time environment sensing result of the vision sensor and a preset scene classification algorithm; accordingly, the control instruction may be generated according to the category information of the identified target scene.
In a specific implementation, in order to achieve the above purpose, when learning the scene classification algorithm, training image samples of multiple different scene categories may be provided in advance, so that the algorithm can identify a corresponding target scene category according to specific image features.
As described above, the category information of the target scene may include: the protective component is shielded by rainwater; in this case, since the rain process usually lasts for a while, and the magnitude of the rain amount may change at any time, the rotation speed of the wiper blade required for different rain amounts may be different. Therefore, after the target scene of the protective component shielded by rainwater is identified, the degree of shielding of the protective component by rainwater can be continuously identified according to a preset frequency; then, the required rotation speed information of the windscreen wiper can be determined according to the identified shielding degree information; further, the control command may be generated based on the rotational speed information. By the mode, the rotating speed of the windscreen wiper can be adjusted as required, and under the condition that the rainfall is not too large, the rotating speed of the windscreen wiper can be relatively slow, so that the influence of the windscreen wiper on an environment perception result can be reduced.
It should be noted that, since various scenes may occur randomly, the detection of scene recognition may be performed continuously. However, the specific detection frequency may be longer, for example, 30 seconds/time, until the specific target scene is not detected. However, after a scene of rain blocking is detected, in order to more accurately monitor the state of rain, to adjust the rotation speed of the wiper blade, the detection frequency may be changed to 1 second/time, and so on. The detection or recognition algorithm employed may be the same, whether before or after the detection of a particular target scene.
In addition, the category information of the target scene may also include: the protective component is shielded by dirt; at this time, a control instruction for starting the wiper and the detergent sprayer may be generated according to scene information that the protection member is blocked by dirt.
In this case, the control command may further include information on the operating time of the wiper device, and of course, may further include information on the rotational speed, for example, 10S per operation, 2 rotations per second, and the like. In this case, after the expiration of the working time, the detection can also be continued, and if there is still a situation in which the guard member is obscured by dirt, the control instruction for activating the detergent injector simultaneously with the activation of the wiper device can be regenerated. After repeated starting for a preset number of times, if the situation that the protection component is blocked by dirt still exists, alarm information can be generated to prompt that manual cleaning is carried out on the protection component.
In addition, if the category information of the target scene is: the target area of the protection component is shielded by rain and fog, and the rain and fog are usually static and are not as difficult to clean as dirt, so that a control instruction for starting the windscreen wiper and rotating for a preset number of times can be generated directly according to scene information that the target area of the protection component is shielded by rain and fog. That is, in this case, the influence of the foreign matter can be removed only after the wiper blade rotates several times.
Of course, as described above, since the wiper device in the embodiment of the present application is disposed outside the protective part, the environmental sensing data collected by the visual sensor may be affected to some extent when the wiper device is in the operating state. For this situation, in an optional embodiment of the present application, the rotation angle information of the wiper may also be obtained in real time in the working state of the wiper device, so that the position information of the image of the wiper device, which is acquired by the visual sensor, in the whole frame of image may be determined according to the rotation angle information, and after the image content at the position is deleted, the environment sensing calculation may be performed. In particular, the wiper device may swing by rotation of a gear or the like, and thus, the rotation angle information may be generated according to information such as the number of rotations and the position of the rotating gear.
It should be noted that, since the width of the wiper blade is fixed, and the distance between the wiper blade and the vision sensor is also known in advance, the width of the image formed after the wiper blade is captured by the vision sensor is fixed. Therefore, the position of the wiper image in the whole frame image can be determined as long as the rotation angle information of the wiper is obtained. For example, as shown in fig. 2, a portion between two dotted lines may indicate a position area of the wiper device image in the entire frame image at the corresponding rotation angle.
In addition, as described above, the category information of the specifically identified target scene may also be: the target area of the protective component is shielded by frost; at this time, the specific foreign substance cleaning member may further include an electric heating member; correspondingly, a control instruction for starting the electric heating part can be generated according to scene information that a target area of the protection part is blocked by frost. Therefore, the target area can be shielded by the frost in an electric heating deicing mode.
In summary, with the embodiments of the present application, a specific vision sensor can still be disposed outside the vehicle, such as on the roof of the vehicle, to achieve a more complete perception of the surrounding environment. Meanwhile, a protective component is arranged outside the visual sensor, for example, a glass cover and the like, wiper equipment is also arranged outside the protective component, in addition, a scene recognition module can be used for recognizing a target scene of the protective component, which is shielded by foreign matters, the recognition operation can be carried out based on an environment sensing result of the visual sensor instead of simply determining according to a rainfall sensor, and therefore, various different target scenes can be recognized. After the target scene is identified, the windscreen wiper device is started to enter the working state, so that the windscreen wiper device can be started as few as possible, and the influence of the windscreen wiper device on the accuracy of an environment sensing result in the working state is reduced.
Example two
The second embodiment corresponds to the first embodiment, and provides a method for controlling a vision sensor of a vehicle, which may specifically include:
s310: obtaining environment perception result information of a vision sensor;
s320: according to the environment sensing result of the visual sensor, a target scene of a target area of a protective component, which is shielded by foreign matters, is identified, wherein the protective component is arranged outside hardware equipment of the visual sensor, the target area of the protective component has a transparent property, the target area corresponds to the sensing range of the visual sensor, and the protective component is also provided with a foreign matter cleaning component;
s330: and generating a control instruction according to the identified target scene information, so as to control the foreign matter cleaning component to work through a control module, so as to clean the foreign matter on the target area.
During specific implementation, when the target scene shielded by the foreign matters is identified, the target scene shielded by the foreign matters and influencing the accuracy of the environment sensing result can be identified according to the definition and/or the integrity of the real-time environment sensing result of the visual sensor.
The target scene and the category of the target scene can be identified according to the real-time environment sensing result of the visual sensor and a preset scene classification algorithm; at this time, the control command may be generated according to the category information of the identified target scene.
Wherein the category information of the target scene includes: the target area of the protective component is shielded by rainwater;
at this time, the foreign matter cleaning member includes a wiper blade; after a target scene that the target area of the protection component is shielded by rainwater is identified, the degree that the target area of the protection component is shielded by rainwater can be continuously identified according to a preset frequency; determining required rotation speed information of the windscreen wiper according to the identified shielding degree information; the control command may then be generated based on the rotational speed information.
In addition, the category information of the target scene includes: the target area of the protective component is shielded by dirt; at this time, the foreign matter cleaning member includes a wiper blade and a detergent sprayer; according to the scene information that the target area of the protection component is blocked by dirt, a control instruction for starting the wiper device and starting the cleaning agent ejector at the same time can be generated.
At this time, the control instruction may further include working time information of the wiper device; after the working time has expired, if there is still a situation in which the shielding member is obscured by dirt, the control command for activating the detergent injector simultaneously with the wiper device can be regenerated.
And after repeated starting for preset times, if the situation that the protection part is blocked by dirt still exists, generating alarm information to prompt that the protection part is manually cleaned.
In addition, the category information of the target scene may further include: the target area of the protective component is shielded by rain and fog; at this time, the foreign matter cleaning member includes a wiper blade; specifically, a control instruction for starting the windscreen wiper and rotating the windscreen wiper for a preset number of times can be generated according to scene information that a target area of the protection component is shielded by rain and fog.
In order to reduce the influence of the wiper on the accuracy of an environment sensing result in a working state, in the working state of the wiper device, the rotation angle information of the wiper can be acquired in real time, so that the position information of an image of the wiper device, which is acquired by the visual sensor, in the whole frame of image is determined according to the rotation angle information, and after the image content at the position is deleted, environment sensing calculation is carried out.
In addition, the category information of the target scene may further include: the target area of the protective component is shielded by frost; at this time, the foreign matter cleaning member includes an electric heating member; specifically, a control instruction for starting the electric heating part may be generated according to scene information that a target area of the protection part is blocked by frost.
For the parts of the second embodiment that are not described in detail, reference may be made to the descriptions of the first embodiment, and the details are not repeated here.
Corresponding to the second embodiment, the embodiment of the present application further provides a vision sensor control device for a vehicle, and referring to fig. 4, the device may specifically include:
an environmental perception result information obtaining unit 410 for obtaining environmental perception result information of the visual sensor;
a scene recognition unit 420, configured to recognize, according to an environment sensing result of a visual sensor, a target scene in which a target area of a protective component is blocked by a foreign object, where the protective component is disposed outside a hardware device of the visual sensor, the target area of the protective component has a transparent property, the target area corresponds to a sensing range of the visual sensor, and the protective component is further provided with a foreign object cleaning component;
and the instruction control unit 430 is used for generating a control instruction according to the identified target scene information, so as to control the foreign matter cleaning component to work through a control module, so as to clean the foreign matter on the target area.
The scene recognition unit may be specifically configured to:
and identifying a target scene which is shielded by foreign matters and affects the accuracy of the environment sensing result according to the definition and/or the integrity of the real-time environment sensing result of the vision sensor.
In addition, the scene recognition unit may be specifically configured to:
identifying the target scene and the category of the target scene according to the real-time environment sensing result of the visual sensor and a preset scene classification algorithm;
the instruction control unit may specifically be configured to:
and generating the control instruction according to the identified category information of the target scene.
Wherein the category information of the target scene includes: the target area of the protective component is shielded by rainwater;
the foreign matter cleaning component comprises a windscreen wiper;
the apparatus may further include:
the continuous shielding degree identification unit is used for continuously identifying the degree of shielding of the target area of the protection component by rainwater according to a preset frequency after identifying the target scene of the target area of the protection component shielded by rainwater;
the rotating speed determining unit is used for determining the required rotating speed information of the windscreen wiper according to the identified shielding degree information;
the instruction control unit may specifically be configured to:
and generating the control instruction according to the rotation speed information.
Or, the category information of the target scene includes: the target area of the protective component is shielded by dirt;
the foreign matter cleaning component comprises a wiper and a cleaning agent ejector;
the instruction control unit may specifically be configured to:
and generating a control instruction for starting the windscreen wiper and the cleaning agent ejector according to the scene information that the target area of the protection part is blocked by dirt.
The control instruction further comprises working time information of the wiper device;
the device further comprises:
and the command regenerating unit is used for regenerating control commands for starting the wiper device and the cleaning agent ejector if the target area of the protection component is still shielded by dirt after the working time is over.
In addition, the apparatus may further include:
and the alarm unit is used for generating alarm information to prompt that the target area of the protection component is manually cleaned if the target area of the protection component is still shielded by dirt after repeated starting for preset times.
Wherein the category information of the target scene includes: the target area of the protective component is shielded by rain and fog;
the foreign matter cleaning component comprises a windscreen wiper;
the instruction control unit may specifically be configured to:
and generating a control instruction for starting the windscreen wiper and rotating for preset times according to the scene information that the target area of the protection part is shielded by rain and fog.
In addition, the apparatus may further include:
and the rotation angle information acquisition unit is used for acquiring the rotation angle information of the windscreen wiper in real time in the working state of the windscreen wiper device so as to determine the position information of the image of the windscreen wiper device in the whole frame of image acquired by the visual sensor according to the rotation angle information, delete the image content at the position and then carry out environment perception calculation.
In addition, the category information of the target scene includes: the target area of the protective component is shielded by frost;
the foreign matter cleaning member includes an electric heating member;
the instruction control unit may specifically be configured to:
and generating a control instruction for starting the electric heating part according to the scene information that the target area of the protection part is blocked by the frost.
In addition, an embodiment of the present application further provides a computer system, including:
one or more processors; and
a memory associated with the one or more processors for storing program instructions that, when read and executed by the one or more processors, perform operations comprising:
obtaining environment perception result information of a vision sensor;
according to the environment sensing result of the visual sensor, a target scene of a target area of a protective component, which is shielded by foreign matters, is identified, wherein the protective component is arranged outside hardware equipment of the visual sensor, the target area of the protective component has a transparent property, the target area corresponds to the sensing range of the visual sensor, and the protective component is also provided with a foreign matter cleaning component;
and generating a control instruction according to the identified target scene information, so as to control the foreign matter cleaning component equipment to work through a control module, so as to clean the foreign matter on the target area.
Fig. 5 illustrates an architecture of a computer system, which may specifically include a processor 510, a video display adapter 511, a disk drive 512, an input/output interface 513, a network interface 514, and a memory 520. The processor 510, the video display adapter 511, the disk drive 512, the input/output interface 513, the network interface 514, and the memory 520 may be communicatively connected by a communication bus 530.
The processor 510 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an Application Specific Integrated Circuit (ASIC), or one or more Integrated circuits, and is configured to execute related programs to implement the technical solution provided in the present Application.
The Memory 520 may be implemented in the form of a ROM (Read Only Memory), a RAM (Random access Memory), a static storage device, a dynamic storage device, or the like. The memory 520 may store an operating system 521 for controlling the operation of the computer system 500, a Basic Input Output System (BIOS) for controlling low-level operations of the computer system 500. In addition, a web browser 523, a data storage management system 524, and an information processing system 525, among others, may also be stored. The information processing system 525 may be an application program that implements the operations of the foregoing steps in this embodiment of the present application. In summary, when the technical solution provided in the present application is implemented by software or firmware, the relevant program codes are stored in the memory 520 and called to be executed by the processor 510.
The input/output interface 513 is used for connecting an input/output module to realize information input and output. The i/o module may be configured as a component in a device (not shown) or may be external to the device to provide a corresponding function. The input devices may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output devices may include a display, a speaker, a vibrator, an indicator light, etc.
The network interface 514 is used for connecting a communication module (not shown in the figure) to realize communication interaction between the device and other devices. The communication module can realize communication in a wired mode (such as USB, network cable and the like) and also can realize communication in a wireless mode (such as mobile network, WIFI, Bluetooth and the like).
In addition, the computer system 500 may also obtain information of specific pickup conditions from the virtual resource object pickup condition information database 541 for performing condition judgment, and the like.
It should be noted that although the above-mentioned devices only show the processor 510, the video display adapter 511, the disk drive 512, the input/output interface 513, the network interface 514, the memory 520, the bus 530, etc., in a specific implementation, the device may also include other components necessary for normal operation. Furthermore, it will be understood by those skilled in the art that the apparatus described above may also include only the components necessary to implement the solution of the present application, and not necessarily all of the components shown in the figures.
From the above description of the embodiments, it is clear to those skilled in the art that the present application can be implemented by software plus necessary general hardware platform. Based on such understanding, the technical solutions of the present application may be essentially or partially implemented in the form of a software product, which may be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the embodiments or some parts of the embodiments of the present application.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, the system or system embodiments are substantially similar to the method embodiments and therefore are described in a relatively simple manner, and reference may be made to some of the descriptions of the method embodiments for related points. The above-described system and system embodiments are only illustrative, wherein the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
The method, the device and the system for controlling the vehicle vision sensor provided by the present application are described in detail above, and specific examples are applied herein to illustrate the principle and the implementation manner of the present application, and the description of the above embodiments is only used to help understand the method and the core idea of the present application; meanwhile, for a person skilled in the art, according to the idea of the present application, the specific embodiments and the application range may be changed. In view of the above, the description should not be taken as limiting the application.
Claims (14)
1. A vision sensor control system of a vehicle,
a protective component is arranged outside hardware equipment of the visual sensor, a target area of the protective component has a transparent property, the target area corresponds to the sensing range of the visual sensor, and the protective component is also provided with a foreign matter cleaning component;
the control system includes:
a scene recognition module and a control module;
the scene recognition module is used for recognizing a target scene of which a target area of the protection component is shielded by foreign matters according to an environment perception result of the visual sensor and generating a control instruction according to recognized target scene information;
and the control module is used for controlling the foreign matter cleaning component to work according to the control instruction so as to clean the foreign matter on the target area.
2. The system of claim 1,
the scene recognition module is specifically configured to recognize a category to which the target scene belongs according to a real-time environment sensing result of the visual sensor and a preset scene classification algorithm, and generate the control instruction according to category information of the target scene.
3. A vision sensor control method of a vehicle, characterized by comprising:
obtaining environment perception result information of a vision sensor;
according to an environment sensing result, a target scene of a target area of a protection component, which is shielded by foreign matters, is identified, wherein the protection component is arranged outside hardware equipment of the visual sensor, the target area of the protection component has a transparent property, the target area corresponds to the sensing range of the visual sensor, and the protection component is also provided with a foreign matter cleaning component;
and generating a control instruction according to the identified target scene information, so as to control the foreign matter cleaning component to work through a control module, so as to clean the foreign matter on the target area.
4. The method of claim 3, wherein the identifying the target scene with the protective component blocked by the foreign object comprises:
and identifying a target scene which is shielded by foreign matters and affects the accuracy of the environment sensing result according to the definition and/or the integrity of the real-time environment sensing result of the vision sensor.
5. The method of claim 3,
the identifying of the target scene of which the protection component is shielded by the foreign matters comprises the following steps:
identifying the target scene and the category of the target scene according to the real-time environment sensing result of the visual sensor and a preset scene classification algorithm;
the generating of the control instruction according to the identified target scene information includes:
and generating the control instruction according to the identified category information of the target scene.
6. The method of claim 5,
the category information of the target scene includes: the target area of the protective component is shielded by rainwater;
the foreign matter cleaning component comprises a windscreen wiper;
the method further comprises the following steps:
after a target scene that a target area of the protection component is shielded by rainwater is identified, continuously identifying the degree of shielding of the target area of the protection component by the rainwater according to a preset frequency;
determining required rotation speed information of the windscreen wiper according to the identified shielding degree information;
the generating the control instruction according to the identified category information of the target scene includes:
and generating the control instruction according to the rotation speed information.
7. The method of claim 5,
the category information of the target scene includes: the target area of the protective component is shielded by dirt;
the foreign matter cleaning component comprises a wiper and a cleaning agent ejector;
the generating the control instruction according to the identified category information of the target scene includes:
and generating a control instruction for starting the windscreen wiper and the cleaning agent ejector according to the scene information that the target area of the protection part is blocked by dirt.
8. The method of claim 7,
the control instruction also comprises working time information of the wiper device;
the method further comprises the following steps:
after the working time is over, if the target area of the protection component is still covered by dirt, the control instruction for starting the wiper blade device and the cleaning agent injector is regenerated.
9. The method of claim 7, further comprising:
and after repeated starting for preset times, if the condition that the target area of the protection component is still covered by dirt exists, generating alarm information to prompt that the target area of the protection component is manually cleaned.
10. The method of claim 5,
the category information of the target scene includes: the target area of the protective component is shielded by rain and fog;
the foreign matter cleaning component comprises a windscreen wiper;
the generating the control instruction according to the identified category information of the target scene includes:
and generating a control instruction for starting the windscreen wiper and rotating for preset times according to the scene information that the target area of the protection part is shielded by rain and fog.
11. The method of any one of claims 6 to 10, further comprising:
and under the working state of the windscreen wiper device, acquiring the rotation angle information of the windscreen wiper device in real time so as to determine the position information of the image of the windscreen wiper device, which is acquired by the visual sensor, in the whole frame of image according to the rotation angle information, deleting the image content at the position, and then carrying out environment perception calculation.
12. The method of claim 5,
the category information of the target scene includes: the target area of the protective component is shielded by frost;
the foreign matter cleaning member includes an electric heating member;
the generating the control instruction according to the identified category information of the target scene includes:
and generating a control instruction for starting the electric heating part according to the scene information that the target area of the protection part is blocked by the frost.
13. A vision sensor control apparatus of a vehicle, characterized by comprising:
an environment perception result information obtaining unit for obtaining environment perception result information of the vision sensor;
the scene recognition unit is used for recognizing a target scene of which a target area of a protection component is shielded by foreign matters according to an environment sensing result of a visual sensor, wherein the protection component is arranged outside hardware equipment of the visual sensor, the target area of the protection component has a transparent property, the target area corresponds to the sensing range of the visual sensor, and the protection component is also provided with a foreign matter cleaning component;
and the instruction control unit is used for generating a control instruction according to the identified target scene information so as to control the foreign matter cleaning component to work through a control module, so that the foreign matters on the target area are cleaned.
14. A computer system, comprising:
one or more processors; and
a memory associated with the one or more processors for storing program instructions that, when read and executed by the one or more processors, perform operations comprising:
obtaining environment perception result information of a vision sensor;
according to the environment sensing result of the visual sensor, a target scene of a target area of a protective component, which is shielded by foreign matters, is identified, wherein the protective component is arranged outside hardware equipment of the visual sensor, the target area of the protective component has a transparent property, the target area corresponds to the sensing range of the visual sensor, and the protective component is also provided with a foreign matter cleaning component;
and generating a control instruction according to the identified target scene information, so as to control the foreign matter cleaning component equipment to work through a control module, so as to clean the foreign matter on the target area.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910239422.7A CN111766855A (en) | 2019-03-27 | 2019-03-27 | Method, device and system for controlling vision sensor of vehicle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910239422.7A CN111766855A (en) | 2019-03-27 | 2019-03-27 | Method, device and system for controlling vision sensor of vehicle |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111766855A true CN111766855A (en) | 2020-10-13 |
Family
ID=72718965
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910239422.7A Pending CN111766855A (en) | 2019-03-27 | 2019-03-27 | Method, device and system for controlling vision sensor of vehicle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111766855A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH10203259A (en) * | 1997-01-17 | 1998-08-04 | Nippon Buyuu Tec:Kk | On-vehicle camera device |
| JP2016088192A (en) * | 2014-10-31 | 2016-05-23 | アスモ株式会社 | Foreign matter removal device for on-vehicle optical sensor |
| KR20180102462A (en) * | 2017-03-07 | 2018-09-17 | 현대모비스 주식회사 | Apparatus Cleaning Dust of Moving Camera Lens |
| CN108583453A (en) * | 2018-06-28 | 2018-09-28 | 北京图森智途科技有限公司 | A kind of installation device of sensor |
-
2019
- 2019-03-27 CN CN201910239422.7A patent/CN111766855A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH10203259A (en) * | 1997-01-17 | 1998-08-04 | Nippon Buyuu Tec:Kk | On-vehicle camera device |
| JP2016088192A (en) * | 2014-10-31 | 2016-05-23 | アスモ株式会社 | Foreign matter removal device for on-vehicle optical sensor |
| KR20180102462A (en) * | 2017-03-07 | 2018-09-17 | 현대모비스 주식회사 | Apparatus Cleaning Dust of Moving Camera Lens |
| CN108583453A (en) * | 2018-06-28 | 2018-09-28 | 北京图森智途科技有限公司 | A kind of installation device of sensor |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11565659B2 (en) | Raindrop recognition device, vehicular control apparatus, method of training model, and trained model | |
| CA2301303C (en) | Moisture sensor and windshield fog detector | |
| US7860275B2 (en) | Method for detecting rain on a windscreen | |
| CN110999273B (en) | Attachment detection device and vehicle system having same | |
| CN107128287B (en) | Intelligent windscreen wiper control device and its control method based on image recognition and cloud computing | |
| US9783166B1 (en) | Windscreen clearing using surface monitoring | |
| US20200198587A1 (en) | Vehicle wiper control system for target area cleaning | |
| CN111845647B (en) | Automobile camera cleaning system and method | |
| EP1521694B1 (en) | Rain detection apparatus and method | |
| CN107051924A (en) | A kind of vehicle-mounted camera decontamination method and device | |
| CN112248970A (en) | Sensor cleaning device, sensor cleaning method, sensor identification system and unmanned automobile | |
| CN112298106A (en) | Windscreen wiper control method, device and system | |
| JP6058307B2 (en) | Wiper control device | |
| CN114312672A (en) | Windshield wiper control method and system and automobile data recorder | |
| CN112115968A (en) | Intelligent sweeper garbage identification method and system | |
| CN113787987A (en) | Self-cleaning method and system for field of view area of front-view camera of windshield | |
| CN111766855A (en) | Method, device and system for controlling vision sensor of vehicle | |
| CN111147761A (en) | Vehicle camera equipment cleaning method and system, vehicle and storage medium | |
| CN114655166A (en) | Cleaning method and device for sensor, electronic equipment and storage medium | |
| JPH10261064A (en) | Sticking material discriminating method for raindrop detector | |
| JP7375411B2 (en) | Camera control device, camera control method, and program | |
| CN219382339U (en) | Projection system and vehicle | |
| CN117002448A (en) | Anti-shielding system, method and device of laser radar, domain controller and vehicle | |
| CN115973096A (en) | Method and system for cleaning vehicle windshield, storage medium and vehicle | |
| CN116533929A (en) | Driving visual field monitoring method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20230718 Address after: Room 437, Floor 4, Building 3, No. 969, Wenyi West Road, Wuchang Subdistrict, Yuhang District, Hangzhou City, Zhejiang Province Applicant after: Wuzhou Online E-Commerce (Beijing) Co.,Ltd. Address before: Box 847, four, Grand Cayman capital, Cayman Islands, UK Applicant before: ALIBABA GROUP HOLDING Ltd. |
|
| TA01 | Transfer of patent application right |