WO2021244591A1

WO2021244591A1 - Driving auxiliary device and method, and vehicle and storage medium

Info

Publication number: WO2021244591A1
Application number: PCT/CN2021/098021
Authority: WO
Inventors: 黎建平; 张宣彪; 李激光; 王俊越; 刘卫龙; 孙牵宇; 许亮
Original assignee: 上海商汤临港智能科技有限公司
Priority date: 2020-06-03
Filing date: 2021-06-02
Publication date: 2021-12-09
Also published as: KR20220054659A; JP2022551243A

Abstract

Disclosed are a driving auxiliary device and method, and a vehicle and a storage medium. The device comprises a camera, wherein when the driving auxiliary device is installed on a vehicle, a lens of the camera is arranged facing the forward direction of the vehicle and is used for collecting an image of an environment in which the vehicle is located; a processor, which is used for respectively performing first resolution conversion processing and second resolution conversion processing on the image, so as to obtain a first-resolution image and a second-resolution image, wherein a first resolution is different from a second resolution; a digital video recorder, which is used for storing the first-resolution image; and an auxiliary driving apparatus, which is used for generating intelligent driving control information of the vehicle according to the second-resolution image.

Description

Driving auxiliary equipment, method, vehicle and storage medium

Cross-references to related applications

This application claims the priority of Chinese patent applications with application numbers 202021001117.9 and 202010495894.1 filed on June 3, 2020, the entire contents of which are incorporated herein by reference.

Technical field

The present disclosure relates to the field of computer technology, and in particular to a driving assistance device, a method, a vehicle, and a storage medium.

Background technique

With the development of automotive electronics, more and more vehicles are equipped with ADAS (Advanced Driving Assistance System) and DVR (Digital Video Recorder). At present, most of the ADAS and DVR in vehicles are two. Independent modules.

Summary of the invention

The purpose of one or more embodiments of the present disclosure is to provide a driving assistance device, method, vehicle and storage medium.

According to an aspect of the present disclosure, there is provided a driving assistance device, the device comprising: a camera, a processor, a driving record module, and a driving assistance module, wherein the camera, the driving record module, and the driving assistance module are respectively Connected with the processor; when the driving assistance equipment is installed on the vehicle, the lens of the camera is set forward to the vehicle, and is used to collect images of the environment where the vehicle is located, wherein the images include images And/or video; the processor is configured to perform a first resolution conversion process and a second resolution conversion process on the image to obtain a first resolution image and a second resolution image, the first resolution Rate is different from the second resolution; the driving recorder is used to store the first resolution image; the auxiliary driver is used to generate the intelligent driving of the vehicle according to the second resolution image Control information.

With reference to any one of the embodiments provided in the present disclosure, the driving assistance device further includes a display device connected to the processor; the processor is also configured to perform a third resolution conversion process on the image to obtain a third resolution And output the third resolution image for display on the display device, the third resolution being different from at least one of the first resolution and the second resolution.

With reference to any one of the embodiments provided in the present disclosure, the device further includes a gravity sensor connected to the processor; the processor is further configured to obtain detection information of the gravity sensor when the vehicle is in a parking mode; respond When the detection information satisfies a first set condition, the camera is triggered to collect an image of the environment.

With reference to any one of the embodiments provided in the present disclosure, the processor is further configured to store the image and/or send the image to a set target address.

With reference to any of the embodiments provided in the present disclosure, the device further includes a gravity sensor connected to the processor; the processor is further configured to obtain detection information of the gravity sensor when the vehicle is in a driving mode; The detection information satisfies a second set condition, and triggers the camera to collect an image of the environment.

In conjunction with any one of the embodiments provided in the present disclosure, the processor is further configured to: store the image; send the image to a set target address; and/or retrieve the image of the corresponding time period of the detection information and set The destination address sends the image.

With reference to any one of the embodiments provided in the present disclosure, the device further includes a yaw sensor connected to the processor; the processor is further configured to obtain detection information of the yaw sensor when the vehicle is in a driving mode; In response to the detection information meeting the third set condition, trigger the camera to collect an image of the environment.

With reference to any one of the embodiments provided in the present disclosure, the processor is also used to obtain vehicle operation information; the auxiliary driver is specifically used to: perform the following according to the second resolution image and the vehicle operation information At least one item: outputting the driving warning information of the vehicle, outputting the control information of the on-board equipment installed on the vehicle, outputting the control information for switching the driving mode of the vehicle, and outputting the automatic driving control instruction of the vehicle.

With reference to any of the embodiments provided in the present disclosure, the processor is further configured to respond to receiving image collection sent by at least one of the vehicle's central control screen, the vehicle's triggering device, and the external device communicatively connected with the vehicle A trigger signal that triggers the camera to perform at least one of the following: collect an image of the environment, store the image and/or send the image to a set target address; collect and store a video of the environment; retrieve the image The image captures the image and/or the video of the corresponding time period of the trigger signal and sends the image and/or the video to the set target address.

According to one aspect of the present disclosure, a driving assistance method is provided. The method includes: collecting images of the environment where the vehicle is located through a camera set on the vehicle, wherein the images include images and/or videos; The first resolution conversion processing and the second resolution conversion processing obtain a first resolution image and a second resolution image, where the first resolution is different from the second resolution; storing the first resolution image ; Generate intelligent driving control information of the vehicle according to the second resolution image.

With reference to any of the embodiments provided in the present disclosure, the method further includes: performing a third resolution conversion process on the image to obtain a third resolution image, and outputting the third resolution image for display on a display device , The third resolution is different from at least one of the first resolution and the second resolution.

With reference to any one of the embodiments provided in the present disclosure, the method further includes: when the vehicle is in a parking mode, acquiring detection information of a gravity sensor provided in the vehicle; responding to the detection information meeting the first set condition , Trigger the camera to collect an image of the environment, store the image and/or send the image to a set target address.

With reference to any one of the embodiments provided in the present disclosure, the method further includes: when the vehicle is in a driving mode, acquiring detection information of a gravity sensor provided in the vehicle; in response to the detection information meeting a second set condition, Trigger the camera to collect an image of the environment, store the image and/or send the image to a set target address; and/or, in response to the detection information meeting a second set condition, retrieve the detection Information corresponding time period of the image and send the image to the set target address.

With reference to any one of the embodiments provided in the present disclosure, the method further includes: when the vehicle is in a driving mode, acquiring detection information of a yaw sensor provided in the vehicle; and responding to the detection information that satisfies the third setting condition , Triggering the camera to collect an image of the environment, storing the image and/or sending the image to a set target address; and/or, in response to the detection information meeting a third set condition, retrieving the The image in the corresponding time period of the information is detected and the image is sent to the set target address.

With reference to any one of the embodiments provided in the present disclosure, the method further includes: acquiring vehicle operation information; performing intelligent driving control on the vehicle according to the second resolution image, including: according to the second resolution image and the For the vehicle operation information, at least one of the following is executed: outputting the driving warning information of the vehicle, outputting the control information of the on-board equipment installed on the vehicle, outputting the control information for switching the driving mode of the vehicle, outputting the automatic control of the vehicle Driving control instructions.

With reference to any one of the embodiments provided in the present disclosure, the method further includes: responding to receiving an image capture trigger sent by at least one of the central control screen of the vehicle, the trigger device of the vehicle, and the external device communicatively connected with the vehicle Signal to trigger the camera to perform at least one of the following: collect an image of the environment, store the image and/or send the image to a set target address; collect and store a video of the environment; retrieve the image Collect the image and/or video of the corresponding time period of the trigger signal and send the image and/or the video to the set target address.

According to an aspect of the present disclosure, there is provided a vehicle including the driving assistance device according to any one of the embodiments of the present disclosure.

According to an aspect of the present disclosure, there is provided a computer-readable storage medium on which a computer program is stored, and the program is executed by a processor to implement the driving assistance method according to any embodiment of the present disclosure.

According to the driving assistance device, method, vehicle and storage medium of one or more embodiments of the present disclosure, when the driving assistance device is installed on the vehicle, a camera is used to collect images of the environment where the vehicle is located, and the images are distinguished differently Rate conversion. On the one hand, the converted first resolution image is stored, and on the other hand, the intelligent driving control information of the vehicle is generated according to the converted second resolution. The functions of driving record and intelligent driving are realized through the same camera. The hardware utilization rate is increased, the hardware cost is reduced, and the space occupied by the electronic module in the vehicle is saved.

Description of the drawings

In order to more clearly explain one or more embodiments of this specification or technical solutions in the prior art, the following will briefly introduce the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, in the following description The drawings are only some of the embodiments described in one or more embodiments of this specification. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative labor. .

Fig. 1 is a structural diagram of a driving assistance device provided by at least one embodiment of the present disclosure;

2 is a schematic diagram of image processing of a driving assistance device provided by at least one embodiment of the present disclosure;

FIG. 3 is a method flowchart of a driving assistance method provided by at least one embodiment of the present disclosure.

detailed description

In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the following will combine the drawings in one or more embodiments of this specification to compare The technical solution is described clearly and completely. Obviously, the described embodiments are only a part of the embodiments in this specification, rather than all the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by a person of ordinary skill in the art without creative work shall fall within the protection scope of the present disclosure.

At least one embodiment of the present disclosure provides a driving assistance device, which can be installed on a vehicle, and the vehicle can be any type of vehicle, and can be any vehicle for carrying people or cargo or other purposes. As shown in FIG. 1, the driving assistance device may include a camera 101, a processor 102, a driving recorder 103 and an auxiliary driver 104. The camera 101, the driving recorder 103, and the auxiliary driver 104 are respectively connected to the processor 102.

When the driving assistance equipment is installed on the vehicle, the lens of the camera 101 is set toward the front of the vehicle, such as toward the front of the vehicle, and is used to collect images of the environment where the vehicle is located, including shooting images of the environment, recording videos of the environment, etc. Wait. The camera 101 can also collect sounds in the environment while collecting images of the environment.

The camera 101 may be a camera integrated on the main body of the driving assistance device, that is, integrated with other components in a housing. In the case of an integrated setting, the driving assistance device can be set on the rear view of the vehicle and the camera is facing the forward direction of the vehicle, so that the camera collects environmental images from a better perspective. The camera 101 may also be a separate camera, that is, set outside the main housing of the driving assistance device, so as to facilitate the deployment of the driving assistance device on the vehicle. For example, in the case of separate installation, the main body of the driving assistance equipment can be installed in the dashboard, and the camera is installed inside the rearview mirror, which can collect environmental images with a better perspective while occupying a small space. And beautiful. In the case of a separate arrangement, the camera 101 can be connected to the main housing in a wired or wireless manner, and the present disclosure does not limit the specific connection manner.

The processor 102 is connected to the camera 101 to obtain images collected by the camera, and perform first resolution conversion processing and second resolution conversion processing on the images, respectively, to obtain a first resolution image and a second resolution image , Wherein the first resolution is different from the second resolution.

In the embodiment of the present disclosure, the processor may be a system-level chip with built-in RAM (Random Access Memory), ROM (Read-Only Memory), and capable of running an operating system, such as SOC (System On Chip), or other processors with similar processing capabilities, such as FPGA (Field Programmable Gate Array), ASIC (Application Specific Integrated Circuit, application specific integrated circuit), DSP (Digital Signal Processing, digital signal processing) chip, CPU (Central Processing Unit, central processing unit), etc.

The images collected by the camera may include videos and/or images, and the processor may split the images according to different requirements to perform conversion processing of different resolutions. The splitting refers to performing different conversion processing on multiple copies of the image. For example, videos or images stored for driving records usually require a higher resolution. The video or image used for intelligent driving does not have particularly high requirements for resolution. Therefore, it is possible to split the original video stream collected by the camera, and perform the first resolution conversion processing on a part of the split video stream to obtain the first resolution image for driving recording; split the other part of the video obtained by the split The stream performs a second resolution conversion process to obtain a second resolution image lower than the first resolution for use in intelligent driving.

The driving recorder 103 is connected to the processor 102, and is used to obtain a first resolution image and store the first resolution image for driving recording.

The auxiliary driver 104 is connected to the processor 102, and is configured to obtain a second resolution image, and generate intelligent driving control information of the vehicle according to the second resolution image.

Wherein, the intelligent driving control information may include the driving warning information of the vehicle, such as warning information prompting the driver to be too close to the vehicle ahead, driving information prompting the driver to exceed a safe speed, etc., to assist the driver in safety Driving; it can also include the control information of the on-board equipment set on the vehicle, such as the information to control the turning on or off of the high beam of the vehicle, etc., to achieve the function of assisting driving; it can also include the automatic driving control information of the vehicle, such as controlling the vehicle Speed, driving direction, etc., in order to realize the automatic driving of the vehicle. Those skilled in the art should understand that other types of intelligent driving control information may also be included, which are not limited to the above.

In the embodiment of the present disclosure, when the driving assistance equipment is installed on the vehicle, the camera is used to collect images of the environment where the vehicle is located, and the images are converted to different resolutions. On the one hand, the converted first resolution is Images are stored. On the other hand, the intelligent driving control information of the vehicle is generated according to the converted second resolution. The images collected by the same camera realize the functions of driving record and intelligent driving, which improves the hardware utilization rate and reduces The hardware cost saves the space occupied by the electronic module in the vehicle.

In addition, by converting the resolution of the images collected by the camera, the resolution images suitable for storage and image detection are respectively generated, which reduces the hardware cost while ensuring the quality of driving records and assisted driving.

In some embodiments, the driving assistance device further includes a display device, and the processor is further configured to perform a third resolution conversion process on the image to obtain a third resolution image, and output the third resolution image The image is displayed on the display device, the third resolution is different from at least one of the first resolution and the second resolution, and the third resolution may be based on the display resolution of the display device Sure.

Figure 2 shows a schematic diagram of processing the original video stream collected by the camera. As shown in FIG. 2, on the one hand, the original video stream can be converted into the first resolution image by the processor, and the first resolution image can be stored by the driving recorder, so as to realize the driving recording function. The first resolution is, for example, 920*1080 or 1280*720, and the driving recorder may be an external storage module of the processor, such as an SD card (Secure Digital Memory Card), a TF card (T -Flash, flash memory card) and so on, in order to realize the function of driving record.

On the other hand, the original video stream can be converted into a second resolution image by the processor, and the intelligent driving control information of the vehicle can be generated according to the second resolution image by the auxiliary driver.

Taking the implementation of assisted driving as an example, the original video may be resolution converted according to the second resolution required by the assisted driving related algorithm. For example, for the video used in realizing forward collision warning, the second resolution is usually 1024*576, then the original video stream can be converted to the second resolution at 1024*576 to obtain the second resolution image and Input to the auxiliary driver to generate intelligent driving control information.

In addition, the processor may also perform a third resolution conversion process on the image to obtain a third resolution image, and output the third resolution image for display in the display module.

Take the display device as the vehicle central control screen or mobile phone APP (application) as an example. In this case, the required sub-resolution is usually low, such as 720*480, and the third resolution can be set to 720 *480, and convert the images collected by the camera. For the obtained third conversion rate image, it can be transmitted to the vehicle's central control system through a wireless network, such as WIFI or Bluetooth, or to the cloud or mobile phone, so as to realize the video information or image on the central control screen or mobile phone APP. Information can be browsed, deleted, moved, played back, and downloaded, and the video stream collected by the camera can also be displayed in real time.

In one example, the display device and the processor can also be connected in a wired manner, such as HDMI (High Definition Multimedia Interface, high-definition multimedia interface)/LVDS (Low-Voltage Differential Signaling, low-voltage differential signaling) to output the third resolution image , To display in an external display device, in this case, you can set the third resolution to 1280*720.

The above settings of the first resolution, the second resolution, and the third resolution are only examples. Those skilled in the art should understand that the first resolution, the second resolution, the second resolution, and the The value of the third resolution is not limited in this disclosure.

In the embodiment of the present disclosure, the resolution conversion of the image collected by the camera is performed according to the resolution requirement of the display module, which improves the processing efficiency while ensuring the display effect.

The driving assistance device may also include a gravity sensor (Gravity Sensor) for detecting acceleration information of the vehicle. The gravity sensor may be integrated on the main body of the driving auxiliary equipment, that is, integrated with other components in a housing; or it may be separately arranged from the driving auxiliary equipment, that is, set in the main housing of the driving auxiliary equipment In addition, to facilitate the deployment of the driving assistance equipment on the vehicle.

In some embodiments, when the vehicle is in the parking mode, the detection information of the gravity sensor is acquired; in response to the detection information meeting the first set condition, the camera is triggered to collect images and/or videos of the environment , To store the image and/or video and/or send the image and/or video to a set target address.

The parking mode of the vehicle includes, but is not limited to, at least one of the following situations: a stalled parking state, and a state where the vehicle's gear is in a parking gear or a neutral gear. When the vehicle is in the parking mode, the first set condition includes: the acceleration of the vehicle detected by the gravity sensor changes greatly in a short time, for example, the rate of change of the acceleration is greater than the first set threshold. In this case, there is a high probability that the corresponding vehicle will have an abnormal situation. For example, the body may be collided, or the vehicle may be in a state of being towed. At this time, the camera will be triggered to take an image of the environment and/or record a video of the environment, thereby It can automatically and timely record the abnormal state of the vehicle. The processor acquires the image and/or the video and stores it for use in subsequent processing, such as analyzing or tracing the source of abnormal conditions; or sending it to a set target address, for example, through a smart vehicle terminal (T-BOX) is sent to the owner's mobile phone to promptly remind the owner of the abnormal state of the vehicle; the image and/or video can also be sent to a third party/cloud so that the abnormal situation of the vehicle can be dealt with in time.

When the vehicle is in the driving mode, in response to the detection information meeting the second set condition, trigger the camera to take an image of the environment, store the image and/or send the image to a set target address; and /Or, in response to that the detection information satisfies the second set condition, the video of the corresponding time period of the detection information is retrieved and the video is sent to the set target address.

During the running of the vehicle, the second set condition includes that the acceleration detected by the gravity sensor changes greatly in a short time, for example, the rate of change of the acceleration is greater than the second set threshold. In this case, the probability of an abnormal situation in the corresponding vehicle is high. For example, an abnormal change in the speed of the vehicle may occur. Record the abnormal driving state of the vehicle in time. In addition, the processor obtains the image and/or the video and stores it for use in subsequent processing, such as analyzing or tracing the source of an abnormal state; or sending it to a set target address, for example, through smart The vehicle-mounted terminal (T-BOX) is sent to the owner's mobile phone to promptly remind the owner of the abnormal state of the vehicle; the image and/or video may also be sent to a third party/cloud so that the abnormal situation of the vehicle can be dealt with in time.

The driving assistance device may also include a yaw sensor for detecting yaw information of the vehicle. The yaw sensor may be integrated on the main body of the driving auxiliary equipment, that is, integrated with other components in a housing; it may also be arranged separately from the driving auxiliary equipment, that is, set in the main housing of the driving auxiliary equipment Outside the body to facilitate the deployment of the driving assistance equipment on the vehicle. For example, in the case of separate installation, the yaw sensor may be installed under the shift lever, and the main body of the driving assistance device may be installed in the dashboard.

When the vehicle is in the driving mode, in response to the detection information meeting the third set condition, trigger the camera to take an image of the environment, store the image and/or send the image to a set target address; and /Or, in response to that the detection information satisfies the third set condition, the video of the time period corresponding to the detection information is retrieved and the video is sent to the set target address.

During the driving of the vehicle, the third setting condition includes: the yaw information detected by the yaw sensor indicates that the driving direction of the vehicle has a large deviation in a short time, for example, the change of the yaw information is greater than the first 3. Set the threshold. In this case, the probability of the abnormal situation of the corresponding vehicle is high. If the driving direction of the vehicle is abnormal, the camera will be triggered to take the image of the environment and/or record the video of the environment, so that the abnormality of the vehicle can be automatically and timely recorded Driving state. In addition, the processor obtains the image and/or the video and stores it for use in subsequent processing, such as analyzing or tracing the source of an abnormal state; or sending it to a set target address, for example, through smart The vehicle-mounted terminal (T-BOX) is sent to the owner's mobile phone to promptly remind the owner of the abnormal state of the vehicle; the image and/or video may also be sent to a third party/cloud so that the abnormal situation of the vehicle can be dealt with in time.

Exemplarily, the first set threshold, the second set threshold, and the third set threshold may be numerical values summarized according to various abnormal situations. The first set threshold, the second set threshold, and the third set threshold may be the same, may be partially the same, or may be completely different.

Since the space occupied by the images and/or videos taken under the above-mentioned setting conditions may be relatively small, storing such images and/or videos can be either by storing the images and/or videos in the processor, or by The processor instructs the driving recorder to store the image and/or video, which is not limited in this application.

In some embodiments, the processor also triggers the operation of the camera when an image capture trigger signal is received. The image acquisition trigger signal may be sent through at least one of the vehicle's central control screen, the vehicle's trigger device, and an external device connected in communication with the vehicle, wherein the vehicle's trigger device may be appropriately installed inside the vehicle. Location buttons, and external devices connected to the vehicle in communication include mobile phones and so on. The operation of the triggered camera includes: collecting an image of the environment, storing the image and/or sending the image to a set target address; recording a video of the environment, storing the video and/or sending the The video is sent to the set target address; the image and/or video in the corresponding period of the image acquisition trigger signal is retrieved and the image and/or video is sent to the set target address. For example, the image and video can be sent through a smart vehicle-mounted terminal To the owner’s mobile phone, it can also be sent to a third party/cloud, etc.

In the embodiments of the present disclosure, by sending out an image capture trigger signal to trigger the camera to perform shooting, storage, and sending operations, it is possible to autonomously determine the images that need to be recorded and saved in an active trigger manner, and to actively send the images, which satisfies The diversified needs of users of the driving assistance equipment. Compared with the driving recorder in the related art that is always in the recording and storage state once it is turned on, the driving assistance device proposed in the present disclosure realizes more effective storage of the images collected by the camera, and improves the utilization rate of the storage space.

In some embodiments, the processor is further configured to obtain vehicle operation information, and the auxiliary driver outputs driving warning information of the vehicle according to the second resolution image and the vehicle operation information.

In one example, the front lane line and the vehicle motion trajectory can be monitored in real time according to the second resolution image. When the deviation between the lane line and the vehicle motion trajectory is greater than a certain degree, the auxiliary driver determines that the vehicle If it deviates from the lane, a warning message is output to remind the driver to pay attention.

In one example, the auxiliary driver can detect the front vehicle and the distance between the front vehicle and its own vehicle in real time based on the second resolution image, and calculate the distance between the own vehicle and the front vehicle based on the vehicle speed information in the vehicle operation information. When the potential collision time of the vehicle is less than a certain set threshold, it is judged that there is a risk of collision between the own vehicle and the vehicle in front, and an alarm message is output to remind the driver to pay attention.

In one example, the auxiliary driver can detect the pedestrian in front and the distance between the pedestrian in front and the own vehicle in real time based on the second resolution image, and calculate the distance between the own vehicle and the front pedestrian based on the speed information in the vehicle operation information. The potential collision time of pedestrians, when the potential collision time is less than a certain set threshold, it is judged that there is a risk of collision between the own vehicle and the pedestrian in front, and an alarm message is output to remind the driver to pay attention.

In one example, when the vehicle is in the parking mode, the auxiliary driver can detect the change in the distance between the preceding vehicle and the own vehicle in real time according to the second resolution image, and when the distance change is greater than a certain set threshold, If it is determined that the vehicle ahead has started, a warning message is output to remind the driver to pay attention.

In one example, the auxiliary driver may detect traffic signs, such as speed limit and prohibition signs, according to the second resolution image, and remind the driver to drive according to the detected traffic signs.

In an example, the auxiliary driver can detect the traffic signal light based on the second resolution image, and remind the driver to follow the detected traffic signal light and drive safely.

In some embodiments, the auxiliary driver outputs control information of in-vehicle equipment installed on the vehicle based on the second resolution image and the vehicle operating information.

The vehicle operation information includes state information in the vehicle driving mode and the vehicle parking mode, such as the vehicle's turn signal state, seat belt state, vehicle speed, and other information. Those skilled in the art should understand that the vehicle operation information may also include other information, which is not limited in the present disclosure.

In one example, when the vehicle light switch, such as the high beam switch, is in the AUTO mode, the second resolution image is judged with parameters, and the vehicle lights are automatically turned on or off according to the judgment result to realize The driving environment automatically controls the function of the lights.

In some embodiments, the auxiliary driver outputs an automatic driving control command of the vehicle according to the second resolution image and the vehicle operation information, for example, outputs a command to control the speed of the vehicle, so as to realize the automatic driving of the vehicle It can also output the control command for switching the driving mode of the vehicle, for example, in the automatic driving mode, in the case of confirming that the vehicle is driving abnormally through the second resolution image and the vehicle information, it can output the switching of the driving mode of the vehicle Control instructions to make the vehicle from the automatic driving mode to the manual driving mode.

In the disclosed embodiment, the driver assistance device can detect the safety conditions of the vehicle driving, predict the conditions of the safe driving of the vehicle, etc., based on the second resolution image converted from the image collected by the camera and the vehicle operation information. So as to realize the intelligent driving function of the vehicle.

In some embodiments, the vehicle system may further include at least one coprocessor connected to the processor. In the case that the processor is an SOC, the coprocessor is, for example, an MCU (Micro Controller Unit, single-chip microcomputer), or other processors with corresponding processing capabilities.

The MCU can be connected to at least one vehicle-mounted device in the vehicle, for example, through CAN (Controller Area Network)/CANFD (CAN with Flexible Data-Rate, control area network with flexible data transmission rate)/Ethernet (Ethernet) Communicate with the at least one vehicle-mounted device to obtain the information output by the device, and the SOC obtains the information output by the vehicle-mounted device via the MCU.

In an example, the control information of the in-vehicle equipment installed on the vehicle may be sent to the MCU by the SOC through SPI (Serial Peripheral Interface) communication, so that the MCU can control the in-vehicle equipment according to the control information.

In some embodiments, the same power supply may be used to supply power to the processor and the coprocessor, and the coprocessor may be used to perform power management on the power supply.

At least one embodiment of the present disclosure also proposes a driving assistance method, which can be implemented according to the driving assistance device described in any embodiment of the present disclosure.

FIG. 3 shows a flowchart of the driving assistance method proposed by at least one embodiment of the present disclosure, which may include steps 301 to 304. The method can be used on driving assistance equipment, which may include a camera, a processor, a driving recorder, and an auxiliary driver.

In step 301, an image of the environment where the vehicle is located is collected through a camera set on the vehicle.

In step 302, a first resolution conversion process and a second resolution conversion process are respectively performed on the image to obtain a first resolution image and a second resolution image, and the first resolution and the second resolution are The rates are different.

In step 303, the first resolution image is stored.

In step 304, intelligent driving control information of the vehicle is generated according to the second resolution image.

In the embodiments of the present disclosure, a camera is used to collect images of the environment where the vehicle is located, and the images are converted to different resolutions. The intelligent driving control information of the vehicle is generated with two resolutions, and the functions of driving record and intelligent driving are realized through the same camera, which improves the hardware utilization rate, reduces the hardware cost, and saves the space occupied by the electronic module in the vehicle.

In some embodiments, the method further includes: performing a third resolution conversion process on the image to obtain a third resolution image, and outputting the third resolution image for display on a display device, the first The three resolutions are different from at least one of the first resolution and the second resolution.

In some embodiments, the method further includes: when the vehicle is in a parking mode, acquiring detection information of a gravity sensor provided in the vehicle; and in response to the detection information meeting a first set condition, triggering the camera to collect The image of the environment and/or the video of the environment; storing the image and/or video and/or sending the image and/or video to a set target address.

In some embodiments, the method further includes: when the vehicle is in a driving mode, acquiring detection information of the gravity sensor; and in response to the detection information meeting a second set condition, triggering the camera to collect information about the environment Image, storing the image and/or sending the image to a set target address; and/or, in response to the detection information satisfying the second setting condition, retrieve the video of the corresponding time period of the detection information and send it to the setting The destination address sends the video.

In some embodiments, the method further includes: when the vehicle is in a driving mode, acquiring detection information of a yaw sensor provided in the vehicle; and in response to the detection information meeting a third set condition, triggering the camera to collect The image of the environment, storing the image and/or sending the image to a set target address; and/or, in response to the detection information meeting a third set condition, retrieving the video of the corresponding time period of the detection information And send the video to the set target address.

In some embodiments, the method further includes: acquiring vehicle operating information; performing intelligent driving control on the vehicle according to the second resolution image, including: according to the second resolution image and the vehicle operating information Execute at least one of the following: output the driving warning information of the vehicle, output the control information of the in-vehicle equipment provided on the vehicle, output the control information for switching the driving mode of the vehicle, and output the automatic driving control instruction of the vehicle.

In some embodiments, the method further includes: in response to receiving an image acquisition trigger signal sent by at least one of the vehicle's central control screen, the vehicle's triggering device, and the external device communicatively connected with the vehicle, triggering the The camera performs at least one of the following: collects an image of the environment, stores the image and/or sends the image to a set target address; collects and stores a video of the environment; retrieves the corresponding image acquisition trigger signal And send the video to the set target address.

At least one embodiment of the present disclosure also proposes a vehicle including the driving assistance device according to any embodiment of the present disclosure.

In some embodiments, a rear-view mirror is further provided in the cabin of the vehicle, and the driving assistance device may be provided on the rear-view mirror with the camera facing forward of the vehicle.

By arranging the driving assistance equipment on the rearview mirror of the vehicle and the camera facing the front of the vehicle, the camera can collect images with a better angle of view and save space in the vehicle.

In some embodiments, the vehicle is provided with a central control screen, and the central control screen is communicatively connected with the camera or the processor, and is used to trigger the camera to perform image collection.

In some embodiments, a trigger device communicatively connected with the camera or the processor is provided in the cabin of the vehicle for triggering the camera to perform image collection.

In some embodiments, a terminal device communicatively connected with the camera or the processor is provided on the vehicle to trigger the camera to perform image collection.

Through the vehicle's central control screen to trigger the camera for image collection, the vehicle driver or passenger can autonomously determine the image to be recorded in an active trigger mode, which improves the flexibility of the use of vehicle equipment.

In some embodiments, the vehicle is provided with a gravity sensor connected to the processor, and the gravity sensor outputs a gravity sensing detection signal to the processor, so that the processor is The detection signal controls the camera to perform image collection.

By controlling the camera to collect images according to the gravity sensor detection signal, the abnormal driving state of the vehicle can be automatically and timely recorded.

In some embodiments, the vehicle is provided with a yaw sensor connected to the processor, and the yaw sensor outputs a yaw sensing detection signal to the processor so that the processor can The yaw sensing detection signal controls the camera to collect images.

By controlling the camera to collect images according to the yaw sensor detection signal, the abnormal driving state of the vehicle can be automatically and timely recorded.

Those skilled in the art should understand that one or more embodiments of this specification can be provided as a method, a system, or a computer program product. Therefore, one or more embodiments of this specification may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, one or more embodiments of this specification may adopt a computer program implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The form of the product.

The embodiment of this specification also provides a computer-readable storage medium, and the storage medium can store a computer program, which when executed by a processor realizes the steps of the driving assistance method described in any embodiment of this specification. Wherein, the "and/or" means at least one of the two, for example, "A and/or B" includes three schemes: A, B, and "A and B".

Refer to each other for the same or similar parts in the various embodiments in this specification, and each embodiment focuses on the differences from other embodiments. In particular, for the method embodiment, since each step is basically similar to the function implemented by each module of the driving assistance device, the description is relatively simple, and the relevant parts can be referred to the part of the description of the device embodiment.

The foregoing describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than in the embodiments and still achieve desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or sequential order shown in order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

The embodiments of the subject and functional operations described in this specification can be implemented in the following: digital electronic circuits, tangible computer software or firmware, computer hardware including the structures disclosed in this specification and structural equivalents thereof, or among them A combination of one or more. The embodiments of the subject matter described in this specification can be implemented as one or more computer programs, that is, one or one of the computer program instructions encoded on a tangible non-transitory program carrier to be executed by a data processing device or to control the operation of the data processing device Multiple modules. Alternatively or additionally, the program instructions may be encoded on artificially generated propagated signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated to encode information and transmit it to a suitable receiver device for data transmission. The processing device executes. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them.

The processing and logic flow described in this specification can be executed by one or more programmable computers executing one or more computer programs to perform corresponding functions by operating according to input data and generating output. The processing and logic flow can also be executed by a dedicated logic circuit, such as FPGA (Field Programmable Gate Array) or ASIC (Application Specific Integrated Circuit), and the device can also be implemented as a dedicated logic circuit.

Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices (such as EPROM, EEPROM, and flash memory devices), magnetic disks (such as internal hard disks or flash memory devices). Removable disks), magneto-optical disks, CD ROM and DVD-ROM disks. The processor and the memory can be supplemented by or incorporated into a dedicated logic circuit.

Although this specification contains many specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claimed protection, but are mainly used to describe the features of specific embodiments of a particular invention. Certain features described in multiple embodiments in this specification can also be implemented in combination in a single embodiment. On the other hand, various features described in a single embodiment can also be implemented in multiple embodiments separately or in any suitable sub-combination. In addition, although features may function in certain combinations as described above and even initially claimed as such, one or more features from the claimed combination may in some cases be removed from the combination, and the claimed The combination of protection can be directed to a sub-combination or a variant of the sub-combination.

Similarly, although operations are depicted in a specific order in the drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of various system modules and components in the above embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can usually be integrated together in a single software product. In, or packaged into multiple software products.

Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve desired results. In addition, the processes depicted in the drawings are not necessarily in the specific order or sequential order shown in order to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

The foregoing descriptions are only preferred embodiments of one or more embodiments of this specification, and are not intended to limit one or more embodiments of this specification. All within the spirit and principle of one or more embodiments of this specification, Any modification, equivalent replacement, improvement, etc. made should be included in the protection scope of one or more embodiments of this specification.

Claims

A driving auxiliary equipment, characterized in that the equipment includes:

A camera, a processor, a driving record module and a driving assistance module, wherein the camera, the driving record module and the driving assistance module are respectively connected to the processor;

When the driving assistance equipment is installed on a vehicle, the lens of the camera is set forward to the vehicle, and is used to collect images of the environment where the vehicle is located, where the images include images and/or videos;

The processor is configured to perform a first resolution conversion process and a second resolution conversion process on the image respectively to obtain a first resolution image and a second resolution image, the first resolution and the second resolution image Two different resolutions;

The driving recorder is used to store the first resolution image;

The driver assistance device is configured to generate intelligent driving control information of the vehicle according to the second resolution image.
The device according to claim 1, wherein the driving assistance device further comprises a display device connected to the processor;

The processor is further configured to perform third resolution conversion processing on the image to obtain a third resolution image, and output the third resolution image for display on the display device, the third resolution It is different from at least one of the first resolution and the second resolution.
The device according to claim 1 or 2, wherein the device further comprises a gravity sensor connected to the processor;

The processor is further configured to obtain detection information of the gravity sensor when the vehicle is in a parking mode; in response to the detection information meeting a first set condition, trigger the camera to collect an image of the environment.
The device according to claim 3, wherein the processor is further configured to store the image and/or send the image to a set target address.
The device according to claim 1 or 2, wherein the device further comprises a gravity sensor connected to the processor;

The processor is further configured to obtain detection information of the gravity sensor when the vehicle is in a driving mode; in response to the detection information meeting a second set condition, trigger the camera to collect an image of the environment.
The device according to claim 5, the processor is further configured to:

Store the image;

Send the image to the set target address; and/or

Retrieve the image of the corresponding time period of the detection information and send the image to the set target address.
The device according to any one of claims 1 to 6, wherein the device further comprises a yaw sensor connected to the processor;

The processor is further configured to obtain detection information of the yaw sensor when the vehicle is in a driving mode; in response to the detection information meeting a third set condition, trigger the camera to collect an image of the environment.
According to the device of claim 7, the processor is further configured to:

Store the image;

Send the image to the set target address; and/or

Retrieve the image of the corresponding time period of the detection information and send the image to the set target address.
The device according to any one of claims 1 to 8, characterized in that:

The processor is also used to obtain vehicle operation information;

The driver assistance device is specifically configured to: according to the second resolution image and the vehicle operation information, perform at least one of the following: output driving warning information of the vehicle, output control of on-board equipment installed on the vehicle Information, output of control information for switching the driving mode of the vehicle, and output of an automatic driving control command of the vehicle.
The device according to any one of claims 1 to 9, wherein the processor is further configured to respond to receiving the vehicle’s central control screen, the vehicle’s triggering device, and the external equipment communicatively connected to the vehicle. The image acquisition trigger signal sent by at least one item in triggers the camera to perform at least one of the following:

Acquiring an image of the environment, storing the image and/or sending the image to a set target address;

Collect and store the video of the environment;

Retrieve the image and/or video of the corresponding time period of the image acquisition trigger signal and send the image and/or the video to a set target address.
A driving assistance method, characterized in that the method includes:

Collect an image of the environment where the vehicle is located through a camera set on the vehicle, where the image includes an image and/or video;

Performing a first resolution conversion process and a second resolution conversion process on the image respectively to obtain a first resolution image and a second resolution image, where the first resolution is different from the second resolution;

Storing the first resolution image;

The intelligent driving control information of the vehicle is generated according to the second resolution image.
The method according to claim 11, wherein the method further comprises: performing a third resolution conversion process on the image to obtain a third resolution image, and outputting the third resolution image for display The device displays that the third resolution is different from at least one of the first resolution and the second resolution.
The method according to claim 11 or 12, wherein the method further comprises:

When the vehicle is in the parking mode, acquiring detection information of a gravity sensor provided in the vehicle;

In response to the detection information meeting the first set condition, trigger the camera to collect an image of the environment, store the image and/or send the image to a set target address.
The method according to any one of claims 11 to 13, wherein the method further comprises:

When the vehicle is in the driving mode, acquiring detection information of a gravity sensor provided in the vehicle;

In response to the detection information meeting the second set condition, trigger the camera to collect an image of the environment, store the image and/or send the image to a set target address; and/or, in response to the The detection information satisfies the second setting condition, and the image of the corresponding time period of the detection information is retrieved and the image is sent to the set target address.
The method according to any one of claims 11 to 14, wherein the method further comprises:

When the vehicle is in the driving mode, acquiring the detection information of the yaw sensor provided in the vehicle;

In response to the detection information meeting the third set condition, trigger the camera to collect an image of the environment, store the image and/or send the image to a set target address; and/or, in response to the The detection information satisfies the third setting condition, the image of the corresponding time period of the detection information is retrieved and the image is sent to the set target address.
The method according to any one of claims 11 to 15, wherein the method further comprises:

Obtain vehicle operation information;

Performing intelligent driving control on the vehicle according to the second resolution image includes:

According to the second resolution image and the vehicle operation information, perform at least one of the following: output driving warning information of the vehicle, output control information of on-board equipment installed on the vehicle, output the vehicle driving mode switch The control information of the vehicle is output, and the automatic driving control instruction of the vehicle is output.
The method according to any one of claims 11 to 16, wherein the method further comprises:

In response to receiving an image capture trigger signal sent through at least one of the vehicle's central control screen, the vehicle's trigger device, and an external device communicatively connected with the vehicle, trigger the camera to perform at least one of the following:

Acquiring an image of the environment, storing the image and/or sending the image to a set target address;

Collect and store the video of the environment;

Retrieve the image and/or video of the corresponding time period of the image acquisition trigger signal and send the image and/or the video to a set target address.
A vehicle, characterized by comprising the driving assistance equipment according to any one of claims 1 to 10.
A computer-readable storage medium with a computer program stored thereon, characterized in that the method according to any one of claims 11 to 17 is implemented when the program is executed by a processor.