WO2022127307A1 - 车辆视觉辅助系统、车载图像显示方法及装置 - Google Patents

车辆视觉辅助系统、车载图像显示方法及装置 Download PDF

Info

Publication number
WO2022127307A1
WO2022127307A1 PCT/CN2021/122882 CN2021122882W WO2022127307A1 WO 2022127307 A1 WO2022127307 A1 WO 2022127307A1 CN 2021122882 W CN2021122882 W CN 2021122882W WO 2022127307 A1 WO2022127307 A1 WO 2022127307A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
vehicle
original image
processing
enhanced
Prior art date
Application number
PCT/CN2021/122882
Other languages
English (en)
French (fr)
Inventor
张思伟
Original Assignee
奇瑞汽车股份有限公司
芜湖雄狮汽车科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 奇瑞汽车股份有限公司, 芜湖雄狮汽车科技有限公司 filed Critical 奇瑞汽车股份有限公司
Publication of WO2022127307A1 publication Critical patent/WO2022127307A1/zh

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W30/00—Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
    • B60W30/08—Active safety systems predicting or avoiding probable or impending collision or attempting to minimise its consequences
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/02—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to ambient conditions
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/02—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to ambient conditions
    • B60W40/06—Road conditions
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
    • B60W2050/146—Display means

Definitions

  • the present disclosure belongs to the field of vehicle driving safety, and relates to a vehicle visual assistance system, a vehicle image display method and a device.
  • visual assistance systems refer to systems that provide drivers with visual information to improve driving safety.
  • Traditional visual aid systems include night vision systems, which improve visual effects through infrared cameras.
  • the present disclosure provides a vehicle visual assistance system, a vehicle image display method and a device, which aim to improve the visual effect of the occupants in the vehicle and improve the driving safety under the condition of poor visual conditions.
  • Some embodiments of the present disclosure provide a vehicle visual assistance system, including a high-definition camera; the vehicle visual assistance system is further provided with an AI image processing module using AI image processing technology, and a window display using transparent OLED display technology;
  • the high-definition camera captures the environment around the vehicle, and transmits the captured original image to the AI image processing module; the AI image processing module sends the processed image signal to the window display.
  • the AI image processing technology can convert the image of the road with insufficient light outside the window captured by the high-definition camera into a clear and well-lit image.
  • the window display is a part of the windshield; in the case of no electricity, the window display is a piece of transparent glass; in the case of the display with electricity, the background image can be covered and displayed .
  • the AI image processing module adopts a software system based on artificial intelligence technology; the original image is intercepted by the image, and then calculated by the AI model, and the calculated image data is output to the window display for display; the AI model Including AI image enhancement model and AI image transformation model.
  • the AI image enhancement model is mainly used to process images with low light and blurred vision, and through the calculation of the model, an image with a normal light environment and a clear vision can be obtained.
  • the AI image transformation model is mainly used to transform the surrounding environment of the road, and the original scene image becomes a changed scene image through the calculation of the model.
  • the window display includes a transparent display screen and a host; the host accepts image signals and controls the display to display; the display is embedded in the windshield; the size of the display is different from the size of the vehicle.
  • the windows are the same size and use a transparent OLED display.
  • the vehicle visual aid system is arranged on all windshields of the vehicle to perform the transformation of the overall environment.
  • Some embodiments of the present disclosure provide a vehicle vision assistance system, including:
  • the camera is used to capture the environment around the vehicle to obtain the original image
  • An image processing module configured to acquire the original image captured by the camera, and when the brightness of the original image is lower than a threshold, process the original image to obtain an enhanced image, the brightness of the enhanced image is greater than the brightness of the enhanced image the brightness of the original image;
  • the window display is used for acquiring the enhanced image processed by the image processing module and displaying the enhanced image.
  • the image processing module is configured to perform ambient illumination processing on the original image, and replace the first ambient illumination of the original image with the second ambient illumination to obtain the enhanced image, the second ambient illumination
  • the light intensity of the light is greater than the light intensity of the first ambient light
  • the image processing module is configured to perform ambient illumination processing on the original image by using an ambient illumination processing model
  • the ambient lighting processing model is obtained by using pictures of different ambient lighting in the same scene as samples for training.
  • the image processing module is further configured to perform environment transformation processing on the enhanced image before outputting the enhanced image to the window display, and replace the first road surrounding environment of the original image with The surrounding environment of the second road.
  • the image processing module is configured to perform environment transformation processing on the enhanced image by using an environment transformation processing model
  • the environment transformation processing model is obtained by training pictures with different road surrounding environments as samples.
  • the window display is part of a windshield of the vehicle
  • the vehicle window display is used to be in a transparent state when it is not powered on; when powered on, the enhanced image is displayed.
  • Some embodiments of the present disclosure provide a vehicle-mounted image display method, including:
  • the enhanced image is displayed through a window display.
  • the processing of the original image to obtain an enhanced image includes:
  • the performing ambient lighting processing on the original image includes:
  • the ambient lighting processing model is obtained by using pictures of different ambient lighting in the same scene as samples for training.
  • the vehicle-mounted image display method further includes:
  • an environment transformation process is performed on the enhanced image, and the first road surrounding environment of the original image is replaced with a second road surrounding environment.
  • performing environment transformation processing on the enhanced image includes:
  • the environment transformation processing model is obtained by training pictures with different road surrounding environments as samples.
  • an in-vehicle image display device including:
  • a processor and a memory the memory storing at least one piece of program code, the program code being loaded and executed by the processor to implement the method of any preceding item.
  • Some embodiments of the present disclosure provide a computer-readable storage medium, characterized in that, the computer-readable storage medium stores at least one piece of program code, and the program code is loaded and executed by a processor to implement the above-mentioned item. method described.
  • FIG. 1 is a schematic structural diagram of a vehicle visual aid system provided by some embodiments of the present disclosure
  • FIG. 2 is a functional schematic diagram of an AI image processing system in the present disclosure
  • Fig. 3 is the driving view transformation process diagram of the AI image enhancement function in the present disclosure
  • FIG. 4 is a process diagram of the driving field of view transformation of the AI image transformation function in the present disclosure
  • FIG. 5 is a flowchart of a vehicle-mounted image display method provided by some embodiments of the present disclosure.
  • FIG. 6 is a schematic structural diagram of a vehicle-mounted image display device provided by some embodiments of the present disclosure.
  • infrared cameras use infrared cameras to take pictures at night to assist driving.
  • the pictures taken by the infrared camera are all grayscale pictures, which are quite different from the actual field of view, and the image quality is not high, and it is not clear if it is far away, which does not greatly improve the driver's field of vision.
  • FIG. 1 is a schematic structural diagram of a vehicle visual aid system provided by some embodiments of the present disclosure.
  • the system includes a camera 11 , an image processing module 12 and a window display 13 , and the camera 11 and the window display 13 are respectively connected to the image processor 12 .
  • the camera 11 is used for photographing the environment around the vehicle to obtain the original image; usually, the camera 11 is installed in front of the vehicle, so what is usually photographed is the environment in front of the vehicle.
  • the image processing module 12 is used to obtain the original image captured by the camera, and when the brightness of the original image is lower than the threshold, the original image is processed to obtain an enhanced image, and the brightness of the enhanced image is greater than that of the original image.
  • the window display 13 is used for acquiring the enhanced image processed by the image processing module 12 and displaying the enhanced image.
  • the present disclosure adopts the above-mentioned technical solution, collects the original image of the driving road and the surrounding environment in real time through the camera, processes the original image through image processing technology, and enhances the brightness of the image; the processed image is displayed through the window display.
  • the image is obtained by changing the brightness of the original image, and the image quality is high, so that the driver can improve the driving field of vision to the greatest extent based on the enhanced image; it can greatly improve the safety of the driver, the vehicle and the people outside the vehicle.
  • the original image is directly displayed through the window display 13 .
  • raw images captured on a clear day usually do not need to be processed and are displayed directly on the window display.
  • the raw images captured at night with dim lighting need to be processed and then displayed on the window display.
  • the brightness of the original image may refer to the average brightness of each pixel in the original image, or the intermediate brightness, the maximum brightness, the minimum brightness, and the like.
  • the threshold value can be set according to actual needs, for example, it can be the minimum brightness value of the image under the condition of ensuring a certain range of vision of the driver.
  • the camera 11 is a high-definition camera, so as to ensure the clarity of the captured image.
  • the image processing module 12 is an artificial intelligence (Artificial Intelligence, AI) image processing module, which uses AI technology to perform image processing, thereby ensuring that the brightness of the processed image is enhanced.
  • AI Artificial Intelligence
  • the image processing module 12 changes the overall brightness of the image by replacing the lighting environment of the original image, so that the driver can clearly see the road environment around the vehicle through the enhanced image.
  • the image processing module 12 is configured to perform ambient illumination processing on the original image, and replace the first ambient illumination of the original image with a second ambient illumination to obtain an enhanced image, and the illumination intensity of the second ambient illumination is greater than that of the first ambient illumination. the light intensity.
  • the first ambient light may be the ambient light of a dimly lit nighttime, foggy day, cloudy and rainy day and other scenes with weak light intensity
  • the second ambient light may be a clear daytime, brightly lit nighttime, etc. with a strong light intensity The ambient lighting of the scene.
  • the light intensity refers to the light intensity under specific ambient light, such as the light intensity at night with dim lighting, the light intensity in cloudy and rainy days, the light intensity in sunny daytime, etc.
  • the light intensity of each ambient light There is no need to determine the specific value of the intensity, only the size relationship needs to be clear. For example, the light intensity of a sunny day is greater than the light intensity of other ambient lighting. Therefore, when the ambient lighting is transformed, other ambient lighting is replaced by sunny. daytime ambient lighting.
  • the image processing module 12 is configured to perform ambient illumination processing on the original image by using an ambient illumination processing model
  • the ambient lighting processing model is obtained by training images of different ambient lighting in the same scene as samples.
  • the ambient lighting processing model may be a neural network model, such as a convolutional neural network model.
  • the ambient lighting processing model learns the characteristics of different ambient lighting through the training of a large number of samples, and uses the characteristics of different ambient lighting to realize the conversion of pictures of different ambient lighting, such as converting the original picture at night with dim lighting into a sunny day. enhanced image.
  • the training process using pictures of different ambient lighting in the same scene as samples can make the learned characteristics of each ambient lighting more accurate.
  • the training samples can have pictures in various scenes, and each scene has pictures with different ambient lighting.
  • the samples used for training only need to distinguish the ambient lighting of the pictures, and do not need to pay attention to whether they are in the same scene.
  • the ambient lighting processing model may be formed by cascading two convolutional neural networks, such as a decomposition network + an enhancement network, the decomposition network is used to decompose the input original image into the first illumination The image and the reflection image, the enhancement network is used to perform enhancement processing on the reflection image, and synthesize the reflection image and the second illumination image to obtain an enhanced image.
  • two convolutional neural networks such as a decomposition network + an enhancement network
  • the image processing module 12 is further configured to perform environment transformation processing on the enhanced image before outputting the enhanced image to the window display, and replace the first road surrounding environment of the original image with the second road surrounding environment.
  • This implementation method can transform a single driving roadside scene, improve the visual experience of drivers and passengers, increase the fun of driving, and improve the comfort of driving.
  • the surrounding environment of the first road is desert
  • the surrounding environment of the second road is snow
  • the driver can have different visual experience by replacing the environment.
  • the replacement process it is necessary to ensure that the main part of the road in the picture is not replaced.
  • the image processing module 12 when the image processing module 12 performs the above-mentioned replacement of the surrounding environment of the road, it first identifies the road part through the recognition algorithm, and then replaces the surrounding environment of the road on both sides of the road.
  • the two cannot be realized by using the same model. It can also be implemented by two parts of a model.
  • the road surrounding environment refers to the parts on both sides of the two side lines of the road, and the road part is the two side lines of the road and the part between them. Therefore, when identifying the road part, it is only necessary to perform the road edge detection, and then divide the road part and the road surrounding environment part based on this.
  • the image processing module 12 is configured to perform environment transformation processing on the enhanced image by adopting an environment transformation processing model
  • the environment transformation processing model is obtained by using pictures with different road surrounding environments as samples.
  • the environment transformation processing model here is actually an image synthesis model, which first separates the road and the vehicles on the road, and then combines the road, the vehicles and the environment on the road into one image.
  • FIG. 1 is a schematic diagram of the connection relationship of each component of a vehicle driving visual intelligent assistance system according to a specific embodiment of the present disclosure.
  • the vision system includes high-definition cameras that capture the environment around the vehicle.
  • the HD camera's field of view is larger than the driver's driving field of view.
  • the vehicle visual assistance system is further provided with an AI image processing module using AI image processing technology, and a window display 13 using transparent OLED display technology;
  • the high-definition camera captures the environment around the vehicle, and transmits the captured original image to the AI image processing module; the AI image processing module sends the processed image signal to the window display 13 .
  • the present disclosure collects driving road images in real time through a high-definition camera, processes the road images through AI (artificial intelligence) image processing technology, and displays the processed images through the OLED display screen embedded in the windshield, which can be clearly transformed into a brightly lit image of the same scene, maximizing driving visibility. It can greatly improve the safety of drivers, passengers and third parties. It is also possible to transform a single driving roadside scene to improve the fun of driving.
  • AI artificial intelligence
  • the AI image processing technology described can convert images of roads with insufficient light outside the window captured by high-definition cameras into clear, well-lit images.
  • the window display 13 is part of the windshield of the vehicle.
  • the window display 13 is used to be in a transparent state (completely transparent state) when it is not powered on; when it is powered on, it displays the enhanced image (usually in a semi-transparent state at this time).
  • the window display 13 is realized by using organic light emitting diode (organic light emitting diode, OLED) technology, and the window display 13 can be a transparent OLED display, that is, a self-luminous and transparent organic OLED screen, the material is OLED, but a transparent process is adopted.
  • OLED organic light emitting diode
  • the window display 13 is a part of the windshield; in the case of no electricity, the window display 13 is a piece of transparent glass; in the case of the display with electricity, the background image can be covered and displayed.
  • the AI image processing module adopts a software system based on artificial intelligence technology; the original image is intercepted 21 by the image, and then calculated by the AI model 22, and the calculated image data is output to the window display 13 for display;
  • the AI model 22 includes an AI image enhancement 23 model and an AI image transformation 24 model.
  • the AI model includes the aforementioned environment illumination processing model and environment transformation processing model.
  • the intelligent assistance system for vehicle driving vision of the present disclosure due to the adoption of AI image processing technology, does not require additional technical means of light supplementation, but processes the dim and blurry scene seen by the current driver through the AI model, and outputs a set of A clearer and brighter scene is displayed to the driver, thereby restoring the driver's driving vision to the greatest extent and ensuring driving safety (as shown in Figure 3).
  • the AI image enhancement model 23 (that is, the ambient lighting processing model) is mainly used to process images 31 with low light and blurred vision. image 32.
  • the AI image transformation 24 model (ie, the environment transformation processing model) is mainly used to transform the surrounding environment of the road, and the original scene image 41 becomes the changed scene image 42 through the calculation of the model.
  • the AI image processing technology can change the driving scene, and can convert the actual driving scene of the vehicle (such as a desert road section) into a virtual driving road section (such as a snow road section).
  • the present disclosure can replace driving scenes through the technology of AI image processing to increase the entertainment of driving.
  • the window display 13 includes a transparent display screen and a host; the host accepts image signals and controls the display to display; the display is embedded in the windshield; the size of the display is consistent with the size of the window , using a transparent OLED display. Usually it is a normal windshield, when the image is displayed, the windshield becomes a display. The driver drives through the images on the display.
  • the vehicle vision assistance system is installed on all windshields (front, rear, left and right) of the vehicle to transform the overall environment.
  • FIG. 5 is a flowchart of a method for displaying an in-vehicle image provided by some embodiments of the present disclosure. Referring to Figure 5, the method includes:
  • This step is performed by the aforementioned camera 11 , and for the detailed process, please refer to the foregoing description of the camera 11 .
  • This step is performed by the aforementioned image processing module 12, and for the detailed process, please refer to the above description of the image processing module 12.
  • S53 Display the enhanced image through the window display.
  • This step is performed by the aforementioned window display 13 , and for the detailed process, please refer to the foregoing description of the vehicle window display 13 .
  • the present disclosure adopts the above-mentioned technical solution, collects the original image of the driving road and the surrounding environment in real time through the camera, processes the original image through image processing technology, and enhances the brightness of the image; the processed image is displayed through the window display.
  • the image is obtained by changing the brightness of the original image, and the image quality is high, so that the driver can improve the driving field of vision to the greatest extent based on the enhanced image; it can greatly improve the safety of the driver, the vehicle and the people outside the vehicle.
  • the original image is directly displayed through the window display.
  • raw images captured on a clear day usually do not need to be processed and are displayed directly on the window display.
  • the raw images captured at night with dim lighting need to be processed and then displayed on the window display.
  • the brightness of the original image may refer to the average brightness of each pixel in the original image, or the intermediate brightness, the maximum brightness, the minimum brightness, and the like.
  • the threshold value can be set according to actual needs, such as the minimum brightness value of the image under the condition of ensuring a certain range of vision of the driver.
  • the original image is processed to obtain an enhanced image, including:
  • the first ambient light may be the ambient light of a dimly lit nighttime, foggy day, cloudy and rainy day and other scenes with weak light intensity
  • the second ambient light may be a clear daytime, brightly lit nighttime, etc. with a strong light intensity The ambient lighting of the scene.
  • the light intensity refers to the light intensity under specific ambient light, such as the light intensity at night with dim lighting, the light intensity in cloudy and rainy days, the light intensity in sunny daytime, etc.
  • the light intensity of each ambient light There is no need to determine the specific value of the intensity, only the size relationship needs to be clear. For example, the light intensity of a sunny day is greater than the light intensity of other ambient lighting. Therefore, when the ambient lighting is transformed, other ambient lighting is replaced by sunny. daytime ambient lighting.
  • ambient lighting processing is performed on the original image, including:
  • the ambient lighting processing model is obtained by training images of different ambient lighting in the same scene as samples.
  • the ambient lighting processing model may be a neural network model, such as a convolutional neural network model.
  • the ambient lighting processing model learns the characteristics of different ambient lighting through the training of a large number of samples, and uses the characteristics of different ambient lighting to realize the conversion of pictures of different ambient lighting, for example, convert the original picture in the dimly lit night into the sunny day enhanced image.
  • the training process using pictures of different ambient lighting in the same scene as samples can make the learned characteristics of each ambient lighting more accurate.
  • the training samples can have pictures in various scenes, and each scene has pictures with different ambient lighting.
  • the samples used for training only need to distinguish the ambient lighting of the pictures, and do not need to pay attention to whether they are in the same scene.
  • the ambient lighting processing model may be formed by cascading two convolutional neural networks, such as a decomposition network + an enhancement network, the decomposition network is used to decompose the input original image into the first illumination The image and the reflection image, the enhancement network is used to perform enhancement processing on the reflection image, and synthesize the reflection image and the second illumination image to obtain an enhanced image.
  • two convolutional neural networks such as a decomposition network + an enhancement network
  • the vehicle image display method further includes:
  • an environment transformation process is performed on the enhanced image to replace the first road surrounding environment of the original image with the second road surrounding environment.
  • This implementation method can transform a single driving roadside scene, improve the visual experience of drivers and passengers, increase the fun of driving, and improve the comfort of driving.
  • the surrounding environment of the first road is desert
  • the surrounding environment of the second road is snow
  • the driver can have different visual experience by replacing the environment.
  • the replacement process it is necessary to ensure that the main part of the road in the picture is not replaced.
  • the road part is first identified by the recognition algorithm, and then the surrounding environment of the road on both sides of the road is replaced. Implemented in two parts.
  • the road surrounding environment refers to the parts on both sides of the two side lines of the road, and the road part is the two side lines of the road and the part between them. Therefore, when identifying the road part, only the road edge detection is actually required, and then the road part and the road surrounding environment part can be divided according to this.
  • Exemplarily, performing environment transformation processing on the enhanced image including:
  • the environment transformation processing model is obtained by training pictures with different road surrounding environments as samples.
  • the environment transformation processing model here is actually an image synthesis model, which first separates the road and the vehicles on the road, and then combines the road, the vehicles and the environment on the road into one image.
  • Embodiments of the present disclosure also provide an in-vehicle image display device.
  • the in-vehicle image display device may include a processor and a memory, the memory storing at least one piece of program code, the program code being loaded and executed by the processor to implement the aforementioned method.
  • FIG. 6 is a schematic structural diagram of an in-vehicle image display device provided by an embodiment of the present disclosure.
  • the in-vehicle image display device 600 includes a central processing unit (Central Processing Unit, CPU) 601, a system including a random access memory (Random Access Memory, RAM) 602 and a read-only memory (Read-Only Memory, ROM) 603 memory 604 , and a system bus 605 connecting the system memory 604 and the central processing unit 601 .
  • CPU Central Processing Unit
  • RAM random access memory
  • ROM Read-Only Memory
  • the in-vehicle image display device 600 also includes a basic input/output system (Input/Output, I/O system) 606 that helps transmit information between various devices in the computer, and is used to store an operating system 613, application programs 614 and other program modules 615 of the mass storage device 607.
  • I/O system input/output system
  • Basic input/output system 606 includes a display 608 for displaying information and input devices 609 such as control keys for user input of information. Both the display 608 and the input device 609 are connected to the central processing unit 601 through the input and output controller 610 connected to the system bus 605 .
  • Mass storage device 607 is connected to central processing unit 601 through a mass storage controller (not shown) connected to system bus 605 .
  • Mass storage device 607 and its associated computer-readable media provide non-volatile storage for in-vehicle image display device 600 . That is, mass storage device 607 may include a computer-readable medium (not shown) such as a hard disk or a CD-ROM drive.
  • Computer-readable media can include computer storage media and communication media.
  • Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.
  • Computer storage media include RAM, ROM, Erasable Programmable Read Only Memory (EPROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other solid-state storage Its technology, Compact Disc Read-Only Memory (CD-ROM), Digital Video Disc (DVD) or other optical storage, cassette, magnetic tape, magnetic disk storage or other magnetic storage devices.
  • RAM random access memory
  • ROM read only Memory
  • EPROM Erasable Programmable Read Only Memory
  • EEPROM Electrically Erasable Programmable Read Only Memory
  • flash memory or other solid-state storage Its technology, Compact Disc Read-Only Memory (CD-ROM), Digital Video Disc (DVD) or other optical storage, cassette, magnetic tape, magnetic disk storage or other magnetic storage devices.
  • CD-ROM Compact
  • the in-vehicle image display apparatus 600 may also be operated by connecting to a remote computer on the network through a network such as the Internet. That is, the in-vehicle image display device 600 can be connected to the network 612 through the network interface unit 611 connected to the system bus 605, or, in other words, the network interface unit 611 can also be used to connect to other types of networks or remote computer systems (not shown). ).
  • the above-mentioned memory also includes one or more programs, and the one or more programs are stored in the memory and configured to be executed by the CPU.
  • the CPU 601 implements the aforementioned in-vehicle image display method by executing the one or more programs.
  • FIG. 6 does not constitute a limitation on the vehicle-mounted image display device 600, and may include more or less components than the one shown, or combine some components, or use different components layout.
  • Embodiments of the present disclosure also provide a computer-readable storage medium, where at least one piece of program code is stored in the computer-readable storage medium, and the program code is loaded and executed by the processor to implement the above method.
  • the computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like.
  • Embodiments of the present disclosure also provide a computer program product, where at least one piece of program code is stored in the computer program product, and the program code is loaded and executed by the processor to implement the above method.

Landscapes

  • Engineering & Computer Science (AREA)
  • Automation & Control Theory (AREA)
  • Transportation (AREA)
  • Mechanical Engineering (AREA)
  • Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Human Computer Interaction (AREA)
  • Fittings On The Vehicle Exterior For Carrying Loads, And Devices For Holding Or Mounting Articles (AREA)
  • Closed-Circuit Television Systems (AREA)
  • Controls And Circuits For Display Device (AREA)

Abstract

一种车辆视觉辅助系统、车载图像显示方法及装置（600），该系统包括：摄像头(11)，用于拍摄车辆周围的环境，得到原始图像；图像处理模块(12)，用于获取摄像头(11)拍摄的原始图像，在原始图像的亮度低于阈值时，对原始图像进行处理，得到增强图像，增强图像的亮度大于原始图像的亮度；车窗显示器(13)，用于获取图像处理模块(12)处理的增强图像，显示增强图像。

Description

车辆视觉辅助系统、车载图像显示方法及装置
本公开要求于2020年12月18日提交的申请号为202011510914.4、发明名称为“一种车辆智能辅助视觉系统”的中国专利申请的优先权，其全部内容通过引用结合在本公开中。
技术领域
本公开属于车辆驾驶安全领域，涉及一种车辆视觉辅助系统、车载图像显示方法及装置。
背景技术
在汽车领域，视觉辅助系统是指给驾驶员提供视觉信息，从而提高驾驶安全性的系统。传统的视觉辅助系统包括夜视系统，夜视系统通过红外摄像头来提高视像效果。
发明内容
本公开提供一种车辆视觉辅助系统、车载图像显示方法及装置，目的是在视觉条件较差情况下改善车内人员的视觉效果，提高行车的安全性。
为了实现上述目的，本公开采取的技术方案为：
本公开一些实施例提供了一种车辆视觉辅助系统，包括高清摄像头；所述的车辆视觉辅助系统还设有采用AI图像处理技术的AI图像处理模块、采用透明OLED显示技术的车窗显示器；所述的高清摄像头对车辆周围的环境进行拍摄，并将所拍摄的原始图像传递给AI图像处理模块；所述的AI图像处理模块将处理过的图像信号发送给车窗显示器。
可选地，所述的AI图像处理技术能够将高清摄像头拍摄的窗外的光照不足路面图像，转换成清晰的、光照充足的图像。
可选地，所述的车窗显示器是挡风玻璃的一部分；在不通电的情况下，所述的车窗显示器是一块透明的玻璃；在通电显示的情况下，可以对背景图像进行覆盖显示。
可选地，所述的AI图像处理模块采用基于人工智能技术的软件系统；原始 图像经过图像截取，再经过AI模型计算，输出计算后的图像数据给车窗显示器进行显示；所述的AI模型包括AI图像增强模型和AI图像变换模型。
可选地，所述的AI图像增强模型主要用于处理弱光和视线模糊的图像，经过该模型的计算，可以得到正常光线环境和视野清晰的图像。
可选地，所述的AI图像变换模型主要用于进行道路周边环境的变换，原始场景图像经过该模型的计算成为改变后的场景图像。
可选地，所述的车窗显示器包括透明的显示屏和主机两个部分；所述的主机接受图像信号并控制显示屏进行显示；显示屏内嵌在挡风玻璃中；显示屏大小和车窗大小一致，采用透明OLED显示屏。
可选地，所述的车辆视觉辅助系统设置在车辆的所有挡风玻璃上，进行整体环境的变换。
本公开一些实施例提供了一种车辆视觉辅助系统，包括：
摄像头，用于拍摄车辆周围的环境，得到原始图像；
图像处理模块，用于获取所述摄像头拍摄的所述原始图像，在所述原始图像的亮度低于阈值时，对所述原始图像进行处理，得到增强图像，所述增强图像的亮度大于所述原始图像的亮度；
车窗显示器，用于获取所述图像处理模块处理的增强图像，显示所述增强图像。
可选地，所述图像处理模块，用于对所述原始图像进行环境光照处理，将所述原始图像的第一环境光照替换成第二环境光照，得到所述增强图像，所述第二环境光照的光照强度大于所述第一环境光照的光照强度。
可选地，所述图像处理模块，用于采用环境光照处理模型对所述原始图像进行环境光照处理；
其中，所述环境光照处理模型采用同一场景下的不同环境光照的图片作为样本训练得到。
可选地，所述图像处理模块，还用于在向所述车窗显示器输出所述增强图像之前，对所述增强图像进行环境变换处理，将所述原始图像的第一道路周边环境替换成第二道路周边环境。
可选地，所述图像处理模块，用于采用环境变换处理模型对所述增强图像进行环境变换处理；
其中，所述环境变换处理模型采用具有不同道路周边环境的图片作为样本训练得到。
可选地，所述车窗显示器属于所述车辆的挡风玻璃的一部分；
所述车窗显示器，用于在不通电时呈透明状态；在通电时，显示所述增强图像。
本公开一些实施例提供了一种车载图像显示方法，包括：
拍摄车辆周围的环境，得到原始图像；
在所述原始图像的亮度低于阈值时，对所述原始图像进行处理，得到增强图像，所述增强图像的亮度大于所述原始图像的亮度；
通过车窗显示器显示所述增强图像。
可选地，所述对所述原始图像进行处理，得到增强图像，包括：
对所述原始图像进行环境光照处理，将所述原始图像的第一环境光照替换成第二环境光照，得到所述增强图像，所述第二环境光照的光照强度大于所述第一环境光照的光照强度。
可选地，所述对所述原始图像进行环境光照处理，包括：
采用环境光照处理模型对所述原始图像进行环境光照处理；
其中，所述环境光照处理模型采用同一场景下的不同环境光照的图片作为样本训练得到。
可选地，所述车载图像显示方法，还包括：
在向所述车窗显示器输出所述增强图像之前，对所述增强图像进行环境变换处理，将所述原始图像的第一道路周边环境替换成第二道路周边环境。
可选地，所述对所述增强图像进行环境变换处理，包括：
采用环境变换处理模型对所述增强图像进行环境变换处理；
其中，所述环境变换处理模型采用具有不同道路周边环境的图片作为样本训练得到。
本公开一些实施例提供了一种车载图像显示装置，包括：
处理器和存储器，所述存储器存储有至少一条程序代码，所述程序代码由所述处理器加载并执行以实现如前任一项所述的方法。
本公开一些实施例提供了一种计算机可读存储介质，其特征在于，所述计算机可读存储介质存储有至少一条程序代码，所述程序代码由处理器加载并执行以实现如前任一项所述的方法。
附图说明
图1是本公开一些实施例提供的车辆视觉辅助系统的结构示意图；
图2是本公开中的AI图像处理系统的功能示意图；
图3是本公开中的AI图像增强功能的驾驶视野变换过程图；
图4是本公开中的AI图像变换功能的驾驶视野变换过程图；
图5是本公开一些实施例提供的车载图像显示方法的流程图；
图6是本公开一些实施例提供的车载图像显示装置的结构示意图。
具体实施方式
下面对照附图，通过对实施例的描述，对本公开的具体实施方式作进一步详细的说明，以帮助本领域的技术人员对本公开的发明构思、技术方案有更完整、准确和深入的理解。
相关技术通过红外摄像头来拍摄夜间图片，辅助行车。红外摄像头拍摄的图片都是灰度图片和实际的视野有较大差别，而且成像质量不高，稍远就不清晰，对驾驶人员视野提升不大。
图1是本公开一些实施例提供的一种车辆视觉辅助系统的结构示意图。参见图1，该系统包括：摄像头11、图像处理模块12和车窗显示器13，摄像头11以及车窗显示器13分别和图像处理器12连接。
其中，摄像头11，用于拍摄车辆周围的环境，得到原始图像；通常，摄像头11安装在车辆的前方，因此拍摄到的通常是车辆前方的环境。
图像处理模块12，用于获取摄像头拍摄的原始图像，在原始图像的亮度低于阈值时，对原始图像进行处理，得到增强图像，增强图像的亮度大于原始图像的亮度。
车窗显示器13，用于获取图像处理模块12处理的增强图像，显示增强图像。
本公开采用上述技术方案，通过摄像头实时采集行车道路及周边环境的原始图像，通过图像处理技术对原始图像进行处理，增强图像的亮度；处理后的 图像通过车窗显示器进行显示，由于显示的增强图像是从原始图像变化亮度而来，图像质量高，使得驾驶员能够基于该增强图像，最大程度改善驾驶视野；能够大大提高驾乘人员、车辆以及车外人员的安全性。
可选地，在原始图像的亮度不低于阈值时，直接通过车窗显示器13显示原始图像。
例如，在晴朗的白天拍摄到的原始图像通常不需要处理，直接通过车窗显示器进行显示。而在灯光昏暗的夜间拍摄到的原始图像则需要处理，经过处理后再通过车窗显示器进行显示。
在本公开实施例中，原始图像的亮度可以是指原始图像中各个像素点的平均亮度，或者中间亮度、最大亮度、最小亮度等。阈值可以根据实际需要设置，例如可以是在保证驾驶员一定范围视野情况下的图像的最低亮度值等。
示例性地，摄像头11为高清摄像头，从而保证拍摄到的图像的清晰度。
示例性地，图像处理模块12为人工智能(Artificial Intelligence，AI)图像处理模块，利用AI技术进行图像处理，从而保证处理后的图像的亮度得到增强。
在一些可能的实现方式中，图像处理模块12通过替换原始图像的光照环境，从而改变图像的整体亮度，从而使得驾驶员能够通过增强图像看清楚车辆周围的道路环境。
示例性地，图像处理模块12，用于对原始图像进行环境光照处理，将原始图像的第一环境光照替换成第二环境光照，得到增强图像，第二环境光照的光照强度大于第一环境光照的光照强度。
其中，第一环境光照可以是灯光昏暗的夜间、雾天、阴雨天等光照强度较弱的场景时的环境光照，而第二环境光照可以是晴朗的白天、灯光明亮的夜间等光照强度较强的场景时的环境光照。
其中，光照强度是指在特定环境光照下的光照强度，例如灯光昏暗的夜间的光照强度、阴雨天的光照强度、晴朗的白天的光照强度等，在本公开实施例中，各个环境光照的光照强度不需要确定具体数值，只需要明确大小关系即可，例如，晴朗的白天的光照强度大于其他环境光照下的光照强度，因此在进行环境光照的变换时，都是将其他环境光照替换成晴朗的白天的环境光照。
示例性地，图像处理模块12，用于采用环境光照处理模型对原始图像进行环境光照处理；
其中，环境光照处理模型采用同一场景下的不同环境光照的图片作为样本 训练得到。
示例性地，该环境光照处理模型可以为神经网络模型，例如卷积神经网络模型。该环境光照处理模型通过大量样本的训练，从而学习到不同环境光照的特征，利用不同环境光照的特征实现不同环境光照的图片的转换，例如将灯光昏暗的夜间时的原始图片转换成晴朗白天时的增强图片。
在训练过程中，利用同样场景下的不同环境光照的图片作为样本，能够使得学习到的各个环境光照的特征更加准确。这里，训练样本中又可以有多种场景下的图片，每种场景下都有不同环境光照的图片。
当然，在其他实现方式中，训练所用的样本只需要区分图片的环境光照即可，无需关注是否是同一场景下。
在一种示例性实现方式中，该环境光照处理模型可以由2个卷积神经网络级联而成，例如一个分解网络+一个增强网络，分解网络用于将输入的原始图像分解为第一光照图像和反射图像，增强网络用于对反射图像进行增强处理，使反射图像和第二光照图像合成，得到增强图像。
可选地，图像处理模块12，还用于在向车窗显示器输出增强图像之前，对增强图像进行环境变换处理，将原始图像的第一道路周边环境替换成第二道路周边环境。
该实现方式可以对单一的驾驶路边场景进行变换，改善驾乘人员视觉感受，增加行车的乐趣，提高乘车的舒适性。
例如，第一道路周边环境为沙漠，第二道路周边环境为雪地，通过替换环境给驾驶员不同的视觉感受。当然，在替换过程中需要保证，图片中的道路主体部分不被替换。
因此，该图像处理模块12，在执行上述道路周边环境替换时，先通过识别算法识别出道路部分，然后对道路两侧的道路周边环境进行替换，这里的两个不可可以采用同一个模型实现，也可以由一个模型的两个部分实现。
其中，道路周边环境是指道路的两条边线两侧的部分，道路部分是道路的两条边线及其之间的部分。因此，在识别道路部分时，实际只需要进行道路边线检测，然后以此进行道路部分和道路周边环境部分的划分即可。
示例性地，图像处理模块12，用于采用环境变换处理模型对增强图像进行环境变换处理；
其中，环境变换处理模型采用具有不同道路周边环境的图片作为样本训练 得到。
这里的环境变换处理模型实际是一种图像合成模型，先将道路及道路上的车辆分离出来，再将道路及道路上的车辆和环境合成一幅图像。
如图1、图2所表达的本公开的结构，为一种车辆视觉辅助系统，涉及汽车智能电子技术。其中，图1为本公开的一个具体实施方式的车辆驾驶视觉智能辅助系统的各部件的连接关系示意图。该视觉系统包括高清摄像头，用于拍摄车辆周围的环境。高清摄像头的拍摄视野要大于驾驶员的驾驶视野。
为了克服现有技术的缺陷，实现在视觉条件较差情况下改善车内人员的视觉效果，提高行车的安全性的发明目的，本公开采取的技术方案为：
如图1、图2所示，本公开的车辆视觉辅助系统，所述的车辆视觉辅助系统还设有采用AI图像处理技术的AI图像处理模块、采用透明OLED显示技术的车窗显示器13；所述的高清摄像头对车辆周围的环境进行拍摄，并将所拍摄的原始图像传递给AI图像处理模块；所述的AI图像处理模块将处理过的图像信号发送给车窗显示器13。
本公开通过高清摄像头实时采集行车道路图像，通过AI(人工智能)的图像处理技术，对道路图像进行处理，处理后的图像通过内嵌在挡风玻璃中的OLED显示屏进行显示，可以清晰变换成同一场景下的光线明亮的图像，从最大程度改善驾驶视野。能够大大提高驾驶员，乘客以及第三方的安全性。也可以对单一的驾驶路边场景进行变换，提高驾驶的乐趣。
所述的AI图像处理技术能够将高清摄像头拍摄的窗外的光照不足路面图像，转换成清晰的、光照充足的图像。
示例性地，所述车窗显示器13属于所述车辆的挡风玻璃的一部分；
所述车窗显示器13，用于在不通电时呈透明状态(完全透明状态)；在通电时，显示所述增强图像(此时通常为半透明状态)。
其中，车窗显示器13采用有机发光二极管(organic light emitting diode，OLED)技术实现，车窗显示器13可以为透明OLED显示屏，即自发光透明的有机OLED屏幕，材质为OLED，但是采用透明工艺。
所述的车窗显示器13是挡风玻璃的一部分；在不通电的情况下，所述的车窗显示器13是一块透明的玻璃；在通电显示的情况下，可以对背景图像进行覆盖显示。
如图2所示，所述的AI图像处理模块采用基于人工智能技术的软件系统； 原始图像经过图像截取21，再经过AI模型22计算，输出计算后的图像数据给车窗显示器13进行显示；所述的AI模型22包括AI图像增强23模型和AI图像变换24模型。
其中，AI模型包括前述环境光照处理模型和环境变换处理模型。
本公开的汽车驾驶视觉智能辅助系统，由于采用AI图像处理的技术，不需要通过额外补光的技术手段，而是通过AI模型对当前驾驶员所看到的昏暗模糊场景进行处理，输出一套更清晰更明亮的场景显示给驾驶员，从而最大程度恢复驾驶员的驾驶视野，保证驾驶安全(如图3所示)。
如图3所示，所述的AI图像增强23模型(也即环境光照处理模型)主要用于处理弱光和视线模糊的图像31，经过该模型的计算，可以得到正常光线环境和视野清晰的图像32。
如图4所示，所述的AI图像变换24模型(也即环境变换处理模型)主要用于进行道路周边环境的变换，原始场景图像41经过该模型的计算成为改变后的场景图像42。
该AI图像处理技术可以改变驾驶场景，能够把车辆实际行驶的场景(比如沙漠路段)，转换成车辆虚拟行驶的路段(比如雪地路段)。本公开可以通过AI图像处理的技术，进行驾驶场景的替换，增加驾驶的娱乐性。
所述的车窗显示器13包括透明的显示屏和主机两个部分；所述的主机接受图像信号并控制显示屏进行显示；显示屏内嵌在挡风玻璃中；显示屏大小和车窗大小一致，采用透明OLED显示屏。平时就是正常的档风玻璃，当进行图像显示的时候，挡风就变成了显示器。驾驶员通过显示器上的图像进行驾驶。
特别地，所述的车辆视觉辅助系统设置在车辆的所有挡风玻璃上(前、后、左、右)，进行整体环境的变换。
图5是本公开一些实施例提供的一种车载图像显示方法的流程图。参见图5，该方法包括：
S51：拍摄车辆周围的环境，得到原始图像。
该步骤由前述摄像头11执行，详细过程参见前文关于摄像头11的描述。
S52：在原始图像的亮度低于阈值时，对原始图像进行处理，得到增强图像，增强图像的亮度大于原始图像的亮度。
该步骤由前述图像处理模块12执行，详细过程参见前文关于图像处理模块 12的描述。
S53：通过车窗显示器显示增强图像。
该步骤由前述车窗显示器13执行，详细过程参见前文关于车窗显示器13的描述。
本公开采用上述技术方案，通过摄像头实时采集行车道路及周边环境的原始图像，通过图像处理技术对原始图像进行处理，增强图像的亮度；处理后的图像通过车窗显示器进行显示，由于显示的增强图像是从原始图像变化亮度而来，图像质量高，使得驾驶员能够基于该增强图像，最大程度改善驾驶视野；能够大大提高驾乘人员、车辆以及车外人员的安全性。
可选地，在原始图像的亮度不低于阈值时，直接通过车窗显示器显示原始图像。
例如，在晴朗的白天拍摄到的原始图像通常不需要处理，直接通过车窗显示器进行显示。而在灯光昏暗的夜间拍摄到的原始图像则需要处理，经过处理后再通过车窗显示器进行显示。
在本公开实施例中，原始图像的亮度可以是指原始图像中各个像素点的平均亮度，或者中间亮度、最大亮度、最小亮度等。阈值可以根据实际需要设置，例如在保证驾驶员一定范围视野情况下的图像的最低亮度值等。
在一些可能的实现方式中，对原始图像进行处理，得到增强图像，包括：
对原始图像进行环境光照处理，将原始图像的第一环境光照替换成第二环境光照，得到增强图像，第二环境光照的光照强度大于第一环境光照的光照强度。
其中，第一环境光照可以是灯光昏暗的夜间、雾天、阴雨天等光照强度较弱的场景时的环境光照，而第二环境光照可以是晴朗的白天、灯光明亮的夜间等光照强度较强的场景时的环境光照。
其中，光照强度是指在特定环境光照下的光照强度，例如灯光昏暗的夜间的光照强度、阴雨天的光照强度、晴朗的白天的光照强度等，在本公开实施例中，各个环境光照的光照强度不需要确定具体数值，只需要明确大小关系即可，例如，晴朗的白天的光照强度大于其他环境光照下的光照强度，因此在进行环境光照的变换时，都是将其他环境光照替换成晴朗的白天的环境光照。
示例性地，对原始图像进行环境光照处理，包括：
采用环境光照处理模型对原始图像进行环境光照处理；
其中，环境光照处理模型采用同一场景下的不同环境光照的图片作为样本训练得到。
示例性地，该环境光照处理模型可以为神经网络模型，例如卷积神经网络模型。该环境光照处理模型通过大量样本的训练，从而学习到不同环境光照的特征，利用不同环境光照的特征实现不同环境光照的图片的转换，例如将灯光昏暗的夜间时的原始图片转换成晴朗白天时的增强图片。
在训练过程中，利用同样场景下的不同环境光照的图片作为样本，能够使得学习到的各个环境光照的特征更加准确。这里，训练样本中又可以有多种场景下的图片，每种场景下都有不同环境光照的图片。
当然，在其他实现方式中，训练所用的样本只需要区分图片的环境光照即可，无需关注是否是同一场景下。
在一种示例性实现方式中，该环境光照处理模型可以由2个卷积神经网络级联而成，例如一个分解网络+一个增强网络，分解网络用于将输入的原始图像分解为第一光照图像和反射图像，增强网络用于对反射图像进行增强处理，使反射图像和第二光照图像合成，得到增强图像。
在一些可能的实现方式中，车载图像显示方法，还包括：
在向车窗显示器输出增强图像之前，对增强图像进行环境变换处理，将原始图像的第一道路周边环境替换成第二道路周边环境。
该实现方式可以对单一的驾驶路边场景进行变换，改善驾乘人员视觉感受，增加行车的乐趣，提高乘车的舒适性。
例如，第一道路周边环境为沙漠，第二道路周边环境为雪地，通过替换环境给驾驶员不同的视觉感受。当然，在替换过程中需要保证，图片中的道路主体部分不被替换。
因此，在执行上述道路周边环境替换时，先通过识别算法识别出道路部分，然后对道路两侧的道路周边环境进行替换，这里的两个不可可以采用同一个模型实现，也可以由一个模型的两个部分实现。
其中，道路周边环境是指道路的两条边线两侧的部分，道路部分是道路的两条边线及其之间的部分。因此，在识别道路部分时，实际只需要进行道路边线检测，然后以此进行道路部分和道路周边环境部分的划分即可。
示例性地，对增强图像进行环境变换处理，包括：
采用环境变换处理模型对增强图像进行环境变换处理；
其中，环境变换处理模型采用具有不同道路周边环境的图片作为样本训练得到。
这里的环境变换处理模型实际是一种图像合成模型，先将道路及道路上的车辆分离出来，再将道路及道路上的车辆和环境合成一幅图像。
本公开实施例还提供了一种车载图像显示装置。该车载图像显示装置可以包括处理器和存储器，所述存储器存储有至少一条程序代码，所述程序代码由所述处理器加载并执行以实现前述方法。
图6是本公开实施例提供的一种车载图像显示装置的结构示意图。参见图6，车载图像显示装置600包括中央处理单元(Central Processing Unit，CPU)601、包括随机存取存储器(Random Access Memory，RAM)602和只读存储器(Read-Only Memory，ROM)603的系统存储器604，以及连接系统存储器604和中央处理单元601的系统总线605。车载图像显示装置600还包括帮助计算机内的各个器件之间传输信息的基本输入/输出系统(Input/Output，I/O系统)606，和用于存储操作系统613、应用程序614和其他程序模块615的大容量存储设备607。
基本输入/输出系统606包括有用于显示信息的显示器608和用于用户输入信息的诸如控制按键之类的输入设备609。其中显示器608和输入设备609都通过连接到系统总线605的输入输出控制器610连接到中央处理单元601。
大容量存储设备607通过连接到系统总线605的大容量存储控制器(未示出)连接到中央处理单元601。大容量存储设备607及其相关联的计算机可读介质为车载图像显示装置600提供非易失性存储。也就是说，大容量存储设备607可以包括诸如硬盘或者CD-ROM驱动器之类的计算机可读介质(未示出)。
不失一般性，计算机可读介质可以包括计算机存储介质和通信介质。计算机存储介质包括以用于存储诸如计算机可读指令、数据结构、程序模块或其他数据等信息的任何方法或技术实现的易失性和非易失性、可移动和不可移动介质。计算机存储介质包括RAM、ROM、可擦除可编程只读存储器(Erasable Programmable Read Only Memory，EPROM)、带电可擦可编程只读存储(Electrically Erasable Programmable read only memory，EEPROM)、闪存或其他固态存储其技术，只读光盘(Compact Disc Read-Only Memory，CD-ROM)、数字通用光盘(Digital Video Disc，DVD)或其他光学存储、磁带盒、磁带、磁盘 存储或其他磁性存储设备。当然，本领域技术人员可知计算机存储介质不局限于上述几种。上述的系统存储器604和大容量存储设备607可以统称为存储器。
根据本公开的各种实施例，车载图像显示装置600还可以通过诸如因特网等网络连接到网络上的远程计算机运行。也即车载图像显示装置600可以通过连接在系统总线605上的网络接口单元611连接到网络612，或者说，也可以使用网络接口单元611来连接到其他类型的网络或远程计算机系统(未示出)。
上述存储器还包括一个或者一个以上的程序，一个或者一个以上程序存储于存储器中，被配置由CPU执行。CPU 601通过执行该一个或一个以上程序来实现前述车载图像显示方法。
本领域技术人员可以理解，图6中示出的结构并不构成对车载图像显示装置600的限定，可以包括比图示更多或更少的组件，或者组合某些组件，或者采用不同的组件布置。
本公开实施例还提供了一种计算机可读存储介质，该计算机可读存储介质存储有至少一条程序代码，所述程序代码由所述处理器加载并执行以实现如上所述的方法。例如，所述计算机可读存储介质可以是ROM、随机存取存储器(RAM)、CD-ROM、磁带、软盘和光数据存储设备等。
本公开实施例还提供了一种计算机程序产品，该计算机程序产品存储有至少一条程序代码，所述程序代码由所述处理器加载并执行以实现如上所述的方法。
上面结合附图对本公开进行了示例性描述，显然本公开具体实现并不受上述方式的限制，只要采用了本公开的方法构思和技术方案进行的各种非实质性的改进，或未经改进将本公开的构思和技术方案直接应用于其它场合的，均在本公开的保护范围之内。

Claims (13)

  1. 一种车辆视觉辅助系统，包括：
    摄像头(11)，用于拍摄车辆周围的环境，得到原始图像；
    图像处理模块(12)，用于获取所述摄像头(11)拍摄的所述原始图像，在所述原始图像的亮度低于阈值时，对所述原始图像进行处理，得到增强图像，所述增强图像的亮度大于所述原始图像的亮度；
    车窗显示器(13)，用于获取所述图像处理模块(12)处理的增强图像，显示所述增强图像。
  2. 根据权利要求1所述的车辆视觉辅助系统，所述图像处理模块(12)，用于对所述原始图像进行环境光照处理，将所述原始图像的第一环境光照替换成第二环境光照，得到所述增强图像，所述第二环境光照的光照强度大于所述第一环境光照的光照强度。
  3. 根据权利要求2所述的车辆视觉辅助系统，所述图像处理模块(12)，用于采用环境光照处理模型对所述原始图像进行环境光照处理；
    其中，所述环境光照处理模型采用同一场景下的不同环境光照的图片作为样本训练得到。
  4. 根据权利要求2或3所述的车辆视觉辅助系统，所述图像处理模块(12)，还用于在向所述车窗显示器(13)输出所述增强图像之前，对所述增强图像进行环境变换处理，将所述原始图像的第一道路周边环境替换成第二道路周边环境。
  5. 根据权利要求4所述的车辆视觉辅助系统，所述图像处理模块(12)，用于采用环境变换处理模型对所述增强图像进行环境变换处理；
    其中，所述环境变换处理模型采用具有不同道路周边环境的图片作为样本训练得到。
  6. 根据权利要求1至5任一项所述的车辆视觉辅助系统，所述车窗显示器(13)属于所述车辆的挡风玻璃的一部分；
    所述车窗显示器(13)，用于在不通电时呈透明状态；在通电时，显示所述增强图像。
  7. 一种车载图像显示方法，包括：
    拍摄车辆周围的环境，得到原始图像；
    在所述原始图像的亮度低于阈值时，对所述原始图像进行处理，得到增强图像，所述增强图像的亮度大于所述原始图像的亮度；
    通过车窗显示器显示所述增强图像。
  8. 根据权利要求7所述的车载图像显示方法，所述对所述原始图像进行处理，得到增强图像，包括：
    对所述原始图像进行环境光照处理，将所述原始图像的第一环境光照替换成第二环境光照，得到所述增强图像，所述第二环境光照的光照强度大于所述第一环境光照的光照强度。
  9. 根据权利要求8所述的车载图像显示方法，所述对所述原始图像进行环境光照处理，包括：
    采用环境光照处理模型对所述原始图像进行环境光照处理；
    其中，所述环境光照处理模型采用同一场景下的不同环境光照的图片作为样本训练得到。
  10. 根据权利要求8或9所述的车载图像显示方法，所述车载图像显示方法，还包括：
    在向所述车窗显示器输出所述增强图像之前，对所述增强图像进行环境变换处理，将所述原始图像的第一道路周边环境替换成第二道路周边环境。
  11. 根据权利要求10所述的车载图像显示方法，所述对所述增强图像进行环境变换处理，包括：
    采用环境变换处理模型对所述增强图像进行环境变换处理；
    其中，所述环境变换处理模型采用具有不同道路周边环境的图片作为样本训练得到。
  12. 一种车载图像显示装置，包括：
    处理器和存储器，所述存储器存储有至少一条程序代码，所述程序代码由所述处理器加载并执行以实现如权利要求7至11任一项所述的方法。
  13. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质存储有至少一条程序代码，所述程序代码由处理器加载并执行以实现如权利要求7 至11任一项所述的方法。
PCT/CN2021/122882 2020-12-18 2021-10-09 车辆视觉辅助系统、车载图像显示方法及装置 WO2022127307A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011510914.4A CN112606832A (zh) 2020-12-18 2020-12-18 一种车辆智能辅助视觉系统
CN202011510914.4 2020-12-18

Publications (1)

Publication Number Publication Date
WO2022127307A1 true WO2022127307A1 (zh) 2022-06-23

Family

ID=75241125

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/122882 WO2022127307A1 (zh) 2020-12-18 2021-10-09 车辆视觉辅助系统、车载图像显示方法及装置

Country Status (2)

Country Link
CN (1) CN112606832A (zh)
WO (1) WO2022127307A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112606832A (zh) * 2020-12-18 2021-04-06 芜湖雄狮汽车科技有限公司 一种车辆智能辅助视觉系统

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104272345A (zh) * 2012-05-18 2015-01-07 日产自动车株式会社 车辆用显示装置、车辆用显示方法以及车辆用显示程序
CN108515909A (zh) * 2018-04-04 2018-09-11 京东方科技集团股份有限公司 一种汽车抬头显示系统及其障碍物提示方法
CN108528339A (zh) * 2017-03-01 2018-09-14 京东方科技集团股份有限公司 一种显示系统及其显示方法和车辆
CN110610463A (zh) * 2019-08-07 2019-12-24 深圳大学 一种图像增强方法及装置
DE102020003668A1 (de) * 2020-06-19 2020-08-20 Daimler Ag Verfahren zum Anzeigen eines augmentierten Bildes
CN112606832A (zh) * 2020-12-18 2021-04-06 芜湖雄狮汽车科技有限公司 一种车辆智能辅助视觉系统
DE102019220168A1 (de) * 2019-12-19 2021-06-24 Conti Temic Microelectronic Gmbh Helligkeits-Umwandlung von Bildern einer Kamera

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030095080A1 (en) * 2001-11-19 2003-05-22 Koninklijke Philips Electronics N.V. Method and system for improving car safety using image-enhancement
CN104627078B (zh) * 2015-02-04 2017-03-08 上海咔酷咔新能源科技有限公司 基于柔性透明oled的汽车驾驶虚拟系统及其控制方法
CN105216715A (zh) * 2015-10-13 2016-01-06 湖南七迪视觉科技有限公司 一种汽车驾驶员视觉辅助增强系统
WO2017134861A1 (ja) * 2016-02-05 2017-08-10 日立マクセル株式会社 ヘッドアップディスプレイ装置
CN109658519B (zh) * 2018-12-28 2022-07-12 吉林大学 基于现实路况信息图像处理的车载多模式增强现实系统
CN109636924B (zh) * 2018-12-28 2022-11-22 吉林大学 基于现实路况信息三维建模的车载多模式增强现实系统
CN110807740B (zh) * 2019-09-17 2023-04-18 北京大学 一种面向监控场景车窗图像的图像增强方法与系统
CN111625457A (zh) * 2020-05-27 2020-09-04 多伦科技股份有限公司 基于改进的dqn算法的虚拟自动驾驶测试优化方法

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104272345A (zh) * 2012-05-18 2015-01-07 日产自动车株式会社 车辆用显示装置、车辆用显示方法以及车辆用显示程序
CN108528339A (zh) * 2017-03-01 2018-09-14 京东方科技集团股份有限公司 一种显示系统及其显示方法和车辆
CN108515909A (zh) * 2018-04-04 2018-09-11 京东方科技集团股份有限公司 一种汽车抬头显示系统及其障碍物提示方法
CN110610463A (zh) * 2019-08-07 2019-12-24 深圳大学 一种图像增强方法及装置
DE102019220168A1 (de) * 2019-12-19 2021-06-24 Conti Temic Microelectronic Gmbh Helligkeits-Umwandlung von Bildern einer Kamera
DE102020003668A1 (de) * 2020-06-19 2020-08-20 Daimler Ag Verfahren zum Anzeigen eines augmentierten Bildes
CN112606832A (zh) * 2020-12-18 2021-04-06 芜湖雄狮汽车科技有限公司 一种车辆智能辅助视觉系统

Also Published As

Publication number Publication date
CN112606832A (zh) 2021-04-06

Similar Documents

Publication Publication Date Title
Binas et al. DDD17: End-to-end DAVIS driving dataset
US10504214B2 (en) System and method for image presentation by a vehicle driver assist module
US10063786B2 (en) Vehicle vision system with enhanced low light capabilities
US11891023B1 (en) Using camera data to manage a vehicle parked outside in cold climates
CN108515909B (zh) 一种汽车抬头显示系统及其障碍物提示方法
US11970156B1 (en) Parking assistance using a stereo camera and an added light source
US10616488B2 (en) Image display method, vehicle display device, vehicle sun visor, and related vehicle
US11341614B1 (en) Emirror adaptable stitching
JP2007142624A (ja) 車両搭載撮像装置
CN208479822U (zh) 一种车用全景环视系统
CN114338958B (zh) 一种图像处理的方法及相关设备
US20150042802A1 (en) Vehicle safety control apparatus and method using cameras
WO2022127307A1 (zh) 车辆视觉辅助系统、车载图像显示方法及装置
CN113866983A (zh) 抬头显示装置、抬头显示装置的显示方法
US11531197B1 (en) Cleaning system to remove debris from a lens
CN112740264A (zh) 用于处理红外图像的设计
KR20170010534A (ko) 차량 사이드 영상 제공 장치 및 그 방법
CN113840123B (zh) 一种车载图像的图像处理装置、汽车
CN205193902U (zh) 一种驱雾行车记录仪
CN211628409U (zh) 一种基于鱼眼镜头的大场景识别相机
CN203528379U (zh) 一种短波红外驾驶员视觉增强系统
TWM608886U (zh) 汽車ａ柱視覺盲區顯示裝置
CN202491757U (zh) 一种虚拟倒车镜系统
KR102339358B1 (ko) 악천후 상황에서의 시인성 향상을 위한 차량용 영상처리 장치
WO2024146088A1 (zh) 图像处理方法、电子设备、计算机可读存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21905236

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21905236

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 21905236

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 23/01/2024)