CN116615357A

CN116615357A - Image processing method, image processing device, storage medium, electronic rearview mirror system and vehicle

Info

Publication number: CN116615357A
Application number: CN202180021631.XA
Authority: CN
Inventors: 陈秀波
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2021-12-16
Filing date: 2021-12-16
Publication date: 2023-08-18
Also published as: WO2023108560A1

Abstract

An image processing method, an image processing device, a storage medium, an electronic rearview mirror system and a vehicle. The method comprises the following steps: the electronic rearview mirror control unit (101) acquires acquired first image data from one or more cameras (111; 112; 113) and sends the first image data to the domain controller (102); the domain controller (102) receives first image data sent by the electronic rearview mirror control unit (101); the electronic rear view mirror control unit (101) processes the first image data to obtain processed image data, and outputs the processed image data to the display device (121; 122; 123). The safety requirement of intelligent driving is guaranteed, and the user experience is improved.

Description

Image processing method, image processing device, storage medium, electronic rearview mirror system and vehicle

Technical Field

The present application relates to the field of automotive technologies, and in particular, to an image processing method, an image processing device, a storage medium, an electronic rearview mirror system, and a vehicle.

Background

With the development of technology, electronic rearview mirrors are installed on vehicles to provide image information around the vehicles and assist equipment in driving safely. An electronic rearview mirror, which may also be referred to as a camera monitoring system (camera monitor system, CMS), includes a camera and a display screen, and real-time images may be captured by the camera and transmitted to the display screen for display. Compared with the optical rearview mirror, the optical rearview mirror has the advantages of widening the angle of view, being free from environmental influence and the like.

Currently, a special electronic control unit (electronic control unit, ECU) is generally required to process data collected by a camera to display the data on a display screen when the electronic rearview mirror is implemented on a vehicle, and the scheme of installing the electronic rearview mirror on the vehicle is still immature, so that slow starting, high time delay and influence on user experience and large-scale application often exist. Therefore, a safer and more efficient image processing technology is needed to improve the above problems and enhance the user experience, so that the technology can be widely applied.

Disclosure of Invention

In view of this, an image processing method, apparatus, storage medium, electronic rearview mirror system, and vehicle are proposed.

In a first aspect, an embodiment of the present application provides an image processing method. The method is for an electronic rear view mirror control unit, the method comprising: acquiring first image data acquired by one or more cameras from the one or more cameras; transmitting the first image data to a domain controller DC; and processing the first image data to obtain processed image data, and outputting the processed image data to display equipment corresponding to the electronic rearview mirror control unit.

According to the embodiment of the application, the electronic rearview mirror control unit is used for acquiring the image data acquired by the camera and outputting the processed image data to the display device, so that the camera can be directly connected to the electronic rearview mirror control unit to realize the function of the electronic rearview mirror without depending on a domain controller, the function of the electronic rearview mirror can be normally realized when the domain controller fails, the safety requirement of intelligent driving is ensured, and in such a case, the electronic rearview mirror is started quickly and has small time delay, and the user experience is improved. Meanwhile, the electronic rearview mirror control unit sends the first image data to the domain controller, so that the electronic rearview mirror and the domain controller can share the image data collected by the camera, and cost and space allocation are saved.

In a first possible implementation manner of the image processing method according to the first aspect, the method further includes: receiving second image data sent by the domain controller, wherein the second image data is determined by the domain controller according to the first image data and/or sensing data acquired by one or more sensors; and processing the second image data to obtain processed image data, and outputting the processed image data to the display equipment.

According to the embodiment of the application, the first image data is sent to the domain controller for processing through the electronic rearview mirror control unit to obtain the second image data, so that the domain controller can calculate by utilizing the calculation capability of the domain controller, the function of the electronic rearview mirror is enhanced, and the safety and the comfort in the driving process are further improved.

In a first possible implementation manner of the first aspect, in a second possible implementation manner of the image processing method, receiving second image data sent by the domain controller includes: and receiving the second image data sent by the domain controller through a Controller Area Network (CAN) interface or an Ethernet (ETH) interface.

Thereby, the transmission efficiency when the electronic rear view mirror control unit receives the second image data can be made higher.

In a third possible implementation manner of the image processing method according to the first or second possible implementation manner of the first aspect, the field of view of the second image data corresponds to a vehicle driving scene.

Therefore, the image information can be more refined, and the experience is better during driving.

In a fourth possible implementation form of the image processing method according to the first or second or third possible implementation form of the first aspect, the field of view of the second image data comprises a 360 degree panoramic environment around the vehicle.

Therefore, the visual field range of the image is larger, and the experience is better during running.

In a fifth possible implementation manner of the image processing method according to the first aspect or the first or the second or the third or the fourth possible implementation manner of the first aspect, the method further includes: receiving indication information sent by the domain controller, wherein the indication information is determined by the domain controller according to the first image data and/or sensing data acquired by one or more sensors; and outputting corresponding display information to the display equipment according to the indication information.

According to the embodiment of the application, the electronic rearview mirror control unit sends the first image data to the domain controller for processing to obtain the indication information, so that the domain controller can calculate by utilizing the calculation capability of the domain controller, the function of the electronic rearview mirror is enhanced, and the safety and the comfort in the driving process are further improved.

In a sixth possible implementation manner of the image processing method according to the first aspect or the first or the second or the third or the fourth or the fifth possible implementation manner of the first aspect, the electronic rear view mirror control unit includes a deserializer and a processing unit, the deserializer is configured to connect with the one or more cameras, acquire the first image data, and send the first image data to the processing unit; the processing unit is used for processing the first image data to obtain processed image data, and outputting the processed image data to the display equipment corresponding to the electronic rearview mirror control unit.

Thus, the electronic rearview mirror function can be independently realized.

In a seventh possible implementation manner of the image processing method according to the sixth possible implementation manner of the first aspect, the sending the first image data to the domain controller DC includes: the deserializer transmits the first image data to the domain controller.

Therefore, a mode that the electronic rearview mirror control unit sends the first image data to the domain controller can be realized, so that the domain controller can calculate by utilizing the calculation capability of the domain controller, the function of the electronic rearview mirror is enhanced, and the user experience is improved.

In a sixth possible implementation manner of the first aspect, in an eighth possible implementation manner of the image processing method, the sending the first image data to the domain controller DC includes: the processing unit transmits the first image data to the domain controller.

Therefore, another mode of sending the first image data to the domain controller by the electronic rearview mirror control unit can be realized, so that the domain controller can calculate by utilizing the calculation capability of the domain controller, the function of the electronic rearview mirror is enhanced, and the user experience is improved.

In a ninth possible implementation manner of the image processing method according to the first aspect or the first or the second or the third or the fourth or the fifth or the sixth or the seventh or the eighth possible implementation manner of the first aspect, the sending the first image data to the domain controller DC includes: and sending the first image data to the domain controller through a Low Voltage Differential Signaling (LVDS) interface.

Therefore, the domain controller can utilize the computing capability to perform computation, so that the function of the electronic rearview mirror is enhanced, and the user experience is improved.

In a tenth possible implementation manner of the image processing method according to the first aspect or the first or second or third or fourth or fifth or sixth or seventh or eighth or ninth possible implementation manner of the first aspect, the one or more cameras include a first camera for photographing a rear left side of the vehicle, a second camera for photographing a rear right side of the vehicle, and a third camera for photographing a right rear side of the vehicle, and the display device includes a first display device corresponding to the first camera, for displaying an electronic rearview mirror image photographed by the first camera; the second display device is used for displaying the electronic rearview mirror picture shot by the second camera, and the third display device is used for displaying the electronic rearview mirror picture shot by the third camera.

Therefore, the electronic rearview mirror can be better realized by utilizing the cameras and the display equipment, and the user experience is improved.

In a second aspect, embodiments of the present application provide an image processing method. The method is for a domain controller DC, the method comprising: and receiving first image data sent by an electronic rearview mirror control unit, wherein the first image data is acquired by the electronic rearview mirror control unit from one or more cameras and acquired by the one or more cameras, the first image data is used for being processed by the electronic rearview mirror control unit to obtain processed image data, and the processed image data is used for being output to display equipment corresponding to the electronic rearview mirror control unit by the electronic rearview mirror control unit.

According to the embodiment of the application, the first image data is received, and the first image data is the image data acquired by the camera and acquired by the electronic rearview mirror control unit, and the image data is used for being processed and output to the display device, so that the camera can be directly connected to the electronic rearview mirror control unit to realize the function of the electronic rearview mirror, a domain controller is not needed, when the domain controller fails, the function of the electronic rearview mirror can be normally realized, the safety requirement of intelligent driving is ensured, and in this case, the electronic rearview mirror is started quickly and has small time delay, and the user experience is improved. Meanwhile, by receiving the first image data sent by the electronic rearview mirror control unit, the image data collected by the camera can be shared with the electronic rearview mirror, so that the cost and the space deployment are saved.

In a first possible implementation manner of the image processing method according to the second aspect, the method further includes: acquiring sensing data acquired by one or more sensors; determining second image data according to the first image data and/or sensing data acquired by the one or more sensors; and sending the second image data to the electronic rearview mirror control unit, wherein the second image data is used for being processed by the electronic rearview mirror control unit to obtain processed image data, and the processed image data is used for being output to the display equipment by the electronic rearview mirror control unit.

According to the embodiment of the application, the domain controller acquires the sensing data, and utilizes the first image data and/or the sensing data to determine the second image data, so that the domain controller can calculate by utilizing the calculation capability of the domain controller, the function of the electronic rearview mirror is enhanced, and the safety and the comfort in the driving process are further improved.

In a second possible implementation manner of the image processing method according to the first possible implementation manner of the second aspect, the sending the second image data to the electronic rear view mirror control unit includes: and sending the second image data to the electronic rearview mirror control unit through a Controller Area Network (CAN) interface or an Ethernet (ETH) interface.

Thereby, the transmission efficiency when the domain controller transmits the second image data can be made higher.

In a third possible implementation manner of the image processing method according to the first or second possible implementation manner of the second aspect, the field of view of the second image data corresponds to a vehicle driving scene.

In a fourth possible implementation manner of the image processing method according to the first or second or third possible implementation manner of the second aspect, the field of view of the second image data includes a 360-degree panoramic environment around the vehicle.

In a fifth possible implementation manner of the image processing method according to the second aspect or the first or the second or the third or the fourth possible implementation manner of the second aspect, the method further includes: acquiring sensing data acquired by one or more sensors; determining indication information according to the first image data and/or sensing data acquired by the one or more sensors; and sending the indication information to the electronic rearview mirror control unit, wherein the indication information is used for the electronic rearview mirror control unit to output corresponding display information to the display equipment according to the indication information.

According to the embodiment of the application, the domain controller acquires the sensing data, and utilizes the first image data and/or the sensing data to determine the indication information, so that the domain controller can calculate by utilizing the calculation capability of the domain controller, the function of the electronic rearview mirror is enhanced, and the safety and the comfort in the driving process are further improved.

In a sixth possible implementation manner of the image processing method according to the second aspect or the first or second or third or fourth or fifth possible implementation manner of the second aspect, the electronic rear view mirror control unit includes a deserializer, receives the first image data sent by the electronic rear view mirror control unit, and includes: and receiving the first image data sent by the deserializer.

Therefore, a mode that the domain controller receives the first image data can be realized, so that the domain controller can calculate by utilizing the calculation capability of the domain controller, the function of the electronic rearview mirror is enhanced, and the user experience is improved.

In a seventh possible implementation manner of the image processing method according to the second aspect or the first or second or third or fourth or fifth possible implementation manner of the second aspect, the electronic rear view mirror control unit includes a processing unit, and the receiving the first image data sent by the electronic rear view mirror control unit includes: and receiving the first image data sent by the processing unit.

Therefore, another mode of receiving the first image data by the domain controller can be realized, so that the domain controller can utilize the computing capability of the domain controller to calculate, the function of the electronic rearview mirror is enhanced, and the user experience is improved.

In an eighth possible implementation manner of the image processing method according to the second aspect or the first or second or third or fourth or fifth or sixth or seventh possible implementation manner of the second aspect, receiving the first image data sent by the electronic rear view mirror control unit includes: and receiving the first image data sent by the electronic rearview mirror control unit through a Low Voltage Differential Signaling (LVDS) interface.

In a ninth possible implementation manner of the image processing method according to the second aspect or the first or the second or the third or the fourth or the fifth or the sixth or the seventh or the eighth possible implementation manner of the second aspect, the one or more cameras include a first camera for photographing a rear left side of the vehicle, a second camera for photographing a rear right side of the vehicle, and a third camera for photographing a right rear side of the vehicle, and the display device includes a first display device corresponding to the first camera, and is configured to display an electronic rearview mirror image photographed by the first camera; the second display device is used for displaying the electronic rearview mirror picture shot by the second camera, and the third display device is used for displaying the electronic rearview mirror picture shot by the third camera.

In a third aspect, an embodiment of the present application provides an image processing apparatus. The device is used for an electronic rearview mirror control unit, and comprises: the first acquisition module is used for acquiring first image data acquired by one or more cameras from the one or more cameras; a first transmitting module for transmitting the first image data to a domain controller DC; and the first processing output module is used for processing the first image data to obtain processed image data and outputting the processed image data to the display equipment corresponding to the electronic rearview mirror control unit.

According to a third aspect, in a first possible implementation manner of the image processing apparatus, the apparatus further includes: the first receiving module is used for receiving second image data sent by the domain controller, wherein the second image data is determined by the domain controller according to the first image data and/or sensing data acquired by one or more sensors; and the second processing output module is used for processing the second image data to obtain processed image data and outputting the processed image data to the display equipment.

According to a first possible implementation manner of the third aspect, in a second possible implementation manner of the image processing apparatus, a first receiving module is configured to: and receiving the second image data sent by the domain controller through a Controller Area Network (CAN) interface or an Ethernet (ETH) interface.

In a third possible implementation manner of the image processing apparatus according to the first or second possible implementation manner of the third aspect, the field of view of the second image data corresponds to a vehicle driving scene.

In a fourth possible implementation manner of the image processing apparatus according to the first or the second or the third possible implementation manner of the third aspect, the field of view of the second image data includes a 360-degree panoramic environment around the vehicle.

In a fifth possible implementation manner of the image processing apparatus according to the third aspect or the first or the second or the third or the fourth possible implementation manner of the third aspect, the apparatus further includes: the second receiving module is used for receiving indication information sent by the domain controller, wherein the indication information is determined by the domain controller according to the first image data and/or sensing data acquired by one or more sensors; and the third processing output module is used for outputting corresponding display information to the display equipment according to the indication information.

In a sixth possible implementation manner of the image processing apparatus according to the third aspect or the first or the second or the third or the fourth or the fifth possible implementation manner of the third aspect, the electronic rear view mirror control unit includes a deserializer and a processing unit, the deserializer is configured to connect with the one or more cameras, acquire the first image data, and send the first image data to the processing unit; the processing unit is used for processing the first image data to obtain processed image data, and outputting the processed image data to the display equipment corresponding to the electronic rearview mirror control unit.

According to a sixth possible implementation manner of the third aspect, in a seventh possible implementation manner of the image processing apparatus, a first sending module is configured to: the deserializer transmits the first image data to the domain controller.

According to a sixth possible implementation manner of the third aspect, in an eighth possible implementation manner of the image processing apparatus, a first sending module is configured to: the processing unit transmits the first image data to the domain controller.

In a ninth possible implementation manner of the image processing apparatus according to the third aspect or the first or the second or the third or the fourth or the fifth or the sixth or the seventh or the eighth possible implementation manner, the first sending module is configured to: and sending the first image data to the domain controller through a Low Voltage Differential Signaling (LVDS) interface.

In a tenth possible implementation manner of the image processing apparatus according to the third aspect or the first or the second or the third or the fourth or the fifth or the sixth or the seventh or the eighth or the ninth possible implementation manner, the one or more cameras include a first camera for photographing a rear left side of the vehicle, a second camera for photographing a rear right side of the vehicle, and a third camera for photographing a right side of the vehicle, and the display device includes a first display device corresponding to the first camera, for displaying an electronic rearview mirror image photographed by the first camera; the second display device is used for displaying the electronic rearview mirror picture shot by the second camera, and the third display device is used for displaying the electronic rearview mirror picture shot by the third camera.

In a fourth aspect, an embodiment of the present application provides an image processing apparatus. The apparatus is for a domain controller DC, the apparatus comprising: the third receiving module is used for receiving first image data sent by the electronic rearview mirror control unit, the first image data are acquired by the electronic rearview mirror control unit from one or more cameras and acquired by the one or more cameras, the first image data are used for being processed by the electronic rearview mirror control unit to obtain processed image data, and the processed image data are used for being output to display equipment corresponding to the electronic rearview mirror control unit by the electronic rearview mirror control unit.

In a first possible implementation manner of the image processing apparatus according to the fourth aspect, the apparatus further includes: the second acquisition module is used for acquiring sensing data acquired by one or more sensors; a first determining module, configured to determine second image data according to the first image data and/or sensing data acquired by the one or more sensors; and the second sending module is used for sending the second image data to the electronic rearview mirror control unit, the second image data is used for being processed by the electronic rearview mirror control unit to obtain processed image data, and the processed image data is used for being output to the display device by the electronic rearview mirror control unit.

In a second possible implementation manner of the image processing apparatus according to the first possible implementation manner of the fourth aspect, the second sending module is configured to: and sending the second image data to the electronic rearview mirror control unit through a Controller Area Network (CAN) interface or an Ethernet (ETH) interface.

In a third possible implementation form of the image processing apparatus according to the first or second possible implementation form of the fourth aspect, the field of view of the second image data corresponds to a vehicle driving scene.

In a fourth possible implementation form of the image processing apparatus according to the first or second or third possible implementation form of the fourth aspect, the field of view of the second image data comprises a 360 degree panoramic environment around the vehicle.

In a fifth possible implementation manner of the image processing apparatus according to the fourth aspect or the first or the second or the third or the fourth possible implementation manner of the fourth aspect, the apparatus further includes: the third acquisition module is used for acquiring sensing data acquired by one or more sensors; a second determining module, configured to determine indication information according to the first image data and/or sensing data acquired by the one or more sensors; and the third sending module is used for sending the indication information to the electronic rearview mirror control unit, and the indication information is used for the electronic rearview mirror control unit to output corresponding display information to the display equipment according to the indication information.

In a sixth possible implementation manner of the image processing apparatus according to the fourth aspect or the first or the second or the third or the fourth or the fifth possible implementation manner of the fourth aspect, the electronic rear view mirror control unit includes a deserializer, and a third receiving module is configured to: and receiving the first image data sent by the deserializer.

In a seventh possible implementation manner of the image processing apparatus according to the fourth aspect or the first or the second or the third or the fourth or the fifth possible implementation manner of the fourth aspect, the electronic rear view mirror control unit includes a processing unit, and a third receiving module is configured to: and receiving the first image data sent by the processing unit.

In an eighth possible implementation manner of the image processing apparatus according to the fourth aspect or the first or the second or the third or the fourth or the fifth or the sixth or the seventh possible implementation manner of the fourth aspect, a third receiving module is configured to: and receiving the first image data sent by the electronic rearview mirror control unit through a Low Voltage Differential Signaling (LVDS) interface.

In a ninth possible implementation manner of the image processing apparatus according to the fourth aspect or the first or the second or the third or the fourth or the fifth or the sixth or the seventh or the eighth possible implementation manner of the fourth aspect, the one or more cameras include a first camera for photographing a rear left side of the vehicle, a second camera for photographing a rear right side of the vehicle, and a third camera for photographing a right rear side of the vehicle, and the display device includes a first display device corresponding to the first camera, for displaying an electronic rearview mirror image photographed by the first camera; the second display device is used for displaying the electronic rearview mirror picture shot by the second camera, and the third display device is used for displaying the electronic rearview mirror picture shot by the third camera.

In a fifth aspect, an embodiment of the present application provides an image processing apparatus including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the image processing method of the first aspect or one or several of the possible implementations of the first aspect when executing the instructions, or to implement the image processing method of the second aspect or one or several of the possible implementations of the second aspect.

In a sixth aspect, embodiments of the present application provide a non-transitory computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the image processing method of the first aspect or one or more of the possible implementations of the first aspect, or implement the image processing method of the second aspect or one or more of the possible implementations of the second aspect.

In a seventh aspect, embodiments of the present application provide a terminal device, which may perform the image processing method of the first aspect or one or more of the multiple possible implementation manners of the first aspect, or perform the image processing method of the second aspect or one or more of the multiple possible implementation manners of the second aspect.

In an eighth aspect, embodiments of the present application provide a computer program product comprising a computer readable code, or a non-transitory computer readable storage medium carrying computer readable code, which when run in an electronic device, a processor in the electronic device performs an image processing method of the first aspect or one or more of the possible implementations of the first aspect, or performs an image processing method of the second aspect or one or more of the possible implementations of the second aspect.

In a ninth aspect, embodiments of the present application provide an electronic rear view mirror system comprising: one or more cameras; display devices corresponding to the one or more cameras; an electronic rear view mirror control unit for performing the image processing method of the first aspect or one or several of the possible implementations of the first aspect.

In a tenth aspect, embodiments of the present application provide a vehicle comprising one or more cameras; display devices corresponding to the one or more cameras; an electronic rear view mirror control unit for performing the image processing method of the first aspect or one or more of the plurality of possible implementations of the first aspect; a domain controller DC for performing the image processing method of the second aspect described above or one or more of the many possible implementations of the second aspect.

In an eleventh aspect, an embodiment of the application provides a vehicle comprising an electronic rear view mirror system as in the ninth aspect above; a domain controller DC for performing the image processing method of the second aspect described above or one or more of the many possible implementations of the second aspect.

These and other aspects of the application will be apparent from and elucidated with reference to the embodiment(s) described hereinafter.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments, features and aspects of the application and together with the description, serve to explain the principles of the application.

Fig. 1 shows one possible deployment of an intelligent driving area controller and an electronic rear view mirror sharing camera.

Fig. 2 shows a schematic diagram of a system architecture according to an embodiment of the application.

Fig. 3 shows a flowchart of an image processing method according to an embodiment of the present application.

Fig. 4 is a schematic diagram showing a transmission manner of first image data according to an embodiment of the present application.

Fig. 5 shows a flowchart of an image processing method according to an embodiment of the present application.

Fig. 6 is a schematic diagram showing a transmission manner of the second image data according to an embodiment of the present application.

Fig. 7 shows a flowchart of an image processing method according to an embodiment of the present application.

Fig. 8 shows a block diagram of an image processing apparatus according to an embodiment of the present application.

Fig. 9 shows a block diagram of an image processing apparatus according to an embodiment of the present application.

Fig. 10 shows a block diagram of an electronic device 100 according to an embodiment of the application.

Detailed Description

Various exemplary embodiments, features and aspects of the application will be described in detail below with reference to the drawings. In the drawings, like reference numbers indicate identical or functionally similar elements. Although various aspects of the embodiments are illustrated in the accompanying drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

The word "exemplary" is used herein to mean "serving as an example, embodiment, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

In addition, numerous specific details are set forth in the following description in order to provide a better illustration of the application. It will be understood by those skilled in the art that the present application may be practiced without some of these specific details. In some instances, well known methods, procedures, components, and circuits have not been described in detail so as not to obscure the present application.

Along with the development of technology, cameras are often arranged at different positions of a vehicle respectively and used for collecting images around the vehicle, and an intelligent driving domain controller performs fusion processing after acquiring data from sensors comprising the cameras and performs vehicle control according to the fused data. After the electronic rearview mirror is installed on the vehicle, partial positions of the camera connected with the electronic rearview mirror and the camera connected with the intelligent driving area controller may overlap, and the problem that the intelligent driving area controller and the camera of the electronic rearview mirror at corresponding positions are required to be shared is solved. Fig. 1 shows one possible deployment of an intelligent driving area controller and an electronic rear view mirror sharing camera. Referring to fig. 1, in this deployment manner, the camera may send the collected image data to the intelligent driving domain controller, and the intelligent driving domain controller may perform fusion processing on the image data and the collected data of other sensors, so as to perform vehicle control, and implement a function of intelligent driving. The image data can be sent to the electronic rearview mirror ECU by the intelligent driving domain controller, and the electronic rearview mirror ECU processes the image data and sends the processed data to a corresponding display screen for display so as to realize the function of the electronic rearview mirror.

In this way, when the intelligent driving area controller fails, the electronic rearview mirror ECU cannot acquire image data acquired by the camera, so that the electronic rearview mirror function cannot be realized, and therefore, the electronic rearview mirror fails, and the safety requirement cannot be guaranteed. Under the condition, after the vehicle is electrified, the electronic rearview mirror can be normally used after the intelligent driving domain controller is started, so that the starting is slow, the time delay is high, and the user experience is influenced.

In order to solve the above technical problems, the present application further provides an image processing method, where the image processing method according to the embodiments of the present application can enable an electronic rearview mirror control unit to obtain first image data collected by one or more cameras from the one or more cameras, process the first image data by the electronic rearview mirror control unit to obtain processed image data, and output the processed image data to a display device corresponding to the electronic rearview mirror control unit, so as to implement a function of the electronic rearview mirror. The electronic rear view mirror control unit may also send the first image data to an intelligent driving domain controller (domain controller, DC), which receives the first image data by the domain controller DC to realize other intelligent driving functions. According to the embodiment of the application, the camera can be directly connected to the electronic rearview mirror control unit to realize the function of the electronic rearview mirror without depending on a domain controller, so that the function of the electronic rearview mirror can be normally realized when the domain controller fails, the safety requirement of intelligent driving is ensured, and under the condition, the electronic rearview mirror is started quickly and has small time delay, and the user experience is improved. Meanwhile, the first image data is sent to the domain controller, so that the electronic rearview mirror and the domain controller can share the image data acquired by the camera, and cost and space arrangement of the camera are saved.

Fig. 2 shows a schematic diagram of a system architecture according to an embodiment of the application. As shown in fig. 2, the electronic rearview mirror system of an embodiment of the application may be deployed on a vehicle and may include one or more of an electronic rearview mirror control unit 101, one or more cameras, and one or more display devices. For example, the electronic rear view mirror system may include only the electronic rear view mirror control unit 101, or include the electronic rear view mirror control unit 101 and one or more display devices, or include the electronic rear view mirror control unit 101 and one or more cameras, or include the electronic rear view mirror control unit 101, one or more cameras, and one or more display devices, as the application is not limited in this respect. Wherein, the one or more cameras may include cameras 111-113, some or all of the cameras 111-113 may be cameras dedicated to implementing electronic rearview mirror functions, some or all of the cameras 111-113 may also be a part of the sensors 103, and may be used to provide image data to a domain controller for vehicle control to implement intelligent driving functions in addition to implementing electronic rearview mirror functions, which is not limited by the present application. The one or more display devices may include display devices 121-123, some or all of which display devices 121-123 may be display devices dedicated to implementing electronic rearview mirror functionality, and some or all of which display devices 121-123 may also be display devices dedicated to implementing other intelligent driving functionality in addition to electronic rearview mirror functionality, as the application is not limited in this respect. The cameras 111-113 may be disposed outside the vehicle, connected to the electronic rearview mirror control unit 101, and used for shooting the rear of the vehicle, obtaining first image data, and sending the first image data to the electronic rearview mirror control unit 101. The electronic rearview mirror control unit 101 can be specially used for combining a camera and a display device to realize the function of the electronic rearview mirror, can be independently deployed on an ECU or a CDC, can be integrated with other vehicle-mounted domain controllers, can be used for receiving first image data acquired by the cameras 111-113, processing the first image data to obtain processed image data, and sending the processed image data to the display devices 121-123 for display, and can also be used for sending the first image data to the domain controller 102 deployed on a vehicle. The domain controller 102 may be connected to the electronic rear view mirror control unit 101 for receiving the first image data transmitted by the electronic rear view mirror control unit 101. The domain controller 102 may be a domain controller, such as an in-vehicle computing device, for implementing intelligent driving-related functions. The display devices 121-123 may be screens, projections, etc., may be deployed inside the vehicle, may be connected to the electronic rear view mirror control unit 101, and may be configured to receive and display processed image data transmitted by the electronic rear view mirror control unit 101.

In one possible implementation, the sensor 103 may be connected to the domain controller 102 as other sensors are also deployed on the vehicle, as shown by sensor 103. The sensor 103 may include one or more of the following: vehicle-mounted radars (such as millimeter wave radars, laser radars, ultrasonic radars, etc.), rainfall sensors, cameras, vehicle attitude sensors (such as gyroscopes), inertial measurement units (inertial measurement unit, IMU), global navigation satellite systems (global navigation satellite system, GNSS), etc. The cameras may be other than the cameras 111 to 113, or may be some or all of the cameras 111 to 113. In the embodiment of the present application, the sensor 103 may be used to enhance the function of the electronic rearview mirror, for example, after the electronic rearview mirror control unit 101 sends the first image data to the domain controller 102, the domain controller 102 may determine new image data according to the first image data and the sensing data collected by the sensor 103, so as to implement a better function of the electronic rearview mirror.

In one possible implementation, the image processing method of the present application may be used in a smart driving scenario. For example, in an automatic driving scenario, if the domain controller 102 fails, the driver may take over the vehicle control, and at this time, the electronic rearview mirror control unit 101 may also work normally, so as to implement the function of the electronic rearview mirror, that is, the driver may perform safe driving according to the surrounding situation of the vehicle indicated by the electronic rearview mirror. It should be noted that the image processing method of the present application may also be used in other driving scenarios of vehicles, for example, in a scenario of non-intelligent driving, to assist a driver in driving, which is not limited by the present application.

The image processing method according to the embodiment of the present application is described in detail below with reference to fig. 3 to 7 on the basis of the system architecture shown in fig. 2.

Fig. 3 shows a flowchart of an image processing method according to an embodiment of the present application. As shown in fig. 3, the method includes:

in step S201, the electronic rearview mirror control unit acquires first image data acquired by one or more cameras from the one or more cameras.

The one or more cameras may include a first camera (e.g., camera 111 described above) for capturing images of the rear left side of the vehicle, a second camera (e.g., camera 112 described above) for capturing images of the rear right side of the vehicle, and a third camera (e.g., camera 113 described above) for capturing images of the right rear side of the vehicle. The first image data may include image data of the rear left, rear right, and right of the vehicle acquired by the cameras 111 to 113, respectively. The camera may also include only some of the cameras described above or more cameras than described above, which is not limiting in this regard.

Wherein the electronic rear view mirror control unit may comprise a deserializer and a processing unit. The processing unit may be a system on chip (SoC), the deserializer may be connected to one or more cameras (such as the cameras 111-113 described above), the deserializer obtains the first image data collected by the camera, and the first image data is sent to the processing unit through the deserializer, and the processing unit performs step S204.

In one possible implementation, the one or more cameras may send the first image data to the deserializer via a low voltage differential signal (low voltage differential signaling, LVDS).

Step S202, the electronic rear view mirror control unit transmits the first image data to the domain controller.

Wherein, in order to enable the electronic rear view mirror control unit to send the first image data to the domain controller for intelligent driving, and in order to meet functional safety requirements of intelligent driving, the automotive safety integrity rating (automotive safety integration level, ASIL) of the electronic rear view mirror control unit may be ASIL B or higher.

Wherein the electronic rear view mirror control unit may send the first image data to the domain controller through a low voltage differential signaling LVDS interface. Referring to fig. 4, fig. 4 is a schematic diagram illustrating a transmission manner of first image data according to an embodiment of the present application. As shown in fig. 4, the domain controller may include a deserializer and a processing unit, which may be a system-on-chip SoC (e.g., soC2 in the figure).

In a possible implementation, in step S202, the electronic rear view mirror control unit may send the first image data to the domain controller through a deserializer. Wherein, referring to fig. 4 (a) and 4 (b), two ways of the electronic rear view mirror control unit transmitting the first image data to the domain controller through the deserializer are shown, respectively.

In the first manner, as shown in fig. 4 (a), after receiving the first image data that is parallel data and is sent by the camera, the deserializer of the electronic rearview mirror control unit may not convert the first image data from parallel data to serial data, and send the first image data to the deserializer of the domain controller through the LVDS interface, process the data by the deserializer of the domain controller, convert the first image data from parallel data to serial data, and send the processed image data to the SoC2 through the mobile industry processor interface (mobile industry processor interface, MIPI), so that the processing unit of the subsequent domain controller may process the image data to implement the relevant intelligent driving function.

In a second way, as shown in fig. 4 (b), the electronic rear view mirror control unit may further comprise a serializer. The deserializer of the electronic rearview mirror control unit can process first image data, convert the first image data into serial data from parallel data, send the serial data to the deserializer through MIPI, the deserializer can convert the first image data into parallel data from serial data, send the first image data to the deserializer of the domain controller through LVDS interface, the deserializer of the domain controller can process the data, convert the first image data into serial data from parallel data, and send the processed image data to the SoC2 through the mobile industry processor interface (mobile industry processor interface, MIPI), so that the processing unit of the subsequent domain controller can process the image data to realize relevant intelligent driving functions.

In a possible implementation, in step S202, the electronic rear view mirror control unit may send the first image data to the domain controller via a processing unit of the electronic rear view mirror control unit. Therein, reference is made to fig. 4 (c), which shows one way of the electronic rear view mirror control unit sending the first image data to the domain controller via the processing unit. The processing unit of the electronic rearview mirror control unit can be a system-on-chip SoC (such as SoC1 in the figure)

As shown in fig. 4 (c), the electronic rear view mirror control unit may further include a serializer. The SoC1 may receive the first image data processed and transmitted by the deserializer as serial data through the MIPI and transmit the first image data to the deserializer through the MIPI, the deserializer may convert the first image data from serial data to parallel data, and transmit the first image data to the deserializer of the domain controller through the LVDS interface, the deserializer of the domain controller may process the data, the first image data is converted from parallel data to serial data, and transmit the processed image data to the SoC2 through the MIPI, so that the processing unit of the subsequent domain controller may process the image data to implement the relevant intelligent driving function.

In step S203, the domain controller receives the first image data transmitted by the electronic rear view mirror control unit.

The domain controller can receive the first image data sent by the electronic rearview mirror control unit through a Low Voltage Differential Signaling (LVDS) interface.

In a possible implementation manner, in step S203, the domain controller may receive the first image data sent by the deserializer of the electronic rear view mirror control unit, and the domain controller may also receive the first image data sent by the processing unit of the electronic rear view mirror control unit. The above procedure corresponds to that in step S202, and reference is made to the description of step S202.

Step S204, the electronic rearview mirror control unit processes the first image data to obtain processed image data, and outputs the processed image data to a display device corresponding to the electronic rearview mirror control unit.

Wherein step S204 may be performed by the processing unit of the electronic rear view mirror control unit.

For example, the processing unit may convert the color format of the first image data, such as converting YUV format to RGB format.

The electronic rearview mirror control unit can further comprise a serializer, and the processing unit can send the processed image data to the serializer for processing through MIPI, convert the image data from serial data into parallel data and send the parallel data to corresponding display equipment for display through LVDS interface.

The display device may be a display device dedicated to implementing the electronic rearview mirror function, or may be a display device simultaneously implementing the electronic rearview mirror function and other intelligent driving functions, for example, a display screen of a center console, or may be used to implement other functions such as global positioning system (global positioning system, GPS) navigation, which is not limited in this aspect of the application.

In one possible implementation, the display device may include: a first display device (such as the display device 121) corresponding to the first camera, which may be disposed on the left front side in the vehicle, and is used for displaying the electronic rearview mirror image of the rear left side of the vehicle shot by the first camera; a second display device (e.g., display device 122 described above) corresponding to the second camera may be disposed on the front right side in the vehicle for displaying the electronic rear view mirror image of the rear right side of the vehicle captured by the second camera, and a third display device (e.g., display device 123 described above) corresponding to the third camera may be disposed above the center control in the vehicle for displaying the electronic rear view mirror image of the right rear side of the vehicle captured by the third camera. The display device may also include only a portion of the display devices described above or more display devices than described above, as the application is not limited in this regard.

According to the embodiment of the application, the electronic rearview mirror control unit is used for acquiring the image data acquired by the camera and outputting the processed image data to the display device, so that the camera can be directly connected to the electronic rearview mirror control unit to realize the function of the electronic rearview mirror without depending on a domain controller, the function of the electronic rearview mirror can be normally realized when the domain controller fails, the safety requirement of intelligent driving is ensured, and in such a case, the electronic rearview mirror is started quickly and has small time delay, and the user experience is improved. The electronic rearview mirror control unit is used for sending the first image data to the domain controller, so that the electronic rearview mirror and the domain controller can share the image data collected by the camera, and cost and space allocation are saved.

In a possible implementation manner, on the basis of the above procedure, the domain controller may further process the first image data by using the sensing data, so as to implement functional enhancement of the electronic rearview mirror, and further improve safety and comfort in a driving process. Two ways of implementing the enhancement of the electronic rear view mirror function can be seen in fig. 5 and 6 described below.

Fig. 5 shows a flowchart of an image processing method according to an embodiment of the present application. As shown in fig. 5, the method further includes:

In step S401, the domain controller acquires sensing data acquired by one or more sensors.

The sensing data may be used to implement a function related to intelligent driving, for example, radar data collected by a vehicle radar, image data collected by a camera (which may include some or all of the cameras 111-113, or other cameras except the cameras 111-113, such as a 360-degree look-around camera, etc.), and the application is not limited thereto.

In step S402, the domain controller determines second image data according to the first image data and/or the sensing data acquired by the one or more sensors.

For example, the sensing data may include image data acquired by a camera in the sensor, where the image data may include image data of a portion of the first image data that is blocked (i.e., a blind area of a field of view in the first image data, such as a portion blocked by pillars on both sides of a front windshield of the vehicle), and may include other portions of the first image data that are not captured, and the first image data may be supplemented according to the image data, so as to determine second image data, where the second image data may be image data having a field of view larger than that of the first image data. Wherein the second image data may be determined by a processing unit of the domain controller.

In one possible implementation, the field of view of the second image data may include a 360 degree panoramic environment around the vehicle. Therefore, the visual field is wider during running, and the vehicle is safer and more comfortable during running.

In one possible implementation, the field of view of the second image data may correspond to a vehicle driving scene.

For example, the domain controller may determine the current driving state of the vehicle, such as a straight-going, reversing, turning left, turning right, changing a lane, etc., through sensing data collected by a vehicle posture sensor (such as a gyroscope) in the sensor, and determine the field of view range (such as including different directions, angles, etc.) of the different second image data according to different scenes. For example, in the case where the running state of the current vehicle is straight, the second image data may be image data of the left and right sides of the vehicle (including portions blocked by the pillars on both sides of the front windshield); in the case where the current running state of the vehicle is a lane change to the left, the second image data may be image data on the left and rear of the vehicle among the first image data. The second image data may be determined by supplementing the blocked portion of the first image data according to the image data acquired by the camera in the sensor. Wherein the second image data may be determined by a processing unit in the domain controller.

Step S403, the domain controller transmits the second image data to the electronic rear view mirror control unit.

In one possible implementation manner, the interface used when the domain controller sends the second image data to the electronic rearview mirror control unit may be referred to as an interface used when the first image data is transmitted in fig. 4, and only the transmission direction when the domain controller sends the second image data to the electronic rearview mirror control unit is opposite to the transmission direction when the first image data is sent in fig. 4, which is not described herein again.

In one possible implementation, the domain controller may also send the second image data to the electronic rear view mirror control unit via a controller area network (controller area network, CAN) or an Ethernet (ETH) interface.

Fig. 6 is a schematic diagram showing a transmission manner of the second image data according to an embodiment of the present application. As shown in fig. 6, the second image data may be sent by the processing unit of the domain controller (e.g., soc2 in the figure) to the processing unit of the electronic rear view mirror control unit (e.g., soc1 in the figure) through the CAN or ETH interface.

Thereby, the transmission efficiency when the second image data is transmitted can be made higher.

Step S404, the electronic rearview mirror control unit receives the second image data sent by the domain controller.

In a possible implementation, the electronic rear view mirror control unit receives said second image data that the domain controller CAN send over the controller area network CAN interface or the ethernet ETH interface. This process can be seen in fig. 6 described above, and will not be described again here.

Step S405, the electronic rearview mirror control unit processes the second image data to obtain processed image data, and outputs the processed image data to the display device.

The step S405 may be executed by the processing unit of the electronic rearview mirror control unit, and the manner in which the processing unit of the electronic rearview mirror control unit processes the second image data may refer to the manner in which the processing unit processes the first image data in the step S204, for example, converting the color format of the second image data.

For example, if the field of view of the second image data is a 360-degree panoramic environment around the vehicle, the 360-degree panoramic environment may be displayed on all display devices in a split screen manner, for example, if there are three display devices, the environment around the vehicle per 120 degrees after the second image data is processed may be displayed on each device.

For example, if the field of view of the second image data is different in different vehicle driving scenes, the corresponding image may be displayed on the corresponding display device. For example, if there are three display devices, the second image data transmitted by the domain controller may include image data of the left, right, and right rear of the vehicle, corresponding to the three display devices in the left, right, and center and upper sides of the vehicle, respectively. In the case where the domain controller determines that the running state of the current vehicle is straight, the second image data transmitted by the domain controller to the electronic rear view mirror control unit may include image data of left and right sides of the vehicle, the image data corresponding to the right rear of the vehicle may be empty, and the electronic rear view mirror control unit may display the image data corresponding to the left and right sides of the vehicle after processing the second image data on the display devices of the left and right sides of the vehicle, respectively, after receiving the second image data, without displaying the empty image data on the display devices of the upper center of the vehicle.

It should be noted that, the display device may display the image data in other manners, which are not displayed in the present application.

Fig. 7 shows a flowchart of an image processing method according to an embodiment of the present application. As shown in fig. 7, the method further includes:

in step S601, the domain controller acquires sensing data acquired by one or more sensors.

The process can be described with reference to step S401.

In step S602, the domain controller determines indication information according to the first image data and/or the sensing data acquired by the one or more sensors.

Wherein the indication information may be used to indicate a 360 degree condition around the vehicle.

For example, the sensing data may include data collected by a vehicle-mounted radar, which may indicate a distance of a vehicle from a related obstacle, the domain controller may determine whether there is an obstacle around the vehicle that is too close (may be determined to be too close when the distance is less than a preset threshold) through the data collected by the vehicle-mounted radar (e.g., a laser radar), and determine a position of the obstacle, which may be a relative position of the obstacle and the vehicle, and after determining the position of the obstacle, may further perform positioning in the first image data according to the position of the obstacle to determine a type of the obstacle (e.g., other vehicles, pedestrians, etc.), and so on, thereby determining the indication information. In a possible implementation manner, the domain controller may further determine second image data according to the first image data and/or the sensing data collected by the one or more sensors, refer to the method in step S402, and after determining the position of the obstacle, may further perform positioning in the second image data according to the position of the obstacle to determine the type of the obstacle, thereby determining the indication information. The indication information may include information indicating a position of an obstacle whose surrounding distance is too close to the vehicle, a type of the obstacle, and the like, such as image data indicating corresponding obstacle position information on the first image data or the second image data, distance information indicating a distance between the obstacle and the vehicle, and the like, to which the present application is not limited. Wherein the indication information may be determined by a processing unit in the domain controller.

Step S603, the domain controller sends the indication information to the electronic rearview mirror control unit.

The manner in which the domain controller sends the indication information to the electronic rearview mirror control unit may refer to the manner in which the domain controller sends the second image data to the electronic rearview mirror control unit in step S403, which is not described herein.

In step S604, the electronic rearview mirror control unit receives the indication information sent by the domain controller.

The manner in which the electronic rearview mirror control unit receives the indication information may refer to the manner in which the electronic rearview mirror control unit receives the second image data in step S404, which is not described herein.

Step S605, the electronic rearview mirror control unit outputs corresponding display information to the display device according to the indication information.

Wherein step S605 may be performed by the processing unit of the electronic rear view mirror control unit.

The display information may be displayed on the display device in the form of text, images, video, etc., which the present application is not limited to.

For example, the display information may be, for example, the position, type, etc. of the obstacle. For example, if the obstacle is a pedestrian on the left side of the vehicle, the display information may include an image on the left side of the vehicle, and the image may be highlighted for the pedestrian as the obstacle, or may include text of the distance of the obstacle from the vehicle. After being output to the display device, the left picture image of the vehicle can be displayed on the display device at the left side of the interior of the vehicle, the pedestrians which are obstacles in the picture are highlighted, and the current distance between the pedestrians and the vehicle is displayed in a text mode.

Fig. 8 shows a block diagram of an image processing apparatus according to an embodiment of the present application. The device is used for an electronic rear view mirror control unit, as shown in fig. 8, and comprises:

a first obtaining module 701, configured to obtain first image data collected by one or more cameras from the one or more cameras;

a first transmitting module 702 for transmitting the first image data to a domain controller DC;

a first processing output module 703, configured to process the first image data to obtain processed image data, and output the processed image data to a display device corresponding to the electronic rearview mirror control unit.

In a possible implementation manner, the electronic rearview mirror control unit comprises a deserializer and a processing unit, wherein the deserializer is used for being connected with the one or more cameras, acquiring the first image data and sending the first image data to the processing unit; the processing unit is used for processing the first image data to obtain processed image data, and outputting the processed image data to the display equipment corresponding to the electronic rearview mirror control unit.

Thus, the electronic rearview mirror function can be independently realized.

In one possible implementation, the apparatus further includes: the first receiving module is used for receiving second image data sent by the domain controller, wherein the second image data is determined by the domain controller according to the first image data and/or sensing data acquired by one or more sensors; and the second processing output module is used for processing the second image data to obtain processed image data and outputting the processed image data to the display equipment.

In one possible implementation, the first receiving module is configured to: and receiving the second image data sent by the domain controller through a Controller Area Network (CAN) interface or an Ethernet (ETH) interface.

In one possible implementation, the field of view of the second image data corresponds to a vehicle driving scene.

In one possible implementation, the field of view of the second image data includes a 360 degree panoramic environment around the vehicle.

In one possible implementation, the apparatus further includes: the second receiving module is used for receiving indication information sent by the domain controller, wherein the indication information is determined by the domain controller according to the first image data and/or sensing data acquired by one or more sensors; and the third processing output module is used for outputting corresponding display information to the display equipment according to the indication information.

In one possible implementation manner, the first sending module is configured to: the deserializer transmits the first image data to the domain controller.

In one possible implementation manner, the first sending module is configured to: the processing unit transmits the first image data to the domain controller.

In one possible implementation manner, the first sending module is configured to: and sending the first image data to the domain controller through a Low Voltage Differential Signaling (LVDS) interface.

In one possible implementation manner, the one or more cameras comprise a first camera for shooting the rear left side of the vehicle, a second camera for shooting the rear right side of the vehicle and a third camera for shooting the right rear of the vehicle, and the display device comprises a first display device corresponding to the first camera and used for displaying an electronic rearview mirror picture shot by the first camera; the second display device is used for displaying the electronic rearview mirror picture shot by the second camera, and the third display device is used for displaying the electronic rearview mirror picture shot by the third camera.

Fig. 9 shows a block diagram of an image processing apparatus according to an embodiment of the present application. The apparatus is for a domain controller DC, as shown in fig. 9, and comprises:

The third receiving module 801 is configured to receive first image data sent by the electronic rearview mirror control unit, where the first image data is acquired by the electronic rearview mirror control unit from one or more cameras and acquired by the one or more cameras, and the first image data is used for being processed by the electronic rearview mirror control unit to obtain processed image data, and the processed image data is used for being output by the electronic rearview mirror control unit to a display device corresponding to the electronic rearview mirror control unit.

In one possible implementation, the apparatus further includes: the second acquisition module is used for acquiring sensing data acquired by one or more sensors; a first determining module, configured to determine second image data according to the first image data and/or sensing data acquired by the one or more sensors; and the second sending module is used for sending the second image data to the electronic rearview mirror control unit, the second image data is used for being processed by the electronic rearview mirror control unit to obtain processed image data, and the processed image data is used for being output to the display device by the electronic rearview mirror control unit.

In one possible implementation manner, the second sending module is configured to: and sending the second image data to the electronic rearview mirror control unit through a Controller Area Network (CAN) interface or an Ethernet (ETH) interface.

In one possible implementation, the apparatus further includes: the third acquisition module is used for acquiring sensing data acquired by one or more sensors; a second determining module, configured to determine indication information according to the first image data and/or sensing data acquired by the one or more sensors; and the third sending module is used for sending the indication information to the electronic rearview mirror control unit, and the indication information is used for the electronic rearview mirror control unit to output corresponding display information to the display equipment according to the indication information.

In a possible implementation manner, the electronic rearview mirror control unit includes a deserializer, and a third receiving module is configured to: and receiving the first image data sent by the deserializer.

In a possible implementation manner, the electronic rearview mirror control unit includes a processing unit, a third receiving module, configured to: and receiving the first image data sent by the processing unit.

In one possible implementation manner, the third receiving module is configured to: and receiving the first image data sent by the electronic rearview mirror control unit through a Low Voltage Differential Signaling (LVDS) interface.

Fig. 10 shows a block diagram of an electronic device 100 according to an embodiment of the application. As shown in fig. 10, the electronic device 100 may be the above-described electronic mirror control unit 101 or the above-described domain controller 102, and performs the functions of the electronic mirror control unit or the domain controller in the image processing method shown in any one of fig. 2 to 7. The electronic device 100 includes at least one processor 1801, at least one memory 1802, and at least one communication interface 1803. The electronic device may further comprise common components such as an antenna, which are not described in detail herein.

The respective constituent elements of the electronic apparatus 100 are specifically described below with reference to fig. 10.

The processor 1801 may be a general purpose Central Processing Unit (CPU), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the above program schemes. The processor 1801 may include one or more processing units, such as: the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), a controller, a video codec, a digital signal processor (digital signal processor, DSP), a baseband processor, and/or a neural network processor (neural-network processing unit, NPU), etc. Wherein the different processing units may be separate devices or may be integrated in one or more processors.

A communication interface 1803 for communicating with other electronic devices or communication networks, such as ethernet, radio Access Network (RAN), core network, wireless local area network (Wireless Local Area Networks, WLAN), etc. For example, referring to fig. 2, the electronic device 100 is an electronic rearview mirror control unit 101, and the communication interface 1803 may be used to communicate with the domain controller 102. As another example, the digital electronic device 100 is a domain controller 102 and the communication interface 1803 may be used to communicate with the electronic rear view mirror control unit 101.

The Memory 1802 may be, but is not limited to, a read-Only Memory (ROM) or other type of static storage device that can store static information and instructions, a random access Memory (random access Memory, RAM) or other type of dynamic storage device that can store information and instructions, an electrically erasable programmable read-Only Memory (ElectricallyErasable Programmable Read-Only Memory, EEPROM), a compact disc read-Only Memory (Compact Disc Read-Only Memory) or other optical disc storage, a compact disc storage (including compact disc, laser disc, optical disc, digital versatile disc, blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. The memory may be stand alone and coupled to the processor via a bus. The memory may also be integrated with the processor.

Wherein the memory 1802 is configured to store application program codes for performing the above schemes and is controlled to be executed by the processor 1801. The processor 1801 is configured to execute application code stored in the memory 1802.

In the foregoing embodiments, the descriptions of the embodiments are emphasized, and for parts of one embodiment that are not described in detail, reference may be made to related descriptions of other embodiments.

An embodiment of the present application provides an image processing apparatus including: a processor and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above-described method when executing the instructions.

Embodiments of the present application provide a non-transitory computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the above-described method.

The embodiment of the application provides terminal equipment, which can execute the method.

Embodiments of the present application provide a computer program product comprising a computer readable code, or a non-transitory computer readable storage medium carrying computer readable code, which when run in a processor of an electronic device, performs the above method.

An embodiment of the present application provides an electronic rear view mirror system including: one or more cameras; display devices corresponding to the one or more cameras; and an electronic rear view mirror control unit, which can be used to perform the above related method.

Embodiments of the present application provide a vehicle including one or more cameras; display devices corresponding to the one or more cameras; an electronic rear view mirror control unit operable to perform the above-described related method; the domain controller DC may be used to perform the above-described related methods.

An embodiment of the present application provides a vehicle including: the electronic rearview mirror system; and a domain controller DC, which may be used to perform the above-described related methods.

The computer readable storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: portable computer disk, hard disk, random Access Memory (Random Access Memory, RAM), read Only Memory (ROM), erasable programmable Read Only Memory (Electrically Programmable Read-Only-Memory, EPROM or flash Memory), static Random Access Memory (SRAM), portable compact disk Read Only Memory (Compact Disc Read-Only Memory, CD-ROM), digital versatile disk (Digital Video Disc, DVD), memory stick, floppy disk, mechanical coding devices, punch cards or in-groove protrusion structures having instructions stored thereon, and any suitable combination of the foregoing.

The computer readable program instructions or code described herein may be downloaded from a computer readable storage medium to a respective computing/processing device or to an external computer or external storage device over a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmissions, wireless transmissions, routers, firewalls, switches, gateway computers and/or edge servers. The network interface card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium in the respective computing/processing device.

Computer program instructions for carrying out operations of the present application may be assembly instructions, instruction set architecture (Instruction Set Architecture, ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, c++ or the like and conventional procedural programming languages, such as the "C" language or similar programming languages. The computer readable program instructions may be executed entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a local area network (Local Area Network, LAN) or a wide area network (Wide Area Network, WAN), or it may be connected to an external computer (e.g., through the internet using an internet service provider). In some embodiments, aspects of the present application are implemented by personalizing electronic circuitry, such as programmable logic circuitry, field programmable gate arrays (Field-Programmable Gate Array, FPGA), or programmable logic arrays (Programmable Logic Array, PLA), with state information for computer readable program instructions.

Various aspects of the present application are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.

These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable medium having the instructions stored therein includes an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.

It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by hardware (e.g., circuits or ASICs (Application Specific Integrated Circuit, application specific integrated circuits)) which perform the corresponding functions or acts, or combinations of hardware and software, such as firmware, etc.

Although the application is described herein in connection with various embodiments, other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed application, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

The foregoing description of embodiments of the application has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope of the various embodiments described. The terminology used herein was chosen in order to best explain the principles of the embodiments, the practical application, or the improvement of technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

An image processing method for an electronic rear view mirror control unit, the method comprising:

acquiring first image data acquired by one or more cameras from the one or more cameras;

transmitting the first image data to a domain controller DC;

and processing the first image data to obtain processed image data, and outputting the processed image data to display equipment corresponding to the electronic rearview mirror control unit.
The method according to claim 1, wherein the method further comprises:

receiving second image data sent by the domain controller, wherein the second image data is determined by the domain controller according to the first image data and/or sensing data acquired by one or more sensors;

and processing the second image data to obtain processed image data, and outputting the processed image data to the display equipment.
The method of claim 2, wherein receiving the second image data transmitted by the domain controller comprises:

and receiving the second image data sent by the domain controller through a Controller Area Network (CAN) interface or an Ethernet (ETH) interface.
A method according to claim 2 or 3, wherein the field of view of the second image data corresponds to a vehicle driving scene.
The method of any of claims 2-4, wherein the field of view of the second image data includes a 360 degree panoramic environment around the vehicle.
The method according to any one of claims 1-5, further comprising:

receiving indication information sent by the domain controller, wherein the indication information is determined by the domain controller according to the first image data and/or sensing data acquired by one or more sensors;

and outputting corresponding display information to the display equipment according to the indication information.
The method according to any one of claims 1-6, wherein the electronic rear view mirror control unit comprises a deserializer and a processing unit, the deserializer being adapted to connect with the one or more cameras, obtain the first image data, and send the first image data to the processing unit;

the processing unit is used for processing the first image data to obtain processed image data, and outputting the processed image data to the display equipment corresponding to the electronic rearview mirror control unit.
The method of claim 7, wherein the transmitting the first image data to a domain controller DC comprises:

the deserializer transmits the first image data to the domain controller.
The method of any of claims 7, wherein the transmitting the first image data to a domain controller DC comprises:

the processing unit transmits the first image data to the domain controller.
The method according to any one of claims 1-9, wherein said transmitting said first image data to a domain controller DC comprises:

and sending the first image data to the domain controller through a Low Voltage Differential Signaling (LVDS) interface.
The method according to any one of claims 1 to 10, wherein,

the one or more cameras include a first camera for photographing a rear left side of the vehicle, a second camera for photographing a rear right side of the vehicle, and a third camera for photographing a right rear side of the vehicle,

the display equipment comprises first display equipment corresponding to the first camera and is used for displaying an electronic rearview mirror picture shot by the first camera; the second display device is used for displaying the electronic rearview mirror picture shot by the second camera, and the third display device is used for displaying the electronic rearview mirror picture shot by the third camera.
An image processing method for a domain controller DC, the method comprising:

and receiving first image data sent by an electronic rearview mirror control unit, wherein the first image data is acquired by the electronic rearview mirror control unit from one or more cameras and acquired by the one or more cameras, the first image data is used for being processed by the electronic rearview mirror control unit to obtain processed image data, and the processed image data is used for being output to display equipment corresponding to the electronic rearview mirror control unit by the electronic rearview mirror control unit.
The method according to claim 12, wherein the method further comprises:

acquiring sensing data acquired by one or more sensors;

determining second image data according to the first image data and/or sensing data acquired by the one or more sensors;

and sending the second image data to the electronic rearview mirror control unit, wherein the second image data is used for being processed by the electronic rearview mirror control unit to obtain processed image data, and the processed image data is used for being output to the display device by the electronic rearview mirror control unit.
The method according to claim 13, wherein transmitting the second image data to the electronic rear view mirror control unit comprises:

and sending the second image data to the electronic rearview mirror control unit through a Controller Area Network (CAN) interface or an Ethernet (ETH) interface.
The method according to claim 13 or 14, wherein the field of view of the second image data corresponds to a vehicle driving scene.
The method of any of claims 13-15, wherein the field of view of the second image data includes a 360 degree panoramic environment around the vehicle.
The method according to any one of claims 12-16, further comprising:

acquiring sensing data acquired by one or more sensors;

determining indication information according to the first image data and/or sensing data acquired by the one or more sensors;

and sending the indication information to the electronic rearview mirror control unit, wherein the indication information is used for the electronic rearview mirror control unit to output corresponding display information to the display equipment according to the indication information.
The method according to any one of claims 12-17, wherein the electronic rear view mirror control unit comprises a deserializer, and the receiving the first image data sent by the electronic rear view mirror control unit comprises:

And receiving the first image data sent by the deserializer.
Method according to any of claims 12-17, wherein the electronic rear view mirror control unit comprises a processing unit, the receiving first image data sent by the electronic rear view mirror control unit comprising:

and receiving the first image data sent by the processing unit.
Method according to any one of claims 12-19, wherein said receiving the first image data sent by the electronic rear view mirror control unit comprises:

and receiving the first image data sent by the electronic rearview mirror control unit through a Low Voltage Differential Signaling (LVDS) interface.
The method according to any one of claims 12 to 20, wherein,

the one or more cameras include a first camera for photographing a rear left side of the vehicle, a second camera for photographing a rear right side of the vehicle, and a third camera for photographing a right rear side of the vehicle,

the display equipment comprises first display equipment corresponding to the first camera and is used for displaying an electronic rearview mirror picture shot by the first camera; the second display device is used for displaying the electronic rearview mirror picture shot by the second camera, and the third display device is used for displaying the electronic rearview mirror picture shot by the third camera.
An image processing apparatus for an electronic rear view mirror control unit, the apparatus comprising:

a first acquisition module for acquiring first image data acquired by one or more cameras from the one or more cameras;

a first transmitting module for transmitting the first image data to a domain controller DC;

and the first processing output module is used for processing the first image data to obtain processed image data and outputting the processed image data to the display equipment corresponding to the electronic rearview mirror control unit.
An image processing apparatus for a domain controller DC, the apparatus comprising:

the third receiving module is used for receiving first image data sent by the electronic rearview mirror control unit, the first image data are acquired by the electronic rearview mirror control unit from one or more cameras and acquired by the one or more cameras, the first image data are used for being processed by the electronic rearview mirror control unit to obtain processed image data, and the processed image data are used for being output to display equipment corresponding to the electronic rearview mirror control unit by the electronic rearview mirror control unit.
An image processing apparatus, comprising:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to implement the method of any of claims 1-11 or the method of any of claims 12-21 when executing the instructions.
A non-transitory computer readable storage medium having stored thereon computer program instructions, which when executed by a processor, implement the method of any of claims 1-11 or the method of any of claims 12-21.
A computer program product comprising computer readable code, or a non-transitory computer readable storage medium carrying computer readable code, which when run in an electronic device, a processor in the electronic device performs the method of any one of claims 1-11, or performs the method of any one of claims 12-21.
An electronic rearview mirror system comprising: one or more cameras; display devices corresponding to the one or more cameras; and an electronic rear view mirror control unit for performing the method according to any one of claims 1-11.
A vehicle, characterized by comprising:

one or more cameras;

display devices corresponding to the one or more cameras;

electronic rear view mirror control unit for performing the method according to any one of claims 1-11;

domain controller DC for performing the method according to any of claims 12-21.
A vehicle, characterized by comprising:

the electronic rearview mirror system of claim 27; and

domain controller DC for performing the method according to any of claims 12-21.