CN115520100A

CN115520100A - Automobile electronic rearview mirror system and vehicle

Info

Publication number: CN115520100A
Application number: CN202211153961.7A
Authority: CN
Inventors: 喻超; 俞波
Original assignee: Beijing Binli Information Technology Co Ltd
Current assignee: Beijing Binli Information Technology Co Ltd
Priority date: 2022-09-21
Filing date: 2022-09-21
Publication date: 2022-12-27

Abstract

The application discloses automotive electronics rear-view mirror system and vehicle. The system comprises: a vision sensor for sensing an environment around the vehicle and generating image data; the electronic rearview mirror corresponding to the vision sensor comprises a display screen, and the display screen is used for displaying the image data; a memory for storing a computer program; a processor for performing the following steps when the computer program is executed by the processor: acquiring the image data; determining a target object in the image data and a semantic region of the target object; acquiring movement information of the target object relative to the vehicle, and determining a risk target in the target object; highlighting an outline and/or a region of the risk objective in the display screen based on the semantic region. The system and the vehicle can effectively integrate the display function of the electronic rearview mirror and the automobile auxiliary driving function, provide accurate and rich interactive information for the electronic rearview mirror, and improve the driving experience.

Description

Automobile electronic rearview mirror system and vehicle

Technical Field

The application relates to the field of automatically driven vehicles, in particular to an automotive electronic rearview mirror system and a vehicle.

Background

Conventional rearview mirrors for automobiles include an interior rearview mirror and a side rearview mirror, i.e., the mirror surface reflects light to allow a driver to observe the right back and side back of the vehicle. However, such conventional rear view mirror solutions have not been well integrated with advanced driver assistance or automatic driving functions. For example, the conventional lane-changing auxiliary function is only a small prompting area added to the rear-view mirror area on the basis of the conventional rear-view mirror scheme, and whether to prompt needs to be determined by another detecting device such as an ultrasonic radar to determine whether a vehicle exists in a nearby lane, that is, the lane-changing auxiliary function can only provide a "yes or no" prompt.

With the development of technology and regulations, there has been a trend to replace glass rearview mirrors with camera streaming media rearview mirrors. On the other hand, camera-based driving assistance systems are also becoming popular. The traditional whole-vehicle-factory solution is that different suppliers provide different modules, and the requirements of the streaming media rearview mirror are not exactly the same as the requirements of auxiliary driving. Therefore, the streaming media rearview mirror only realizes the function of electronic display, and the real function is not much different from the traditional rearview mirror.

Therefore, there is a need to provide an electronic rearview mirror for a vehicle to better integrate the advanced driving assistance function or the automatic driving function of the vehicle to provide more effective information for the driver.

Disclosure of Invention

In view of this, the automotive electronic rearview mirror system and the vehicle provided by the application can effectively realize the fusion between the display of the streaming media rearview mirror and the auxiliary driving function or the automatic driving function, and simultaneously provide more effective information for the driver, thereby improving the driving experience and the driving safety.

In order to solve the above technical problem, a first aspect of the present application provides an electronic rearview mirror system for an automobile, comprising: a vision sensor for sensing an environment around the vehicle and generating image data; the electronic rearview mirror corresponding to the vision sensor comprises a display screen, and the display screen is used for displaying the image data; a memory for storing a computer program; a processor for performing the following steps when the computer program is executed by the processor: acquiring the image data; determining a target object in the image data and a semantic region of the target object; acquiring movement information of the target object relative to the vehicle, and determining a risk target in the target object; highlighting an outline and/or a region of the risk objective in the display screen based on the semantic region.

According to a preferred embodiment of the present application, the electronic rear view mirror is configured to be mounted inside the vehicle and comprises an interior electronic rear view mirror and/or a side electronic rear view mirror.

According to a preferred embodiment of the present application, the determining the target object and the semantic region of the target object in the image data further includes: and performing semantic analysis on the image data to acquire the target object and the semantic area.

According to a preferred embodiment of the present application, the acquiring movement information of the target object relative to the vehicle further includes: determining the movement information based on a plurality of frames of images in time sequence in the image data; or determining the movement information based on multi-frame images at the same moment in the image data, wherein the vision sensor is a multi-view vision sensor; wherein the movement information comprises at least one of: distance data, velocity data, or motion trends.

According to a preferred embodiment of the present application, the laser radar corresponding to the vision sensor is configured to sense the surrounding environment of the vehicle and generate point cloud data; the determining the target object in the image data and the semantic region of the target object further includes: acquiring the point cloud data; performing semantic analysis on the point cloud data to determine the target object and a point cloud area of the target object; and determining a corresponding area of the point cloud area in the image data as the semantic area based on the calibration position relation between the laser radar and the visual sensor.

According to a preferred embodiment of the present application, the acquiring movement information of the target object relative to the vehicle further includes: acquiring the movement information based on the point cloud data; wherein the movement information comprises at least one of: distance data, velocity data, or motion trends.

According to a preferred embodiment of the present application, the determining a risk target in the target object further comprises: determining the target object as a risk target when the distance data is not greater than a distance threshold and/or when the speed data is not less than a speed threshold.

According to a preferred embodiment of the present application, said highlighting in said display screen the outline and/or area of said risk target further comprises: determining a risk level of the risk objective; determining a display level of the highlighting based on the risk level; the highlighting is performed using a display configuration of the display grade.

According to a preferred embodiment of the present application, the movement information of the risk target is displayed in a position in the display screen corresponding to the semantic area.

According to a preferred embodiment of the present application, said highlighting in said display screen the outline and/or area of said risk target further comprises: acquiring a driving state of the vehicle; determining a display type of the highlighting based on the driving status; the highlighting is performed using a display configuration of the display type.

According to a preferred embodiment of the present application, the running state of the vehicle includes at least one of: a high speed state, a lane change state, a turning state, or a parking state.

According to a preferred embodiment of the present application, the system further comprises: a rotation mechanism configured to connect with the vision sensor to adjust an orientation of the vision sensor; before the image data is acquired, the method further comprises: acquiring a running state of the vehicle; based on the driving state, controlling the rotation mechanism to adjust the vision sensor to a default orientation in the driving state.

In order to solve the above technical problem, a second aspect of the present application provides an electronic rearview mirror system for an automobile, comprising: a vision sensor for sensing an environment around the vehicle and generating image data; the electronic rearview mirror corresponding to the vision sensor comprises a display screen, and the display screen is used for displaying the image data; the display screen comprises a global display area, wherein the global display area comprises at least one risk target and a first display area corresponding to the risk target; the contour of the first display region matches the contour of the risk target in the image data.

According to a preferred embodiment of the present application, the global display area further includes a second display area; the second display area is configured to be disposed corresponding to the first display area and to display movement information of the risk target.

In order to solve the above technical problem, a third aspect of the present application provides a vehicle including the automotive electronic rearview mirror system as provided in the first or second aspect of the present application.

The application provides an automotive electronics rear-view mirror system and vehicle can enough realize the electronic display of streaming media rear-view mirror, also can realize the demand of driver assistance or autopilot function for the two can realize simultaneously through a set of hardware, and the conflict appears or can't match in whole car configuration in the two of avoiding. Meanwhile, richer and more effective prompt information can be provided for the driver, and the driving experience of the driver is greatly improved.

Drawings

In order to make the technical problems solved, technical means adopted and technical effects achieved by the embodiments of the present application clearer, specific embodiments of the present application will be described in detail below with reference to the accompanying drawings. It should be noted, however, that the drawings described below are only for exemplary embodiments of the present application and that other embodiments may be obtained by those skilled in the art without inventive faculty.

Fig. 1 and 2 are schematic system frames of an electronic rearview mirror system for an automobile according to the present application.

Fig. 3 is a schematic view of an application of an automotive electronic rearview mirror system according to the present application.

FIG. 4 is a flow chart of steps of a method performed by a processor of an automotive electronic rearview mirror system according to the present application.

FIG. 5 is a schematic view of a screen interface of another automotive electronic rearview mirror system according to the present application.

Detailed Description

Exemplary embodiments of the present application will now be described more fully hereinafter with reference to the accompanying drawings. The exemplary embodiments, however, may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these exemplary embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept to those skilled in the art. The same reference numerals denote the same or similar elements, components, or portions in the drawings, and thus, a repetitive description thereof will be omitted.

Features, structures, characteristics or other details described in a particular embodiment do not preclude the fact that the features, structures, characteristics or other details may be combined in any suitable manner in one or more other embodiments while remaining within the technical spirit of the embodiments of the present application.

In describing particular embodiments, the features, structures, characteristics or other details of the embodiments of the present application are described in order to provide a full understanding of the embodiments to those skilled in the art. One skilled in the relevant art will recognize, however, that the embodiments of the present application can be practiced without one or more of the specific features, structures, characteristics, or other details.

The flowcharts shown in the figures are illustrative only and do not necessarily include all of the contents and operations/steps, nor do they necessarily have to be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the actual execution sequence may be changed according to the actual situation.

The block diagrams shown in the figures are functional entities only and do not necessarily correspond to physically separate entities. I.e. these functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor means and/or microcontroller means.

It will be understood that, although the terms first, second, third, etc. may be used herein to describe various elements, components, or sections, these terms should not be construed as limiting. These phrases are used to distinguish one from another. For example, a first device may also be referred to as a second device without departing from the spirit of the embodiments of the present application.

The term "and/or" and/or "includes any and all combinations of one or more of the associated listed items.

Referring to fig. 1 and 2, an automotive electronic rearview mirror system 100 according to the present application is shown, including a processor 10, a memory (not shown), a vision sensor 20, and an electronic rearview mirror 30. The electronic rearview mirror system 100 for an automobile can be integrally arranged in a vehicle, namely, the electronic rearview mirror system is already taken as a part of the vehicle when being installed in the front of the factory; or can be arranged in the vehicle in a mode of adding and modifying as a set of vehicle after-loading systems. Thus, the processor 10 may be either an aftermarket separately provided processor or the vehicle's own computing processing device. It will be appreciated that the processor 10 may be various types of processing units, such as a separate computing chip or a domain controller of the vehicle, etc. As shown in fig. 1, in an exemplary embodiment, the vision sensor 20 may be disposed at a side rear view mirror of a vehicle to assist or replace a conventional side rear view mirror of the vehicle and sense a surrounding environment of the vehicle to generate image data; accordingly, the electronic rear view mirror 30 corresponds to the vision sensor 20, and is disposed in a vehicle inside area corresponding to the side rear view mirror for displaying the image data, i.e., the side electronic rear view mirror. It will be appreciated that the same arrangement of vision sensors and corresponding electronic mirrors may be provided at the side mirrors to the left of the vehicle in figure 1. Likewise, the interior rear view mirror of the vehicle may also be an electronic rear view mirror, i.e. an interior electronic rear view mirror, for displaying image data of the environment behind the vehicle; the corresponding vision sensor can be arranged at the tail part of the vehicle, or above the rear glass of the vehicle, and the like.

Further, referring to fig. 3, a scenario that the electronic rearview mirror system for automobile provided by the present application is applied to a vehicle is schematically shown. For example, the vision sensor 20 may be provided at a conventional side rear view mirror and replace the conventional side rear view mirror. Accordingly, an electronic rearview mirror 30 may be provided at a position corresponding to the vision sensor 20 on the vehicle inside. The electronic rearview mirror 30 includes a display screen for displaying image data of the vehicle side rear side sensed by the vision sensor 20. The processor and the memory in the system can be hidden and installed in the vehicle; or directly using the existing computing processing equipment of the vehicle and connecting with the electronic rearview mirror 30 and the vision sensor 20 through vehicle-mounted internal communication.

Referring to fig. 4, the processor 10 of the automotive electronic rearview mirror system 100 can execute the computer program stored in the memory and realize the following steps:

step S100: the image data is acquired.

The image data is perceptually generated by the vision sensor 20 and may be retrieved from the vision sensor 20 by the processor 10 via a communication link. The communication connection may be a point-to-point communication connection between the processor 10 and the vision sensor 20, or a communication connection using an in-vehicle internal communication protocol.

Step S200: determining a target object in the image data and a semantic region of the target object.

When the vehicle is traveling on the road, the image data generated by the vision sensor 20 is often included with other nearby vehicles, pedestrians, etc., and these other objects in the vicinity of the vehicle are often objects of great interest in vehicle intelligence. Therefore, the target objects can be determined by semantic recognition, and the corresponding semantic regions can be determined.

In one embodiment, the semantic recognition and other processing may be directly performed on the image data, and the target object and the semantic region of the target object in the image data may be directly acquired. For example, semantic segmentation, instance segmentation, or panorama segmentation may be performed on the image to determine various types of target objects in the image, such as vehicles, bicycles, pedestrians, and to determine semantic regions corresponding to the various target objects at the same time.

In another embodiment, the target object and the semantic area may also be determined by means of a lidar. At this time, the electronic rearview mirror system 100 for an automobile may further include a laser radar for sensing the surroundings of the vehicle and generating point cloud data. The lidar corresponds to the vision sensor 20, i.e. the lidar and the vision sensor have similar fields of view, e.g. the lidar is mounted beside the side electronic mirror and has the same orientation. Both having similar fields of view means that the lidar field of view is substantially covered by the vision sensor, or that the vision sensor field of view is substantially covered by the lidar, or that the fields of view of both have a large overlap overall. In this way, the lidar and the vision sensor may be considered to be sensing the same environmental region.

In this case, the target object and the semantic region corresponding to the target object may be determined by processing the point cloud data. Specifically, semantic analysis may be performed on the point cloud data, for example, in a form of cluster analysis or neural network processing, to determine that a certain part of the point cloud in the point cloud data belongs to a certain target object, and then, a point cloud area corresponding to the point cloud of the target object may be determined at the same time. Then, based on the calibration position relationship between the laser radar and the visual sensor and the respective internal references of the laser radar and the visual sensor, the target object and the point cloud area identified from the point cloud data can be corresponded to the image data of the visual sensor, so that the target object is correspondingly determined from the image data, and the area corresponding to the point cloud data is used as a semantic area.

In this embodiment, the laser radar is introduced to identify the target object and the semantic region thereof, and although the overall processing is more troublesome than directly processing the image data, the movement information required in the subsequent steps can be better provided due to the three-dimensional information of the point cloud data of the laser radar.

Step S300: and acquiring the movement information of the target object relative to the vehicle, and determining a risk target in the target object.

After the target object is determined in the foregoing steps, movement information of the target object relative to the vehicle may be further determined, so as to identify which moving objects are risk targets that need to be focused on according to the movement information.

In one embodiment, as described above, the target object and the semantic information may be obtained by performing semantic recognition or the like on the image data. And further processing the image data to obtain the movement information corresponding to the target object. Further, in one embodiment, the movement information of the target object is determined through a plurality of frames of images in time sequence in the image data, that is, the monocular distance measurement method is used for realizing the determination. In another embodiment, when the visual sensor is a multi-view visual sensor, the movement information of the target object is determined through multi-frame images at the same time in the image data and by combining the position calibration relationship among a plurality of cameras of the multi-view visual sensor, that is, the method is realized by using a binocular distance measurement method. Specifically, the movement information includes at least one of distance data, speed data, or a movement tendency, i.e., a distance, a speed of the target object with respect to the vehicle, or a movement tendency in which the target object is located, such as acceleration, deceleration, passing, or the like.

In another embodiment, the automotive electronic rearview mirror system can further include a lidar, as previously described. At this time, the point cloud data of the target object acquired by the laser radar includes three-dimensional information, through which the movement information of the target object may be directly acquired, and the movement information includes distance data and/or speed data.

After the movement information of the target objects is obtained, which target objects are risk targets can be determined. In particular, the target object is determined to be a risk target when the distance data is not greater than a distance threshold and/or when the speed data is not less than a speed threshold. The distance threshold and the speed threshold are determined in advance by human beings. The distance data of the target object is not greater than the distance threshold, indicating that the target object is very close to the vehicle, and the important attention should be paid to the target object no matter how fast the target object is. Since the vision sensor in the electronic rearview mirror system of the automobile usually detects the environment behind the vehicle, the speed of the target object is not less than the speed threshold value, which means that the target object usually approaches the vehicle quickly, but not moves away from the vehicle quickly, and the important attention should be paid. Further, if the environment behind the vehicle includes an opposite lane, and the vehicle in the opposite lane drives in an opposite direction, so that the speed data of the target object is too large, the target object in the opposite lane can be removed and is not taken as a risk target.

Step S400: highlighting an outline and/or a region of the risk objective in the display screen based on the semantic region.

Having determined the risk objective in the image data, the outline and/or the area of the risk objective may be highlighted in the display screen based on the semantic area that has been obtained. For example, when the risk target is another vehicle close to the side and the rear, the outline of the other vehicle is highlighted, that is, the outer edge of the outline of the vehicle is displayed in a red, yellow and highlighted manner; and highlighting the other vehicle region, i.e., coloring a portion of the vehicle in the image data for display. In this way, the situation of some risk targets around the driver can be effectively prompted. The highlighting provided by the application is determined based on semantic recognition of the risk target, so that the highlighting can move along with the movement of the target object, and a more accurate and intelligent prompting function can be provided for a driver.

The highlighting of risk targets has different forms. In one embodiment, in particular, the method comprises the following steps:

step S411: determining a risk level of the risk objective.

Step S412: determining a display level of the highlighting based on a risk level.

Step S413: the highlighting is performed using a display configuration of the display grade.

The risk level indicates how much the risk target around the vehicle affects the vehicle. For example, the risk level may be divided into three levels, and other vehicles that illustratively approach the adjacent lane quickly behind the vehicle may be considered high risk.

Further, a display level of the highlight display is determined according to the different risk levels, and the highlight display is performed using a display configuration of the display level. The display levels and corresponding display configurations may include different colors, different brightness, different modes, etc. For example, when the risk level is divided into three levels, the outline and/or the area may be highlighted using red, yellow, and green colors, respectively, to distinguish the different risk levels; it is also possible to use different luminances for distinction; different ways may also be used, such as flashing the highlighted outline and/or area when the risk level is high. The display configuration of the various display levels may be determined based on actual needs, which are not limited in this application.

Further, movement information of the risk target may also be displayed in the display screen at a position corresponding to the semantic region of the risk target. For example, a small area is additionally displayed beside the highlighted outline and/or area to display information of speed, distance, motion trend, etc. of the risk target.

In another embodiment, the risk target highlighting has a different form, and may further include the steps of:

step S421: and acquiring the running state of the vehicle.

Step S422: determining a display type of the highlighting based on the driving status.

Step S423: the highlighting is performed using a display configuration of the display type.

The running state of the vehicle includes at least one of a high speed state, a lane change state, a turning state, or a parking state. Specifically, the running state of the vehicle may be acquired from a central computing processing device or the like of the vehicle, and the display type of the highlight may be determined. For example, when the vehicle is in a high speed state, the vehicle is driving on an expressway or a highway, and the display type is adjusted to be suitable for the high speed type, for example, green is used for highlighting by default to avoid causing excessive influence on the driver; and when the vehicle is in the lane change state, the display type is adjusted to be suitable for the lane change type, such as the red is used by default for highlighting in an electronic rearview mirror at the lane change side to strongly remind a risk target. The display configurations of the various display types may be determined based on actual needs, which are not limited in this application.

The automobile electronic rearview mirror system can highlight the outline and/or the area of the risk target in the electronic rearview mirror, so that more accurate and rich information of the surrounding environment of a vehicle is provided for a driver, the driving assistance function or the automatic driving function of the vehicle is fused, and the driving experience is greatly improved.

In another embodiment, the present application provides an electronic rearview mirror system for a vehicle, further comprising a rotation mechanism configured to be connected to the vision sensor to adjust an orientation of the vision sensor. Further, before acquiring the image data, it is also possible to first acquire the traveling state of the vehicle and control the rotation mechanism based on the traveling state to adjust the visual sensor to the default orientation in the traveling state.

For example, when the vehicle is in a parking state, the rotating mechanism is controlled to adjust the orientation of the visual sensor to be biased downwards so as to provide more environmental information of the side, rear and lower parts of the vehicle; when the vehicle is in a turning state or a lane changing state, the rotating mechanism is controlled to follow the steering of the vehicle to rotate in the yaw direction, so that the orientation of the vision sensor can be adjusted to always face to the right back of the lane, and therefore the blind vision area which is right behind the side face of the vehicle is avoided when the vehicle turns or changes lanes.

Referring to fig. 5, the present application further provides another automotive electronic rearview mirror system, and particularly relates to a display interface. The electronic rearview mirror system for the automobile similarly comprises a processor, a memory, a vision sensor (none of the figures show), and a display screen 30, wherein the display screen 30 is used for displaying image data generated by perception of the vision sensor. The display screen 30 includes a global display area 301, which includes at least one risk target 200 and a first display area 302 corresponding to the risk target. Wherein the contour of the first display area 302 matches the contour of the risk target in the image data.

This automotive electronic rearview mirror system is capable of identifying the presence of a risk target 200 in the global display area 301 of the electronic rearview mirror 30, determining the outline and/or area of the risk target, and displaying the risk target using the first display area 302. Since the outline of the first display area 302 matches the risk target, when the risk target moves in the global display area 301, the first display area 302 moves similarly, and it is possible to effectively reflect what the risk target is. For example, when the risk target is a vehicle, the outline of the first display area 302 is the outline of the vehicle; when the risk target is a pedestrian, the outline of the first display area 302 is the outline of the pedestrian. The display of the first display area 302 may be an outline display, such as highlighting an outline without displaying the middle part; it may also be a regional display, such as semi-transparently displaying the middle portion and not displaying the surrounding outline.

In some embodiments, the global display area 301 also includes a second display area 303. The second display area 303 is disposed corresponding to the first display area 302, i.e., disposed beside the first display area 302. The second display area 303 is used to display movement information of the risk target, including distance data, velocity data, or a movement trend. Exemplarily, "30km/h" in fig. 5 represents the speed of the risk vehicle, "20m" represents the distance of the risk vehicle from the host vehicle, and "↓" represents the distance of the risk vehicle being reduced, i.e., the risk vehicle is approaching the host vehicle, in the overtaking tendency. It is understood that the second display area 303 may display only one type of movement information or simultaneously display a plurality of types of movement information, and may also use text to indicate the movement tendency of the risky vehicle, which is not limited in this application.

The application provides this electron rear-view mirror system can provide the display mode of the automobile electron rear-view mirror that content is more abundant, and this display mode is realized based on the technique of the driving assistance function or the autopilot function of vehicle itself, therefore can fuse the driving function of the application of automobile electron rear-view mirror and car effectively to provide better driving experience for the driver.

The application also provides a vehicle, which comprises the automobile electronic rearview mirror system provided by the application.

Those skilled in the art will appreciate that all or part of the steps for implementing the above-described embodiments are implemented as programs executed by data processing apparatuses (including computers), i.e., computer programs. When the computer program is executed, the method provided by the application can be realized. Also, the computer program may be stored in a computer readable storage medium, which may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a magnetic disk, an optical disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing such as a storage array of multiple storage media, e.g., a magnetic disk or tape storage array. The computer program, when executed by one or more data processing devices, enables the computer-readable storage medium to implement the above-described methods of the present application. Further, the storage medium is not limited to a centralized storage, but may be a distributed storage, such as a cloud storage based on cloud computing. It should be appreciated that in the foregoing description of exemplary embodiments of the present application, various features of the present application are sometimes described in a single embodiment or with reference to a single figure, in order to streamline the application and assist those skilled in the art in understanding various aspects of the present application. However, the present application should not be construed that the features included in the exemplary embodiments are all the essential technical features of the present patent claims.

Further, those skilled in the art will readily appreciate that the exemplary embodiments described herein may be implemented in software, or in combination with hardware as necessary. Therefore, the technical solution according to the embodiments of the present application may be embodied in the form of a software product, which may be stored in a computer-readable storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a data processing device (which may be a personal computer, a server, or a network device, etc.) to execute the above method according to the present application. The computer readable storage medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable storage medium may also be any readable storage medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing. Program code for carrying out operations of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the C language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).

Thus, the present application may be embodied as a method, system, electronic device, or computer-readable storage medium that executes a computer program. Some or all of the functions of the present application may be implemented in practice using general purpose data processing devices such as microprocessors or Digital Signal Processors (DSPs).

It should be understood that the modules, units, components, and the like included in the device of one embodiment of the present application may be adaptively changed to be provided in a device different from the embodiment. The different modules, units or components comprised by the apparatus of an embodiment may be combined into one module, unit or component or they may be divided into a plurality of sub-modules, sub-units or sub-components. The modules, units or components in the embodiments of the present application may be implemented in hardware, may be implemented in software running on one or more processors, or may be implemented in a combination thereof.

The above-mentioned embodiments are further described in detail for the purpose of illustrating the invention, and it should be understood that the above-mentioned embodiments are only illustrative of the present invention and are not intended to limit the present invention, and any modifications, equivalents, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

In summary, the present application may be implemented as a method, apparatus, system, or computer-readable storage medium that executes a computer program. Some or all of the functions of the present application may be implemented in practice using general purpose data processing devices such as microprocessors or Digital Signal Processors (DSPs). The foregoing detailed description of the embodiments, and the objects, technical solutions and advantages of the embodiments of the present application have been described in further detail, and it should be understood that the embodiments of the present application are not inherently related to any particular computer, virtual device or electronic system, and that various general-purpose devices may implement the embodiments of the present application. The above description is only exemplary of the embodiments of the present application and should not be construed as limiting the embodiments of the present application, and any modifications, equivalents, improvements, etc. made within the spirit and principle of the embodiments of the present application should be included in the scope of the embodiments of the present application.

Claims

1. An automotive electronic rearview mirror system, comprising:

a vision sensor for sensing an environment around the vehicle and generating image data;

the electronic rearview mirror corresponding to the vision sensor comprises a display screen, and the display screen is used for displaying the image data;

a memory for storing a computer program;

a processor for performing the following steps when the computer program is executed by the processor:

acquiring the image data;

determining a target object in the image data and a semantic region of the target object;

acquiring movement information of the target object relative to the vehicle, and determining a risk target in the target object;

highlighting an outline and/or a region of the risk objective in the display screen based on the semantic region.

2. The system of claim 1,

the electronic rearview mirror is configured to be mounted inside the vehicle and includes an interior electronic rearview mirror and/or a side electronic rearview mirror.

3. The system of claim 1, wherein the determining a target object and a semantic region of the target object in the image data further comprises:

and performing semantic analysis on the image data to acquire the target object and the semantic region.

4. The system of claim 3, wherein the obtaining movement information of the target object relative to the vehicle further comprises:

determining the movement information based on a plurality of frames of images in time sequence in the image data; or

Determining the movement information based on multi-frame images at the same moment in the image data, wherein the vision sensor is a multi-view vision sensor;

wherein the movement information comprises at least one of: distance data, velocity data, or motion trends.

5. The system of claim 1, further comprising:

the laser radar is corresponding to the vision sensor and used for sensing the surrounding environment of the vehicle and generating point cloud data;

the determining a target object in the image data and a semantic region of the target object further includes:

acquiring the point cloud data;

performing semantic analysis on the point cloud data to determine the target object and a point cloud area of the target object;

and determining a corresponding area of the point cloud area in the image data as the semantic area based on the calibration position relation between the laser radar and the visual sensor.

6. The system of claim 5, wherein said obtaining movement information of the target object relative to the vehicle further comprises:

acquiring the movement information based on the point cloud data;

7. The system of claim 4 or 6, wherein said determining a risk objective in said target object further comprises:

determining the target object as a risk target when the distance data is not greater than a distance threshold and/or when the speed data is not less than a speed threshold.

8. The system of claim 7, wherein said highlighting in said display screen an outline and/or area of said risk target further comprises:

determining a risk level of the risk objective;

determining a display level of the highlighting based on the risk level;

the highlighting is performed using a display configuration of the display grade.

9. The system of claim 8, further comprising:

displaying movement information of the risk target at a position in the display screen corresponding to the semantic area.

10. The system of claim 7, wherein said highlighting contours and/or regions of said risk targets in said display screen further comprises:

acquiring a driving state of the vehicle;

determining a display type of the highlighting based on the driving status;

the highlighting is performed using a display configuration of the display type.

11. The system of claim 10, wherein the driving state of the vehicle comprises at least one of: a high speed state, a lane change state, a turning state, or a parking state.

12. The system of claim 1, further comprising:

a rotation mechanism configured to connect with the vision sensor to adjust an orientation of the vision sensor; before the image data is acquired, the method further comprises:

acquiring a driving state of the vehicle;

based on the driving state, controlling the rotation mechanism to adjust the vision sensor to a default orientation in the driving state.

13. An automotive electronic rearview mirror system, comprising:

the display screen comprises a global display area, wherein the global display area comprises at least one risk target and a first display area corresponding to the risk target;

the contour of the first display region matches the contour of the risk target in the image data.

14. The system of claim 13, wherein the global display area further comprises a second display area;

the second display area is configured to be disposed corresponding to the first display area and to display movement information of the risk target.

15. A vehicle comprising an automotive electronic rearview mirror system as claimed in any one of claims 1 to 14.