CN112046470A - Method, device, medium, controller and vehicle for controlling display of rear view image - Google Patents

Method, device, medium, controller and vehicle for controlling display of rear view image Download PDF

Info

Publication number
CN112046470A
CN112046470A CN202010747148.7A CN202010747148A CN112046470A CN 112046470 A CN112046470 A CN 112046470A CN 202010747148 A CN202010747148 A CN 202010747148A CN 112046470 A CN112046470 A CN 112046470A
Authority
CN
China
Prior art keywords
vehicle
steering wheel
steering angle
rear view
view image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010747148.7A
Other languages
Chinese (zh)
Inventor
赫连晓会
田俊涛
金大鹏
师小五
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beiqi Foton Motor Co Ltd
Original Assignee
Beiqi Foton Motor Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beiqi Foton Motor Co Ltd filed Critical Beiqi Foton Motor Co Ltd
Priority to CN202010747148.7A priority Critical patent/CN112046470A/en
Publication of CN112046470A publication Critical patent/CN112046470A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W30/00—Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units, or advanced driver assistance systems for ensuring comfort, stability and safety or drive control systems for propelling or retarding the vehicle
    • B60W30/08—Active safety systems predicting or avoiding probable or impending collision or attempting to minimise its consequences
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
    • B60W2050/146—Display means

Abstract

The present disclosure relates to a method, an apparatus, a medium, a controller, and a vehicle for controlling display of a rear view image, the method applied to a vehicle, including: the method comprises the steps of obtaining a rear view image of the vehicle through a camera, obtaining a steering wheel steering angle of the vehicle, determining a target image from the rear view image according to the steering wheel steering angle, and displaying the target image. In other words, the target image to be displayed can be determined according to the steering angle of the steering wheel of the vehicle in the driving process of the vehicle, so that the visual field of the rear view image presented by the vehicle can be adaptively displayed according to the steering angle of the steering wheel, the visual field coverage of the vehicle body is reduced, and the accident potential can be reduced.

Description

Method, device, medium, controller and vehicle for controlling display of rear view image
Technical Field
The present disclosure relates to the field of vehicle technologies, and in particular, to a method, an apparatus, a medium, a controller, and a vehicle for controlling display of a rear view image.
Background
With the development of automobile technology, digital rearview mirrors are more and more popular among people. The digital rearview mirror is characterized in that a display screen is used for replacing a traditional reflector, the road condition behind a vehicle is shot in real time through a camera, and the shot image is displayed on the display screen.
However, in the process of turning the vehicle, because the included angle exists between the vehicle head and the vehicle body, the view imaging of the rearview mirror is blocked by the vehicle body, the original view range in the driving direction of the vehicle cannot be displayed, so that a driver cannot know the conditions around the vehicle body in real time, and the accident potential is caused.
Disclosure of Invention
In order to solve the above problems, the present disclosure provides a method, an apparatus, a medium, a controller, and a vehicle for controlling display of a rear view image.
In a first aspect, the present disclosure provides a method of controlling a rear view image display, applied to a vehicle, the method comprising: acquiring a rear view image of the vehicle through a camera; acquiring a steering angle of a steering wheel of the vehicle; determining a target image from the rear view image according to the steering angle of the steering wheel; and displaying the target image.
Optionally, the determining a target image from the rear view image according to the steering wheel steering angle includes: determining a target area from a picture corresponding to the rear view image according to the steering angle of the steering wheel; and taking the rear view image corresponding to the target area as the target image.
Optionally, the determining a target area from a frame corresponding to the rear-view image according to the steering angle of the steering wheel includes: according to preset resolution and position information of the camera on the vehicle, determining an initial area from a picture corresponding to the rear view image, wherein the initial area is an area displayed by the vehicle in a straight-ahead state; and determining the target area according to the steering angle of the steering wheel and the initial area.
Optionally, the determining the target region according to the steering wheel steering angle and the initial region includes: taking the initial region as the target region when the steering angle of the steering wheel is smaller than a preset steering angle; or, when the steering wheel steering angle is greater than or equal to the preset steering angle, determining the number of pixels to be moved and the moving direction according to the steering wheel steering angle, moving the initial region according to the number of pixels and the moving direction, and taking the region reached by the initial region as the target region.
Optionally, the determining the number of pixels to be moved and the moving direction according to the steering wheel steering angle includes: acquiring the number of pixels to be moved corresponding to the steering wheel steering angle through a preset pixel incidence relation, wherein the pixel incidence relation comprises the corresponding relation between the steering wheel steering angle and the number of pixels; and determining the turning direction of the vehicle according to the steering wheel steering angle, and taking the turning direction as the moving direction.
Optionally, before the moving the initial region according to the number of pixels and the moving direction, the method further includes: acquiring the steering wheel steering speed of the vehicle and the running speed of the vehicle; determining the moving speed of the initial area according to the steering speed of the steering wheel and the driving speed; the moving the initial region according to the number of pixels and the moving direction includes: and moving the initial area according to the moving speed according to the number of the pixels and the moving direction.
In a second aspect, the present disclosure provides an apparatus for controlling display of a rear view image, applied to a vehicle, the apparatus comprising: the image acquisition module is used for acquiring a rear view image of the vehicle through a camera; the angle acquisition module is used for acquiring the steering angle of a steering wheel of the vehicle; the image determining module is used for determining a target image from the rear view image according to the steering angle of the steering wheel; and the image display module is used for displaying the target image.
Optionally, the image determining module is specifically configured to: determining a target area from a picture corresponding to the rear view image according to the steering angle of the steering wheel; and taking the rear view image corresponding to the target area as the target image.
Optionally, the image determining module is further configured to: according to preset resolution and position information of the camera on the vehicle, determining an initial area from a picture corresponding to the rear view image, wherein the initial area is an area displayed by the vehicle in a straight-ahead state; and determining the target area according to the steering angle of the steering wheel and the initial area.
Optionally, the image determining module is further configured to: taking the initial region as the target region when the steering angle of the steering wheel is smaller than a preset steering angle; or, when the steering wheel steering angle is greater than or equal to the preset steering angle, determining the number of pixels to be moved and the moving direction according to the steering wheel steering angle, moving the initial region according to the number of pixels and the moving direction, and taking the region reached by the initial region as the target region.
Optionally, the image determining module is further configured to: acquiring the number of pixels to be moved corresponding to the steering wheel steering angle through a preset pixel incidence relation, wherein the pixel incidence relation comprises the corresponding relation between the steering wheel steering angle and the number of pixels; and determining the turning direction of the vehicle according to the steering wheel steering angle, and taking the turning direction as the moving direction.
Optionally, the apparatus further comprises: the speed acquisition module is used for acquiring the steering wheel steering speed of the vehicle and the running speed of the vehicle; the speed determining module is used for determining the moving speed of the initial area according to the steering speed of the steering wheel and the driving speed; the image determination module is further configured to: and moving the initial area according to the moving speed according to the number of the pixels and the moving direction.
In a third aspect, the present disclosure provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the method of the first aspect of the present disclosure.
In a fourth aspect, the present disclosure provides a controller comprising: a memory having a computer program stored thereon; a processor for executing the computer program in the memory to implement the steps of the method of the first aspect of the disclosure.
In a fifth aspect, the present disclosure provides a vehicle including the controller of the fourth aspect of the present disclosure.
According to the technical scheme, the rearview image of the vehicle is acquired through the camera, the steering angle of the steering wheel of the vehicle is acquired, the target image is determined from the rearview image according to the steering angle of the steering wheel, and the target image is displayed. In other words, the target image to be displayed can be determined according to the steering angle of the steering wheel of the vehicle in the driving process of the vehicle, so that the visual field of the rear view image presented by the vehicle can be adaptively displayed according to the steering angle of the steering wheel, the visual field coverage of the vehicle body is reduced, and the accident potential can be reduced.
Additional features and advantages of the disclosure will be set forth in the detailed description which follows.
Drawings
The accompanying drawings, which are included to provide a further understanding of the disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the description serve to explain the disclosure without limiting the disclosure. In the drawings:
fig. 1 is a flowchart of a method of controlling a rear view image display provided by an embodiment of the present disclosure;
FIG. 2 is a schematic illustration of a vehicle field of view provided by an embodiment of the present disclosure;
FIG. 3 is a flow chart of another method of controlling a rear view image display provided by an embodiment of the present disclosure;
FIG. 4 is a schematic diagram illustrating the connections of a system for controlling the display of a rear view image according to an embodiment of the present disclosure;
fig. 5 is a schematic diagram of region division provided by the embodiment of the present disclosure;
FIG. 6 is a schematic diagram of a region movement provided by an embodiment of the present disclosure;
FIG. 7 is a schematic view of another region movement provided by embodiments of the present disclosure;
fig. 8 is a schematic structural diagram of an apparatus for controlling a rear view image display according to an embodiment of the present disclosure;
FIG. 9 is a schematic structural diagram of another apparatus for controlling a rear view image display provided in an embodiment of the present disclosure;
FIG. 10 is a block diagram of a controller provided by embodiments of the present disclosure;
fig. 11 is a block diagram of a vehicle according to an embodiment of the present disclosure.
Detailed Description
The following detailed description of specific embodiments of the present disclosure is provided in connection with the accompanying drawings. It should be understood that the detailed description and specific examples, while indicating the present disclosure, are given by way of illustration and explanation only, not limitation.
In the description that follows, the terms "first," "second," and the like are used for descriptive purposes only and are not intended to indicate or imply relative importance nor order to be construed.
First, an application scenario of the present disclosure will be explained. The visual mirror type monitoring system can be applied to various vehicles such as trucks and cars, the types of the visual mirrors on the vehicles are diversified in order to solve the problem of the blind area of the vehicle, and the monitoring of the surrounding environment of the whole vehicle is particularly important. The traditional rearview mirror has limited angle of view coverage and complex structure, so that an electronic rearview mirror product formed by a camera, a display screen and the like can be produced at any time, and pictures shot by the camera can be displayed through the display screen.
However, the cameras of the electronic rearview mirrors are fixedly mounted on the left side and the right side of the vehicle, the positions of the cameras cannot be adjusted adaptively, and the vehicle turns, so that the included angle between the vehicle head and the vehicle body can cause the visual field of the cameras to be blocked by the vehicle body, and the visual field of the cameras is reduced. For example, in the process of straight-ahead driving of the vehicle, the picture shot by the camera can be a view range 15 meters away from the vehicle body, and after the vehicle turns, the picture shot by the camera can only include the view range 10 meters away from the vehicle body, so that the view range in the original driving direction of the vehicle cannot be displayed, a driver cannot know the situation around the vehicle body in real time, and the hidden danger of accidents is caused.
In order to solve the above problems, the present disclosure provides a method, an apparatus, a medium, a controller, and a vehicle for controlling display of a rear view image, in which a target image to be displayed is determined from a rear view image captured by a camera according to a steering angle of a steering wheel of the vehicle during driving of the vehicle, so that a view of the rear view image presented by the vehicle can be adaptively displayed according to the steering angle of the steering wheel, thereby reducing coverage of the view of a vehicle body and reducing potential accidents.
The present disclosure is described below with reference to specific examples.
Fig. 1 is a flowchart of a method for controlling a rear view image display, which is applied to a vehicle according to an embodiment of the present disclosure. As shown in fig. 1, the method includes:
s101, acquiring a rear view image of the vehicle through the camera.
Wherein the camera may comprise a left camera installed on the left side of the vehicle and a right camera installed on the right side of the vehicle, for example, the left camera may be installed at the position of a conventional left rearview mirror, and the right camera may be installed at the position of a conventional right rearview mirror. Fig. 2 is a schematic view of a vehicle field of view provided by an embodiment of the present disclosure, as shown in fig. 2, the left camera is used for acquiring a left rear view image of the left side of the vehicle, the right camera is used for acquiring a right rear view image of the right side of the vehicle, and the left rear view image and the right rear view image correspond to a second-type field of view and a fourth-type field of view in a vehicle GB15084 (motor vehicle indirect view device performance and installation requirements).
It should be noted that the camera may further include a front camera mounted right in front of the vehicle for acquiring a front view image of the front vision blind zone of the vehicle, and a front right camera mounted right in front of the vehicle for acquiring a front right view image of the front right vision blind zone of the vehicle, where the front view image corresponds to six types of vision in the GB15084 requirement, and the front right view image corresponds to five types of vision in the GB15084 requirement. The present disclosure does not limit the position where the cameras are installed, the number of cameras, and the type of camera. The front camera and the right front camera are similar to the left camera and the right camera in processing mode, so the rear view image corresponding to the left camera and the right camera is taken as an example in the disclosure to explain the scheme of the disclosure.
In this step, after the ignition of the vehicle is started, a start instruction may be sent to the camera, and the camera starts to acquire a rear view image of the vehicle after receiving the start instruction.
And S102, acquiring the steering angle of the steering wheel of the vehicle.
In this step, the steering angle of the steering wheel of the vehicle may be obtained through a CAN (Controller Area Network) bus of the vehicle. Here, the steering wheel steering angle of the vehicle may be periodically acquired, and the period for acquiring the steering wheel steering angle may be 20ms, for example.
And S103, determining a target image from the rear-view image according to the steering angle of the steering wheel.
The target image comprises an image which is displayed on a display screen and is used for representing the rear view field of the vehicle, and a driver can observe the surrounding environment of the vehicle through the target image.
In this step, since the field of view corresponding to the rear view image of the vehicle captured by the camera is large, the driver does not need to observe the entire field of view, and the large field of view also affects the concentration of the driver, after the rear view image of the vehicle is acquired, the target image corresponding to a part of the field of view can be determined from the rear view image, and only the target image can be displayed during the driving of the vehicle, so that the driver can observe the surrounding environment of the vehicle according to the target image. For example, during the straight-ahead running of the vehicle, the right camera of the vehicle can shoot the surrounding environment including part of the vehicle body and within 15 meters from the vehicle body, but during the running of the vehicle, only the obstacle within 10 meters from the vehicle can interfere with the running of the vehicle, so that after the rear view image of the vehicle is acquired, the target image within 10 meters from the vehicle body can be determined from the rear view image.
In addition, when the vehicle turns, the vehicle head and the vehicle body are not in the same straight line, so that the visual field range shot by the camera changes, for example, for the right camera of the vehicle, if the vehicle can shoot the surrounding environment from the vehicle body to the range of forming an angle of 90 degrees with the vehicle body in the straight process, when the vehicle turns to the right for 45 degrees, the right camera can shoot more vehicle body parts, and can only shoot the surrounding environment from the vehicle body to the range of forming an angle of 45 degrees with the vehicle body, so that the effective visual field range is reduced, and the driver cannot observe more surrounding environments, thereby increasing the potential safety hazard.
For the above reasons, after the rear view image of the vehicle is acquired and the steering wheel steering angle of the vehicle is acquired, the target image most suitable for the driver to observe the surrounding environment can be determined from the rear view image of the vehicle according to the steering wheel steering angle. In a possible implementation manner, a target area may be determined from a picture corresponding to the rear view image according to the steering angle of the steering wheel, and the rear view image corresponding to the target area is taken as the target image. When the target area is determined, a preset area association relationship may be obtained, where the area association relationship may include a correspondence between the steering angle of the steering wheel and the target area. After the rear-view image and the steering angle of the steering wheel are obtained, the target area can be determined through the area association relation according to the steering angle of the steering wheel. For example, the target region may include a middle portion of the rear view image in a case where the vehicle is in a straight traveling state, and the target region may include a rightmost portion of the rear view image in a case where the vehicle is in a 45-degree turn.
It should be noted that the area association relationship may be obtained according to a test, and the area association relationships corresponding to vehicles of different models may be different. For example, the position of the target region corresponding to each steering angle of 1 degree in the rear view image may be determined, or the position of the target region corresponding to the steering angle of the steering wheel in the rear view image may be determined at intervals of 5 degrees, for example, when the steering angle of the steering wheel is 0 degree, the position of the target region corresponding to the steering angle of the steering wheel in the rear view image is determined, when the steering angle of the steering wheel is 5 degrees, the position of the target region corresponding to the steering angle of the steering wheel in the rear view image is determined, when the steering angle of the steering wheel is 10 degrees, the position of the target region corresponding to the steering angle of the steering wheel in the rear view image is determined, and so on, the position of the target region corresponding to the maximum steering angle of the steering wheel in the rear view image is finally determined. The interval of the steering angle of the steering wheel is not limited in the present disclosure.
And S104, displaying the target image.
In this step, after the target image is determined, the target image may be displayed through a display screen so that the driver of the vehicle can observe the surroundings of the vehicle through the target image.
By adopting the method, the target image to be displayed can be determined from the rear-view image shot by the camera according to the steering angle of the steering wheel of the vehicle, so that the visual field of the rear-view image presented by the vehicle can be adaptively displayed according to the steering angle of the steering wheel, the visual field coverage of the vehicle body is reduced, and the accident potential can be reduced.
Fig. 3 is a flowchart of another method for controlling a rear view image display according to an embodiment of the present disclosure. As shown in fig. 3, the method includes:
s301, acquiring a rear view image of the vehicle through the camera.
It should be noted that fig. 4 is a connection schematic diagram of a system for controlling rear view image display provided in an embodiment of the present disclosure, as shown in fig. 4, the system for controlling rear view image display may include a left camera, a left controller, a left display screen, a right camera, a right controller, and a right display screen, where the left controller is connected to the left camera and the left display screen, the right controller is connected to the right camera and the right display screen, and the left controller and the right controller are connected to a CAN bus of the vehicle. The connection mode in the system for controlling the display of the rear view image is only an example, the system for controlling the display of the rear view image may also include only one camera controller, and the left camera and the right camera are controlled by the camera controller to acquire the rear view image of the vehicle. In addition, the size of this left display screen and this right display screen can be different, and for example, this left display screen can be 12.3 cun, and this right display screen can be 9 cun, and the size of this left display screen and this right display screen also can be the same, and for example, this left display screen and this right display screen all can be 12.3 cun, and this disclosure does not do the restriction to the size of this display screen.
In this step, after the ignition of the vehicle is started, the right controller of the vehicle may control the right camera to acquire a right rear view image of the vehicle, and the left controller of the vehicle may control the left camera to acquire a left rear view image of the vehicle. Since the left and right rear-view images are processed in a similar manner in the present disclosure, the left and right rear-view images are not distinguished in the following steps.
And S302, acquiring the steering angle of the steering wheel of the vehicle.
In this step, since there may be signal interference when the steering wheel steering angle signal is acquired from the CAN bus, resulting in a deviation of the acquired steering wheel steering angle, after the steering wheel steering angle of the vehicle is acquired, the steering wheel steering angle may be filtered through a linear kalman filter algorithm or a band pass filter algorithm, and the steering wheel corner angle may be smoothed through a correlation technique, so that a more accurate steering wheel steering angle may be acquired.
And S303, determining an initial area from a picture corresponding to the rear view image according to the preset resolution and the position information of the camera on the vehicle.
Wherein the preset resolution may be a resolution of a display screen displaying the target image, and the initial area may be an area where the vehicle is displayed in a straight-ahead state.
In this step, the positions where the cameras are installed may be different for different types of vehicles, and the viewing ranges of the rear view images captured by the cameras at different positions are also different, and the viewing ranges at least include the ranges corresponding to the second type of viewing and the fourth type of viewing in the GB15084 requirements. Here, the field of view of the rear view image acquired by the camera, which may be a distance between a left edge or a right edge of the rear view image and the vehicle body of the vehicle, may be determined according to the position information of the camera on the vehicle in a case where the vehicle is in a straight traveling state. For example, the visual field range corresponding to the camera can be determined according to the position information of the left edge and the right edge of the rearview image shot by the camera under the condition that the vehicle is in a static state and the head and the body of the vehicle are in a straight line.
Further, after the view range corresponding to the camera is determined, the preset resolution may be obtained, and according to the preset resolution, an initial region may be determined from the picture corresponding to the back view image, where the view range corresponding to the initial region may be a range corresponding to a second type view and a fourth type view in the GB15084 requirement. Because the rear-view mirror of vehicle includes two parts, to commercial car (freight train), the rear-view mirror includes main rear-view mirror and wide angle rear-view mirror, and to the car, the rear-view mirror passes through the vertical line and divides this rear-view mirror region into two parts, and the vertical line outside is used for showing the environmental information far away from the vehicle, and the vertical line inboard is used for showing the environmental information that is close apart from the vehicle, consequently, when showing this target image through the display screen, also can show this target image according to two parts.
Illustratively, taking a commercial vehicle as an example, the display screen of the vehicle can comprise two parts, namely a part A and a part B, wherein the part A corresponds to a second type of visual field in the GB15084 requirement, and the part B corresponds to a fourth type of visual field in the GB15084 requirement. If the resolution of the display screen of the vehicle is 720 × 1920, the resolution corresponding to the part a may be 720 × 1280, and the resolution corresponding to the part B may be 720 × 640. In determining the initial region, a first initial region corresponding to the part a and a second initial region corresponding to the part B may be determined, respectively. Since the range of the two types of views corresponding to the part a is small, here, when the first initial region is determined, only the first initial region of 360 × 640 size may be determined, and then the first initial region is stretched to obtain the region of 720 × 1280. The range of the four types of views corresponding to the part B is larger, and the second initial region of 720 x 640 can be directly determined. Fig. 5 is a schematic diagram of region division provided by the embodiment of the present disclosure, as shown in fig. 5, a maximum region is a picture corresponding to the back-view image, a narrower region is a first initial region corresponding to the part a, and a wider region is a second initial region corresponding to the part B.
And S304, determining the target area according to the steering angle of the steering wheel and the initial area.
In this step, after determining the initial region, a preset steering angle may be obtained, and in the case that it is determined that the steering angle of the steering wheel is smaller than the preset steering angle, step S305 is performed; if it is determined that the steering wheel steering angle is greater than or equal to the preset steering angle, steps S306 to S309 are performed.
In a possible implementation manner, the preset steering angle may be preset according to a change of the view range corresponding to the initial region, for example, the preset steering angle may be an angle of a second type view and a fourth type view that the initial region meets requirements of GB15084, for example, if the steering angle of the steering wheel is 5 degrees, the view range corresponding to the initial region may meet requirements of GB15084, and if the steering angle of the steering wheel is 6 degrees, the view range corresponding to the initial region does not meet requirements of GB15084, the preset steering angle may be 5 degrees.
In another possible implementation, the preset steering angle may be determined according to a fastest rotation speed and a persistence time of vision of the vehicle. For example, the fastest rotational speed of the vehicle may be determined by equation (1):
formula (1): rf＝(Rw*360)/T
Wherein R isfAt the fastest speed of the vehicle, RwIs the fastest rotation speed of the steering wheel of the vehicle, and T is the steering system transmission ratio of the vehicle。
For example, if the fastest speed of the steering wheel of the vehicle is 1.5 revolutions per second and the steering gear ratio is 20, then the fastest speed of the vehicle can be calculated to be 0.027 degrees per millisecond. According to conventional experience, the one-time persistence time of a person is 100 milliseconds to 400 milliseconds, the disclosure takes 100 milliseconds as an example, and the locomotive can rotate 2.7 degrees when the vehicle rotates at the fastest rotating speed within the one-time persistence time of the driver. In consideration of communication delay, mechanical inertia and response errors, the actual turning angle of the vehicle head is smaller than 2 degrees, so that the image to be updated can be extracted once every 2.5 degrees, and after the image to be updated is extracted twice, the image displayed on the display screen is updated according to the image to be updated, so that the preset turning angle can be determined to be 5 degrees.
And S305, taking the initial area as the target area.
In this step, when the steering angle of the steering wheel is smaller than the preset steering angle, it indicates that the vehicle is in a straight-ahead state, and the view range corresponding to the initial region is unchanged; or the vehicle has a small rotation range, and the change of the visual field range does not influence the observation of the surrounding environment by the driver. In this case, the initial region may be regarded as the target region.
S306, determining the number of pixels to be moved and the moving direction according to the steering wheel steering angle.
In this step, when the steering angle of the steering wheel is greater than or equal to the preset steering angle, it indicates that the view range corresponding to the initial area is changed greatly, which may affect the driver's view of the vehicle surroundings, or the view range corresponding to the initial area does not satisfy the requirement of GB15084 for the view range. In this case, the target area needs to be re-determined according to the steering angle of the steering wheel, so that the rear view image corresponding to the initial area can be updated according to the new rear view image corresponding to the target area, so that the driver can know the situation around the vehicle body in real time, and accidents are reduced.
When the screen displayed on the display screen is updated, if the rear view image corresponding to the initial area is directly updated to the rear view image corresponding to the target area, the screen may not be smooth. For example, when the vehicle turns faster, the rear view image corresponding to the target area is completely different from the rear view image corresponding to the initial area, which may cause screen distortion and affect the judgment of the driver.
In order to solve the problem of poor image quality, the target area may be determined by moving the initial area, and in a possible implementation manner, the number of pixels to be moved corresponding to the steering angle of the steering wheel may be obtained through a preset pixel association relationship, where the pixel association relationship includes a correspondence relationship between the steering angle of the steering wheel and the number of pixels, and the turning direction of the vehicle is determined according to the steering angle of the steering wheel, and the turning direction is taken as the moving direction.
The pixel association relationship may be obtained through a test, for example, each degree of the steering angle may correspond to 8 pixels, and for vehicles of different models, the pixel association relationship may also be different, which is not limited in the present disclosure.
After obtaining the steering wheel steering angle of the vehicle, the pixel data to be moved may be determined according to the steering wheel steering angle, for example, if the steering wheel steering angle is 5 degrees, the number of pixels to be moved may be determined to be 40, and if the steering wheel steering angle is 45 degrees, the number of pixels to be moved may be determined to be 360. Thereafter, the turning direction of the vehicle may also be determined based on the steering wheel steering angle, and the moving direction may be determined to be right when it is determined that the vehicle is turning to the right, and may be determined to be left when it is determined that the vehicle is turning to the left.
The turning direction of the vehicle may be determined by a steering angle sensor, and the manner of determining the turning direction of the vehicle is not limited in the present disclosure.
And S307, acquiring the steering wheel steering speed of the vehicle and the running speed of the vehicle.
When the target area is determined by moving the initial area, if the moving speed is too fast, the area staying time between the initial area and the target area is too short, and the screen switching is too fast, which may also cause the screen to be unsmooth. In this case, the moving rate of the initial area may be determined according to the steering wheel turning rate of the vehicle and the traveling speed of the vehicle, and in the case where the steering wheel turning rate and the traveling speed of the vehicle are fast, a fast moving rate may be set, and in the case where the steering wheel turning rate and the traveling speed of the vehicle are slow, a slow moving rate may be set.
In this step, the steering wheel steering rate of the vehicle and the traveling speed of the vehicle may be acquired through the CAN bus of the vehicle.
And S308, determining the moving speed of the initial area according to the steering wheel steering speed and the running speed.
In this step, after acquiring the steering wheel steering rate of the vehicle and the traveling speed of the vehicle, the moving rate of the initial zone may be determined by equation (2):
formula (2) P ═ a ═ w + b ═ v
Where P is the moving speed of the initial region, w is the steering wheel steering speed of the vehicle, v is the traveling speed of the vehicle, a is a steering speed coefficient, and b is a traveling speed coefficient. a and b may be preset and obtained through experimental tests.
S309, moving the initial area according to the moving speed according to the number of the pixels and the moving direction, and taking the area reached by the initial area as the target area.
In the case where the display screen of the vehicle includes two portions a and B, the visual field range corresponding to the second initial region corresponding to the portion B is large, and the change in the visual field range corresponding to the second initial region has little influence on the driver during turning of the vehicle, so that the target region of the portion B may not be updated during traveling of the vehicle, and only the first initial region may be moved during turning of the vehicle.
In this step, after determining the number of pixels and the moving direction, the initial area may be moved according to the moving rate in the picture corresponding to the rear-view image. Illustratively, fig. 6 is a schematic diagram of a region shift provided by the embodiment of the present disclosure, the first initial region corresponding to the part a is shifted on the basis of the region division shown in fig. 5, as shown in fig. 6, when the steering angle of the steering wheel is 5 degrees, the corresponding number of pixels to be shifted is 40, the region shown by the dotted line is the first initial region corresponding to the part a, and the region shown by the solid line is the first target region that the first initial region finally reaches. Fig. 7 is another schematic diagram of area shifting provided by the embodiment of the present disclosure, where a first initial area corresponding to the part a is shifted based on the area division shown in fig. 5, as shown in fig. 7, when the steering angle of the steering wheel is 45 degrees, the corresponding number of pixels to be shifted is 360, an area shown by a dotted line is the first initial area corresponding to the part a, and an area shown by a solid line is a first target area that the first initial area finally reaches.
And S310, taking the rear view image corresponding to the target area as the target image.
In this step, in the process of moving the initial region, an image corresponding to a region where the initial region arrives is taken as the first target image, and then the first target image and the second target image may be stitched, where the stitching manner of the first target image and the second target image may be determined according to the type of the vehicle. For example, for a commercial vehicle, the first target image and the second target image may be stitched up and down, for example, the first target image is on the top, and the second target image is on the bottom; for a car, the first target image and the second target image may be stitched in a left-right stitching manner, for example, the first target image is on the left side, and the second target image is on the right side.
And S311, displaying the target image.
In this step, after the target image is determined, the target image may be sent to a display screen, and the target image may be displayed through the display screen.
It should be noted that, in the case that the vehicle is turned off or is stopped for a long time, for example, for half an hour, the display screen may be controlled to enter the standby state, so that the energy consumption of the vehicle may be saved.
By adopting the method, after the rear view image of the vehicle and the steering wheel steering angle of the vehicle are obtained, the initial area can be determined from the image corresponding to the rear view image according to the preset resolution and the position information of the camera on the vehicle, the target area is determined according to the steering wheel steering angle and the initial area, the rear view image corresponding to the target area is used as the target image, and the target image is displayed. Therefore, the visual field of the rear view image presented by the vehicle can be adaptively displayed according to the steering angle of the steering wheel, the visual field coverage of the vehicle body is reduced, and the accident potential can be reduced. In addition, when the target region is determined, the initial region may be moved according to the number of pixels to be moved, the moving direction, and the moving rate, so that smooth display of the rear view image may be achieved.
Fig. 8 is a schematic structural diagram of an apparatus for controlling a rear view image display according to an embodiment of the present disclosure, which is applied to a vehicle. As shown in fig. 8, the apparatus includes:
an image obtaining module 801, configured to obtain a rear view image of the vehicle through a camera;
an angle obtaining module 802, configured to obtain a steering angle of a steering wheel of the vehicle;
an image determining module 803, configured to determine a target image from the rear-view image according to the steering angle of the steering wheel;
an image display module 804, configured to display the target image.
Optionally, the image determining module 803 is specifically configured to: determining a target area from a picture corresponding to the rear view image according to the steering angle of the steering wheel; and taking the rear view image corresponding to the target area as the target image.
Optionally, the image determining module 803 is further configured to: according to the preset resolution and the position information of the camera on the vehicle, determining an initial area from a picture corresponding to the rear view image, wherein the initial area is an area displayed by the vehicle in a straight-ahead state; and determining the target area according to the steering angle of the steering wheel and the initial area.
Optionally, the image determining module 803 is further configured to: taking the initial region as the target region under the condition that the steering angle of the steering wheel is smaller than a preset steering angle; or, when the steering wheel steering angle is greater than or equal to the preset steering angle, determining the number of pixels to be moved and the moving direction according to the steering wheel steering angle, moving the initial region according to the number of pixels and the moving direction, and taking the region reached by the initial region as the target region.
Optionally, the image determining module 803 is further configured to: acquiring the number of the pixels to be moved corresponding to the steering wheel steering angle through a preset pixel incidence relation, wherein the pixel incidence relation comprises the corresponding relation between the steering wheel steering angle and the number of the pixels; the turning direction of the vehicle is determined according to the steering wheel steering angle, and the turning direction is taken as the moving direction.
Optionally, fig. 9 is a schematic structural diagram of another apparatus for controlling display of a rear view image according to an embodiment of the present disclosure, and as shown in fig. 9, the apparatus further includes:
a speed acquisition module 805 for acquiring a steering wheel steering speed of the vehicle and a running speed of the vehicle;
a speed determination module 806, configured to determine a moving speed of the initial area according to the steering wheel steering speed and the driving speed;
the image determining module 803 is further configured to: the initial area is moved according to the moving rate based on the number of pixels and the moving direction.
By the aid of the device, the target image to be displayed can be determined from the rear-view image shot by the camera according to the steering angle of the steering wheel of the vehicle, so that the view of the rear-view image presented by the vehicle can be adaptively displayed according to the steering angle of the steering wheel, the view coverage of the vehicle body is reduced, and accident potential can be reduced.
With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
Fig. 10 is a block diagram of a controller 1000 provided by an embodiment of the present disclosure. For example, the controller 1000 may be provided as a server. Referring to fig. 10, the controller 1000 includes a processor 1022, which may be one or more in number, and a memory 1032 for storing computer programs executable by the processor 1022. The computer programs stored in memory 1032 may include one or more modules that each correspond to a set of instructions. Further, the processor 1022 may be configured to execute the computer program to perform the above-described method of controlling the rear view image display.
Additionally, the controller 1000 may also include a power component 1026 and a communication component 1050, the power component 1026 may be configured to perform power management of the controller 1000, and the communication component 1050 may be configured to enable communication, e.g., wired or wireless communication, of the controller 1000. In addition, the controller 1000 may also include an input/output (I/O) interface 1058. The controller 1000 may operate based on an operating system stored in memory 1032, such as Windows Server, Mac OS XTM, UnixTM, Linux, and the like.
In another exemplary embodiment, there is also provided a computer readable storage medium including program instructions which, when executed by a processor, implement the steps of the above-described method of controlling a rear view image display. For example, the computer readable storage medium may be the memory 1032 comprising program instructions executable by the processor 1022 of the controller 1000 to perform the method of controlling the display of the rear view image described above.
In another exemplary embodiment, a computer program product is also provided, which comprises a computer program executable by a programmable apparatus, the computer program having code portions for performing the above-mentioned method of controlling a display of a rear view image when executed by the programmable apparatus.
Fig. 11 is a block diagram of a vehicle provided in an embodiment of the present disclosure, and as shown in fig. 11, the vehicle includes the above controller.
The preferred embodiments of the present disclosure are described in detail with reference to the accompanying drawings, however, the present disclosure is not limited to the specific details of the above embodiments, and various simple modifications may be made to the technical solution of the present disclosure within the technical idea of the present disclosure, and these simple modifications all belong to the protection scope of the present disclosure. It should be noted that, in the foregoing embodiments, various features described in the above embodiments may be combined in any suitable manner, and in order to avoid unnecessary repetition, various combinations that are possible in the present disclosure are not described again.
In addition, any combination of various embodiments of the present disclosure may be made, and the same should be considered as the disclosure of the present disclosure, as long as it does not depart from the spirit of the present disclosure.

Claims (10)

1. A method of controlling a rear view image display, for use in a vehicle, the method comprising:
acquiring a rear view image of the vehicle through a camera;
acquiring a steering angle of a steering wheel of the vehicle;
determining a target image from the rear view image according to the steering angle of the steering wheel;
and displaying the target image.
2. The method of claim 1, wherein determining a target image from the rear view image based on the steering wheel steering angle comprises:
determining a target area from a picture corresponding to the rear view image according to the steering angle of the steering wheel;
and taking the rear view image corresponding to the target area as the target image.
3. The method of claim 2, wherein determining a target area from a frame corresponding to the rear-view image according to the steering angle of the steering wheel comprises:
according to preset resolution and position information of the camera on the vehicle, determining an initial area from a picture corresponding to the rear view image, wherein the initial area is an area displayed by the vehicle in a straight-ahead state;
and determining the target area according to the steering angle of the steering wheel and the initial area.
4. The method of claim 3, wherein the determining the target zone based on the steering wheel steering angle and the initial zone comprises:
taking the initial region as the target region when the steering angle of the steering wheel is smaller than a preset steering angle; alternatively, the first and second electrodes may be,
and under the condition that the steering wheel steering angle is larger than or equal to the preset steering angle, determining the number of pixels to be moved and the moving direction according to the steering wheel steering angle, moving the initial region according to the number of pixels and the moving direction, and taking the region reached by the initial region as the target region.
5. The method of claim 4, wherein determining the number of pixels to be moved and the direction of movement from the steering wheel steering angle comprises:
acquiring the number of pixels to be moved corresponding to the steering wheel steering angle through a preset pixel incidence relation, wherein the pixel incidence relation comprises the corresponding relation between the steering wheel steering angle and the number of pixels;
and determining the turning direction of the vehicle according to the steering wheel steering angle, and taking the turning direction as the moving direction.
6. The method of claim 4, wherein prior to said moving said initial region according to said number of pixels and said moving direction, said method further comprises:
acquiring the steering wheel steering speed of the vehicle and the running speed of the vehicle;
determining the moving speed of the initial area according to the steering speed of the steering wheel and the driving speed;
the moving the initial region according to the number of pixels and the moving direction includes:
and moving the initial area according to the moving speed according to the number of the pixels and the moving direction.
7. An apparatus for controlling display of a rear view image, applied to a vehicle, comprising:
the image acquisition module is used for acquiring a rear view image of the vehicle through a camera;
the angle acquisition module is used for acquiring the steering angle of a steering wheel of the vehicle;
the image determining module is used for determining a target image from the rear view image according to the steering angle of the steering wheel;
and the image display module is used for displaying the target image.
8. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 6.
9. A controller, comprising:
a memory having a computer program stored thereon;
a processor for executing the computer program in the memory to carry out the steps of the method of any one of claims 1 to 6.
10. A vehicle characterized by comprising the controller of claim 9.
CN202010747148.7A 2020-07-29 2020-07-29 Method, device, medium, controller and vehicle for controlling display of rear view image Pending CN112046470A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010747148.7A CN112046470A (en) 2020-07-29 2020-07-29 Method, device, medium, controller and vehicle for controlling display of rear view image

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010747148.7A CN112046470A (en) 2020-07-29 2020-07-29 Method, device, medium, controller and vehicle for controlling display of rear view image

Publications (1)

Publication Number Publication Date
CN112046470A true CN112046470A (en) 2020-12-08

Family

ID=73601836

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010747148.7A Pending CN112046470A (en) 2020-07-29 2020-07-29 Method, device, medium, controller and vehicle for controlling display of rear view image

Country Status (1)

Country Link
CN (1) CN112046470A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113119865A (en) * 2021-04-15 2021-07-16 常州斯伯银车辆科技有限公司 Monitoring method and device for electronic rearview mirror, vehicle-mounted equipment and storage medium
CN113401059A (en) * 2021-08-06 2021-09-17 上海豫兴电子科技有限公司 Display system and method for compensating visual field blind area of trailer rearview mirror
CN113859127A (en) * 2021-09-28 2021-12-31 惠州华阳通用智慧车载系统开发有限公司 Mode switching method for electronic rearview mirror

Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2015098156A1 (en) * 2013-12-27 2015-07-02 株式会社Ｊｖｃケンウッド Visual field support device, visual field support method, and visual field support program
CN105335927A (en) * 2015-11-23 2016-02-17 上海汽车集团股份有限公司 Fisheye image processing method and device as well as vehicle
CN105416286A (en) * 2014-09-12 2016-03-23 爱信精机株式会社 Parking assist device, parking assist method and vehicle assist control program
CN106537905A (en) * 2014-08-12 2017-03-22 索尼公司 Signal processing device, signal processing method and monitoring system
CN109455141A (en) * 2018-12-29 2019-03-12 上海梅克朗汽车镜有限公司 The electronics rearview mirror system automatically adjusted with angle of turn
US20190100245A1 (en) * 2017-10-03 2019-04-04 Aisin Seiki Kabushiki Kaisha Surroundings monitoring apparatus
CN109691088A (en) * 2016-08-22 2019-04-26 索尼公司 Image processing equipment, image processing method and program
CN110386066A (en) * 2018-04-16 2019-10-29 Jvc 建伍株式会社 Vehicle display control unit and method, vehicle display system and storage medium
CN110509851A (en) * 2019-08-09 2019-11-29 上海豫兴电子科技有限公司 A kind of multi-curvature electronics rearview mirror of servo-actuated display
CN110626269A (en) * 2019-10-16 2019-12-31 湖北工业大学 Intelligent imaging driving assistance system and method based on intention identification fuzzy control
CN210139816U (en) * 2019-06-24 2020-03-13 中寰卫星导航通信有限公司 Vehicle vision blind area monitoring system and vehicle
CN111016788A (en) * 2019-12-31 2020-04-17 上海豫兴电子科技有限公司 Electronic rearview mirror system and vehicle

Patent Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2015098156A1 (en) * 2013-12-27 2015-07-02 株式会社Ｊｖｃケンウッド Visual field support device, visual field support method, and visual field support program
CN106537905A (en) * 2014-08-12 2017-03-22 索尼公司 Signal processing device, signal processing method and monitoring system
CN105416286A (en) * 2014-09-12 2016-03-23 爱信精机株式会社 Parking assist device, parking assist method and vehicle assist control program
CN105335927A (en) * 2015-11-23 2016-02-17 上海汽车集团股份有限公司 Fisheye image processing method and device as well as vehicle
CN109691088A (en) * 2016-08-22 2019-04-26 索尼公司 Image processing equipment, image processing method and program
US20190100245A1 (en) * 2017-10-03 2019-04-04 Aisin Seiki Kabushiki Kaisha Surroundings monitoring apparatus
CN110386066A (en) * 2018-04-16 2019-10-29 Jvc 建伍株式会社 Vehicle display control unit and method, vehicle display system and storage medium
CN109455141A (en) * 2018-12-29 2019-03-12 上海梅克朗汽车镜有限公司 The electronics rearview mirror system automatically adjusted with angle of turn
CN210139816U (en) * 2019-06-24 2020-03-13 中寰卫星导航通信有限公司 Vehicle vision blind area monitoring system and vehicle
CN110509851A (en) * 2019-08-09 2019-11-29 上海豫兴电子科技有限公司 A kind of multi-curvature electronics rearview mirror of servo-actuated display
CN110626269A (en) * 2019-10-16 2019-12-31 湖北工业大学 Intelligent imaging driving assistance system and method based on intention identification fuzzy control
CN111016788A (en) * 2019-12-31 2020-04-17 上海豫兴电子科技有限公司 Electronic rearview mirror system and vehicle

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113119865A (en) * 2021-04-15 2021-07-16 常州斯伯银车辆科技有限公司 Monitoring method and device for electronic rearview mirror, vehicle-mounted equipment and storage medium
CN113401059A (en) * 2021-08-06 2021-09-17 上海豫兴电子科技有限公司 Display system and method for compensating visual field blind area of trailer rearview mirror
CN113859127A (en) * 2021-09-28 2021-12-31 惠州华阳通用智慧车载系统开发有限公司 Mode switching method for electronic rearview mirror
CN113859127B (en) * 2021-09-28 2023-07-07 惠州华阳通用智慧车载系统开发有限公司 Electronic rearview mirror mode switching method

Similar Documents

Publication Publication Date Title
CN112046470A (en) Method, device, medium, controller and vehicle for controlling display of rear view image
JP4254887B2 (en) Image display system for vehicles
US20130096820A1 (en) Virtual display system for a vehicle
JP2008077628A (en) Image processor and vehicle surrounding visual field support device and method
WO2017122654A1 (en) Drive assist device and drive assist method
JP2018088642A (en) Mirror system for vehicle
CN112849158B (en) Image display method, vehicle-mounted display system and automobile
CN109591703A (en) A kind of electronic rear view mirror system of automobile and its display methods
JP2005534246A5 (en)
JP2005534246A (en) View system
JP2004194071A (en) Drive support image generator
WO2016163094A1 (en) Display device
CN104276092B (en) A kind of vehicle road turn to blind area to shift to an earlier date visible controlling system
JP2009073250A (en) Vehicle backsight display device
CN108202666A (en) Display method capable of synchronously displaying rear panorama and turning picture during vehicle turning
CN110194109B (en) Method for displaying vehicle external image, image system and vehicle
JPH11129815A (en) Rearward confirming device for vehicle
US20200282909A1 (en) Vehicle imaging system and method for a parking solution
KR101491946B1 (en) Car pillar and car comprising the same
CN113401059B (en) Display system and method for compensating visual field blind area of trailer rearview mirror
JP2021118435A (en) Image processing device and image processing method
US11584299B2 (en) Driver-assisting system for an industrial vehicle
JP2021160681A (en) Vehicular display device
TWM560413U (en) Real-time imaging system for vehicle rear viewing
JP2005086754A (en) Vehicle periphery visual recognition apparatus

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20201208