CN108765490A - Panorama view adjusting method and device, storage medium and electronic equipment - Google Patents

Panorama view adjusting method and device, storage medium and electronic equipment Download PDF

Info

Publication number
CN108765490A
CN108765490A CN201810300003.5A CN201810300003A CN108765490A CN 108765490 A CN108765490 A CN 108765490A CN 201810300003 A CN201810300003 A CN 201810300003A CN 108765490 A CN108765490 A CN 108765490A
Authority
CN
China
Prior art keywords
vehicle
initial
range
view
visual field
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201810300003.5A
Other languages
Chinese (zh)
Other versions
CN108765490B (en
Inventor
龙思源
杨珩
郭涛
郑祥祥
胡金水
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
iFlytek Co Ltd
Original Assignee
iFlytek Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by iFlytek Co Ltd filed Critical iFlytek Co Ltd
Priority to CN201810300003.5A priority Critical patent/CN108765490B/en
Priority to PCT/CN2018/104419 priority patent/WO2019192145A1/en
Publication of CN108765490A publication Critical patent/CN108765490A/en
Application granted granted Critical
Publication of CN108765490B publication Critical patent/CN108765490B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/24—Aligning, centring, orientation detection or correction of the image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle
    • G06T2207/30261—Obstacle

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Traffic Control Systems (AREA)
  • Closed-Circuit Television Systems (AREA)

Abstract

The disclosure provides a method and a device for adjusting a field of view of a panorama, a storage medium and an electronic device. The method comprises the following steps: determining dangerous obstacles around the vehicle and an initial view range of the panoramic image; and determining an adjusted visual field range of the panoramic image according to the initial visual field range and the position information of the dangerous barrier relative to the vehicle, wherein the adjusted visual field range comprises the dangerous barrier. According to the scheme, the view range displayed by the panoramic image can be flexibly adjusted, redundant information is reduced, and information loss of dangerous obstacles is avoided.

Description

Panorama view adjusting method and device, storage medium and electronic equipment
Technical Field
The present disclosure relates to the field of intelligent technologies, and in particular, to a method and an apparatus for adjusting a field of view of a panorama, a storage medium, and an electronic device.
Background
With the continuous development of intelligent technology, automatic driving and auxiliary driving are widely applied to the field of automobiles, and a vehicle-mounted 360-degree panoramic system plays a great role in the safe driving of the automobiles as a basic function.
At present, a vehicle-mounted 360-degree panoramic system mainly collects images around a vehicle body through fisheye cameras arranged in four directions of front, back, left and right of the vehicle; then through a predefined front field range dfRear field of view dbLeft field of view dlRight visual field range drAnd generating an overlooking panoramic image with a fixed visual field range through homography transformation.
In general, df、db、dl、drThe parameters are set by the manufacturer and configured in the system and are kept unchanged in the subsequent use process, so that the actually generated overlook panorama can only show a certain range near the vehicle body to the userFixed field of view, the flexibility is relatively poor, and the following problem easily appears: a plurality of redundant information which has no reference value to the current driving behavior of the user is shown in the panorama; due to the limited field of view, some obstacles that are of reference value to the user's current driving behavior are not shown in the panoramic view, particularly obstacles in blind areas of the vehicle. Whether redundant information is contained or valuable obstacles are omitted, certain influence is caused on safe driving, and safety accidents are more prone to happen to users who rely on the panoramic system.
Disclosure of Invention
The present disclosure provides a method and an apparatus for adjusting a field of view of a panorama, a storage medium, and an electronic device, which can flexibly adjust a field of view range displayed by the panorama, thereby reducing redundant information and avoiding information loss of dangerous obstacles.
In order to achieve the above object, the present disclosure provides a method of adjusting a field of view of a panorama, the method including:
determining dangerous obstacles around the vehicle and an initial view range of the panoramic image;
and determining an adjusted visual field range of the panoramic image according to the initial visual field range and the position information of the dangerous barrier relative to the vehicle, wherein the adjusted visual field range comprises the dangerous barrier.
Optionally, the determining a dangerous obstacle around the vehicle includes:
obtaining collision distance s and collision time t between each obstacle and the vehicle according to the running data of the vehicle and the obstacle information of the obstacles around the vehicle;
making the collision distance s smaller than the preset distance s0And/or the time of collision t is less than the preset time t0Is determined to be the dangerous obstacle.
Optionally, the obtaining a collision distance s and a collision time t between each obstacle and the vehicle according to the driving data of the vehicle and the obstacle information of the obstacles around the vehicle includes:
determining a passing area of the vehicle according to the running data;
and obtaining the collision distance s and the collision time t between each obstacle and the vehicle according to the passing area and the obstacle information.
Optionally, the manner of acquiring the obstacle information is:
acquiring image data around a vehicle body through a camera; detecting obstacles according to the image data to obtain obstacle information of obstacles around the vehicle;
and/or the presence of a gas in the gas,
and acquiring obstacle information of obstacles around the vehicle through a radar.
Optionally, the determining an initial field of view of the panoramic image includes:
determining an initial visual field range of the panoramic image according to a preset fixed visual field range; or,
and determining the initial view range of the panoramic view according to the steering wheel rotating angle of the vehicle.
Optionally, if a coordinate system is established with the center of the vehicle body as an origin, the lateral direction of the vehicle body to the right as the positive X-axis direction, and the longitudinal direction of the vehicle body toward the vehicle head as the positive Y-axis direction, determining the initial view range of the panoramic view according to the steering wheel angle of the vehicle includes:
initial forward field of view range df0Initial back field of view db0Initial left visual field range dl0Initial right field of view dr0The following relationship is satisfied:
df0+db0+lcar≥hmin
dr0+dl0+wcar≥wmin
dr can be obtained based on the steering wheel angle of the vehicle0Or dl0：
Wherein h isminRepresenting the longitudinal length covered by the panorama at said minimum field of view; w is aminRepresenting the lateral width covered by the panorama at said minimum field of view; lcarRepresenting the length of the vehicle; w is acarRepresenting the width of the vehicle; p represents an adjustable parameter;and the steering angle of the front wheels of the vehicle is represented and calculated by the steering wheel rotating angle.
Optionally, the determining an adjusted view range of the panoramic image according to the initial view range and the position information of the dangerous obstacle relative to the vehicle includes:
judging whether the dangerous barrier in the visual field direction which needs to be paid attention by the user is positioned in the initial visual field range;
if the dangerous obstacles in the visual field direction needing attention are all located in the initial visual field range, not adjusting the initial visual field range; otherwise, adjusting the initial view range of the view direction needing attention to enable the dangerous obstacle to be located in the adjusted view range.
Optionally, when the vehicle is reversed, the visual field direction to be focused includes a left side, a right side and a rear side of the vehicle, and the adjusting the initial visual field range of the visual field direction to be focused makes the dangerous obstacle located in the adjusted visual field range includes:
if the dangerous obstacle on the left side of the vehicle exceeds the initial left visual field range, adjusting the initial left visual field range to enable the dangerous obstacle on the left side of the vehicle to be located in the adjusted left visual field range;
if the dangerous barrier on the right side of the vehicle exceeds the initial right view range, adjusting the initial right view range to enable the dangerous barrier on the right side of the vehicle to be located in the adjusted right view range;
if the dangerous barrier behind the vehicle exceeds the initial rear view range, adjusting the initial rear view range to enable the dangerous barrier behind the vehicle to be located in the adjusted rear view range;
or,
when the vehicle advances, the view direction needing attention comprises the left side, the right side and the front of the vehicle, and the initial view range of the view direction needing attention is adjusted to enable the dangerous obstacle to be located in the adjusted view range, and the method comprises the following steps:
if the dangerous obstacle on the left side of the vehicle exceeds the initial left visual field range, adjusting the initial left visual field range to enable the dangerous obstacle on the left side of the vehicle to be located in the adjusted left visual field range;
if the dangerous barrier on the right side of the vehicle exceeds the initial right view range, adjusting the initial right view range to enable the dangerous barrier on the right side of the vehicle to be located in the adjusted right view range;
and if the dangerous obstacle in front of the vehicle exceeds the initial front field range, adjusting the initial front field range to enable the dangerous obstacle in front of the vehicle to be positioned in the adjusted front field range.
Optionally, if a maximum view range is preset, the adjusted view range of the panoramic image is not greater than the maximum view range.
The present disclosure provides a field of view adjustment apparatus for a panorama, the apparatus comprising:
the dangerous obstacle determining module is used for determining dangerous obstacles around the vehicle;
the initial visual field range determining module is used for determining the initial visual field range of the panoramic image;
and the adjusted visual field range determining module is used for determining the adjusted visual field range of the panoramic image according to the initial visual field range and the position information of the dangerous obstacle relative to the vehicle, wherein the adjusted visual field range comprises the dangerous obstacle.
Optionally, the dangerous obstacle determination module comprises:
the collision distance and time obtaining module is used for obtaining collision distances s and collision times t between each obstacle and the vehicle according to the driving data of the vehicle and the obstacle information of the obstacles around the vehicle;
a dangerous obstacle determination submodule for making the collision distance s smaller than the preset distance s0And/or the time of collision t is less than the preset time t0Is determined to be the dangerous obstacle.
Optionally, the collision distance and time obtaining module is configured to determine a passing area of the vehicle according to the driving data; and obtaining the collision distance s and the collision time t between each obstacle and the vehicle according to the passing area and the obstacle information.
Optionally, the apparatus further comprises:
the obstacle information acquisition module is used for acquiring image data around the vehicle body through the camera; detecting obstacles according to the image data to obtain obstacle information of obstacles around the vehicle; and/or acquiring obstacle information of obstacles around the vehicle by radar.
Optionally, the initial view range determining module is configured to determine an initial view range of the panoramic image according to a preset fixed view range; or determining the initial view range of the panoramic view according to the steering wheel angle of the vehicle.
Optionally, if a coordinate system is established with the center of the vehicle body as an origin, the lateral direction of the vehicle body to the right as the positive X-axis direction, and the longitudinal direction of the vehicle body toward the vehicle head as the positive Y-axis direction, determining the initial view range of the panoramic view according to the steering wheel angle of the vehicle includes:
initial forward field of view range df0Initial back field of view db0Initial left visual field range dl0Initial right field of view dr0The following relationship is satisfied:
df0+db0+lcar≥hmin
dr0+dl0+wcar≥wmin
dr can be obtained based on the steering wheel angle of the vehicle0Or dl0：
Wherein h isminRepresenting the longitudinal length covered by the panorama at said minimum field of view; w is aminRepresenting the lateral width covered by the panorama at said minimum field of view; lcarRepresenting the length of the vehicle; w is acarRepresenting the width of the vehicle; p represents an adjustable parameter;and the steering angle of the front wheels of the vehicle is represented and calculated by the steering wheel rotating angle.
Optionally, the adjusted view range determining module is configured to determine whether a dangerous obstacle in a view direction that a user needs to pay attention to is located in the initial view range; if the dangerous obstacles in the visual field direction needing attention are all located in the initial visual field range, not adjusting the initial visual field range; otherwise, adjusting the initial view range of the view direction needing attention to enable the dangerous obstacle to be located in the adjusted view range.
Optionally, when the vehicle is reversed, the visual field direction to be focused includes a left side, a right side and a rear side of the vehicle, and the adjusting the initial visual field range of the visual field direction to be focused makes the dangerous obstacle located in the adjusted visual field range includes:
if the dangerous obstacle on the left side of the vehicle exceeds the initial left visual field range, adjusting the initial left visual field range to enable the dangerous obstacle on the left side of the vehicle to be located in the adjusted left visual field range;
if the dangerous barrier on the right side of the vehicle exceeds the initial right view range, adjusting the initial right view range to enable the dangerous barrier on the right side of the vehicle to be located in the adjusted right view range;
if the dangerous barrier behind the vehicle exceeds the initial rear view range, adjusting the initial rear view range to enable the dangerous barrier behind the vehicle to be located in the adjusted rear view range;
or,
when the vehicle advances, the view direction needing attention comprises the left side, the right side and the front of the vehicle, and the initial view range of the view direction needing attention is adjusted to enable the dangerous obstacle to be located in the adjusted view range, and the method comprises the following steps:
if the dangerous obstacle on the left side of the vehicle exceeds the initial left visual field range, adjusting the initial left visual field range to enable the dangerous obstacle on the left side of the vehicle to be located in the adjusted left visual field range;
if the dangerous barrier on the right side of the vehicle exceeds the initial right view range, adjusting the initial right view range to enable the dangerous barrier on the right side of the vehicle to be located in the adjusted right view range;
and if the dangerous obstacle in front of the vehicle exceeds the initial front field range, adjusting the initial front field range to enable the dangerous obstacle in front of the vehicle to be positioned in the adjusted front field range.
Optionally, if a maximum view range is preset, the adjusted view range of the panoramic image is not greater than the maximum view range.
The present disclosure provides a storage medium having stored therein a plurality of instructions, which are loaded by a processor, for performing the steps of the above-described panorama view adjusting method.
The present disclosure provides an electronic device, comprising;
the storage medium described above; and
a processor to execute the instructions in the storage medium.
According to the technical scheme, the dangerous barrier having reference value for the current driving behavior and the initial view range of the panoramic image can be determined, and the adjusted view range of the panoramic image is obtained by combining the initial view range and the position information of the dangerous barrier relative to the vehicle, so that the adjusted view range comprises the dangerous barrier. The adjusted view range is beneficial to reducing redundant information, avoiding information loss of dangerous obstacles and improving the usability of the panoramic image in the actual driving process.
Additional features and advantages of the disclosure will be set forth in the detailed description which follows.
Drawings
The accompanying drawings, which are included to provide a further understanding of the disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the description serve to explain the disclosure without limiting the disclosure. In the drawings:
FIG. 1 is a schematic flow chart of a method for adjusting a field of view of a panoramic view according to the present disclosure;
FIG. 2 is a schematic flow chart illustrating the determination of a dangerous obstacle according to the present disclosure;
FIG. 3 shows a scheme l of the present disclosurem、wcar、db、dfSchematic diagram of the relationship between;
FIG. 4 is a schematic view of a traffic zone during reversing in accordance with the disclosed aspects;
FIG. 5 is a schematic view of a traffic zone while traveling in the disclosed arrangement;
FIG. 6 is a graph of the trajectory of an obstacle relative to a vehicle in accordance with the present disclosure;
FIG. 7 is a schematic illustration of a minimum field of view in an embodiment of the present disclosure;
fig. 8 is a corresponding relationship between a steering wheel angle and an initial field of view in reversing according to the present disclosure;
FIG. 9 is a corresponding relationship between steering wheel angle and initial field of view for forward travel in accordance with aspects of the present disclosure;
FIG. 10 is a schematic view of the panoramic view field adjustment apparatus according to the present disclosure;
fig. 11 is a schematic structural diagram of an electronic device for adjusting a field of view of a panoramic view according to the present disclosure.
Detailed Description
The following detailed description of specific embodiments of the present disclosure is provided in connection with the accompanying drawings. It should be understood that the detailed description and specific examples, while indicating the present disclosure, are given by way of illustration and explanation only, not limitation.
Referring to fig. 1, a flow chart of the method for adjusting the field of view of the panorama of the present disclosure is shown. May include the steps of:
s101, determining dangerous obstacles around the vehicle and an initial view range of the panoramic image.
In the scheme of the disclosure, the dangerous obstacles around the vehicle can be determined at least according to the following modes:
(1) in order to avoid missing valuable obstacles, all obstacles detected around the vehicle may be determined as dangerous obstacles.
(2) Partial obstacles detected around the vehicle may be determined as dangerous obstacles in combination with the attributes of the obstacles. For example, the attributes of the obstacle may be embodied as: whether the device is located within a preset detection range, whether a living body exists, whether the movement speed is too fast, whether the volume is too large, and the like, and the scheme of the disclosure may not be particularly limited to this.
(3) Partial obstacles detected around the vehicle may be determined as dangerous obstacles in combination with the current driving behavior of the user. Refer specifically to the schematic flow diagram of FIG. 2. May include the steps of:
s201, according to the running data of the vehicle and the obstacle information of the obstacles around the vehicle, the collision distance S and the collision time t between each obstacle and the vehicle are obtained.
As an example, the travel data in the aspect of the present disclosure may include: the gear position of the vehicle, the travel speed, and the steering wheel angle. The process of acquiring the driving data can be implemented by referring to the related art, and will not be described in detail here.
As an example, the obstacle information in the present disclosure may include: barrierPosition information of the obstacle with respect to the vehicle, and motion state information of the obstacle. For example, the position information of the N obstacles with respect to the vehicle may be expressed as Location { (x)1,y1),(x2,y2),…,(xi,yi),…,(xN,yN) Motion state information may be expressed as Motion { (v)1,θ1),(v2,θ2),…,(vi,θi),…,(vN,θN)}. The coordinate system corresponding to the position information can take the center of the vehicle body as an origin, the transverse direction to the right of the vehicle body is taken as the positive direction of an X axis, and the longitudinal direction of the vehicle body towards the vehicle head is taken as the positive direction of a Y axis; v represents the instantaneous velocity of the obstacle relative to the vehicle, and θ represents the direction of motion in the world coordinate system.
For example, the disclosed solution may obtain obstacle information of obstacles around the vehicle at least in the following ways:
the method comprises the steps that firstly, image data around a vehicle body are collected through a camera; and detecting an obstacle according to the image data to obtain obstacle information of the obstacle around the vehicle.
In practical application, at least four cameras can be arranged in the front, back, left and right directions of the vehicle, or at least one camera capable of rotating 360 degrees can be arranged on the roof of the vehicle, as long as the camera can shoot image data around the vehicle body. For example, the type of the camera may be a fisheye camera and/or a depth camera and/or a binocular camera, for example, 4 fisheye cameras may be installed around the vehicle body, and then other types of cameras may be used to collect image data around the vehicle body. The type, the installation position and the like of the camera can be not specifically limited by the scheme, the process of collecting image data through the camera and the like can be realized by referring to related technologies, and detailed description is omitted here.
It should be noted that, in order to enhance the relevant information in the image, the acquired image data may be preprocessed, for example, the image noise may be removed through a median filtering or gaussian filtering algorithm, and the contrast may be enhanced through an image histogram equalization method.
For example, the disclosed aspects can perform obstacle detection by at least the following methods: object detection based on deep learning, and moving object detection based on machine vision. Taking an object detection method based on deep learning as an example, model training can be performed in advance by using sample image data with labeled information to generate a target detection and recognition model, wherein the labeled information can be embodied as the type of an obstacle, such as a pedestrian, an automobile, a non-motor vehicle, and the like, and can be specifically determined by combining with practical application conditions. The trained target detection and recognition model can detect and recognize specific class objects according to image data acquired by the camera, and the positions of the obstacles in the image and the classes of the obstacles are obtained.
After the position of the obstacle in the image is obtained, the position of the obstacle on the image can be converted into a world coordinate system through coordinate system conversion; and then combining the position of the same obstacle in the previous frame of image or using an optical flow method to calculate the instantaneous speed v and the movement direction theta of the current frame relative to the previous frame, namely the movement state information of the obstacle. In addition, in the practical application, considering that the obstacle usually has a certain size, the position information of the obstacle relative to the vehicle can be determined from any point of the obstacle, for example, the point of the obstacle closest to the vehicle body is determined as the position of the obstacle relative to the vehicle, and the position information (x) can be obtainedi,yi) The scheme of the present disclosure may not be specifically limited to this, and may be determined in accordance with practical application requirements.
Taking the example of installing at least four cameras around the vehicle body, each camera can shoot one piece of corresponding image data, namely, at least four pieces of image data can be obtained through the cameras, and in the practical application process, obstacle detection can be carried out on all the image data; alternatively, in order to reduce the amount of calculation, obstacle detection may be performed on part of the image data according to the driving demand. For example, it can be known from the driving data that the user is backing up, and at this time, the user is more concerned with the obstacle information in the three directions of the rear, left, and right of the vehicle, so that the obstacle detection can be performed on the image data in these three directions.
And secondly, acquiring obstacle information of obstacles around the vehicle through a radar. The obstacle detection process based on the data collected by the radar can be implemented by referring to the related art, and is not described in detail herein. In the practical application process, the radar precision may be determined in combination with the use requirement, for example, a millimeter wave radar may be adopted, and this may not be specifically limited by the present disclosure. As an example, the radar may be installed around the vehicle body, or may be installed on the roof to perform 360 ° scanning, as long as the obstacle information around the vehicle is collected by the radar.
In practical application, the two modes can be combined, namely the camera and the radar are used for acquiring the obstacle information of the obstacles around the vehicle, so that the obstacles around the vehicle can be found as comprehensively as possible, and the obstacles can be positioned as accurately as possible. The implementation manner of obtaining the obstacle information according to the scheme of the present disclosure may not be specifically limited.
As an example, after obtaining the travel data and the obstacle information, the collision distance s and the collision time t between the obstacle and the vehicle may be calculated as follows.
Firstly, the possible running track and the passing area of the vehicle can be calculated by combining the running data of the vehicle.
Similarly, if the center of the vehicle body is used as the origin, the lateral direction of the vehicle body is taken as the positive direction of the X axis, and the longitudinal direction of the vehicle body is taken as the positive direction of the Y axis. Correspondingly, the motion trail equation of the four vertexes of the left back, the right back, the left front and the right front of the vehicle can be embodied as follows:
wherein (x)bL,ybL) Representing the trajectory coordinates of the left back vertex; (x)bR,ybR) Track coordinates representing the right back vertex; (x)fL,yfL) The track coordinates representing the left front vertex; (x)fR,yfR) Track coordinates representing the right front vertex;the steering angle of the front wheels of the vehicle can be represented, the right turn can be defined as a positive direction angle, and the angle can be obtained by calculating the steering wheel angle; lmRepresenting the fore-aft wheelbase of the vehicle; w is acarRepresenting the width of the vehicle; dbIndicating the distance from the rear wheel axle to the tail of the vehicle; dfThe distance from the front wheel axle to the vehicle head is shown, and the specific relation can be seen in a schematic diagram shown in fig. 3.
After the trajectory lines of the four vertexes are calculated according to the motion trajectory equation, the driving direction of the vehicle, such as front or rear, can be combined; and the direction of turning of the vehicle, e.g., left or right, results in the traffic zone of the vehicle. As an example, fig. 4 shows a schematic view of a traffic zone when the vehicle is reversing, i.e. driving backwards to the left; fig. 5 shows a schematic view of the traffic region when the vehicle is traveling forward, i.e., to the left and forward.
Secondly, after the passing area of the vehicle is obtained, a motion model can be established by combining the motion state information of the obstacle, the collision point of the obstacle when colliding with the vehicle under the condition of maintaining the motion state is calculated, and the collision distance s and the collision time t from the current position of the obstacle to the collision point are further obtained. It is understood that if the obstacle does not collide with the vehicle, s ∞, and t ∞.
For example, a vehicle can be used as a reference system, and since the vehicle performs instant uniform-speed circular motion and the obstacle performs instant uniform-speed linear motion, the speed of the obstacle relative to the vehicle can be decomposed into: linear velocity v of uniform circular motionrConstant linear motion velocity vl. Corresponding to the obstacleThe parametric equation of the trajectory coordinates with respect to time can be expressed as:
wherein (x)i,yi) The current position coordinates of the ith obstacle are represented, namely the position information of the obstacle relative to the vehicle; r represents the motion radius of circular motion and can be obtained through a trajectory equation; ω denotes an angular velocity of the circular motion, and ω ═ vr/R；θiRepresenting an angle between the current position and the X-axis; v. oflxRepresenting linear movement velocity vlThe x component of (a); v. oflyRepresenting linear movement velocity vlThe y component of (a).
Based on the above parametric equation, a motion trajectory curve of the obstacle relative to the vehicle can be obtained, which is specifically shown in fig. 6.
Thus, the coordinates of the collision point and the corresponding collision time t can be solved by combining the following rectangular box equation representing the edge of the vehicle body:
wherein lcarRepresenting the length of the vehicle; w is acarIndicating the width of the vehicle.
It can be understood that, the intersection point of the motion trajectory curve of the obstacle relative to the vehicle and the rectangular frame at the edge of the vehicle body is the collision point, and after the coordinates of the collision point are obtained, the collision distance s from the current position of the obstacle to the collision point can be calculated through integral operation:
thus, the distance between each obstacle and the collision point of the vehicle can be obtainedSet of collision distances S { S }1,s2,…,sNAnd a corresponding set of collision times T T1,t2,…,tN}。
S202, enabling the collision distance S to be smaller than a preset distance S0And/or the time of collision t is less than the preset time t0Is determined to be the dangerous obstacle.
After the collision distance set and the collision time set are obtained, the dangerous barrier which has a large influence on the current driving behavior can be determined according to s and/or t. Specifically, s may be smaller than the preset distance s0And/or t is less than a preset time t0Is determined to be a dangerous obstacle. S of the pair of the presently disclosed embodiments0、t0The value is not particularly limited and can be set in combination with the requirements of practical application.
In the scheme of the present disclosure, the initial field of view of the panoramic image may be determined at least in the following manner:
(1) the initial field of view of the panorama can be determined according to a preset fixed field of view.
For example, the fixed field of view may be a fixed value; alternatively, a usable range of values may be provided, and the size of the fixed field of view in the present disclosure may not be particularly limited.
In the practical application process, the size of each view field direction can be determined in advance by combining the size of the fixed view field, and the initial view field of the panoramic image is obtained. Or, the size of each view field direction can be determined in real time by combining the size of the fixed view field and the position information of the dangerous obstacle relative to the vehicle, so as to obtain the initial view field of the panoramic image. For example, if there are fewer dangerous obstacles on the left side of the vehicle and closer to the vehicle, and there are more dangerous obstacles on the right side of the vehicle and more dispersed, the size of the initial left view range can be made smaller than the size of the initial right view range. The method for determining the initial field of view of the panoramic image using the fixed field of view according to the present disclosure may not be specifically limited.
(2) The initial field of view of the panoramic view can be determined according to the steering wheel angle of the vehicle. See the description below for details.
In the practical application process, in order to ensure the safety of all directions, the minimum visual field range of a panoramic image can be preset, so that the initial visual field range determined by the scheme of the disclosure is not less than the minimum visual field range.
The minimum field of view of the panorama may include: minimum frontal field extent dfminMinimum rear field of view dbminMinimum left visual field range dlminMinimum right field of view drmin. Wherein df ismin+dbmin+lcar＝hminThe longitudinal length covered by the panorama at the minimum view field is shown; dl (dl)min+drmin+wcar＝wminThe lateral width covered by the panorama at the minimum view range is shown, and particularly, refer to the schematic diagram of the minimum view range shown in fig. 7.
As an example, the minimum field of view may satisfy the following condition:
for example, the minimum left field of view may be set to dlmin≥d1，d1Indicating the maximum steering angle when the vehicle is turning to the rightCorresponding to the maximum view to the left of the vehicle.
For example, the minimum right field of view range may be set to drmin≥d2，d2Indicating maximum steering angle when the vehicle is turning leftCorresponding to the maximum view on the right side of the vehicle.
For example, the minimum front field of view range may be set to lcar/8≤dfmin≤lcar/4, e.g. dfmin＝lcarAnd/6, the scheme of the present disclosure may not be specifically limited thereto.
For example, the minimum rear field of view range may be set to lcar/6≤dbmin≤lcar/3, e.g. dbmin＝lcarAnd/4, the scheme of the present disclosure may not be specifically limited thereto.
The following explains a process of determining an initial field of view of a panorama according to a steering wheel angle of a vehicle in the present disclosure with reference to a minimum field of view.
For example, the initial forward field range df0Initial back field of view db0Initial left visual field range dl0Initial right field of view dr0The following relationship can be satisfied:
df0+db0+lcar≥hmin
dr0+dl0+wcar≥wmin
in addition, referring to the correspondence relationship between the steering wheel angle and the initial visual field range in reverse shown in fig. 8 and the correspondence relationship between the steering wheel angle and the initial visual field range in forward shown in fig. 9, df is known0、db0Can be set to a constant value, dl0、dr0It may vary sinusoidally with the steering wheel angle. Therefore, dl is determined0、dr0An initial field of view may be determined based on the steering wheel angle, e.g., dr may be determined0Is then based on dr0+dl0+wcar≥wminDetermining another initial field of view, e.g. dl0Satisfy dl0≥wmin-wcar-dr0And (4) finishing.
Specifically, if a coordinate system is established with the center of the vehicle body as the origin, the lateral right of the vehicle body as the positive X-axis direction, and the longitudinal direction of the vehicle body toward the vehicle head as the positive Y-axis direction, then
Initial right field of view dr corresponding to the current driving behavior in combination with the steering wheel angle0May be embodied as:
or an initial left visual field range dl corresponding to the current driving behavior in conjunction with the steering wheel angle0May be embodied as:
where p represents an adjustable parameter. In the practical application process, the sine function can be stretched by adjusting the value of p, so that a better visual effect is achieved. Specifically, the sinusoidal image is compressed in the horizontal direction when p increases, and is stretched in the horizontal direction when p decreases. As an example, in the practical application process, the interval is locatedInner sinusoid, not more than one cycle and more than half a cycle, i.e.
In the present disclosure, the order of determining the dangerous obstacle and determining the initial field of view may not be specifically limited, as long as the dangerous obstacle and the initial field of view are obtained before the field of view is adjusted.
S102, determining an adjusted view range of the panoramic image according to the initial view range and the position information of the dangerous obstacle relative to the vehicle, wherein the adjusted view range comprises the dangerous obstacle.
After the dangerous barrier and the initial view range are obtained, the adjusted view range of the panoramic image can be determined according to the obtained dangerous barrier and initial view range. In the practical application process, all dangerous obstacles can be included in the adjusted visual field range; alternatively, after sorting the dangerous obstacles, a part of the dangerous obstacles may be selected and included in the adjusted visual field range, which may not be specifically limited by the present disclosure.
As an example, in addition to sorting dangerous obstacles by using s and t, if the obstacle type is determined during obstacle detection, sorting may be performed based on the priority of the obstacle type, for example, the priority of pedestrians is higher than that of non-motor vehicles, and the priority of non-motor vehicles is higher than that of automobiles, which is not limited in the present disclosure.
Specifically, the adjustment process of the visual field range may be embodied as: judging whether the dangerous barrier in the visual field direction which needs to be paid attention by the user is positioned in the initial visual field range; if the dangerous obstacles in the visual field direction needing attention are all located in the initial visual field range, not adjusting the initial visual field range; otherwise, adjusting the initial view range of the view direction needing attention to enable the dangerous obstacle to be located in the adjusted view range.
As an example, the visual field direction in which the user needs to pay attention to may be four directions of front, back, left, and right of the vehicle; or, the user may set the view direction that needs to be focused on according to the user's own needs, for example, if the user wants to view the scene behind the vehicle, the view direction that needs to be focused on may be set behind the vehicle; alternatively, the direction of the field of view that needs attention may also be determined in conjunction with the direction of travel of the vehicle, for example, when the vehicle is reversing, the direction of the field of view that needs attention may include: left, right, and rear of the vehicle; the directions of the field of view that need attention when the vehicle is moving forward may include: left, right, and front of the vehicle.
As an example, the driving direction of the vehicle may be determined by a vehicle gear, which may not be specifically limited by the present disclosure.
For example, when the vehicle is reversed, the initial view range of the view direction that needs to be focused is adjusted, so that the dangerous obstacle is located in the adjusted view range, which may be embodied as:
if the dangerous barrier on the left side of the vehicle exceeds the initial left visual field range, adjusting the initial left visual field range to enable the dangerous barrier on the left side of the vehicle to be located in the adjusted left visual field range;
if the dangerous obstacle on the right side of the vehicle exceeds the initial right view range, adjusting the initial right view range to enable the dangerous obstacle on the right side of the vehicle to be located in the adjusted right view range;
and if the dangerous obstacle behind the vehicle exceeds the initial rear view range, adjusting the initial rear view range to enable the dangerous obstacle behind the vehicle to be located in the adjusted rear view range.
That is, the adjusted field of view of the panorama can be:
for the adjusted left visual field range dl1In other words, if the dangerous obstacle located on the left side of the vehicle does not exceed dl0Then dl is1＝dl0(ii) a OtherwiseFor example,
for the adjusted right viewing range dr1In other words, if the dangerous obstacle on the right side of the vehicle does not exceed dr0Then dr1＝dr0(ii) a OtherwiseFor example,
for adjusted rear field of view db1In other words, if the dangerous obstacle located behind the vehicle does not exceed db0Then db1＝db0(ii) a OtherwiseFor example,
for adjusting the rear field of view range df1In other words, the user does not need to pay more attention to the information of the front field range when backing, so the field range can be df after adjustment1≥df0. For example df1＝df0。
For example, when the vehicle moves forward, the initial field of view of the field of view direction that needs to be focused is adjusted, so that the dangerous obstacle is located in the adjusted field of view, which may be embodied as:
if the dangerous barrier on the left side of the vehicle exceeds the initial left visual field range, adjusting the initial left visual field range to enable the dangerous barrier on the left side of the vehicle to be located in the adjusted left visual field range;
if the dangerous obstacle on the right side of the vehicle exceeds the initial right view range, adjusting the initial right view range to enable the dangerous obstacle on the right side of the vehicle to be located in the adjusted right view range;
and if the dangerous barrier in front of the vehicle exceeds the initial front field range, adjusting the initial front field range to enable the dangerous barrier in front of the vehicle to be positioned in the adjusted front field range.
That is, the adjusted field of view of the panorama can be:
adjusted left visual field range dl1And the adjusted right visual field range dr1The same thing as when the vehicle is reversed can be found in the above description, and the details are not repeated here.
For adjusting the rear field of view range df1In other words, if a dangerous obstacle located in front of the vehicle does not exceed df0Then df is1＝df0(ii) a OtherwiseFor example,
for adjusted rear field of view db1In other words, the vehicle does not need to pay much attention to the information of the rear view range when moving forward, so the adjusted rear view range can be db1≥db0. E.g. db1＝db0。
It will be appreciated that, in the above formula,
| min (X) | represents the absolute value of the minimum value of the X axis in the position information of the dangerous obstacle relative to the vehicle;
| max (X) | represents an absolute value of the maximum value of the X axis in the positional information of the dangerous obstacle relative to the vehicle;
| min (Y) | represents the absolute value of the minimum value of the Y axis in the position information of the dangerous obstacle relative to the vehicle;
| max (Y) | represents the absolute value of the maximum value of the Y axis in the positional information of the dangerous obstacle relative to the vehicle.
As an example, when generating a panorama, the field of view may reach infinity theoretically, that is, the maximum field of view of the panorama is not limited, but when acquiring obstacle information through image data collected by a camera in a practical application process, due to different degrees of sharpness of the camera, the distant obstacle pixels are stretched to the full extentThe scene may become more blurred and appear to the user with less effect. To this end, in order to ensure the overall visual effect of the panoramic image, on the premise that the panoramic image shows valuable information without being affected, the maximum view range of the panoramic image may be preset in the present disclosure, and the preset view range may include: maximum front field extent dfmaxMaximum rear field of view dbmaxMaximum left visual field range dlmaxMaximum right field of view drmax。
As an example, the maximum viewing range may satisfy the following condition:
for example, the maximum front field of view range may be set to lcar≤dfmax≤2.5×lcarE.g. dfmax＝1.5×lcarThis may not be particularly limited in the present disclosure.
For example, the maximum rear field of view may be set to lcar≤dbmax≤2.5×lcarE.g. dbmax＝1.5×lcarThis may not be particularly limited in the present disclosure.
For example, the maximum left field of view may be set to 1.5 × lcar≤dlmax≤3×lcarE.g. dlmax＝2×lcarThis may not be particularly limited in the present disclosure.
For example, the maximum right field of view range may be set to 1.5 × lcar≤drmax≤3×lcarE.g. drmax＝2×lcarThis may not be particularly limited in the present disclosure.
Accordingly, the adjusted viewing range of the panorama in the present disclosure may not be greater than the maximum viewing range.
For example, when the vehicle is reversing, the adjusted field of view of the panorama can be:
if the dangerous obstacle on the left side of the vehicle exceeds dl0Then, thenFor example, the adjusted left view range specifically includes:
if a dangerous obstacle located on the right side of the vehicle exceeds dr0Then, then For example, the right viewing range after adjustment is specifically:
if the dangerous obstacle located at the rear of the vehicle exceeds db0Then, thenFor example, the adjusted rear view range specifically includes:
the field range after adjustment can be dfmax≥df1≥df0. For example, the adjusted front visual field range specifically includes: df is a1＝df0。
For example, when the vehicle is moving forward, the adjusted field of view of the panorama may be:
adjusted left visual field range dl1And the adjusted right visual field range dr1The same thing as when the vehicle is reversed can be found in the above description, and the details are not repeated here.
If a dangerous obstacle located in front of the vehicle exceeds df0Then, thenFor example, the adjusted front visual field range specifically includes:
the adjusted rear visual field range can be dbmax≥db1≥db0. For example, the adjusted rear view range specifically includes: db1＝db0。
In summary, compared with the prior art that homography transformation is carried out only through four fisheye images to generate a panoramic image with a fixed view field range, the panoramic image with the variable view field range can be flexibly adjusted to generate the panoramic image with the variable view field range, the panoramic image with the variable view field range can better show information related to current driving behaviors to a user, the usability of the panoramic image in the actual driving process is improved, and the possibility of safety accidents of the user depending on a panoramic system is reduced as much as possible.
As an example, the panorama can better display the information related to the current driving behavior by minimizing redundant information having no reference value to the current driving behavior and/or maximizing obstacle information having a reference value to the current driving behavior.
Referring to fig. 10, a schematic view of the view field adjusting apparatus of the panorama of the present disclosure is shown. The apparatus may include:
a dangerous obstacle determining module 301, configured to determine dangerous obstacles around the vehicle;
an initial view range determining module 302, configured to determine an initial view range of the panoramic image;
an adjusted view range determining module 303, configured to determine an adjusted view range of the panoramic image according to the initial view range and the position information of the dangerous obstacle relative to the vehicle, where the adjusted view range includes the dangerous obstacle.
Optionally, the dangerous obstacle determination module comprises:
the collision distance and time obtaining module is used for obtaining collision distances s and collision times t between each obstacle and the vehicle according to the driving data of the vehicle and the obstacle information of the obstacles around the vehicle;
a dangerous obstacle determination submodule for making the collision distance s smaller than the preset distance s0And/or the time of collision t is less than the preset time t0Is determined to be the dangerous obstacle.
Optionally, the collision distance and time obtaining module is configured to determine a passing area of the vehicle according to the driving data; and obtaining the collision distance s and the collision time t between each obstacle and the vehicle according to the passing area and the obstacle information.
Optionally, the apparatus further comprises:
the obstacle information acquisition module is used for acquiring image data around the vehicle body through the camera; detecting obstacles according to the image data to obtain obstacle information of obstacles around the vehicle; and/or acquiring obstacle information of obstacles around the vehicle by radar.
Optionally, the initial view range determining module is configured to determine an initial view range of the panoramic image according to a preset fixed view range; or determining the initial view range of the panoramic view according to the steering wheel angle of the vehicle.
Optionally, if a coordinate system is established with the center of the vehicle body as an origin, the lateral direction of the vehicle body to the right as the positive X-axis direction, and the longitudinal direction of the vehicle body toward the vehicle head as the positive Y-axis direction, determining the initial view range of the panoramic view according to the steering wheel angle of the vehicle includes:
initial forward field of view range df0Initial back field of view db0Initial left visual field range dl0Initial right field of view dr0The following relationship is satisfied:
df0+db0+lcar≥hmin
dr0+dl0+wcar≥wmin
dr can be obtained based on the steering wheel angle of the vehicle0Or dl0：
Wherein h isminRepresenting the longitudinal length covered by the panorama at said minimum field of view; w is aminRepresenting the lateral width covered by the panorama at said minimum field of view; lcarRepresenting the length of the vehicle; w is acarRepresenting the width of the vehicle; p represents an adjustable parameter;and the steering angle of the front wheels of the vehicle is represented and calculated by the steering wheel rotating angle.
Optionally, the adjusted view range determining module is configured to determine whether a dangerous obstacle in a view direction that a user needs to pay attention to is located in the initial view range; if the dangerous obstacles in the visual field direction needing attention are all located in the initial visual field range, not adjusting the initial visual field range; otherwise, adjusting the initial view range of the view direction needing attention to enable the dangerous obstacle to be located in the adjusted view range.
Optionally, when the vehicle is reversed, the visual field direction to be focused includes a left side, a right side and a rear side of the vehicle, and the adjusting the initial visual field range of the visual field direction to be focused makes the dangerous obstacle located in the adjusted visual field range includes:
if the dangerous obstacle on the left side of the vehicle exceeds the initial left visual field range, adjusting the initial left visual field range to enable the dangerous obstacle on the left side of the vehicle to be located in the adjusted left visual field range;
if the dangerous barrier on the right side of the vehicle exceeds the initial right view range, adjusting the initial right view range to enable the dangerous barrier on the right side of the vehicle to be located in the adjusted right view range;
if the dangerous barrier behind the vehicle exceeds the initial rear view range, adjusting the initial rear view range to enable the dangerous barrier behind the vehicle to be located in the adjusted rear view range;
or,
when the vehicle advances, the view direction needing attention comprises the left side, the right side and the front of the vehicle, and the initial view range of the view direction needing attention is adjusted to enable the dangerous obstacle to be located in the adjusted view range, and the method comprises the following steps:
if the dangerous obstacle on the left side of the vehicle exceeds the initial left visual field range, adjusting the initial left visual field range to enable the dangerous obstacle on the left side of the vehicle to be located in the adjusted left visual field range;
if the dangerous barrier on the right side of the vehicle exceeds the initial right view range, adjusting the initial right view range to enable the dangerous barrier on the right side of the vehicle to be located in the adjusted right view range;
and if the dangerous obstacle in front of the vehicle exceeds the initial front field range, adjusting the initial front field range to enable the dangerous obstacle in front of the vehicle to be positioned in the adjusted front field range.
Optionally, if a maximum view range is preset, the adjusted view range of the panoramic image is not greater than the maximum view range.
With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
Referring to fig. 11, a schematic structural diagram of an electronic device 400 for performing a field of view adjustment of a panoramic view according to the present disclosure is shown. The electronic device 400 may include at least a processor 401 and a storage medium 402, and as an example, the processor 401 and the storage medium 402 may be connected by a bus or other means, and the connection by the bus is illustrated in fig. 11 as an example. The number of the processors 401 may be one or more, and one processor is illustrated in fig. 11 as an example. The storage medium 402 represents a storage device resource for storing instructions, such as application programs, that are executable by the processor 401. Further, the processor 401 may be configured to load instructions in a storage medium to perform the above-described panorama view adjusting method.
The preferred embodiments of the present disclosure are described in detail with reference to the accompanying drawings, however, the present disclosure is not limited to the specific details of the above embodiments, and various simple modifications may be made to the technical solution of the present disclosure within the technical idea of the present disclosure, and these simple modifications all belong to the protection scope of the present disclosure.
It should be noted that, in the foregoing embodiments, various features described in the above embodiments may be combined in any suitable manner, and in order to avoid unnecessary repetition, various combinations that are possible in the present disclosure are not described again.
In addition, any combination of various embodiments of the present disclosure may be made, and the same should be considered as the disclosure of the present disclosure, as long as it does not depart from the spirit of the present disclosure.

Claims (20)

1. A method for adjusting a field of view of a panorama, the method comprising:
determining dangerous obstacles around the vehicle and an initial view range of the panoramic image;
and determining an adjusted visual field range of the panoramic image according to the initial visual field range and the position information of the dangerous barrier relative to the vehicle, wherein the adjusted visual field range comprises the dangerous barrier.
2. The method of claim 1, wherein the determining of the dangerous obstacle around the vehicle comprises:
obtaining collision distance s and collision time t between each obstacle and the vehicle according to the running data of the vehicle and the obstacle information of the obstacles around the vehicle;
making the collision distance s smaller than the preset distance s0And/or the time of collision t is less than the preset time t0Is determined to be the dangerous obstacle.
3. The method according to claim 2, wherein the obtaining of the collision distance s and the collision time t between each obstacle and the vehicle based on the traveling data of the vehicle and the obstacle information of the obstacles around the vehicle comprises:
determining a passing area of the vehicle according to the running data;
and obtaining the collision distance s and the collision time t between each obstacle and the vehicle according to the passing area and the obstacle information.
4. The method according to claim 2 or 3, characterized in that the obstacle information is acquired by:
acquiring image data around a vehicle body through a camera; detecting obstacles according to the image data to obtain obstacle information of obstacles around the vehicle;
and/or the presence of a gas in the gas,
and acquiring obstacle information of obstacles around the vehicle through a radar.
5. The method of claim 1, wherein determining an initial field of view of the panoramic image comprises:
determining an initial visual field range of the panoramic image according to a preset fixed visual field range; or,
and determining the initial view range of the panoramic view according to the steering wheel rotating angle of the vehicle.
6. The method of claim 5, wherein if a coordinate system is established with a body center as an origin, a body lateral direction to the right as an X-axis positive direction, and a body longitudinal direction toward a vehicle head as a Y-axis positive direction, the determining an initial field of view of the panorama based on a steering wheel angle of the vehicle comprises:
initial forward field of view range df0Initial back field of view db0Initial left visual field range dl0Initial right field of view dr0The following relationship is satisfied:
df0+db0+lcar≥hmin
dr0+dl0+wcar≥wmin
dr can be obtained based on the steering wheel angle of the vehicle0Or dl0：
Wherein h isminRepresenting the longitudinal length covered by the panorama at said minimum field of view; w is aminRepresenting the lateral width covered by the panorama at said minimum field of view; lcarRepresenting the length of the vehicle; w is acarRepresenting the width of the vehicle; p represents an adjustable parameter;and the steering angle of the front wheels of the vehicle is represented and calculated by the steering wheel rotating angle.
7. The method of claim 1, wherein determining an adjusted field of view of the panoramic view based on the initial field of view and the location information of the hazardous obstacle relative to the vehicle comprises:
judging whether the dangerous barrier in the visual field direction which needs to be paid attention by the user is positioned in the initial visual field range;
if the dangerous obstacles in the visual field direction needing attention are all located in the initial visual field range, not adjusting the initial visual field range; otherwise, adjusting the initial view range of the view direction needing attention to enable the dangerous obstacle to be located in the adjusted view range.
8. The method of claim 7,
when the vehicle is reversed, the visual field direction needing attention comprises the left side, the right side and the rear side of the vehicle, and the initial visual field range of the visual field direction needing attention is adjusted to enable the dangerous obstacle to be positioned in the adjusted visual field range, and the method comprises the following steps:
if the dangerous obstacle on the left side of the vehicle exceeds the initial left visual field range, adjusting the initial left visual field range to enable the dangerous obstacle on the left side of the vehicle to be located in the adjusted left visual field range;
if the dangerous barrier on the right side of the vehicle exceeds the initial right view range, adjusting the initial right view range to enable the dangerous barrier on the right side of the vehicle to be located in the adjusted right view range;
if the dangerous barrier behind the vehicle exceeds the initial rear view range, adjusting the initial rear view range to enable the dangerous barrier behind the vehicle to be located in the adjusted rear view range;
or,
when the vehicle advances, the view direction needing attention comprises the left side, the right side and the front of the vehicle, and the initial view range of the view direction needing attention is adjusted to enable the dangerous obstacle to be located in the adjusted view range, and the method comprises the following steps:
if the dangerous obstacle on the left side of the vehicle exceeds the initial left visual field range, adjusting the initial left visual field range to enable the dangerous obstacle on the left side of the vehicle to be located in the adjusted left visual field range;
if the dangerous barrier on the right side of the vehicle exceeds the initial right view range, adjusting the initial right view range to enable the dangerous barrier on the right side of the vehicle to be located in the adjusted right view range;
and if the dangerous obstacle in front of the vehicle exceeds the initial front field range, adjusting the initial front field range to enable the dangerous obstacle in front of the vehicle to be positioned in the adjusted front field range.
9. The method according to claim 7 or 8, wherein if a maximum viewing range is preset, the adjusted viewing range of the panoramic image is not greater than the maximum viewing range.
10. A panorama view adjusting apparatus, comprising:
the dangerous obstacle determining module is used for determining dangerous obstacles around the vehicle;
the initial visual field range determining module is used for determining the initial visual field range of the panoramic image;
and the adjusted visual field range determining module is used for determining the adjusted visual field range of the panoramic image according to the initial visual field range and the position information of the dangerous obstacle relative to the vehicle, wherein the adjusted visual field range comprises the dangerous obstacle.
11. The apparatus of claim 10, wherein the hazardous obstacle determination module comprises:
the collision distance and time obtaining module is used for obtaining collision distances s and collision times t between each obstacle and the vehicle according to the driving data of the vehicle and the obstacle information of the obstacles around the vehicle;
a dangerous obstacle determination submodule for making the collision distance s smaller than the preset distance s0And/or the time of collision t is less than the preset time t0Is determined to be the dangerous obstacle.
12. The apparatus of claim 11,
the collision distance and time obtaining module is used for determining a passing area of the vehicle according to the running data; and obtaining the collision distance s and the collision time t between each obstacle and the vehicle according to the passing area and the obstacle information.
13. The apparatus of claim 11 or 12, further comprising:
the obstacle information acquisition module is used for acquiring image data around the vehicle body through the camera; detecting obstacles according to the image data to obtain obstacle information of obstacles around the vehicle; and/or acquiring obstacle information of obstacles around the vehicle by radar.
14. The apparatus of claim 10,
the initial visual field range determining module is used for determining an initial visual field range of the panoramic image according to a preset fixed visual field range; or determining the initial view range of the panoramic view according to the steering wheel angle of the vehicle.
15. The apparatus of claim 14, wherein if a coordinate system is established with a center of a vehicle body as an origin, a lateral right of the vehicle body as an X-axis positive direction, and a longitudinal direction of the vehicle body toward a vehicle head as a Y-axis positive direction, the determining an initial field of view of the panorama based on a steering wheel angle of the vehicle comprises:
initial forward field of view range df0Initial back field of view db0Initial left visual field range dl0Initial right field of view dr0The following relationship is satisfied:
df0+db0+lcar≥hmin
dr0+dl0+wcar≥wmin
dr can be obtained based on the steering wheel angle of the vehicle0Or dl0：
Wherein h isminRepresenting the longitudinal length covered by the panorama at said minimum field of view; w is aminRepresenting the lateral width covered by the panorama at said minimum field of view; lcarRepresenting the length of the vehicle; w is acarRepresenting the width of the vehicle; p represents an adjustable parameter;and the steering angle of the front wheels of the vehicle is represented and calculated by the steering wheel rotating angle.
16. The apparatus of claim 10,
the adjusted visual field range determining module is used for judging whether the dangerous barrier in the visual field direction which needs to be paid attention by the user is positioned in the initial visual field range; if the dangerous obstacles in the visual field direction needing attention are all located in the initial visual field range, not adjusting the initial visual field range; otherwise, adjusting the initial view range of the view direction needing attention to enable the dangerous obstacle to be located in the adjusted view range.
17. The apparatus of claim 16,
when the vehicle is reversed, the visual field direction needing attention comprises the left side, the right side and the rear side of the vehicle, and the initial visual field range of the visual field direction needing attention is adjusted to enable the dangerous obstacle to be positioned in the adjusted visual field range, and the method comprises the following steps:
if the dangerous obstacle on the left side of the vehicle exceeds the initial left visual field range, adjusting the initial left visual field range to enable the dangerous obstacle on the left side of the vehicle to be located in the adjusted left visual field range;
if the dangerous barrier on the right side of the vehicle exceeds the initial right view range, adjusting the initial right view range to enable the dangerous barrier on the right side of the vehicle to be located in the adjusted right view range;
if the dangerous barrier behind the vehicle exceeds the initial rear view range, adjusting the initial rear view range to enable the dangerous barrier behind the vehicle to be located in the adjusted rear view range;
or,
when the vehicle advances, the view direction needing attention comprises the left side, the right side and the front of the vehicle, and the initial view range of the view direction needing attention is adjusted to enable the dangerous obstacle to be located in the adjusted view range, and the method comprises the following steps:
if the dangerous obstacle on the left side of the vehicle exceeds the initial left visual field range, adjusting the initial left visual field range to enable the dangerous obstacle on the left side of the vehicle to be located in the adjusted left visual field range;
if the dangerous barrier on the right side of the vehicle exceeds the initial right view range, adjusting the initial right view range to enable the dangerous barrier on the right side of the vehicle to be located in the adjusted right view range;
and if the dangerous obstacle in front of the vehicle exceeds the initial front field range, adjusting the initial front field range to enable the dangerous obstacle in front of the vehicle to be positioned in the adjusted front field range.
18. The apparatus of claim 16 or 17, wherein if a maximum viewing range is preset, the adjusted viewing range of the panoramic view is not greater than the maximum viewing range.
19. A storage medium having stored therein a plurality of instructions, wherein said instructions are loaded by a processor for performing the steps of the method of any of claims 1 to 9.
20. An electronic device, characterized in that the electronic device comprises;
the storage medium of claim 19; and
a processor to execute the instructions in the storage medium.
CN201810300003.5A 2018-04-04 2018-04-04 Panorama view adjusting method and device, storage medium and electronic equipment Active CN108765490B (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201810300003.5A CN108765490B (en) 2018-04-04 2018-04-04 Panorama view adjusting method and device, storage medium and electronic equipment
PCT/CN2018/104419 WO2019192145A1 (en) 2018-04-04 2018-09-06 Method and apparatus for adjusting field of view of panoramic image, storage medium, and electronic device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810300003.5A CN108765490B (en) 2018-04-04 2018-04-04 Panorama view adjusting method and device, storage medium and electronic equipment

Publications (2)

Publication Number Publication Date
CN108765490A true CN108765490A (en) 2018-11-06
CN108765490B CN108765490B (en) 2021-04-30

Family

ID=63980925

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810300003.5A Active CN108765490B (en) 2018-04-04 2018-04-04 Panorama view adjusting method and device, storage medium and electronic equipment

Country Status (2)

Country Link
CN (1) CN108765490B (en)
WO (1) WO2019192145A1 (en)

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109466449A (en) * 2018-11-21 2019-03-15 北汽福田汽车股份有限公司 Display control method, system and the vehicle of vehicle
CN109743539A (en) * 2018-12-07 2019-05-10 吉林大学 A kind of adjustable panorama driving assistant device in the visual field and its adjusting method
CN110614951A (en) * 2019-09-25 2019-12-27 的卢技术有限公司 Lens conversion method and system for reconstructing surrounding environment of vehicle
CN111169381A (en) * 2019-10-10 2020-05-19 中国第一汽车股份有限公司 Vehicle image display method and device, vehicle and storage medium
CN111368641A (en) * 2020-02-11 2020-07-03 北京百度网讯科技有限公司 Obstacle screening method, obstacle screening device, electronic device and storage medium
CN111712827A (en) * 2019-04-30 2020-09-25 深圳市大疆创新科技有限公司 Method, device and system for adjusting observation field, storage medium and mobile device
CN112954263A (en) * 2019-12-10 2021-06-11 北京车和家信息技术有限公司 Road condition image acquisition method and system and vehicle
CN113442831A (en) * 2020-03-25 2021-09-28 斑马智行网络(香港)有限公司 Visual field blind area display method and device and navigation system adopting method
CN114633694A (en) * 2022-03-24 2022-06-17 岚图汽车科技有限公司 Obstacle display control method, device and equipment and readable storage medium
CN114743433A (en) * 2021-12-23 2022-07-12 中国科学院软件研究所 Multi-channel alarm presenting method and device for simulating threats in flight training environment

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112016039B (en) * 2020-08-19 2024-02-09 深圳安智杰科技有限公司 Vehicle passing area construction method and device, electronic equipment and readable storage medium
CN113092133B (en) * 2021-04-07 2024-03-22 冒坚 Ultrasonic radar in-loop automatic driving test method based on Gaussian clustering

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104786933A (en) * 2015-03-04 2015-07-22 江苏大学 Panoramic image driving auxiliary device and panoramic image driving auxiliary method
CN106097357A (en) * 2016-06-17 2016-11-09 深圳市灵动飞扬科技有限公司 The bearing calibration of auto-panorama photographic head
CN106355546A (en) * 2015-07-13 2017-01-25 比亚迪股份有限公司 Vehicle panorama generating method and apparatus
US20170032197A1 (en) * 2015-07-29 2017-02-02 Mando Corporation Camera device for vehicle
CN107757479A (en) * 2016-08-22 2018-03-06 何长伟 A kind of drive assist system and method based on augmented reality Display Technique
CN108068697A (en) * 2016-11-11 2018-05-25 法乐第(北京)网络科技有限公司 Camera system, method and automobile based on automobile

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104786933A (en) * 2015-03-04 2015-07-22 江苏大学 Panoramic image driving auxiliary device and panoramic image driving auxiliary method
CN106355546A (en) * 2015-07-13 2017-01-25 比亚迪股份有限公司 Vehicle panorama generating method and apparatus
US20170032197A1 (en) * 2015-07-29 2017-02-02 Mando Corporation Camera device for vehicle
CN106097357A (en) * 2016-06-17 2016-11-09 深圳市灵动飞扬科技有限公司 The bearing calibration of auto-panorama photographic head
CN107757479A (en) * 2016-08-22 2018-03-06 何长伟 A kind of drive assist system and method based on augmented reality Display Technique
CN108068697A (en) * 2016-11-11 2018-05-25 法乐第(北京)网络科技有限公司 Camera system, method and automobile based on automobile

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109466449A (en) * 2018-11-21 2019-03-15 北汽福田汽车股份有限公司 Display control method, system and the vehicle of vehicle
CN109743539A (en) * 2018-12-07 2019-05-10 吉林大学 A kind of adjustable panorama driving assistant device in the visual field and its adjusting method
CN109743539B (en) * 2018-12-07 2020-05-29 吉林大学 Panoramic driving auxiliary device with adjustable visual field and adjusting method thereof
CN111712827A (en) * 2019-04-30 2020-09-25 深圳市大疆创新科技有限公司 Method, device and system for adjusting observation field, storage medium and mobile device
CN110614951A (en) * 2019-09-25 2019-12-27 的卢技术有限公司 Lens conversion method and system for reconstructing surrounding environment of vehicle
CN111169381A (en) * 2019-10-10 2020-05-19 中国第一汽车股份有限公司 Vehicle image display method and device, vehicle and storage medium
CN112954263A (en) * 2019-12-10 2021-06-11 北京车和家信息技术有限公司 Road condition image acquisition method and system and vehicle
CN111368641A (en) * 2020-02-11 2020-07-03 北京百度网讯科技有限公司 Obstacle screening method, obstacle screening device, electronic device and storage medium
CN113442831A (en) * 2020-03-25 2021-09-28 斑马智行网络(香港)有限公司 Visual field blind area display method and device and navigation system adopting method
CN114743433A (en) * 2021-12-23 2022-07-12 中国科学院软件研究所 Multi-channel alarm presenting method and device for simulating threats in flight training environment
CN114633694A (en) * 2022-03-24 2022-06-17 岚图汽车科技有限公司 Obstacle display control method, device and equipment and readable storage medium

Also Published As

Publication number Publication date
CN108765490B (en) 2021-04-30
WO2019192145A1 (en) 2019-10-10

Similar Documents

Publication Publication Date Title
CN108765490B (en) Panorama view adjusting method and device, storage medium and electronic equipment
JP6675448B2 (en) Vehicle position detecting method and device
CN108638999B (en) Anti-collision early warning system and method based on 360-degree look-around input
JP4425495B2 (en) Outside monitoring device
US6734787B2 (en) Apparatus and method of recognizing vehicle travelling behind
JP4530060B2 (en) Parking support apparatus and method
CN105678787A (en) Heavy-duty lorry driving barrier detection and tracking method based on binocular fisheye camera
CN107627959B (en) Panoramic video monitoring method and system for motor vehicle
CN105828028B (en) A kind of method and device carrying out safety detection to vehicle bottom
CN112158197B (en) Vehicle blind area obstacle avoiding method, device and system
CN105539290A (en) System and method for displaying 3D panorama image of vehicle
JP2017520133A (en) Vehicle periphery image generation apparatus and method
JP2010218226A (en) Measurement map generation device and traveling environment confirmation device
JP6084048B2 (en) Object detection apparatus, object detection system, and object detection method
JP2018509799A (en) Method and apparatus for distortion-free display of the vehicle periphery of a vehicle
CN201402131Y (en) Control assistant device for split type vehicle
CN113508574A (en) Imaging system and method
Kim et al. Vehicle path prediction based on radar and vision sensor fusion for safe lane changing
JP4967758B2 (en) Object movement detection method and detection apparatus
KR20090036795A (en) Light stripe detection method for indoor navigation and parking assist apparatus using same
JP2008213647A (en) Parking assist method and parking assist system
CN105787455A (en) Lane-keeping control algorithm introducing image virtual guide line
KR102625203B1 (en) Driving assistance apparatus for vehicle and controlling method of driving assistance apparatus for vehicle
Nobori et al. A surround view image generation method with low distortion for vehicle camera systems using a composite projection
JP2004040523A (en) Surveillance apparatus for vehicle surroundings

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant