CN110109552B - Modeling method of virtual driving scene based on real environment - Google Patents

Abstract

The invention discloses a virtual driving scene modeling method based on a real environment, comprising the steps of: 1) using a panoramic camera to shoot a panoramic video of a real driving scene at a fixed moving speed; 2) establishing a spherical model in the virtual space of Unity3D , paste the captured panoramic video on the inside of the sphere; 3) Put a car body model into the center of the sphere scene, as a virtual driving operation vehicle; 4) Dynamic sphere scene modeling, including sphere scene speed adjustment and scene left and right Offset adjustment. The present invention builds a virtual driving scene through a real panoramic video, and solves the technical problem that the existing automobile simulated driving product adopts 3D modeling to build the scene, which takes a long time, and the built scene has a large gap with the real scene.

Description

Modeling method of virtual driving scene based on real environment

technical field

The invention relates to the technical field of automobile virtual driving, in particular to a virtual driving scene modeling method based on a real environment.

Background technique

Traditional driving simulation products usually use the following methods when building a scene: 1) Use 3dsMax and other 3D model making software to create the geometric shape of the target model, and combine it with Photoshop and other image processing software to map; 2) Use games such as Unity3d The engine development tool builds a virtual space, imports each model made in the previous step into the virtual space for placement, and then sets the physical properties corresponding to each model. One of the biggest advantages of using this method to build a scene is that all objects in the scene can have physical logic and natural characteristics (such as physical collision, natural fall, force acceleration, etc.) according to the functions contained in the development engine used. etc.), the developer does not need to spend effort on this.

But this scene model also has its inherent disadvantage - unreal. Use software such as 3D Max and Photoshop to make models. If you want good visual effects, you need experienced professionals to invest a lot of time to achieve them, and this good visual effect is only a beautiful model, which is different from real objects. very large. For simulated driving equipment, if the difference between the user's experience and the actual driving experience is too large, the training effect will be greatly reduced.

SUMMARY OF THE INVENTION

In view of this, the purpose of the present invention is to provide a virtual driving scene modeling method based on a real environment, so as to solve the problem that the existing automobile simulation driving products adopt 3D modeling to build a scene, which takes a long time and has a large gap between the built scene and the real scene. question.

The virtual driving scene modeling method based on the real environment of the present invention comprises the following steps:

1) Use a panoramic camera to shoot a panoramic video of a real driving scene at a fixed moving speed, and the setting height of the panoramic camera in the shooting process is consistent with the driver's eye height on the vehicle;

2) Create a sphere model in the virtual space of Unity3D, and paste the captured panoramic video inside the sphere by editing the sphere model shader;

3) Put a car body model into the center of the spherical scene as a virtual driving vehicle;

4) Put a virtual camera in the driver's position of the vehicle body model, and the setting height of the virtual camera is consistent with the height of the driver's eyes on the vehicle body model; and realize the synchronization in the development kit of the VR helmet worn by the driver. The script is added to the virtual camera, so that the coordinate and angle information collected by the 3D space tracking locator that detects the coordinate position and rotation angle of the VR headset in the 3D space can be used as the reference input of the synchronization script of the virtual camera, so that the virtual camera can follow The VR helmet rotates synchronously; and the scene image in the current corner direction captured by the virtual camera is input to the VR helmet.

Further, the virtual driving scene modeling method based on the real environment also includes the following steps:

5) Establish the relationship between the scene change speed and the input signal of the simulated driving pedal. The specific steps are as follows:

a) According to the dynamic parameters of a certain actual vehicle, calculate the average acceleration value of the vehicle, including the average acceleration a _c1 of the accelerator and the average acceleration a _c2 of the brake:

In the above formula, v ₁₀ and v ₁₁ represent the initial speed and final speed of the accelerator acceleration process, respectively, in km/h, and the accelerator acceleration time is Δt ₁ second; v ₂₀ and v ₂₁ represent the initial speed and final speed of the braking deceleration process, respectively, The unit is km/h, and the braking deceleration time is Δt ₂ seconds;

b) Calculate the accelerator acceleration a ₁ , the braking acceleration a ₂ and the current acceleration a of the virtual driving vehicle of this model according to the input signal of the simulated driving pedal and the average acceleration value of a certain actual vehicle:

In the above _formula : δ1 simulates the input signal value _of the accelerator pedal, δ2 simulates the input signal value of the driving brake pedal, and the value range of the input signal _of δ1 and _δ2 is [-32767, 32767];

c) Calculate the current speed v _D of the virtual driving vehicle according to the current acceleration a:

v _D = v _D0 +(a·Δt)×3.6 (6)

In the above formula, v _D and v _D0 represent the vehicle speed at the current moment and the previous moment, respectively, in km/h, and the interval between the two calculations before and after is Δt seconds;

d) according to the current speed and the attitude deflection angle of the virtual driving vehicle, dynamically change the scene video playback rate coefficient k, and the expression of the scene video playback rate coefficient k is as follows:

In the above formula, k _max is the maximum playback rate coefficient, and its value depends on the computer hardware configuration, and is generally set between 2 and 3; v ₀ is the moving speed of the camera when shooting panoramic videos of real driving scenes, and v _D is virtual driving The current running speed of the vehicle, v ₀ km _max is the maximum allowed running speed of the virtual driving vehicle; θ is the attitude deflection angle of the virtual driving vehicle, and its value is calculated by the following formula (8).

Further, the virtual driving scene modeling method based on the real environment also includes the following steps:

6) Establish the relationship between the input signal of the simulated driving steering wheel and the pose and lateral position of the virtual driving vehicle in the spherical scene. The specific steps are as follows:

a) Set the steering coefficient q=ω _s /θ _s , where ω _s is the unilateral maximum rotation angle of the simulated driving steering wheel, and θ _s is the maximum steering angle of the steering wheel of the virtual driving vehicle;

b) Set the attitude deflection angle θ of the virtual driving vehicle in the spherical scene as:

In the above formula, ω is the input angle signal of the simulated driving steering wheel;

c) According to the current speed and attitude deflection angle of the virtual driving vehicle, obtain the lateral translation speed v _x of the vehicle:

v _x = v _D · sinθ (9)

d) Set the left and right translation range of the virtual driving vehicle in the spherical scene as R _c . When the virtual driving vehicle translation range is within R _c , that is, x∈[-R _c ,R _c ], the virtual driving vehicle is in the current lane. The scene video moves left and right; when the range of translation exceeds R _c , that is, x<-R _c or x>R _c , the virtual driving vehicle switches to the scene video of the adjacent lane during the moving process.

Beneficial effects of the present invention:

1. The present invention is based on the virtual driving scene modeling method of the real environment. The virtual driving scene is built through the real panoramic video, and the modeling efficiency is high. It also solves the problem that the existing automobile simulation driving products adopt 3D modeling to build the scene, which is time-consuming and difficult to build. The technical problem is that there is a large gap between the scene and the real scene; by associating the virtual camera with the VR helmet, the driver can observe the driving scene in the spherical model immersively, and the driving reality is strong.

2. The virtual driving scene modeling method based on the real environment of the present invention combines the acceleration and deceleration of the virtual driving vehicle with the playback rate of the scene video, and associates the posture and lateral position changes of the virtual driving vehicle with the steering wheel input. In the built driving scene, the virtual driving vehicle interacts with people, and the driving reality is strong.

Description of drawings

Figure 1 shows the sphere scene model.

Figure 2 is the rendering of the sphere scene model.

FIG. 3 is a schematic diagram of a deflection action of a virtual driving vehicle.

Detailed ways

The present invention will be further described below with reference to the accompanying drawings and embodiments.

The virtual driving scene modeling method based on the real environment in this embodiment includes the following steps:

1) Use a panoramic camera to shoot a panoramic video of the real driving scene at a fixed moving speed, the video is several consecutive pictures; the setting height of the panoramic camera during the shooting process is consistent with the driver's eye height on the vehicle. In this embodiment, the panoramic camera is Insta360ONE produced by Insta360 Company. The camera has an anti-shake function, which can largely eliminate vibration and shaking when the camera moves during the shooting process. Of course, other types of panoramic cameras can also be used in different embodiments.

2) Create a sphere model in the virtual space of Unity3D, and paste the panoramic video shot inside the sphere by editing the sphere model shader.

3) Put a car body model into the center of the spherical scene as a virtual driving vehicle.

4) Put a virtual camera in the driver's position of the vehicle body model, and the setting height of the virtual camera is consistent with the height of the driver's eyes on the vehicle body model; and realize the synchronization in the development kit of the VR helmet worn by the driver. The script is added to the virtual camera, so that the coordinate and angle information collected by the 3D space tracking locator that detects the coordinate position and rotation angle of the VR headset in the 3D space can be used as the reference input of the synchronization script of the virtual camera, so that the virtual camera can follow The VR helmet rotates synchronously; and the scene image in the current corner direction captured by the virtual camera is input into the VR helmet.

In this embodiment, a sphere model is established in the virtual space, and the captured panoramic video is pasted inside the sphere using the Unity3D game engine. In this example, the video material in the 3840*1920@30fps format shot by the Install360ONE panoramic camera is selected, and the video material is imported into the project of the Unity3D engine, and then the sphere scene is created. Create a sphere model object in the virtual space of Unity3D, and add a VideoPlayer component to the object. VideoPlayer is a built-in component of Unity3D. Using this component and with the corresponding API, you can play video in Unity3d and perform functions such as switching, pausing, and adjusting the playback rate. After adding the component, set it. The rendering mode RenderMode indicates the way the image is displayed, including Camera Far Plane, Camera Near Plane, Render texture and Material Override. Combined with the actual situation, the screen needs to be displayed on the sphere model, and the Material Override mode is applicable. Similarly, select the sphere in the Renderer item. In the construction, Source represents the video source, and the video source can be selected by directly selecting the video file in the project or by selecting the storage location where the video is stored in the computer through the URL.

After the VideoPlayer component is set up, the video will be displayed on the surface of the sphere model object. The scene construction is to place the subjective perspective at the center of the sphere, so that you can watch the surrounding scene, so you need to modify the Shader file to change the rendering position of the video to the inner surface of the sphere. Shader is part of the computer graphics rendering pipeline, it uses a piece of code to indicate how objects in the scene are rendered. This process involves calculating colors and shading areas, etc., and then giving them to objects to display them. By modifying the Shader file, change the rendering position of the sphere model to the inside of the sphere, and change the rendering brightness to always bright (that is, cancel the influence of the lighting system in Unity3d).

After the video components and Shader files are processed, a virtual camera Camera object is placed in the driver's position of the vehicle body model as the driver's subjective perspective; and the implementation synchronization script in the VR development kit is added to the virtual camera Camera object, In this way, when the program is running, the locator will detect the angle and position of the VR headset in the real environment, and then use the detected angle and position as the reference input of the synchronization script to control the angle and position of the virtual camera, so that the virtual camera follows the VR The helmet rotates synchronously, and the scene image in the current corner direction captured by the virtual camera is input to the VR helmet. When the system is running, the driver can watch the real environment around the spherical scene by wearing the VR helmet, which is very close to reality.

As an improvement to this embodiment, the described real environment-based virtual driving scene modeling method further includes the following steps:

5) Establish the relationship between the scene change speed and the input signal of the simulated driving pedal. The specific steps are as follows:

a) According to the dynamic parameters of a certain actual vehicle, calculate the average acceleration value of the vehicle, including the average acceleration a _c1 of the accelerator and the average acceleration a _c2 of the brake:

In the above formula, v ₁₀ and v ₁₁ represent the initial speed and final speed of the accelerator acceleration process, respectively, in km/h, and the accelerator acceleration time is Δt ₁ second; v ₂₀ and v ₂₁ represent the initial speed and final speed of the braking deceleration process, respectively, The unit is km/h, and the braking deceleration time is Δt ₂ seconds;

b) Calculate the accelerator acceleration a ₁ , the braking acceleration a ₂ and the current acceleration a of the virtual driving vehicle of this model according to the input signal of the simulated driving pedal and the average acceleration value of a certain actual vehicle:

In the above _formula : δ1 simulates the input signal value _of the accelerator pedal, δ2 simulates the input signal value of the driving brake pedal, and the value range of the input signal _of δ1 and _δ2 is [-32767, 32767];

c) Calculate the current speed v _D of the virtual driving vehicle according to the current acceleration a:

v _D = v _D0 +(a·Δt)×3.6 (6)

In the above formula, v _D and v _D0 represent the vehicle speed at the current moment and the previous moment, respectively, in km/h, and the interval between the two calculations before and after is Δt seconds;

d) according to the current speed and the attitude deflection angle of the virtual driving vehicle, dynamically change the scene video playback rate coefficient k, and the expression of the scene video playback rate coefficient k is as follows:

In the above formula, k _max is the maximum playback rate coefficient, and its value depends on the computer hardware configuration, and is generally set between 2 and 3; v ₀ is the moving speed of the camera when shooting panoramic videos of real driving scenes, and v _D is virtual driving The current running speed of the vehicle, v ₀ km _max is the maximum allowed running speed of the virtual driving vehicle; θ is the attitude deflection angle of the virtual driving vehicle, and its value is calculated by the following formula (8).

In this embodiment, the moving speed v ₀ of the camera during material recording is kept at 20km/h, and the frame rate of the captured panoramic video is 30fps. Since the virtual driving system is mainly oriented to driving students, the virtual driving vehicle in the system is set in this embodiment. The maximum travel speed v ₀ km _max is 60km/h (of course, in the specific implementation, the maximum travel speed v ₀ km _max can also be set to other values as required). , the maximum video playback rate km _max is 3. By dynamically adjusting the playback rate of the video, the virtual driving vehicle can be driven at different speeds.

As a further improvement to this embodiment, the virtual driving scene modeling method based on the real environment further includes the following steps:

6) Establish the relationship between the input signal of the simulated driving steering wheel and the pose and lateral position of the virtual driving vehicle in the spherical scene. The specific steps are as follows:

a) Set the steering coefficient q=ω _s /θ _s , where ω _s is the unilateral maximum rotation angle of the simulated driving steering wheel, and θ _s is the maximum steering angle of the steering wheel of the virtual driving vehicle;

b) Set the attitude deflection angle θ of the virtual driving vehicle in the spherical scene as:

In the above formula, ω is the input angle signal of the simulated driving steering wheel;

c) According to the current speed and attitude deflection angle of the virtual driving vehicle, obtain the lateral translation speed v _x of the vehicle:

v _x = v _D · sinθ (9)

d) Set the left and right translation range of the virtual driving vehicle in the spherical scene as R _c . When the virtual driving vehicle translation range is within R _c , that is, x∈[-R _c ,R _c ], the virtual driving vehicle is in the current lane. The scene video moves left and right; when the range of translation exceeds R _c , that is, x<-R _c or x>R _c , the virtual driving vehicle switches to the scene video of the adjacent lane during the moving process.

In this embodiment, the R _c =0.5r, where r is the radius of the spherical model S, and setting R _c =0.5r in actual use can obtain a natural lane change effect.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be Modifications or equivalent substitutions without departing from the spirit and scope of the technical solutions of the present invention should be included in the scope of the claims of the present invention.

Claims (1)

1. The virtual driving scene modeling method based on the real environment is characterized by comprising the following steps:

1) shooting a panoramic video of a real driving scene at a fixed moving speed by using a panoramic camera, wherein the set height of the panoramic camera in the shooting process is consistent with the height of eyes of a driver on a vehicle;

2) establishing a sphere model in a virtual space of Unity3D, and pasting a shot panoramic video to the inner side of a sphere through an editing sphere model shader;

3) putting a vehicle body model into the sphere center position of a sphere scene to serve as a virtual driving vehicle;

4) placing a virtual camera at the position of a driver of a vehicle body model, wherein the set height of the virtual camera is consistent with the eye height of the driver on the vehicle body model, and adding a script which is used for realizing synchronization in a development kit of a VR helmet worn by the driver to the virtual camera so as to enable coordinate and angle information which are acquired by a three-dimensional space tracking positioner for detecting the coordinate position and the rotation angle of the VR helmet in a three-dimensional space to be used as synchronous script reference input of the virtual camera, so that the virtual camera synchronously rotates along with the VR helmet; inputting a scene image captured by the virtual camera in the current corner direction into the VR helmet;

the virtual driving scene modeling method based on the real environment further comprises the following steps:

5) establishing a relation between the scene change speed and the simulated driving pedal input signal, and specifically comprising the following steps of:

a) calculating the average acceleration value of a certain actual vehicle according to the dynamic parameters of the vehicle, including the average acceleration a of the accelerator_c1And average brake acceleration a_c2：

In the above formula v₁₀And v₁₁Respectively representing the initial speed and the final speed of the accelerator in the acceleration process, unit km/h,acceleration time of accelerator is delta t₁Second; v. of₂₀And v₂₁Respectively representing the initial speed and the final speed of the braking deceleration process, the unit km/h and the braking deceleration time delta t₂Second;

b) calculating the acceleration a of the accelerator of a virtual driving vehicle according to the input signal of the simulated driving pedal and the average acceleration value of a certain actual vehicle₁Braking acceleration a₂And current acceleration a:

in the above formula: delta₁Simulating the value of the input signal, delta, of the accelerator pedal in driving₂Input signal value, delta, simulating the braking of a driving brake pedal₁And delta₂The input signals have value ranges of [ -32767, 32767]；

c) Calculating the current speed v of the virtual driving vehicle according to the current acceleration a_D：

v_D＝v_D0+(a·Δt)×3.6 (6)

In the above formula v_DAnd v_D0Respectively representing the vehicle speed at the current moment and the vehicle speed at the previous moment, wherein the unit of km/h is the interval time delta t seconds of the two previous and later calculations;

d) dynamically changing a scene video playing rate coefficient k according to the current speed and the attitude deflection angle of the virtual driving vehicle, wherein the expression of the scene video playing rate coefficient k is as follows:

in the above formula k_maxIs at mostThe playing rate coefficient, the value of which depends on the hardware configuration height of the computer; v. of₀For taking panoramic video of a real driving scene, the moving speed v of the camera_DFor virtually driving the current speed of travel of the vehicle, v₀k_maxA maximum travel speed allowed for the virtual drive vehicle; θ is the attitude yaw angle of the virtually driven vehicle, and its value is calculated by the following equation (8);

the virtual driving scene modeling method based on the real environment further comprises the following steps:

6) establishing a relation between an input signal of a simulated driving steering wheel and the pose and the transverse position of a virtual driving vehicle in a spherical scene, and specifically comprising the following steps of:

a) setting the steering coefficient q ═ ω_s/θ_sWherein ω is_sTo simulate a single maximum rotation angle, θ, of the steering wheel_sA maximum steering angle of a steering wheel for the virtual driving vehicle;

b) setting the attitude deflection angle theta of the virtual driving vehicle in the spherical scene as follows:

in the above formula, ω is an input angle signal of the simulated driving steering wheel;

c) obtaining the lateral translation speed v of the automobile according to the current speed and the attitude deflection angle of the virtual driving vehicle_x：

v_x＝v_D·sinθ (9)

d) Setting the left-right translation range of the virtual driving vehicle in the spherical scene as R_cWhen the virtual driving vehicle is in the translation range of R_cWithin, i.e. x ∈ [ -R_c,R_c]The virtual driving vehicle moves left and right in the scene video of the current lane; when the range of translation exceeds R_cWhen x < -R_cOr x > R_cAnd switching the virtual driving vehicle to the scene video of the adjacent lane in the moving process.

Images