CN103971392A - Navigation-oriented three-dimensional video data processing method and device, system and terminal - Google Patents

Abstract

The present invention is applicable to the field of car navigation, and provides a navigation-oriented three-dimensional video data processing method, device, system and terminal. The method includes the following steps: searching for a three-dimensional model of the road object according to the extracted road object; Rendering and drawing the 3D model by using the scene rendering view set calculated according to the 3D model of the road object to generate a 3D scene video; associating the road object, the 3D scene video, and the scene rendering view set, Generate navigation-oriented 3D scene videos. The invention provides users with effective three-dimensional navigation information by superimposing the real-time driving information of the vehicle with the pre-generated video recording of the three-dimensional scene, reduces the requirements for hardware, reduces the storage capacity of data, and improves the practicability of the data.

Description

Navigation-oriented 3D video data processing method, device, system and terminal

technical field

The invention is applicable to the field of car navigation, and in particular relates to a navigation-oriented three-dimensional video data processing method, device, system and terminal.

Background technique

How to better guide vehicles in complex scenes has always been a hot issue in the field of navigation. Especially for large-scale overpasses, the spatial relationship between each sub-intersection is intricate, and only using two-dimensional data for guidance often leads to misjudgment by drivers. Displaying complex relationships in a three-dimensional form can help users quickly locate and judge.

At present, in the field of navigation electronic maps, the following methods are mainly used for three-dimensional navigation:

(1) Three-dimensional model map technology: This technology uses a pre-generated static road intersection scene picture to help users locate and orientate. It has the advantage of low hardware requirements. However, the 3D mode map is a static picture, which can neither be fused with the real-time driving information of the vehicle nor provide more spatial relationship information. Therefore, for complex scenes, this technology can only provide limited help to users.

(2) Real-time 3D rendering technology based on 3D model: This technology submits 3D model data to a Graphics Processing Unit (GPU) pipeline for rendering. Its advantage is that it can provide interactive browsing, and the display effect is better. However, this technology has high requirements on model modeling and hardware. And when the vehicle is moving, the driver has no time to appreciate the three-dimensional model outside the guiding path, the model data is not practical, and the data storage capacity is large.

(3) Real-time 3D rendering technology based on panoramic images: This technology provides users with 3D positioning and orientation information by handing over the panoramic images collected in the field to GPU rendering. Its advantage is that it is consistent with the real scene and rich in information. However, there are many details in the panoramic image, which is not conducive to the driver to extract useful information in a short time. During data collection, the viewing angle of the panoramic camera is generally low, and cannot provide users with macroscopic information of the environment they are in. Moreover, this technology requires GPU support, and the hardware conditions are relatively high.

Contents of the invention

The embodiment of the present invention provides a navigation-oriented three-dimensional video data generation method, which aims to solve the problems of high hardware requirements, impractical data and large storage capacity of existing navigation systems.

The embodiment of the present invention is achieved in this way, a navigation-oriented three-dimensional video data generation method, the method includes the following steps:

Finding the three-dimensional model of the road object according to the extracted road object;

Rendering the 3D model by using the scene rendering perspective set calculated according to the 3D model of the road object to generate a 3D scene video;

The road object, the 3D scene video, and the scene rendering perspective set are associated to generate a navigation-oriented 3D scene video.

The embodiment of the present invention also provides a navigation-oriented three-dimensional video data generating device, the device comprising:

a road object 3D model search unit, configured to search for a 3D model of the road object based on the extracted road object;

A 3D scene video configuration unit, configured to use the scene rendering perspective set calculated according to the 3D model of the road object to render and draw the 3D model to generate a 3D scene video; and

The navigation-oriented 3D scene video configuration unit is configured to associate the road object, the 3D scene video, and the scene rendering perspective set to generate a navigation-oriented 3D scene video.

An embodiment of the present invention also provides a car navigation system, which includes the above navigation-oriented three-dimensional video data generation device.

An embodiment of the present invention also provides a car navigation terminal, which includes the above-mentioned navigation-oriented three-dimensional video data generation device.

The embodiment of the present invention also provides a navigation-oriented three-dimensional video data processing method, the method comprising the following steps:

Finding the three-dimensional model of the road object according to the extracted road object;

Rendering the 3D model by using the scene rendering perspective set calculated according to the 3D model of the road object to generate a 3D scene video;

Associating the road object, the 3D scene video, and the scene rendering perspective set to generate a navigation-oriented 3D scene video;

According to the current planned path of the vehicle, extract relevant viewing angle parameters, three-dimensional video frame data, and each frame of the current path-related video from the navigation-oriented three-dimensional scene video;

Rendering the current planned path of the vehicle to generate a planned path in a relevant viewing angle space by using the viewing angle parameters and the three-dimensional video frame data;

Associating the planned path under the relevant perspective space with each frame of the current path-related video to generate and output each frame of the 3D video of the navigation system.

The embodiment of the present invention also provides a navigation-oriented three-dimensional video data processing device, the device comprising:

a road object 3D model extraction unit, configured to search for a 3D model of the road object based on the extracted road object;

A 3D scene video generating unit, configured to render and draw the 3D model by using the scene rendering perspective set calculated according to the 3D model of the road object, to generate a 3D scene video;

A navigation-oriented 3D scene video generating unit, configured to associate the road object, 3D scene video, and the scene rendering perspective set to generate a navigation-oriented 3D scene video;

A relevant video viewing angle parameter extraction unit, configured to extract relevant viewing angle parameters, three-dimensional video frame data, and each frame of the current path-related video from the navigation-oriented three-dimensional scene video according to the current planned path of the vehicle;

A planning path configuration unit in a relevant viewing angle space, configured to use the viewing angle parameters and 3D video frame data to render the current planned path of the vehicle to generate a planning path in a relevant viewing angle space; and

The navigation video output unit is configured to associate the planned path in the relevant perspective space with each frame of the current path-related video, and generate and output each frame of the 3D video of the navigation system.

An embodiment of the present invention also provides a car navigation system, which includes the above-mentioned navigation-oriented three-dimensional video data processing device.

An embodiment of the present invention also provides a car navigation terminal, which includes the above-mentioned navigation-oriented three-dimensional video data processing device.

In the embodiment of the present invention, effective 3D navigation information is provided for the user by superimposing the real-time driving information of the vehicle with the pre-generated video recording of the 3D scene, which reduces the requirements for hardware, reduces the storage capacity of data, and improves the data storage capacity. practicality.

Description of drawings

Fig. 1 represents the implementation flowchart of the navigation-oriented three-dimensional video data generating method provided by one embodiment of the present invention;

Fig. 2 shows the implementation flowchart of the navigation-oriented three-dimensional video data generation method provided by another embodiment of the present invention;

Fig. 3 shows the implementation flowchart of the navigation-oriented three-dimensional video data processing method provided by one embodiment of the present invention;

Fig. 4 shows the implementation flowchart of the navigation-oriented three-dimensional video data processing method provided by another embodiment of the present invention;

FIG. 5 shows a structural diagram of a navigation-oriented three-dimensional video data generation device provided by an embodiment of the present invention;

FIG. 6 shows a structural diagram of a navigation-oriented three-dimensional video data processing device provided by an embodiment of the present invention.

Detailed ways

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

The embodiment of the present invention provides users with effective 3D navigation information by superimposing the real-time driving information of the vehicle with the video recording of the 3D scene generated in advance, which reduces the requirements for hardware, reduces the storage capacity of data, and improves the practicality of data. sex.

Fig. 1 shows the implementation process of the navigation-oriented three-dimensional video data generation method provided by one embodiment of the present invention, which is described in detail as follows:

In step S101, according to the extracted road object, the three-dimensional model of the road object is searched;

In step S102, use the scene rendering perspective set calculated according to the 3D model of the road object to render and draw the 3D model to generate a 3D scene video;

In the embodiment of the present invention, the scene rendering perspective set is the lens position information and direction information of the virtual 3D camera at the three-dimensional intersection calculated by using the scene rendering perspective selector according to the three-dimensional model of the road object.

In the embodiment of the present invention, the 3D scene video is a frame sequence that is output after the 3D rendering server renders and draws the 3D model of the road object according to the viewing angle parameters in the scene rendering viewing angle set.

In step S103, the road object, the 3D scene video, and the scene rendering perspective set are associated to generate a navigation-oriented 3D scene video.

Fig. 2 shows the implementation process of the navigation-oriented three-dimensional video data generation method provided by another embodiment of the present invention, which is described in detail as follows:

In step S201, the road object is extracted from the navigation electronic map data;

In step S202, according to the extracted road object, search the corresponding 3D model data in the 3D model database, that is, the 3D model of the road object;

In step S203, calculate the scene rendering perspective set according to the three-dimensional model of the road object;

In the embodiment of the present invention, in order to ensure clear and definite intersection traffic features and surrounding environment in the video, it is necessary to select the viewing angle used for 3D rendering. These viewpoints, the cameras, will form the keyframes for the animation of the scene.

These key frames in the embodiment of the present invention should meet the following conditions:

A. The line of sight direction is at a certain angle to the normal direction of the ground.

B. The overpass/intersection is located in the center of the projection surface and can show the connection between the overpass and the overpass.

C. The road under the overpass should be visible in the projection surface.

D. There is a shooting camera at the position of the overpass/intersection object entering and exiting the boundary node.

E. It is usually enough to use 4 shooting lenses at ordinary level crossings.

Based on the above principles, in the embodiment of the present invention, key frames can be selected manually or automatically.

For automatic selection, several discrete positions can be pre-generated for a certain intersection, and then the capping situation of the model projected on the near projection surface at a certain position can be calculated, then this problem can be transformed into a two-dimensional capping calculation problem. The bird's-eye view at a higher position contains more information. Its corresponding picture can be shared, so 4 pictures are enough.

In step S204, use the scene rendering perspective set to render and draw the 3D model of the road object to generate a 3D scene video;

In the embodiment of the present invention, each frame in the frame sequence of the 3D scene video includes frame color information and frame depth information, and each frame is associated with a corresponding viewing angle parameter.

According to the selected viewing angle, the embodiment of the present invention uses the rendering server to perform three-dimensional rendering on the three-dimensional model, and stores each rendered picture and its related parameter information in the memory. The relevant parameter information includes: camera frustum parameters, camera position and rotation orientation parameters in the frame, time position of the current frame in the entire picture sequence, and depth buffer information of the current frame.

In step S205, the road object, the 3D scene video, and the scene rendering perspective set are associated to generate a navigation-oriented 3D scene video.

The embodiment of the present invention associates road objects, 3D scene videos, and scene rendering perspective sets to generate navigation-oriented 3D scene videos, does not require a navigator to have a GPU, has low hardware requirements, and improves the practicality of data for complex actual scenes. sex.

Fig. 3 is the implementation process of the navigation-oriented three-dimensional video data processing method provided by an embodiment of the present invention, which is described in detail as follows:

In step S301, according to the extracted road object, the three-dimensional model of the road object is searched;

In step S302, use the scene rendering perspective set calculated according to the 3D model of the road object to render and draw the 3D model to generate a 3D scene video;

In step S303, associating the road object, the 3D scene video, and the scene rendering perspective set to generate a navigation-oriented 3D scene video;

In step S304, according to the current planned route of the vehicle, extract relevant viewing angle parameters, three-dimensional video frame data, and each frame of the current route-related video from the navigation-oriented three-dimensional scene video;

In step S305, the current planned path of the vehicle is rendered to generate a planned path in a relevant viewing angle space by using the viewing angle parameters and the three-dimensional video frame data;

In an embodiment of the present invention, when the GPS information shows that the vehicle is about to enter an intersection or an overpass, each frame in the video is processed as follows:

1. Get the frustum of the current 3D scene frame, the transformation matrix of the viewing angle, the depth buffer and other parameter information.

2. Project the planned path onto the projection surface of the virtual camera by using the model-view transformation matrix calculated according to the viewing angle parameters.

3. Draw the operation result to the real-time information frame.

4. Calculate the depth buffer of the planned path.

In step S306, associating the planned path in the relevant perspective space with each frame of the current path-related video, generating each frame of the 3D video for output to the navigation system.

Fig. 4 is the implementation process of the navigation-oriented three-dimensional video data processing method provided by another embodiment of the present invention, which is described in detail as follows:

In step S401, the road object is extracted from the navigation electronic map data;

In step S402, according to the extracted road object, find the corresponding 3D model data in the 3D model database, that is, the 3D model of the road object;

In step S403, calculate the scene rendering perspective set according to the three-dimensional model of the road object;

In step S404, use the scene rendering perspective set to render and draw the 3D model of the road object to generate a 3D scene video;

In step S405, associating the road object, the 3D scene video, and the scene rendering perspective set to generate a navigation-oriented 3D scene video;

In step S406, extract relevant viewing angle parameters, three-dimensional video frame data, and each frame of the current path-related video;

In the embodiment of the present invention, according to the current planned path of the vehicle, the 3D scene video extractor extracts relevant viewing angle parameters, 3D video frame data, and each frame of the current path-related video from the navigation-oriented 3D scene video.

In step S407, using the viewing angle parameters and the 3D video frame data, the renderer of the planned path renders the current planned path of the vehicle to generate a planned path under the relevant viewing angle space;

In the embodiment of the present invention, the color buffer and depth buffer stored in the 3D video frame data and the viewing angle parameters corresponding to the frame are set in the rendering system to generate a planning path in the relevant viewing angle space.

In step S408, associate the planned path in the relevant perspective space with each frame of the current path-related video, generate each frame of the 3D video of the navigation system, and output it on the screen.

In the embodiment of the present invention, the bitmap of the three-dimensional scene frame and the bitmap of the real-time information frame are alpha-blended, and the depth buffers corresponding to the two frames are compared when performing the blending operation.

Fig. 5 shows the structure of the device for generating navigation-oriented 3D video data provided by the embodiment of the present invention. For convenience of description, only the parts related to the embodiment of the present invention are shown.

The device can be integrated in various car navigation systems. These navigation systems can be software units, hardware units or a combination of software and hardware running in terminals such as mobile phones, PDAs, and car navigators. They can also be used as independent pendants. Integrate into the application system of the electronic terminal.

The road object three-dimensional model search unit 51 searches for a three-dimensional model of the road object based on the extracted road object.

The 3D scene video configuration unit 52 renders and draws the 3D model by using the scene rendering perspective set to generate a 3D scene video. Wherein, the scene rendering perspective set is a scene rendering perspective set calculated according to the 3D model of the road object searched by the road object 3D model search unit 51 .

In the embodiment of the present invention, the scene rendering perspective set is the lens position information and direction information of the virtual 3D camera at the three-dimensional intersection calculated by using the scene rendering perspective selector according to the three-dimensional model of the road object.

In the embodiment of the present invention, the 3D scene video is a frame sequence that is output after the 3D rendering server renders and draws the 3D model of the road object according to the viewing angle parameters in the scene rendering viewing angle set.

In the embodiment of the present invention, each frame in the frame sequence includes frame color information and frame depth information, and each frame is associated with a corresponding viewing angle parameter.

The navigation-oriented 3D scene video configuration unit 53 associates the road object, the 3D scene video generated by the 3D scene video configuration unit 52, and the scene rendering perspective set calculated according to the 3D model of the road object searched by the road object 3D model search unit 51, Generate navigation-oriented 3D scene videos.

FIG. 6 shows the structure of the navigation-oriented 3D video data processing provided by the embodiment of the present invention. For the convenience of description, only the parts related to the embodiment of the present invention are shown.

The device can be integrated in various car navigation systems. These navigation systems can be software units, hardware units or a combination of software and hardware running in terminals such as mobile phones, PDAs, and car navigators. They can also be used as independent pendants. Integrate into the application system of the electronic terminal.

The navigation-oriented three-dimensional video data generating device 5 composed of the road object three-dimensional model search unit 51, the three-dimensional scene video configuration unit 52, and the navigation-oriented three-dimensional scene video configuration unit 53 is processed as an independent part with the navigation-oriented three-dimensional video data The devices are integrated together to provide high-quality three-dimensional navigation functions on low-configuration navigation equipment.

The relevant video viewing angle parameter extraction unit 61 extracts relevant viewing angle parameters, three-dimensional video frame data, and each frame of the current path related video from the navigation-oriented three-dimensional scene video in the navigation-oriented three-dimensional scene video configuration unit 53 according to the current planned route of the vehicle. .

The planning path configuration unit 62 in the relevant viewing angle space uses the viewing angle parameters and 3D video frame data extracted by the relevant video viewing angle parameter extraction unit 61 to render the current planned path of the vehicle to generate a planning path in the relevant viewing angle space.

The navigation video output unit 63 associates the planned path under the relevant viewing angle space generated by the planning path configuration unit 62 under the relevant viewing angle space with each frame of the relevant video of the current path extracted by the relevant video viewing angle parameter extraction unit 61, and generates and outputs the three-dimensional view of the navigation system. every frame of the video.

The embodiments of the present invention are very effective for the technical field of vehicle navigation, but are not limited to the field of vehicle navigation. For other fields, if three-dimensional navigation is required due to local changes, the embodiments of the present invention are applicable.

In the embodiment of the present invention, effective 3D navigation information is provided for the user by superimposing the real-time driving information of the vehicle with the pre-generated video recording of the 3D scene, which reduces the requirements for hardware, reduces the storage capacity of data, and improves the data storage capacity. practicality.

The above is only a preferred embodiment of the present invention, it should be pointed out that for those of ordinary skill in the art, without departing from the principle of the present invention, some improvements and modifications can also be made, and these improvements and modifications should also be It is regarded as the protection scope of the present invention.

Claims (20)

1. the three dimensional video data generation method towards navigation, is characterized in that, said method comprising the steps of:

According to the road object extracting, search the three-dimensional model of described road object;

Utilize the scene rendering visual angle collection calculating according to the three-dimensional model of described road object, described three-dimensional model is carried out to color applying drawing, generating three-dimensional scene video;

By described road object, three-dimensional scenic video, and the collection association of described scene rendering visual angle, generate towards the three-dimensional scenic video of navigation.

2. the method for claim 1, is characterized in that, described scene rendering visual angle collection is according to described road object three-dimensional model, utilizes lens location information and the directional information of the virtual 3D camera in three-dimensional crossing of scene rendering visual angle selector switch calculating.

3. the method for claim 1, is characterized in that, described three-dimensional scenic video is the visual angle parameter concentrated according to described scene rendering visual angle, adopts three-dimensional rendering server to carry out to described road object three-dimensional model the frame sequence of exporting after color applying drawing.

4. method as claimed in claim 3, is characterized in that, each frame in described frame sequence includes frame chromatic information, and frame depth information;

The associated corresponding visual angle parameter of each frame.

5. towards a three dimensional video data generating apparatus for navigation, it is characterized in that, described device comprises:

Road object three-dimensional model is searched unit, for according to the road object extracting, searches the three-dimensional model of described road object;

Three-dimensional scenic video configuration unit, for utilizing the scene rendering visual angle collection calculating according to the three-dimensional model of described road object, carries out color applying drawing to described three-dimensional model, generating three-dimensional scene video; And

Towards the three-dimensional scenic video configuration unit of navigation, for by described road object, three-dimensional scenic video, and the collection association of described scene rendering visual angle, generate towards the three-dimensional scenic video of navigation.

6. device as claimed in claim 5, is characterized in that, described scene rendering visual angle collection is according to described road object three-dimensional model, utilizes lens location information and the directional information of the virtual 3D camera in three-dimensional crossing of scene rendering visual angle selector switch calculating.

7. device as claimed in claim 5, is characterized in that, described three-dimensional scenic video is the visual angle parameter concentrated according to described scene rendering visual angle, adopts three-dimensional rendering server to carry out to described road object three-dimensional model the frame sequence of exporting after color applying drawing.

8. device as claimed in claim 7, is characterized in that, each frame in described frame sequence includes frame chromatic information, and frame depth information;

The associated corresponding visual angle parameter of each frame.

9. an auto-navigation system, is characterized in that, described auto-navigation system comprises the three dimensional video data generating apparatus towards navigation described in the arbitrary claim of claim 5 to 8.

10. a car navigation terminal, is characterized in that, described car navigation terminal comprises the three dimensional video data generating apparatus towards navigation described in the arbitrary claim of claim 5 to 8.

11. 1 kinds of three dimensional video data disposal routes towards navigation, is characterized in that, described method comprises:

According to the road object extracting, search the three-dimensional model of described road object;

Utilize the scene rendering visual angle collection calculating according to the three-dimensional model of described road object, described three-dimensional model is carried out to color applying drawing, generating three-dimensional scene video;

By described road object, three-dimensional scenic video, and the collection association of described scene rendering visual angle, generate towards the three-dimensional scenic video of navigation;

According to the current path planning of vehicle, the three-dimensional scenic video from described towards navigation, extract relevant visual angle parameter, 3 D video frame data, and each frame of current path associated video;

Utilize described visual angle parameter and 3 D video frame data, current vehicle path planning is played up to the path planning generating under space, relevant visual angle;

Path planning under space, described relevant visual angle is associated with described each frame of current path associated video, generate each frame of 3 D video of exporting navigational system.

12. methods as claimed in claim 11, is characterized in that, described scene rendering visual angle collection is according to described road object three-dimensional model, utilize lens location information and the directional information of the virtual 3D camera in three-dimensional crossing of scene rendering visual angle selector switch calculating.

13. methods as claimed in claim 11, is characterized in that, described three-dimensional scenic video is the visual angle parameter concentrated according to described scene rendering visual angle, adopt three-dimensional rendering server to carry out to described road object three-dimensional model the frame sequence of exporting after color applying drawing.

14. methods as claimed in claim 13, is characterized in that, each frame in described frame sequence includes frame chromatic information, and frame depth information;

The associated corresponding visual angle parameter of each frame.

15. 1 kinds of three dimensional video data treating apparatus towards navigation, is characterized in that, described device comprises:

Road object three-dimensional model extraction unit, for according to the road object extracting, searches the three-dimensional model of described road object;

Three-dimensional scenic video generation unit, for utilizing the scene rendering visual angle collection calculating according to the three-dimensional model of described road object, carries out color applying drawing to described three-dimensional model, generating three-dimensional scene video;

Towards the three-dimensional scenic video generation unit of navigation, for by described road object, three-dimensional scenic video, and the collection association of described scene rendering visual angle, generate towards the three-dimensional scenic video of navigation;

Associated video visual angle parameter extraction unit, for according to the current path planning of vehicle, extracts relevant visual angle parameter, 3 D video frame data the three-dimensional scenic video from described towards navigation, and each frame of current path associated video;

Path planning dispensing unit under space, relevant visual angle, for utilizing described visual angle parameter and 3 D video frame data, plays up current vehicle path planning the path planning generating under space, relevant visual angle; And

Navigation video output unit, for the path planning under space, described relevant visual angle is associated with described each frame of current path associated video, generates each frame of 3 D video of output navigational system.

16. devices as claimed in claim 15, is characterized in that, described scene rendering visual angle collection is according to described road object three-dimensional model, utilize lens location information and the directional information of the virtual 3D camera in three-dimensional crossing of scene rendering visual angle selector switch calculating.

17. devices as claimed in claim 15, is characterized in that, described three-dimensional scenic video is the visual angle parameter concentrated according to described scene rendering visual angle, adopt three-dimensional rendering server to carry out to described road object three-dimensional model the frame sequence of exporting after color applying drawing.

18. devices as claimed in claim 17, is characterized in that, each frame in described frame sequence includes frame chromatic information, and frame depth information;

The associated corresponding visual angle parameter of each frame.

19. 1 kinds of auto-navigation systems, is characterized in that, described auto-navigation system comprises the three dimensional video data treating apparatus towards navigation described in the arbitrary claim of claim 15 to 18.

20. 1 kinds of car navigation terminals, is characterized in that, described car navigation terminal comprises the three dimensional video data treating apparatus towards navigation described in the arbitrary claim of claim 15 to 18.