WO2013170797A1

WO2013170797A1 - Method and device for producing real-view map

Info

Publication number: WO2013170797A1
Application number: PCT/CN2013/077112
Authority: WO
Inventors: 张丽晓; 尚浩强; 唐磊
Original assignee: 中兴通讯股份有限公司
Priority date: 2012-10-17
Filing date: 2013-06-09
Publication date: 2013-11-21
Also published as: CN102915669A

Abstract

The present invention relates to a method and device for producing a real-view map, comprising: acquiring multiple acquired video images; synthesizing and processing the multiple video images into a panoramic video image; identifying a target object in the panoramic video image and the geographic information of the target object, and labeling the target object to obtain a labeled target object; re-projecting the labeled target object into the panoramic video image; performing map rendering processing on the panoramic video image and generating a real-view map according to the geographic information of the target object. The present invention enables a user to produce a real-view map at real time, and breaks the model in which traditional real-view maps can only be produced by professional teams and users can only use the maps by paying or being connected on-line. The real-view maps produced by the present invention are free for use, and eliminate the reliance to signals and geographic environment during on-line downloading.

Description

Method and device for making real map

The present invention relates to image processing and map making technology in the field of virtual reality, and in particular to a method and apparatus for making a real map. Background technique

Maps are an indispensable tool for people's lives and work. With the development of transportation and the development of tourism industry, more and more people choose to use real-life maps in order to get a better map experience. The real map is a very interesting map roaming mode. People can roam the virtual city streets with a 360 degree view to see the street scene. Combining the superior location query capabilities of the electronic map with the virtual reality experience provided by the panorama will provide great convenience for people to live and travel.

The current real-life map mainly relies on satellite maps, such as: google street map, or some professional website team to make, such as: city bar, etc. Generally, such maps need to pay or network services, and the maps are all pre-made. , can only display standard geographic information, users can not modify, label, etc., these problems are inconvenient for the user's use. Summary of the invention

In order to solve the above technical problem, an embodiment of the present invention provides a method and a device for manufacturing a real-life map, which breaks through a mode in which a real-life map needs to be produced by a professional team, and the user can only use the map for payment or networking.

In order to achieve the above object, the technical solution adopted by the embodiment of the present invention is: a method for manufacturing a real-life map, including:

Acquiring multiple video images collected; Performing a composite processing on the plurality of video images to form a panoramic video image;

Identifying the target object in the panoramic video image and the geographic information of the target object, and performing annotation processing on the target object to obtain the labeled target object;

Re-mapping the annotated target object into the panoramic video image;

And mapping the panoramic video image according to geographic information of the target object to generate a real map.

Further, synthesizing the plurality of video images to form a panoramic video image comprises: performing video synchronization on the plurality of video images;

The plurality of video images after video synchronization are spliced in real time to generate a panoramic video image.

further,

The plurality of video images are video images after being subjected to anti-shake processing.

Further, the method further includes:

Save and display the live map.

Further, labeling the target object, and obtaining the labeled target object includes:

The target object is classified and labeled using different tags according to the attributes of the geographic information.

Further, re-mapping the labeled target object into the panoramic video image includes:

Based on the panoramic video image, the geographic information is mapped to the panoramic video image in real time in conjunction with the geographic information.

Further, performing a map drawing process on the panoramic video image according to the geographic information of the target object, and generating a real-life map includes:

Matching the target object with GPS latitude and longitude information; A mark corresponding to the target object and corresponding GPS latitude and longitude information are attached to corresponding positions in the panoramic video image to generate a live view map.

Further, the mark corresponding to the target object and the corresponding GPS latitude and longitude information are attached to corresponding positions in the panoramic video image, and generating the real map includes:

And uploading a mark corresponding to the target object, corresponding GPS latitude and longitude information, and the user annotation information to a corresponding position in the panoramic video image, under the condition that the user inputs the user annotation information including the traffic condition information, and generates Real map.

An embodiment of the present invention further provides a device for creating a real map, including:

a video acquisition module, which acquires multiple video images collected;

a video synthesizing module, configured to synthesize the plurality of video images to form a panoramic video image;

a target recognition module, configured to identify a target object in the panoramic video image and geographic information of the target object, and perform annotation processing on the target object to obtain a labeled target object;

And a map drawing module, configured to remap the labeled target object into the panoramic video image, and perform map drawing processing on the panoramic video image according to geographic information of the target object to generate a real-life map.

Further, the video synthesizing module is configured to perform video synchronization on the plurality of video images, and perform real-time splicing of the plurality of video images after video synchronization to generate a panoramic video.

Further, the apparatus further includes a video anti-shake module configured to perform anti-shake processing on the plurality of video images.

Further, the device further includes:

a map storage module, configured to save the real map;

A display module, configured to display the real map. Further, the target recognition module is configured to classify the target object by using different tags according to attributes of the geographic information.

Further, the mapping module is configured to match the target object with GPS latitude and longitude information, and attach a mark corresponding to the target object and corresponding GPS latitude and longitude information to a corresponding in the panoramic video image. Location, generate a real map.

Further, the apparatus further includes a human-computer interaction module configured to input user annotation information including traffic condition information.

Further, the mapping module is further configured to add a tag corresponding to the target object, corresponding GPS latitude and longitude information, and the user annotation information to the user when the user inputs the user annotation information including the traffic condition information. A real-life map is generated to a corresponding position in the panoramic video image.

Further, the apparatus further includes a map sharing module configured to implement real-time map sharing between the vehicles.

Further, the map sharing module includes:

a determining unit, configured to determine whether other vehicles have a real-life map;

a sending unit that sends a request for real-time map sharing under the condition that other vehicles have a real-life map;

The receiving unit receives the real-life map sent by other vehicles under the condition that the other vehicles agree to the live map sharing.

The beneficial effects of the embodiments of the present invention are as follows: The embodiment of the present invention can enable the user to realize the real-time production of the real-life map, and break through the traditional real-life map that needs to be produced by a professional team, and the user can only pay or use the map in a network;

In addition to being free to use, the real scene map of the embodiment of the present invention can also avoid the dependence of the network download on the signal and the geographical environment;

The settings of the display module and the map sharing module enable the user to record the current surrounding ring in real time. The real map of the environment is displayed to the user, and at the same time, different vehicles can be shared by wlan and other communication means;

The created real-life map can be saved as a map database of the user through the map storage module for later use, and can also be shared with multiple vehicles through wlan or Bluetooth to realize the update and expansion of the map;

In addition, the embodiment of the present invention is further provided with a human-machine interaction module, which can be set to supplement and update the real-life map, so that the user can obtain the latest information of the current route, thereby greatly improving the user experience. DRAWINGS

1 is a flowchart of a method for manufacturing a live view map according to an embodiment of the present invention;

2 is a flowchart of a method for manufacturing a live view map according to an embodiment of the present invention;

3 is a structural block diagram of a device for creating a live view map according to an embodiment of the present invention;

4 is a structural block diagram of an apparatus for creating a live view map according to an embodiment of the present invention. detailed description

The structure and principle of the present invention are described in detail below with reference to the accompanying drawings, which are intended to illustrate the invention.

In order to break through the traditional real-life map, it is necessary to rely on a professional team to make a model for the user to pay for or use the map. The embodiment of the present invention provides a method for manufacturing a real-life map. As shown in FIG. 1 , the real-life map in the embodiment of the present invention is shown. The production method includes the following steps:

Step 101: Acquire multiple video images collected;

Here, the plurality of video images in the step may be separately collected by the plurality of cameras; Step 102: synthesize the plurality of video images to form a panoramic video image; Step 103: Identify a target object in the panoramic video image And the geographic information of the target object, and labeling the target object to obtain the labeled target object; Step 104: Remap the labeled target object into the panoramic video image. Step 105: Perform a map drawing process on the panoramic video image according to geographic information of the target object to generate a real-life map.

In the embodiment of the present invention, a number of cameras, a number and a position of the camera are installed in front of the vehicle, left and right, to ensure that 360-degree video images around the vehicle can be collected.

The installation position and number of specific cameras should be arranged according to the angle of view of the specific camera. For example: If a dedicated 120° wide-angle camera is used, at least one should be installed in front, left front and right front to ensure 360 around the vehicle. The scenery can be collected. If you install a camera with a small angle of view, you will need to install multiple cameras to ensure a panoramic view of the surrounding area.

When you need to create a real-life map, start the camera around the vehicle, including the camera in front of the car, left and right, and adjust the camera to work normally, that is, the camera can normally capture the current image data.

Since the camera is mounted on the vehicle, the vibration generated by the engine will cause high-frequency vibration of the camera, and during the movement of the vehicle, severe low-frequency vibration will occur due to the road. These vibrations will affect the video captured by the camera and generate video. Blur and picture jitter, etc. Therefore, it is necessary to perform anti-shake processing on the video image output by each camera, that is, motion estimation and special processing on the video image (special processing refers to image post-processing algorithms such as anti-shake processing and image sharpening to make the image clearer), eliminating Video blur and picture jitter caused by body vibration or movement, thus ensuring a clear and stable video image acquired by each camera.

In order to allow the user to see the video effect of the continuous three-dimensional panorama, there is no obvious crease and splicing discontinuity effect. The multiple video images need to be combined to form a panoramic video image, which specifically includes the following steps:

(1) performing video synchronization on the plurality of video images;

The video after anti-shake processing of each camera is not strictly synchronized in time, it must be For video synchronization and video synchronization, a time threshold can be set according to the frame rate of the video. For example, if the frame rate is 30 s, each video image within 33 ms is used as the synchronized frame. The specific threshold can be determined according to the frame rate selected by the user.

(2) real-time splicing the plurality of video images after video synchronization to generate a panoramic video image.

After synchronizing the video images of each channel, video synthesis can be performed. The basic principle of video synthesis is to perform matching and correlation analysis on each video image, and then perform real-time stitching to generate a panoramic video image, which is performed later. Target recognition and the basis for making real-life maps.

In the embodiment of the present invention, the basic principle of video synthesis is specifically to analyze content in each video, extract similar parts, and perform synthetic stitching to form a continuous real-life map. This principle is the same as the principle that the panoramic camera function of a normal camera achieves 360-degree panoramic photography.

In the embodiment of the present invention, in the step of identifying the target object in the panoramic video image and the geographic information of the target object, specifically, identifying the target object in the panoramic video image by using a professional target recognition algorithm, and extracting the Geographic information of the target object.

The target recognition algorithm may be pattern recognition or text scanning, and may of course be other target recognition algorithms as long as the target object in the panoramic video image can be identified.

The target objects are mainly streets, intersections, traffic lights, buildings, and the like in a panoramic video image.

In the embodiment of the present invention, the step of performing the labeling process on the target object to obtain the labeled target object comprises: classifying the target object by using different tags according to the attribute of the geographic information.

The mark may be a symbol mark or a text mark, and the target object may be marked with a voice mark for necessary information such as a red light. Of course, it can also be in other markup forms. No longer here - repeat.

In the embodiment of the present invention, the step of re-mapping the labeled target object into the panoramic video image includes:

In the embodiment of the present invention, the step of performing a map drawing process on the panoramic video image according to the geographic information of the target object, and generating the real map comprises:

Matching the target object with GPS latitude and longitude information;

A mark corresponding to the target object and corresponding GPS latitude and longitude information are attached to corresponding positions in the panoramic video image to generate a live view map.

As shown in FIG. 2, in order to experience a more fresh and accurate real-life map, the method for manufacturing a live view map of the embodiment of the present invention includes the following steps:

Step 201: Acquire multiple video images collected.

Here, the plurality of video images in this step can be separately collected by a plurality of cameras.

Step 202: Perform anti-shake processing on the video image output by each camera.

Step 203: Perform synthesizing processing on the plurality of video images to form a panoramic video image. Step 204: Identify a target object in the panoramic video image and geographic information of the target object, and perform annotation processing on the target object to obtain the labeled target object.

Step 205: Remap the labeled target object into the panoramic video image. Step 206: Perform a map drawing process on the panoramic video image according to geographic information of the target object to generate a real-life map.

The follow-up also includes a correction step 207 to provide the user with a more fresh and accurate real-life map. Step 207: The user inputs the user annotation information under the condition that the user needs to modify on the real map.

The user annotation information may include information such as damage to traffic lights, serious traffic jams, and the like. Step 208: Append a tag corresponding to the target object and corresponding GPS latitude and longitude information to a corresponding position in the panoramic video image to generate a real-life map.

Step 208 specifically includes:

In order to allow the user to have a more complete understanding of the surrounding geographic information, and to have a more accurate judgment on the driving route, the method for manufacturing the live view map of the embodiment of the present invention further includes the following steps:

Save and display the live map.

The drawn real map can be directly transmitted to the display device to provide the real-time map around the vehicle in real time for the user to have a fuller understanding of the surrounding geographic information and a more accurate judgment of the driving route.

The drawn real map is stored, and the real map is used as the user's own map database for later use.

In order to realize the update and expansion of the user map database, the user can detect whether other vehicles have a real-map function through communication methods such as WLAN or Bluetooth, and send to other vehicles through communication methods such as WLAN or Bluetooth under the condition that other vehicles have a real-map function. The real map maps the request, and receives the real map sent by other vehicles under the condition that other vehicles allow the real map to be shared, realizes the real map sharing, and completes the update and expansion of the user's map database.

As shown in FIG. 3, an embodiment of the present invention further provides a device for creating a real-life map, including: a video capture module, which acquires multiple video images collected;

Here, the video capture module can be multiple cameras;

a target recognition module, configured to identify a target object and a location in the panoramic video image Describe the geographic information of the target object, and perform annotation processing on the target object to obtain the labeled target object;

The video capture module is mainly configured to acquire a plurality of video images collected by a plurality of cameras installed in the front, left, and right sides of the vehicle, and the number and position of the camera installations are ensured that the video capture module can acquire 360 degrees around the vehicle. Video image.

Since the camera is mounted on the vehicle, the vibration generated by the engine will cause high-frequency vibration of the camera, and during the movement of the vehicle, severe low-frequency vibration will occur due to the road. These vibrations will affect the video captured by the camera and generate video. Blur and picture jitter, etc. In order to solve the above technical problem, the apparatus for creating a live view of the embodiment of the present invention further includes a video anti-shake module configured to perform anti-shake processing on the plurality of video images to avoid image blur.

The video anti-shake module is configured to perform anti-shake processing on the video image output by each camera, that is, perform motion estimation and special processing on the video image (special processing refers to image post-processing algorithms such as anti-shake processing and image sharpening, Make the image clearer), eliminate video blur and picture jitter caused by body vibration or movement, thus ensuring clear and stable view of each camera Frequency image.

In order to allow the user to see the video effect of the continuous three-dimensional panorama, there is no obvious crease and splicing discontinuity effect. The plurality of video images need to be combined to form a panoramic video image, and the video synthesis module is specifically configured to The plurality of video images are video-synchronized, and the plurality of video images that are synchronized by the video are spliced in real time to generate a panoramic video.

The video after anti-shake processing of each camera is not strictly synchronized in time. Video synchronization must be performed. When video synchronization is performed, a time threshold can be set according to the frame rate of the video. For example, the frame rate is 30 s, then 33 ms. Each video image within the frame is used as a synchronized frame. The specific threshold can be determined based on the frame rate selected by the user.

The target recognition module is specifically configured to identify a target object in the panoramic video image by a professional target recognition algorithm, and extract geographic information of the target object.

The target recognition module is specifically configured to classify the target object by using different tags according to attributes of the geographic information. The mark may be a symbol mark or a text mark, and the target object may be marked with a voice mark for necessary information such as a red light. Of course, it can also be other markup forms, which are not repeated here.

The mapping module is specifically configured to match the target object with GPS latitude and longitude information, and attach a mark corresponding to the target object and corresponding GPS latitude and longitude information to a corresponding position in the panoramic video image to generate Real map.

As shown in FIG. 4, in order to experience a more fresh and accurate real-life map, the device for creating a live view of the embodiment of the present invention further includes a human-computer interaction module, which is configured to input user annotation information including the information of the communication path.

The user annotation information may include information such as damage to traffic lights, serious traffic jams, and the like.

Correspondingly, the mapping module is further configured to: add a tag corresponding to the target object, corresponding GPS latitude and longitude information, and the user annotation information to the user inputting the user annotation information including the traffic condition information to A corresponding position in the panoramic video image generates a real map.

In order to allow the user to have a more complete understanding of the surrounding geographic information, and to have a more accurate judgment on the driving route, the apparatus for making a real map in the embodiment of the present invention further includes a map storage module and a display module, wherein the map storage module is set as a storage location. The live view map; the display module is configured to display the real map.

The drawn real map can be directly transmitted to the display module to provide real-time maps around the vehicle for the user to have a fuller understanding of the surrounding geographic information and a more accurate judgment of the driving route.

The drawn real map is sent to the map storage module for storage, and the real map is used as the user's own map database for later use.

In order to implement the update and extension of the user map database, the device for creating a live view map of the embodiment of the present invention further includes a map sharing module, which is configured to realize real-time map sharing between vehicles. The map sharing module includes:

The judging unit can detect whether other vehicles have a real map function by using a communication method such as WLAN or Bluetooth.

The transmitting unit transmits a request for real-time map sharing to other vehicles through a communication method such as WLAN or Bluetooth under the condition that other vehicles have a real-map function.

The receiving unit receives the real-life map sent by other vehicles under the condition that other vehicles allow real-time map sharing, realizes real-life map sharing, and completes updating and expanding the user's map database.

In the embodiment of the present invention, the display module and the human-machine interaction module are integrated into an integrated structure. When the implementation is implemented, the user can browse the real-life map through the display screen on the vehicle, and perform an operation of inputting the user's annotation information.

In the embodiment of the present invention, the video capture module, the video anti-shake module, the video synthesis module, the target recognition module, the map drawing module, the map storage module, and the map sharing module are integrated in Inside the central processing unit.

The above is a preferred embodiment of the present invention, and it should be noted that those skilled in the art can also make several improvements and retouchings without departing from the principles of the present invention. These improvements and retouchings should also be considered as The scope of protection of the present invention. Industrial applicability

The embodiment of the present invention collects and synthesizes a panoramic video image in real time, identifies the target object, performs labeling processing on the target object, and then remaps the image to the panoramic video image according to the target object. The information is obtained by performing a map drawing process on the panoramic video image to generate a real-life map. By adopting the invention, the user can realize the real-time production of the real-life map, and is completely free, and breaks through the traditional real-life map, which needs to be produced by a professional team, and the user can only use the map of the map for payment or networking.

Claims

claims

1. A method for making a real-life map, the method includes:

Obtain multiple collected video images;

Synthesize the plurality of video images to form a panoramic video image;

Identify the target object in the panoramic video image and the geographical information of the target object, and perform annotation processing on the target object to obtain the annotated target object;

Remap the annotated target object into the panoramic video image;

According to the geographical information of the target object, map drawing processing is performed on the panoramic video image to generate a real scene map.

2. The method of making a real-scene map according to claim 1, wherein said synthesizing the plurality of video images to form a panoramic video image includes:

Video synchronization of the plurality of video images;

The multiple video images after video synchronization are spliced in real time to generate a panoramic video image.

3. The method for making a real-life map according to claim 1, wherein,

The plurality of video images are video images that have undergone anti-shake processing.

4. The method for making a real-life map according to claim 1, wherein the method further includes:

Save and display the live map.

5. The production method of a real-life map according to claim 1, wherein the labeling process on the target object, and the obtained labeled target object includes:

Different tags are used to classify and label the target objects according to the attributes of the geographical information.

6. The method of making a real-scene map according to claim 1, wherein the remapping the annotated target object to the panoramic video image includes: Based on the panoramic video image and combined with the geographical information, the geographical information is mapped to the panoramic video image in real time.

7. The production method of a real-scene map according to claim 1, wherein said performing map drawing processing on the panoramic video image according to the geographical information of the target object and generating a real-scene map includes:

Match the target object with GPS latitude and longitude information;

Attach the mark corresponding to the target object and the corresponding GPS latitude and longitude information to the corresponding position in the panoramic video image to generate a real-view map.

8. The method of making a real-scene map according to claim 7, wherein the mark corresponding to the target object and the corresponding GPS longitude and latitude information are added to the corresponding position in the panoramic video image to generate a real-scene map. include:

Under the condition that the user inputs user annotation information including traffic condition information, a mark corresponding to the target object, the corresponding GPS latitude and longitude information and the user annotation information are appended to the corresponding position in the panoramic video image to generate Real map.

9. A device for making a real-life map, the device includes:

The video collection module is configured to obtain multiple collected video images;

A video synthesis module configured to synthesize the plurality of video images to form a panoramic video image;

The target recognition module is configured to identify the target object in the panoramic video image and the geographical information of the target object, and perform annotation processing on the target object to obtain the annotated target object;

The map drawing module is configured to remap the annotated target object into the panoramic video image, and perform map drawing processing on the panoramic video image according to the geographical information of the target object to generate a real scene map.

10. The device for producing a real scene map according to claim 9, wherein the video combination A module is configured to perform video synchronization on the plurality of video images, and perform real-time splicing of the plurality of video images after video synchronization to generate a panoramic video.

11. The real-scene map production device according to claim 9, wherein the device further includes a video anti-shake module configured to perform anti-shake processing on the plurality of video images.

12. The real-scene map production device according to claim 9, wherein the device further includes:

A map storage module, configured to save the real-life map;

The display module is configured to display the saved real-life map.

13. The real-scene map production device according to claim 9, wherein the target identification module is configured to use different markers to classify and label the target objects according to the attributes of the geographical information.

14. The real-scene map production device according to claim 9, wherein the map drawing module is configured to match the target object with GPS longitude and latitude information, and to match the mark corresponding to the target object with the corresponding The GPS latitude and longitude information is appended to the corresponding location in the panoramic video image to generate a real scene map.

15. The real-scene map production device according to claim 14, wherein the device further includes a human-computer interaction module configured for the user to input user annotation information including traffic condition information.

16. The device for producing a real-scene map according to claim 15, wherein the map drawing module is further configured to, under the condition that the user inputs user annotation information including traffic condition information, draw the map corresponding to the target object. Markers, corresponding GPS latitude and longitude information and the user annotation information are appended to the corresponding positions in the panoramic video image to generate a real scene map.

17. The real-scene map production device according to claim 9, wherein the device further includes a map sharing module configured to realize real-scene map sharing between vehicles.

18. The real-scene map production device according to claim 17, wherein the map sharing module includes: A judgment unit, configured to judge whether other vehicles have real-life maps;

The sending unit sends a request for sharing the real-view map under the condition that other vehicles have real-view maps;

The receiving unit receives the real-view map sent by other vehicles under the condition that other vehicles agree to share the real-view map.