WO2020014832A1

WO2020014832A1 - Map loading method and device, electronic apparatus, and readable storage medium

Info

Publication number: WO2020014832A1
Application number: PCT/CN2018/095824
Authority: WO
Inventors: 易万鑫; 廉士国; 林义闽
Original assignee: 深圳前海达闼云端智能科技有限公司
Priority date: 2018-07-16
Filing date: 2018-07-16
Publication date: 2020-01-23
Also published as: CN109074408B; CN109074408A

Abstract

A map loading method and device, an electronic apparatus, and a readable storage medium The map loading method comprises: acquiring current image data of an environment (101); performing, according to the current image data of the environment, computation to obtain first illumination information currently corresponding to the environment (102); determining, according to the first illumination information, a current time segment of the environment (103); and determining a positioning map to be loaded according to the time segment and a correspondence between the time segment and the positioning map, and loading the positioning map (104). The map loading method enables precise selection of a positioning map when the current illumination of an environment changes, thereby increasing a success rate and precision of positioning using an image captured in real time.

Description

Map loading method, device, electronic device and readable storage medium

Technical field

The present application relates to the field of computer vision, and in particular, to a method, an apparatus, an electronic device, and a readable storage medium for loading a map.

Background technique

If an intelligent robot or a driverless vehicle wants to complete some simple or complex functions in an unknown environment, it needs to know the map information of the entire unknown environment. By acquiring the information of the unknown environment, a map of the unknown environment is established so that the intelligent robot or the unmanned vehicle can be located. Only successful mapping and positioning can guarantee the robot's navigation and other functions.

technical problem

The inventors discovered during the research of the existing technology that currently, visual real-time positioning and map construction (Visual Simultaneous Localization And Mapping (VSLAM)) is used for mapping. However, because VSLAM is an image processing, it is very sensitive to the intensity of the light and vulnerable to light. If the current lighting of the environment changes, the difference between the lighting of the currently loaded positioning map and the current lighting of the environment will become larger, which will cause the currently loaded positioning map to be unavailable and seriously affect the positioning effect.

It can be seen how to replace or update the positioning map corresponding to the current light intensity of the environment to improve the positioning effect when the environment lighting changes, resulting in the current positioning map being unavailable.

Technical solutions

A technical problem to be solved in some embodiments of the present application is how to accurately select a positioning map when the current ambient light changes, thereby improving the success rate and accuracy of positioning images captured in real time.

An embodiment of the present application provides a method for loading a map, including: obtaining current image data of an environment; calculating first current lighting information corresponding to the environment according to the current image data of the environment; and determining the current environment according to the first lighting information. The time zone in which it is located; according to the current time zone of the environment and the correspondence between the time zone and the location map, determine the location map to be loaded and load the location map.

An embodiment of the present application further provides a device for loading a map, including: an obtaining module, a first determining module, a second determining module, and a map loading module; the obtaining module is used to obtain the current image data of the environment; the first determining module It is used to calculate the first lighting information currently corresponding to the environment according to the current image data of the environment; the second determination module is used to determine the current time period of the environment according to the first light information; the map loading module is used to determine the current time period of the environment and The correspondence between the time period and the positioning map, determining the positioning map to be loaded and loading the positioning map.

An embodiment of the present application further provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are at least A processor executes the method to enable at least one processor to execute the above-mentioned map loading.

An embodiment of the present application further provides a computer-readable storage medium storing a computer program, and the computer program is implemented by a processor to implement the foregoing method for loading a map.

Beneficial effect

Compared to the prior art, in some embodiments of the present application, the current first lighting information of the environment is used to determine the current time period of the environment, so that a positioning map corresponding to the current time period of the environment can be determined. Because there is a corresponding relationship between the changes in lighting in the environment and the time period in which the environment is located, there is also a corresponding relationship between the positioning map and the time period. Therefore, the current first light information of the environment can accurately determine the correspondence with the current first light information of the environment Positioning map, so that even if the current lighting of the environment changes, the positioning map can be accurately determined, thereby ensuring that the lighting of the currently loaded positioning map and the environment's lighting in the current period are small, even without lighting The difference improves the success rate and accuracy rate of real-time image positioning.

BRIEF DESCRIPTION OF THE DRAWINGS

One or more embodiments are exemplified by the pictures in the accompanying drawings. These exemplary descriptions do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the drawings in the drawings do not constitute a limitation on scale.

FIG. 1 is a specific flowchart of a map loading method in a first embodiment of the present application;

FIG. 2 is a schematic flowchart of a specific process of constructing a positioning map of an environment, determining a mapping relationship between the positioning map of the environment and a time period, and determining a correspondence relationship between lighting information of the environment and a time period in the first embodiment of the present application;

3 is a specific flowchart of a map loading method in a second embodiment of the present application;

4 is a schematic flowchart of a specific process of constructing a positioning map of an environment and determining a correspondence between the positioning map of the environment and a time period in the second embodiment of the present application;

5 is a schematic structural diagram of a map loading device in a third embodiment of the present application;

FIG. 6 is a schematic structural diagram of an electronic device in a fourth embodiment of the present application.

Embodiments of the invention

In order to make the purpose, technical solution, and advantages of the present application clearer, some embodiments of the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the application, and are not used to limit the application. However, a person of ordinary skill in the art can understand that in the embodiments of the present application, many technical details are provided in order to make the reader better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical solution claimed in this application can be implemented.

The first embodiment of the present application relates to a method for loading a map. The method for loading a map may be applicable to electronic devices that use VSLAM technology to build a map, such as unmanned vehicles, intelligent robots, and the like. The map loading method, the specific process is shown in Figure 1:

Step 101: Acquire current image data of the environment.

Specifically, the image data of the environment at the current moment can be obtained by using a sensor. For example, the image data of the environment at the current moment can be obtained by a camera. The image data includes M images, where M is an integer greater than 1, and shooting in multiple images The content covers a wider area of the environment. For example, if the environment is an outdoor gymnasium, you can shoot the gymnasium every 1 meter away. If you get 5 images of the gymnasium in the current period (assuming 5 images are captured in a period), you will take 5 images of the gymnasium. As the image data of the stadium at the current time. It can be understood that a wide-angle lens may be adopted to increase the shooting range of the camera, so that the captured content contained in the captured image is rich, and the type of the camera is not limited in this embodiment.

In a specific implementation, before acquiring current image data of an environment, it is necessary to construct a positioning map of the environment, and determine the correspondence between the location map of the environment and the time period, and determine the correspondence between the lighting information of the environment and the time period. The process is shown in Figure 2.

Step 2011: Image data of the environment is collected in each period of N cycles, where N is an integer greater than 0.

Specifically, a day can be divided into m periods, each period contains m periods, and m is an integer greater than 1, where N periods can be consecutive N periods or non-continuous N periods. Cycles, this embodiment is described by taking consecutive N cycles as an example.

In a specific implementation, image data of the environment is collected in the same period of N consecutive cycles, where N is an integer greater than 0; in each period, M images of the environment are collected at preset distance intervals, and M is greater than An integer of 1.

Specifically, whether the environment to be collected is an indoor environment or an outdoor environment, the light intensity of the environment changes with time. For example, if the environment to be collected is a road in a park, the light intensity of the road during the day is greater than the light intensity of the road at night. A day can be divided into m periods, m is an integer greater than 1, for example, m is 2, that is, a day is divided into two periods, respectively, daytime and evening periods; and the environment is collected in the same period of N cycles continuously The image data may be, for example, image data of roads in the park collected during the daytime period and image data of roads in the park during the night period.

Collecting image data of the environment in the same period of N consecutive cycles improves the accuracy of the correspondence between the period and the location map of the environment, and the accuracy of the correspondence between the period and the lighting information.

Collect M images of the environment at preset intervals in each period. For example, if you collect image data of a stadium during the daytime period, you can collect an image of the stadium every 1 meter. For the M images in the time period, M can be determined according to the size of the stadium and the angle of view taken by the camera. Of course, for the accuracy of the map to be constructed, the preset distance of collection can be reduced.

Step 2012: According to the image data collected at each time period, a positioning map of the environment is constructed and the corresponding relationship between the time period and the positioning map of the environment is determined.

Specifically, the VSLAM technology is used to construct a positioning map of the environment while collecting the image data of the environment, and when the positioning map of the environment is constructed, the correspondence between the positioning map and the time period is established. For example, the image data of the gymnasium is collected at time t1, and the positioning map constructed using VSLAM technology uses "t1" as the identifier of the constructed positioning map, thereby establishing the correspondence between the time period and the positioning map. It can be understood that other ways can also be used to establish the correspondence between the location map of the environment and the time period. Of course, the form of the correspondence between the time zone and the location map of the environment can also be in other ways, which are no longer in this embodiment List.

Step 2013: The second illumination information corresponding to the environment in different periods is calculated according to the image data collected at each period.

Specifically, the illumination information may be a brightness value or a grayscale value of the acquired image. In this embodiment, the brightness value is used as the illumination information.

In a specific implementation, the process of calculating the second illumination information corresponding to the environment in a period is: based on the image data collected in the same period of N cycles, determining the corresponding first Three lighting information; according to the image data collected in each period, determine the third lighting information corresponding to the environment in the same period of N cycles; calculate the third lighting information corresponding to the environment in the same period of N cycles An average value; and an average value of the third illumination information is used as the second illumination information corresponding to the environment in the same period. The image data collected in the same period of each cycle is processed as follows: the average value of the fourth illumination information corresponding to each of the M images in the period is calculated; and the average value of the fourth illumination information is used as the environment in one The third illumination information corresponding to the same period of the period. The fourth illumination information corresponding to an image is calculated and determined according to the total number of pixels included in the image and the illumination information of each pixel.

The process of the second illumination information corresponding to the computing environment in different periods will be specifically described below.

Take one day as a cycle, and divide the day into m periods, and collect M images of the environment in each period. The images included in the image data are identified by the sequence P _ij , i represents the number of days, and j represents the sequence number of the period. The total number of days is N days. Then the calculation formula of the fourth illumination information h0 corresponding to an image P _ij is shown in Formula 2.1:

Among them, f (x, y) represents the lighting information of each pixel of the image, that is, the pixel value of each pixel, B is the total number of pixels contained in the image, and δ represents a non-zero positive that approaches zero. Numbers, such as 0.001, δ are used to prevent situations where the logarithmic calculation result tends to be negative infinity. By formula 2.1, the fourth illumination information corresponding to the image P _ij can be calculated.

Calculate the average value of the fourth illumination information corresponding to the M images in each period, that is:

Among them, h _ij represents the lighting information of the environment in the j period of the day with the serial number i, that is, the third lighting information corresponding to the environment in a period of a period.

Calculate the average value of the third illumination information corresponding to the environment in the same period of N cycles, that is,

H _j represents the second illumination information corresponding to the environment in the same period j.

Similarly, according to formula 2.1, formula 2.2, and formula 2.3, the second illumination information corresponding to the environment in other periods can be calculated and obtained.

Step 102: Calculate first illumination information currently corresponding to the environment according to the current image data of the environment.

In a specific implementation, the current image data of the environment includes M images, and the fourth lighting information corresponding to each image in the current image data of the environment is separately calculated; and the current environment of the environment is determined according to the fourth lighting information corresponding to each image. Corresponding first lighting information.

Specifically, the process of calculating the fourth illumination information corresponding to each image in the current image data of the environment is substantially the same as the process of step 2013, that is, each of the current image data of the environment can be calculated by formula 2.1. The fourth lighting information corresponding to the image will not be described in detail. The first illumination information corresponding to the current environment can be obtained by bringing the fourth illumination information of each image in the current image data of the environment into Formula 2.2.

Step 103: Determine, according to the first illumination information, a time period in which the environment is currently located.

In a specific implementation, the first illumination information is respectively different from the second illumination information corresponding to the environment in different periods, and the current period of the environment is determined according to the difference result.

Specifically, the first illumination information is respectively different from the second illumination information corresponding to the environment at different periods, and the period corresponding to the smallest difference result is taken as the period in which the environment is currently located. The manner in which the first illumination information is different from the second illumination information corresponding to the environment in a period is as follows:

H _j represents the second illumination information corresponding to the environment in the j period,

Represents the current first illumination information of the environment. By selecting the smallest value of j, the period corresponding to the difference can be determined.

Step 104: Determine the positioning map to be loaded and load the positioning map according to the current time period of the environment and the corresponding relationship between the time period and the positioning map.

The second embodiment of the present application relates to a method for loading a map. The second embodiment is substantially the same as the first embodiment, and the main difference is that this embodiment does not need to acquire data according to different time periods before acquiring the current image data of the environment. Image data of the image, and calculate second illumination information corresponding to the environment in different periods. This embodiment is applied to an environment with a lighting device. The specific process is shown in FIG. 3:

Step 301: Acquire current image data of the environment.

Specifically, this step is substantially the same as step 101 in the first embodiment, and details are not described herein again.

It should be noted that before acquiring the current image data of the environment, it is necessary to construct a positioning map of the environment and determine the correspondence between the positioning map of the environment and the time period. The specific process is shown in Figure 4:

Step 4011: Collect image data of the environment in each period of N cycles, where N is an integer greater than 0.

Step 4012: According to the image data collected at each time period, a positioning map of the environment is constructed and the corresponding relationship between the time period and the positioning map of the environment is determined.

Steps

4011 and 4012 are substantially the same as

steps

2011 and 2012 in the first embodiment, and details are not described herein again.

Step 302: Identify the lighting devices in the current M images of the environment according to the current M images of the environment.

Specifically, the current image data of the environment includes M images, where M is an integer greater than 1. The deep learning method is used to obtain feature points in the M images and identify the lighting device in each image. The lighting device includes various types of lamps.

Step 303: Calculate the lighting device in any of the current M images of the environment or the fifth lighting information within the preset range of the lighting device, and calculate the calculated lighting device or the fifth lighting information within the preset range of the lighting device. The lighting information is used as the first lighting information currently corresponding to the environment.

Specifically, the fifth lighting information of the lighting device in any of the current M images of the environment is calculated, and the calculation method of the fifth lighting information h1 can refer to formula 2.1, such as:

Wherein W is the total pixel points included in the lighting device in the image, and f (x, y) represents the pixel value of the pixel points of the lighting device in the image. It can be understood that other ways of calculating the lighting information may also be adopted.

Alternatively, the fifth lighting information in the preset range of the lighting device in any of the current M images of the environment is calculated. The preset range of the lighting device may be centered on the center of mass of the lighting device in the image, and the preset The area where the pixel is a radius may also be other areas around the lighting device. For example, a circular area with a center of mass of the fluorescent lamp as the center and a radius of 8 pixels as the preset range of the fluorescent lamp. The calculation method of the fifth lighting information in the preset range of the lighting device is substantially the same as the calculation method of the fifth lighting information of the lighting device, and details are not described herein again.

Step 304: Determine whether the lighting device or the fifth lighting information within a preset range of the lighting device exceeds a preset lighting threshold. If yes, go to step 305; otherwise, go to step 306.

Specifically, different periods include a day period and an evening period, and the preset lighting threshold may be determined according to the lighting information of the lighting device during the day period and the evening period. If the fifth lighting information in the lighting device or the preset range of the lighting device exceeds the preset lighting threshold, it indicates that the lighting device is turned on, and the lighting in the lighting device or the preset range of the lighting device is bright and the brightness value is high. The environment is currently at night. If the fifth lighting information in the lighting device or the preset range of the lighting device does not exceed the preset lighting threshold, it indicates that the lighting device is not turned on, and the lighting in the lighting device or the preset range of the lighting device is dark. , The brightness value is low, it can be determined that the environment is currently in the daytime period.

Step 305: Determine that the environment is currently at night.

Step 306: Determine that the environment is currently in the daytime period.

Step 307: Determine the positioning map to be loaded and load the positioning map according to the current time period of the environment and the corresponding relationship between the time period and the positioning map.

Compared with the prior art, the map loading method provided in this embodiment identifies a lighting device in current image data of an environment, and according to the fifth lighting information of the identified lighting device or the fifth lighting information in a preset range of the lighting device To determine whether the lighting device is turned on, so as to determine the current time period of the environment, and then determine the current positioning map corresponding to the environment. This method only needs to calculate the fifth lighting information of the lighting device or the preset range of the lighting device. The calculation and judgment are simple and the processing speed is fast, so that the location map of the environment can be determined quickly and accurately.

The third embodiment of the present application relates to a device 50 for loading a map, including: an obtaining module 501, a first determining module 502, a second determining module 503, and a map loading module 504; the specific structure of the map loading device is shown in FIG. 5 Show:

The acquisition module 501 is configured to acquire current image data of the environment; the first determination module 502 is configured to calculate first lighting information currently corresponding to the environment according to the current image data of the environment; the second determination module 503 is configured to determine according to the first lighting information The current time period of the environment; the map loading module 504 is configured to determine the positioning map to be loaded and load the positioning map according to the current time period of the environment and the corresponding relationship between the time period and the positioning map.

This embodiment is an embodiment of a virtual device corresponding to the above-mentioned map loading method. The technical details in the above-mentioned method embodiment are still applicable in this embodiment, and details are not described herein again.

It should be noted that the device embodiments described above are only schematic and do not limit the scope of protection of this application. In practical applications, those skilled in the art may select some or all of the modules according to actual needs. To achieve the purpose of the solution of this embodiment, there is no limitation here.

A fourth embodiment of the present application relates to an electronic device, whose structure is shown in FIG. 6. It includes: at least one processor 601; and a memory 602 communicatively connected to the at least one processor 601. The memory 602 stores instructions executable by at least one processor 601. The instructions are executed by the at least one processor 601, so that the at least one processor 601 can execute the above-mentioned map loading method.

In this embodiment, the processor uses a central processing unit (CPU) as an example, and the memory uses a readable and writable memory (Random Access Memory) as an example. The processor and the memory may be connected through a bus or other methods. In FIG. 6, the connection through the bus is taken as an example. The memory, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs, and modules. The positioning map is stored in the memory as in the embodiment of the present application. The processor executes various functional applications and data processing of the device by running non-volatile software programs, instructions, and modules stored in the memory, that is, the method for loading the map described above.

The memory may include a storage program area and a storage data area, wherein the storage program area may store an operating system and an application program required for at least one function; the storage data area may store a list of options and the like. In addition, the memory may include a high-speed random access memory, and may further include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory may optionally include a memory remotely set with respect to the processor, and these remote memories may be connected to an external device through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

One or more modules are stored in the memory, and when executed by one or more processors, the method for loading a map in any of the foregoing method embodiments is executed.

The above products can execute the map loading method provided in the embodiment of the present application, and have the corresponding functional modules and beneficial effects of the execution method. For technical details not described in this embodiment, please refer to the map loading provided in the embodiment of the present application. Methods.

A fifth embodiment of the present application relates to a computer-readable storage medium. The readable storage medium is a computer-readable storage medium, and the computer-readable storage medium stores computer instructions that enable a computer to execute the first The method for loading a map involved in the first or second method embodiment.

That is, those skilled in the art can understand that all or part of the steps in the method of the above embodiments can be implemented by a program instructing related hardware. The program is stored in a storage medium and includes several instructions for making a device ( It may be a single-chip microcomputer, a chip, or the like) or a processor to perform all or part of the steps of the method described in each embodiment of the present application. The foregoing storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disks or optical disks and other media that can store program codes .

Those of ordinary skill in the art can understand that the foregoing embodiments are specific embodiments for implementing the present application, and in practical applications, various changes can be made in form and details without departing from the spirit and range.

Claims

A method for loading a map, including:

Get the current image data of the environment;

Calculating, according to current image data of the environment, first illumination information currently corresponding to the environment;

Determining, according to the first lighting information, a time period in which the environment is currently located;

Determine the positioning map to be loaded and load the positioning map according to the current time period and the corresponding relationship between the time period and the positioning map.
The method for loading a map according to claim 1, wherein before acquiring current image data of the environment, the method for loading a map further comprises:

Collecting image data of the environment at each period of N cycles, where N is an integer greater than 0;

According to the image data collected at each time period, a positioning map of the environment is constructed and a corresponding relationship between the time period and the positioning map of the environment is determined.
The method for loading a map according to claim 2, wherein collecting image data of the environment at each period of N cycles specifically comprises:

Collecting image data of the environment in the same period of consecutive N cycles;

In each period, M images of the environment are collected at preset distance intervals, where M is an integer greater than 1.
The method for loading a map according to claim 3, wherein after acquiring image data of the environment in each period of N cycles, before acquiring current image data of the environment, the method for loading the map further comprises:

According to the image data collected in each period, the second illumination information corresponding to the environment in different periods is calculated.
The method for loading a map according to claim 4, wherein calculating the second illumination information corresponding to the environment in different periods according to the image data collected at each period specifically includes:

The following processing is performed on the image data acquired by the environment in the same period of N cycles:

Determining, according to the image data acquired during the same period of N cycles, the third illumination information corresponding to each of the environments during the same period of N cycles;

Calculating an average value of the third illumination information corresponding to each of the environments in the same period of N cycles;

The average value of the third illumination information is used as the second illumination information corresponding to the environment in the same period.
The method for loading a map according to claim 5, wherein determining third illumination information corresponding to each of the environments in the same period of N cycles according to the image data collected during the same period of N cycles specifically includes:

Perform the following processing on the image data collected in the same period of each cycle:

Calculating an average value of the fourth illumination information corresponding to each of the M images in the period;

Using the average value of the fourth illumination information as the third illumination information corresponding to the environment in the period of the period;

The fourth illumination information corresponding to an image is calculated and determined according to the total number of pixels included in the image and the illumination information of each pixel.
The method for loading a map according to any one of claims 4 to 6, wherein the current image data of the environment includes M images;

Calculating the first illumination information currently corresponding to the environment according to the current image data of the environment specifically includes:

Separately calculate fourth illumination information corresponding to each image in the current image data of the environment;

The first lighting information currently corresponding to the environment is determined according to the fourth lighting information corresponding to each image.
The method for loading a map according to claim 7, wherein determining the time period in which the environment is currently located according to the first lighting information specifically includes:

The first illumination information is respectively different from the second illumination information corresponding to the environment in different periods, and the current period of the environment is determined according to the difference result.
The method for loading a map according to claim 2 or 3, wherein the current image data of the environment includes M images, where M is an integer greater than 1;

Calculating the first illumination information currently corresponding to the environment according to the current image data of the environment specifically includes:

Identifying the lighting device in the current M images of the environment according to the current M images of the environment;

Calculate the fifth lighting information of the lighting device or the preset range of the lighting device in any of the current M images of the environment, and calculate the preset range of the lighting device or the lighting device The internal fifth lighting information is used as the first lighting information currently corresponding to the environment.
The method for loading a map according to claim 9, wherein the different periods include a day period and an evening period;

Determining the current time period of the environment according to the first lighting information specifically includes:

It is determined whether the lighting device or the fifth lighting information within a preset range of the lighting device exceeds a preset lighting threshold; if so, it is determined that the environment is currently in the night period; otherwise, it is determined that the environment is currently in the day period.
A device for loading a map, comprising: an obtaining module, a first determining module, a second determining module, and a map loading module;

The acquisition module is configured to acquire current image data of the environment;

The first determining module is configured to calculate first illumination information currently corresponding to the environment according to the current image data of the environment;

The second determining module is configured to determine a time period in which the environment is currently located according to the first lighting information;

The map loading module is configured to determine a positioning map to be loaded and load the positioning map according to the current time period and the corresponding relationship between the time period and the positioning map of the environment.
An electronic device including:

At least one processor; and

A memory connected in communication with the at least one processor; wherein,

The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can execute the method according to any one of claims 1 to 10. Map loading method.
A computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the method for loading a map according to any one of claims 1 to 10 is implemented.