CN116797655A - Visual positioning method, apparatus, medium, computer device and program product - Google Patents

Abstract

The application provides a visual positioning method, a device, a medium, a computer device and a program product, wherein the method comprises the following steps: acquiring a reference image acquired based on a camera; performing corner detection on the reference image to obtain a plurality of corners and scores corresponding to each corner; according to the score and the position information of the plurality of corner points on the reference image, performing first screening operation on the plurality of corner points to determine target feature points, wherein the distance between every two adjacent target feature points is not smaller than a preset distance; and performing visual positioning according to the target characteristic points, so that the characteristic points are prevented from being too concentrated, the uniformity of the characteristic point distribution is improved, and the accuracy of visual positioning is improved.

Description

Visual positioning method, apparatus, medium, computer device and program product

Technical Field

The present application relates to the field of computer technology, and more particularly, to a visual positioning method, apparatus, medium, device and program product.

Background

With the increasing popularity of machine vision applications, machine vision based vision positioning techniques have also gained widespread attention. The visual positioning is mainly to estimate the movement of the camera through the image of the camera, and precisely calculates the pose of the camera based on feature detection. The corner detection algorithm is a feature detection method which is adopted by most of the current visual positioning. According to the corner detection algorithm, a plurality of corners and corresponding scores thereof can be obtained, and usually, the corner with a larger score is selected as a characteristic point, and visual positioning is performed based on the characteristic point.

However, when the image is single or the blank area is more, the feature points screened according to the scores are concentrated in one area, so that the feature points are unevenly distributed, and the accuracy of visual positioning is reduced.

Disclosure of Invention

The application provides a visual positioning method, a visual positioning device, a visual positioning medium, computer equipment and a visual positioning program product, which can reduce the phenomenon that characteristic points are too concentrated and improve the accuracy of visual positioning.

In one aspect, a visual positioning method is provided, the method comprising:

acquiring a reference image acquired based on a camera;

performing corner detection on the reference image to obtain a plurality of corners and scores corresponding to the corners;

according to the scores and the position information of the plurality of corner points on the reference image, performing first screening operation on the plurality of corner points to determine target feature points, wherein the distance between every two adjacent target feature points is not smaller than a preset distance;

and performing visual positioning according to the target feature points.

In another aspect, there is provided a visual positioning device comprising:

the acquisition module is used for acquiring a reference image acquired based on a camera;

the detection module is used for carrying out corner detection on the reference image and obtaining a plurality of corner points and scores corresponding to each corner point;

The screening module is used for carrying out first screening operation on the plurality of corner points according to the scores and the position information of the plurality of corner points on the reference image, and determining target feature points, wherein the distance between every two adjacent target feature points is not smaller than a preset distance;

and the positioning module is used for performing visual positioning according to the target characteristic points.

In another aspect, a computer readable storage medium is provided, comprising instructions which, when run on a terminal device, cause the terminal device to perform the method as described in the first aspect above.

In another aspect, a computer device is provided, the computer device comprising a processor and a memory, the memory having stored therein a computer program for executing the steps of the visual positioning method as described in the first aspect above, by invoking the computer program stored in the memory.

In another aspect, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the steps of the visual positioning method according to the first aspect described above.

The embodiment of the application provides a visual positioning method, which comprises the steps of obtaining a reference image based on camera acquisition, carrying out corner detection on the reference image, obtaining a plurality of corner points and scores corresponding to each corner point, then carrying out first screening operation on the plurality of corner points according to the scores and the position information of the plurality of corner points on the reference image, and determining target feature points, wherein the distance between every two adjacent target feature points is not smaller than a preset distance, and then carrying out visual positioning according to the target feature points, thereby avoiding the feature points from being concentrated in a certain area of the reference image, improving the uniformity of feature point distribution, and further improving the accuracy of visual positioning.

Drawings

Fig. 1 is a schematic structural diagram of a visual positioning system according to an embodiment of the present application.

Fig. 2 is a flow chart of a visual positioning method according to an embodiment of the present application.

Fig. 3 is another flow chart of a speech synthesis method according to an embodiment of the present application.

Fig. 4 is a schematic block diagram of a visual positioning device provided by an embodiment of the present application.

Fig. 5 is a schematic block diagram of a computer device provided by an embodiment of the present application.

Detailed Description

The following description of the technical solutions according to the embodiments of the present application will be given with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art to which the application pertains without inventive faculty, are intended to fall within the scope of the application.

Unless defined otherwise, technical or scientific terms used herein should be given the ordinary meaning as understood by one of ordinary skill in the art to which this disclosure belongs. The terms "first," "second," and the like, as used in this disclosure, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that embodiments of the application described herein may be implemented, for example, in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

First, some terms in the embodiments of the present application are explained below to facilitate understanding by those skilled in the art.

Artificial intelligence (ArtificialIntelligence, AI) is a new technical science to study, develop theories, methods, techniques and application systems for simulating, extending and expanding human intelligence. Artificial intelligence is a branch of computer science that attempts to understand the nature of intelligence and to produce a new intelligent machine that can react in a similar manner to human intelligence, research in this field including robotics, language recognition, image recognition, natural language processing, and expert systems. Since birth, the theory and technology are mature, and the application field is expanding, and it is supposed that the technological product brought by artificial intelligence in the future will be a "container" of human intelligence. Artificial intelligence can simulate the information process of consciousness and thinking of people. Artificial intelligence is not human intelligence, but can think like a human, and may also exceed human intelligence. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and other directions.

Computer vision (ComputerVision), also known as machine vision (machine vision), is the discipline that "teaches" how a computer can "see" the world. Furthermore, the camera and the computer replace human eyes to recognize, track and measure the target, and further perform graphic processing to make the computer process the target into an image more suitable for human eyes to observe or transmit to an instrument to detect. As a scientific discipline, computer vision research-related theory and technology has attempted to build artificial intelligence systems that can acquire information from images or multidimensional data.

Vision sensors are a direct source of information for the entire machine vision system, consisting essentially of one or two graphic sensors, sometimes with light projectors and other auxiliary devices. The primary function of the vision sensor is to acquire the most primitive images that the machine vision system is to process.

In machine vision, a feature point refers to a point in an image that has sharp characteristics and can effectively reflect the nature of the image to identify a target object in the image, i.e., can represent the image or target in other similar images that contain the same scene or target in the same or at least very similar non-deforming fashion, i.e., multiple pictures are taken from different angles for the same object or scene, and if the same place can be identified as the same, the point or block is referred to as a feature point. Such as corner points.

Corner points, which are key local structural features of an image, are generally described as points with discontinuous gray scale intensity variation, and are called singular feature points on a gray scale image. Corner points are important features of images and play an important role in understanding and analyzing the graphics of the images. The corner points can effectively reduce the data volume of information while keeping the important characteristics of the image graph, so that the content of the information is very high, the calculation speed is effectively improved, the reliable matching of the image is facilitated, and the real-time processing is possible.

With the rapid development of the internet of things, AI, big data and computer technology, the visual positioning technology is deeply and widely applied in various industries. In the current visual positioning technology, a common corner detection algorithm is used for carrying out feature detection on a camera image, then feature point screening is carried out according to the score of a corner given by the corner detection algorithm, and then visual positioning is carried out according to the screened feature point. However, if the camera image is single or more blank, feature point screening according to the scores of the corner points may result in feature point concentration, so that the accuracy of visual positioning is biased. In addition, when the current visual positioning technology is applied to the multi-camera visual positioning technology, a plurality of camera images can appear in a common view area, and different camera images have a large number of similar characteristic points, so that the calculation amount of the visual positioning system is large, the time consumption is long, and the real-time performance of visual positioning is influenced. In order to solve the problem, the embodiment of the application provides a visual positioning method, a device, a medium, a computer device and a program product, which ensure that only one characteristic point is in a certain range by taking the corner point with the largest score as the characteristic point, thereby avoiding more characteristic points to be concentrated, improving the accuracy of visual positioning, and in the case of multiple cameras, only preserving the characteristic point on a certain camera image in a common view area according to the score of the characteristic point and the distance from the characteristic point to the center point of the camera image, thereby avoiding a large number of similar characteristic points on different camera images in the common view area, reducing the calculation amount and time consumption of visual positioning, improving the time complexity of visual positioning and improving the real-time performance of visual positioning.

Referring to fig. 1, fig. 1 is a schematic structural diagram of a visual positioning system according to an embodiment of the application. The visual positioning system comprises a terminal 100, a server 200 and the like; the terminal 100 and the server 200 are connected through a network, for example, a wired or wireless network connection. The terminal 100 may be a user device (UserEquipment, UE), a mobile device, a user terminal, a cellular phone, a cordless phone, a personal digital assistant (PersonalDigitalAssistant, PDA), a handheld device, a computing device, a wearable device, an in-vehicle terminal, an unmanned aerial vehicle (unmanned aerial vehicle), a robot, or the like. The server 200 may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or may be a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, a content distribution network (ContentDeliveryNetwork, CDN), basic cloud computing services such as big data and an artificial intelligence platform.

The terminal 100 may be configured to acquire a reference image. For example, the terminal 100 may include a camera, through which reference images may be acquired in real time, wherein the camera may be one or more. The terminal 100 may be configured to collect a reference image and transmit the collected reference image to the server 200. The server 200 may be configured to perform corner detection on the reference image, and obtain a plurality of corners and scores corresponding to each corner; according to the score and the position information of the plurality of corner points on the reference image, performing first screening operation on the plurality of corner points to obtain target feature points, wherein the distance between every two adjacent target feature points is not smaller than a preset distance; visual positioning is carried out according to the target feature points; the visual positioning result is transmitted to the terminal 100.

The following will describe in detail. It should be noted that the following description order of embodiments is not a limitation of the priority order of embodiments.

Referring to fig. 2, fig. 2 shows a flowchart of a visual positioning method according to an embodiment of the present application, where the visual positioning method may be applied to a computer device, and the computer device may be a terminal or a server, and the method mainly includes steps S101 to S104, which are described as follows:

step S101, acquiring a reference image acquired based on a camera.

In particular, the computer device may comprise a vision sensor, which may be used to acquire the reference image. For example, the visual sensor may be a camera and the reference image may be a camera image. It should be noted that, the type of the visual sensor is not limited in this embodiment, and only the image needs to be obtained in real time.

Step S102, detecting angular points of the reference image, and obtaining a plurality of angular points and scores corresponding to each angular point.

The method can adopt an acceleration segmentation test feature (FeaturesfromAcceleratedSegmentTest, FAST) algorithm to perform corner detection, a speedup robust feature (SURF) algorithm to perform corner detection, a fast feature point extraction and description (OrientedFASTandRotatedBRIEF, ORB) algorithm to perform corner detection, and other algorithms to perform corner detection, and it should be noted that in this embodiment, the method is not limited as to which algorithm is used to perform corner detection. It is easy to understand that the angular point detection algorithm only depends on the gray level change of the local area of the reference image, is not limited by space constraint, and causes the local area with strong gray level change to have the distribution phenomenon of angular point aggregation and binding, so that the positioning robustness is weaker.

Step S103, performing first screening operation on the plurality of corner points according to the scores and the position information of the plurality of corner points on the reference image, and determining target feature points, wherein the distance between every two adjacent target feature points is not smaller than a preset distance.

Currently, corner points with larger scores are generally selected as feature points according to the scores of the corner points. If the reference image is single or the blank area is more, the corner points are screened according to the scores of the corner points, so that most of the corner points are concentrated together, and the visual positioning accuracy is deviated. Therefore, the application combines the score of the corner point and the position information of the corner point on the reference image to screen the corner point, so that the distance between every two adjacent target characteristic points screened out is not smaller than the preset distance, thereby avoiding the concentration of the corner point and improving the accuracy of visual positioning.

Specifically, step 103 may mainly include: traversing and screening each corner point in the plurality of corner points according to the order of the scores from large to small, and sequentially determining whether each corner point is a characteristic point or not; in the traversing screening process, determining whether the corner with the largest value in the plurality of corner points is a characteristic point, and if the corner with the largest value in the plurality of corner points is determined to be the characteristic point, deleting the corner which is positioned in the preset pixel range of the characteristic point on the reference image according to the position information of the plurality of corner points on the reference image.

Specifically, according to the order of the values from large to small, whether the corner points are characteristic points is judged, then the corner points in the preset pixel range of the characteristic points are screened out from the ordering order, and the corner points with the highest values in the preset pixel range can be ensured to be determined as the characteristic points. The present application does not limit the preset pixel range. In this embodiment, the preset pixel range may be within one hundred pixels or within twenty-five pixels.

It is easy to understand that the screening operation may be performed on the corners according to the value of the score, for example, the corners with the score not greater than the preset score are all screened out. And then sorting the corner points according to the value, determining whether the corner points are characteristic points according to the sorting order, and screening out the corner points in the preset pixel range of the characteristic points if the corner points are the characteristic points. Thus, the efficiency of corner screening can be improved.

In this embodiment, the method may include: and if the corner with the largest score in the plurality of corner points is not the feature point, deleting the corner with the largest score in the plurality of corner points from the plurality of corner points. Specifically, whether or not the corner is a feature point may be determined according to the score of the corner. If the score of the corner point is not smaller than the preset score, the corner point can be determined to be the characteristic point. If the score of the corner point is smaller than the preset score, determining that the corner point is not the characteristic point. It should be noted that the present application is not limited to the preset score, and may be set as required. The preset score may limit the number and quality of the feature points, and thus, the preset score may be set according to actual requirements, such as the number and quality of the feature points required.

Specifically, according to the number of vision sensors in the system, the vision positioning technology can be divided into multi-vision positioning, binocular vision positioning and monocular vision positioning. Therefore, the number of reference images may be plural. Taking a visual sensor as an example, in the case of a multi-camera, multiple cameras have multiple viewing angles, and multiple cameras have a common viewing area with overlapping fields of view.

In some embodiments, the reference image includes a first image and a second image, the first image and the second image being images acquired by different cameras, respectively, and the first image and the second image including a common view region with overlapping fields of view, after step 103, may further include: acquiring a first characteristic point on a first image and a second characteristic point on a second image in a common view area, wherein the first characteristic point and the second characteristic point are similar characteristic points; and performing a second screening operation according to the first characteristic points and the second characteristic points of the scores.

It will be readily appreciated that within the common view region, different reference images may have a large number of similar feature points, which means corresponding feature points on different reference images, for example, in the common view region, the same object may exist on different reference images, and if the feature points exist on the object, the feature points on the same object on different reference images at the same position are corresponding feature points. Similar feature points increase the computational effort of the system and increase the time consuming of the system, which is a burden for vision positioning with high real-time requirements. Therefore, the second screening operation can be performed on the first characteristic points on the first image and the second characteristic points on the second image in the common view area according to the scores, so that the calculation amount of the system is prevented from being increased due to a large number of similar characteristic points, the time consumption of the system is reduced, and the real-time performance of visual positioning is ensured.

Specifically, the step of performing the second filtering operation on the first feature point and the second feature point corresponding to the first image and the second image in the common view region according to the score may include: and screening out the characteristic points with smaller scores from the scores of the first characteristic points and the scores of the second characteristic points.

It is easy to understand that the feature points with smaller scores in the first feature points and the second feature points are screened out, the feature points with larger scores in the first feature points and the second feature points are reserved, and the best feature point in the similar feature points can be reserved as the target feature point, so that the accuracy of visual positioning can be ensured, the time complexity of multi-camera visual positioning can be reduced, and the real-time performance of visual positioning is improved.

Specifically, the step of performing the second filtering operation on the first feature point and the second feature point according to the score may include: and if the score of the first characteristic point is equal to the score of the second characteristic point, performing a second screening operation on the first characteristic point and the second characteristic point according to the first distance between the first characteristic point and the center of the first image and the second distance between the second characteristic point and the center of the second image.

It will be readily appreciated that, in general, points on the image that are closer to the center point of the image are less distorted, and therefore, if the scores of the corresponding first and second feature points on the first and second images are equal, the second filtering operation may be performed on the first and second feature points according to the first and second distances of the first and second feature points from the center of the first and second images. Specifically, the step of performing a second filtering operation on the first feature point and the second feature point according to a first distance of the first feature point from the center of the first image and a second distance of the second feature point from the center of the second image includes: and screening out the feature points corresponding to the longer distances from the first distances corresponding to the first feature points and the second distances corresponding to the second feature points.

Specifically, it is considered that the distortion of the feature points corresponding to the smaller distance of the first distance and the second distance is smaller, so that the feature points corresponding to the longer distance of the first distance and the second distance are screened out, and the feature points corresponding to the shorter distance of the first distance and the second distance are reserved, so that the accuracy of visual positioning is ensured.

In this embodiment, the method may further include: and determining the feature points obtained after the first screening operation and the second screening operation as target feature points.

Specifically, after the first screening operation, the obtained distance between the feature points is not smaller than a preset distance, for example, the obtained distance between the feature points is not smaller than one hundred pixels, so that accurate reduction of visual positioning caused by feature point concentration can be effectively avoided. Then, under the condition of multi-vision positioning, through the second screening operation, a plurality of reference images can be prevented from having a large number of similar characteristic points in the common vision area, so that the calculated amount and time consumption of the vision positioning are reduced, and the real-time performance of the vision positioning is improved.

Specifically, the step of "acquiring the first feature point and the second feature point on the first image in the common view region" may include: and carrying out matching operation on the characteristic points on the first image and the second image in the common view area to obtain a first characteristic point and a second characteristic point with similar relations.

The feature point matching refers to searching similar feature points among different reference images. Wherein, the feature point matching is performed for descriptors. The descriptors, i.e. the description of the feature points with some mathematical features, are usually a vector, and the distance between the descriptors of two feature points can reflect the degree of similarity, i.e. whether the two feature points are identical. Depending on the descriptor, different distance measures may be chosen. For example, if a descriptor is of the floating point type, its Euclidean distance may be used. For example, the hamming distance for binary descriptors can be used. Specifically, the feature points of the first image and the feature points of the second image are obtained in the foregoing steps, and then, descriptor extraction may be performed. The extraction may be performed using a descriptor extraction algorithm, such as a neighborhood template matching algorithm, SIFT (scale invariant feature transform) algorithm, ORB algorithm. Then, whether the feature points are similar or not can be judged through descriptors of the feature points, and common matching methods include violent matching, kd-Tree (K-dimension Tree), and the like.

It should be understood that the present application is described by taking an example that the reference image includes the first image and the second image, in practical situations, the reference image may further include a third image, a fourth image, and so on, that is, the number of the reference images is not limited by the present application. When the reference image includes a plurality of images, the processing may still be performed according to the method provided by the embodiment of the present application, that is, first, corner detection is performed on each reference image, then, first filtering operation is performed on the corner of each reference image, and then, second filtering operation is performed on the corresponding feature points on different reference images in the common view area, so as to obtain the target feature point.

And step S104, performing visual positioning according to the target feature points.

From the above, the target feature points are corner points after the first screening operation and the second screening operation. And the distance between the reserved characteristic points is not smaller than a preset distance through the first screening operation. Then, if the number of the reference images is multiple, the best feature points among the corresponding feature points on the multiple reference images are reserved in the common view area through the second screening operation. Thus, deviation of visual positioning accuracy due to the concentration of the target feature points can be avoided. Meanwhile, under the condition of binocular or multi-vision positioning, a plurality of reference images can be prevented from having a large number of similar target feature points in a common vision area, so that the calculated amount of vision positioning is reduced, the time complexity of the multi-vision positioning is reduced, the real-time performance of the vision positioning is improved, and the accuracy of the vision positioning is ensured.

In order to better illustrate the visual positioning method provided by the embodiment of the present application, please refer to fig. 3, the visual positioning method can be summarized as the following steps:

in step 201, a first image and a second image are acquired, where the first image and the second image are images acquired by different cameras, and the first image and the second image include a common view area with overlapping fields of view.

Step 202, performing corner detection on the first image, obtaining a plurality of corners of the first image and scores corresponding to each corner, and performing corner detection on the second image, obtaining a plurality of corners of the second image and scores corresponding to each corner.

Step 203, performing a first filtering operation on the plurality of corner points of the first image according to the scores of the plurality of corner points of the first image and the position information of the plurality of corner points of the first image on the first image, and determining a first feature point of the first image.

And 204, performing a first screening operation on the plurality of corner points of the second image according to the scores of the plurality of corner points of the second image and the position information of the plurality of corner points of the second image on the second image, and determining a second characteristic point of the second image.

And 205, performing a second screening operation on the first characteristic points and the second characteristic points corresponding to the first image and the second image in the common view area according to the scores.

And 206, determining the feature points obtained after the first screening operation and the second screening operation as target feature points.

And step 207, performing visual positioning according to the target characteristic points.

It should be noted that, in the embodiment of the present application, the sequence of execution of the steps is not limited, for example, the step 203 and the step 204 may be executed synchronously.

All the above technical solutions may be combined to form an optional embodiment of the present application, and will not be described in detail herein.

According to the embodiment of the application, the reference image acquired based on the camera is acquired, then the reference image is subjected to corner detection to obtain a plurality of corner points and scores corresponding to each corner point, and then the first screening operation is performed on the plurality of corner points according to the scores and the position information of the plurality of corner points on the reference image to obtain the target feature points, wherein the distance between every two adjacent target feature points is not smaller than the preset distance, and then the visual positioning is performed according to the target feature points, so that the feature points are prevented from being too concentrated, the uniformity of feature point distribution is improved, and the accuracy of visual positioning is improved.

Having described in detail the method embodiments of the present application above, and in connection with fig. 4, a detailed description of apparatus embodiments of the present application follows, it being understood that apparatus embodiments correspond to method embodiments, and similar descriptions may refer to method embodiments.

Fig. 4 is a schematic structural view of a visual positioning device 10 according to an embodiment of the present application, and as shown in fig. 4, the visual positioning device 10 may include:

an acquisition module 11 for acquiring a reference image acquired based on a camera;

the detection module 12 is used for performing corner detection on the reference image and acquiring a plurality of corners and scores corresponding to each corner;

the screening module 13 is configured to perform a first screening operation on the multiple corner points according to the scores and the position information of the multiple corner points on the reference image, and determine target feature points, where a distance between every two adjacent target feature points is not less than a preset distance;

and the positioning module 14 is used for performing visual positioning according to the target characteristic points.

Optionally, the screening module 13 may be configured to: traversing and screening each corner point in the plurality of corner points according to the order of the scores from large to small, and sequentially determining whether each corner point is a characteristic point or not; in the traversing screening process, determining whether the corner with the largest value in the plurality of corner points is a characteristic point, and if the corner with the largest value in the plurality of corner points is determined to be the characteristic point, deleting the corner which is positioned in the preset pixel range of the characteristic point on the reference image according to the position information of the plurality of corner points on the reference image.

Optionally, the screening module 13 may be configured to: and if the corner with the largest score in the plurality of corner points is not the feature point, deleting the corner with the largest score in the plurality of corner points from the plurality of corner points.

Optionally, the reference image includes a first image and a second image, where the first image and the second image are images acquired by different cameras, and the first image and the second image include a common view area with overlapping fields of view, and the filtering module 13 is further configured to: after a first screening operation is carried out on a plurality of corner points according to the scores and the position information of the corner points on a reference image, obtaining a first characteristic point on a first image and a second characteristic point on a second image in a common view area, wherein the first characteristic point and the second characteristic point are similar characteristic points; and performing a second screening operation on the first characteristic points and the second characteristic points according to the scores.

Optionally, the screening module 13 may specifically be configured to: and screening out the characteristic points with smaller scores from the scores of the first characteristic points and the scores of the second characteristic points.

Optionally, the screening module 13 may specifically be configured to: and if the score of the first characteristic point is equal to the score of the second characteristic point, performing a second screening operation on the first characteristic point and the second characteristic point according to the first distance between the first characteristic point and the center of the first image and the second distance between the second characteristic point and the center of the second image.

Optionally, the screening module 13 may specifically be configured to: and screening out the feature points corresponding to the longer distances from the first distances corresponding to the first feature points and the second distances corresponding to the second feature points.

Optionally, the screening module 13 may be further configured to: and determining the feature points obtained after the first screening operation and the second screening operation as target feature points.

Optionally, the visual positioning apparatus 10 may further include a matching module that may be configured to: and carrying out matching operation on the characteristic points on the first image and the second image in the common view area to obtain a first characteristic point and a second characteristic point with similar relations.

It should be noted that, the functions of each module in the visual positioning apparatus 10 in the embodiment of the present application may be correspondingly referred to the specific implementation manner in each method embodiment described above, and will not be described herein again.

The various modules in the visual positioning apparatus 10 described above may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in hardware or may be independent of a processor in the terminal device, or may be stored in software in a memory in the terminal device, so that the processor invokes and executes operations corresponding to the above modules.

According to the visual positioning device 10 provided by the embodiment of the application, the reference image acquired based on the camera is acquired through the acquisition module 11, then the detection module 12 detects the corner points of the reference image to acquire a plurality of corner points and the score corresponding to each corner point, the screening module 13 performs a first screening operation on the plurality of corner points according to the score and the position information of the plurality of corner points on the reference image to determine the target feature points, wherein the distance between every two adjacent target feature points is not smaller than the preset distance, and then the positioning module 14 performs visual positioning according to the target feature points, so that the feature points are prevented from being too concentrated, the uniformity of feature point distribution is improved, and the accuracy of visual positioning is improved.

Referring to fig. 5, fig. 5 shows a schematic block diagram of a computer device (e.g., a terminal or a server in fig. 1) according to an embodiment of the present application. The computer device shown in fig. 5 is only an example and should not be construed as limiting the functionality and scope of use of embodiments of the application.

The computer device may include: a processor 21, a memory 22, an input device 23, an output device 24, a communication device 25, a communication bus 26, and an input/output (I/O) interface 27. The processor 21, memory 22, and input/output (I/O) interface 27 communicate with each other via a communication bus 26. In general, the following devices may be connected to the I/O interface 27: input devices 23 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; an output device 24 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, etc.; communication means 25. The communication means 25 may allow the computer device 20 to communicate with other devices wirelessly or by wire to exchange data. While fig. 5 shows computer apparatus 20 having various devices, it should be understood that not all illustrated devices are required to be implemented or provided. More or fewer devices may be implemented or provided instead.

In particular, the memory 22 may be used for storing software programs and modules, and the processor 21 may execute the software programs and modules stored in the memory 22, such as the software programs for corresponding operations in the foregoing method embodiments.

In some embodiments, the processor 21 may call a software program and modules stored in the memory 22 to perform the following operations: acquiring a reference image acquired based on a camera; performing corner detection on the reference image to obtain a plurality of corners and scores corresponding to each corner; according to the score and the position information of the plurality of corner points on the reference image, performing first screening operation on the plurality of corner points to determine target feature points, wherein the distance between every two adjacent target feature points is not smaller than a preset distance; and performing visual positioning according to the target feature points.

In particular, according to embodiments of the present application, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present application include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network via the communication means 25 or from the memory 22. The above-described functions defined in the foregoing method embodiments of the application are performed when the computer program is executed by the processor 21.

It should be noted that, the division of each device and terminal in fig. 5 is merely a division of logic functions, and may be fully or partially integrated into one physical entity or may be physically separated. And these modules may all be implemented in software in the form of calls by the processing element; or can be realized in hardware; the method can also be realized in a form of calling software by a processing element, and the method can be realized in a form of hardware by a part of modules.

The disclosed embodiments also provide a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the visual positioning method described in the method embodiments above. Wherein the storage medium may be a volatile or nonvolatile computer readable storage medium.

Embodiments of the present disclosure further provide a computer program product carrying a program code, where the program code includes instructions for performing the steps of the visual positioning method described in the foregoing method embodiments, and specific reference may be made to the foregoing method embodiments, which are not described herein.

Wherein the above-mentioned computer program product may be realized in particular by means of hardware, software or a combination thereof. In an alternative embodiment, the computer program product is embodied as a computer storage medium, and in another alternative embodiment, the computer program product is embodied as a software product, such as a software development kit (SoftwareDevelopmentKit, SDK), or the like.

It should be appreciated that the processor of an embodiment of the present application may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method embodiments may be implemented by integrated logic circuits of hardware in a processor or instructions in software form. The processor may be a general purpose processor, a digital signal processor (DigitalSignalProcessor, DSP), an application specific integrated circuit (ApplicationSpecificIntegratedCircuit, ASIC), an off-the-shelf programmable gate array (FieldProgrammableGateArray, FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. The disclosed methods, steps, and logic blocks in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present application may be embodied directly in the execution of a hardware decoding processor, or in the execution of a combination of hardware and software modules in a decoding processor. The software modules may be located in a random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. as well known in the art. The storage medium is located in a memory, and the processor reads the information in the memory and, in combination with its hardware, performs the steps of the above method.

It will be appreciated that the memory in embodiments of the application may be volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. The non-volatile memory may be a Read-only memory (ROM), a programmable Read-only memory (ProgrammableROM, PROM), an erasable programmable Read-only memory (ErasablePROM, EPROM), an electrically erasable programmable Read-only memory (ElectricallyEPROM, EEPROM), or a flash memory, among others. The volatile memory may be a random access memory (RandomAccessMemory, RAM) that acts as an external cache. By way of example, and not limitation, many forms of RAM are available, such as Static RAM (SRAM), dynamic random access memory (DynamicRAM, DRAM), synchronous dynamic random access memory (SynchronousDRAM, SDRAM), double data rate synchronous dynamic random access memory (DoubleDataRateSDRAM, DDRSDRAM), enhanced synchronous dynamic random access memory (EnhancedSDRAM, ESDRAM), synchronous link dynamic random access memory (SynchlinkDRAM, SLDRAM), and direct memory bus random access memory (DirectRambusRAM, DRRAM). It should be noted that the memory of the systems and methods described herein is intended to comprise, without being limited to, these and any other suitable types of memory.

It should be appreciated that the above memory is illustrative but not limiting, and for example, the memory in the embodiments of the present application may be Static Random Access Memory (SRAM), dynamic random access memory (dynamicRAM, DRAM), synchronous dynamic random access memory (synchronousDRAM, SDRAM), double data rate synchronous dynamic random access memory (doubledatarateSDRAM, DDRSDRAM), enhanced synchronous dynamic random access memory (enhancedSDRAM, ESDRAM), synchronous link dynamic random access memory (synchlinkDRAM, SLDRAM), direct memory bus random access memory (DirectRambusRAM, DRRAM), and the like. That is, the memory in embodiments of the present application is intended to comprise, without being limited to, these and any other suitable types of memory.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described system and apparatus may refer to corresponding procedures in the foregoing method embodiments, which are not described herein again. In the several embodiments provided in the present disclosure, it should be understood that the disclosed apparatus and method may be implemented in other manners. The above-described apparatus embodiments are merely illustrative, for example, the division of the units is merely a logical function division, and there may be other manners of division in actual implementation, and for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some communication interface, device or unit indirect coupling or communication connection, which may be in electrical, mechanical or other form.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present disclosure may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer readable storage medium executable by a processor. Based on such understanding, the technical solution of the present disclosure may be embodied in essence or a part contributing to the prior art or a part of the technical solution, or in the form of a software product stored in a storage medium, including several instructions for causing an electronic device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in the embodiments of the present disclosure. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-only memory (ROM), a random access memory (RandomAccessMemory, RAM), a magnetic disk, an optical disk, or other various media capable of storing program codes.

The foregoing is merely illustrative of the present application, and the present application is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (13)

1. A method of visual localization, the method comprising:

acquiring a reference image acquired based on a camera;

performing corner detection on the reference image to obtain a plurality of corners and scores corresponding to the corners;

according to the scores and the position information of the plurality of corner points on the reference image, performing first screening operation on the plurality of corner points to determine target feature points, wherein the distance between every two adjacent target feature points is not smaller than a preset distance;

and performing visual positioning according to the target feature points.

2. The visual positioning method according to claim 1, wherein the performing a first filtering operation on the plurality of corner points according to the scores and the position information of the plurality of corner points on the reference image includes:

traversing and screening each corner point in the plurality of corner points according to the order of the scores from large to small, and sequentially determining whether each corner point is a characteristic point or not;

in the traversing screening process, determining whether the corner with the largest score in the plurality of corner points is a characteristic point, and if the corner with the largest score in the plurality of corner points is determined to be the characteristic point, deleting the corner which is positioned in the preset pixel range of the characteristic point on the reference image according to the position information of the plurality of corner points on the reference image.

3. The visual positioning method of claim 2, further comprising:

and if the corner with the largest score in the plurality of corner points is not the feature point, deleting the corner with the largest score in the plurality of corner points from the plurality of corner points.

4. The visual positioning method according to claim 2, wherein the reference image comprises a first image and a second image, the first image and the second image being images acquired by different cameras, respectively, the first image and the second image comprising a common view region with overlapping fields of view;

after the first filtering operation is performed on the plurality of corner points according to the scores and the position information of the plurality of corner points on the reference image, the method further comprises:

acquiring a first characteristic point on the first image and a second characteristic point on the second image in the common view area, wherein the first characteristic point and the second characteristic point are similar characteristic points;

and performing a second screening operation on the first characteristic points and the second characteristic points according to the scores.

5. The visual positioning method according to claim 4, wherein the performing a second filtering operation on the first feature point and the second feature point according to the score includes:

And screening out the characteristic points with smaller scores from the scores of the first characteristic points and the scores of the second characteristic points.

6. The visual positioning method according to claim 4, wherein the performing a second filtering operation on the first feature point and the second feature point according to the score includes:

and if the score of the first characteristic point is equal to the score of the second characteristic point, performing a second screening operation on the first characteristic point and the second characteristic point according to a first distance between the first characteristic point and the center of the first image and a second distance between the second characteristic point and the center of the second image.

7. The visual positioning method according to any one of claim 6, wherein the performing a second filtering operation on the first feature point and the second feature point according to a first distance between the first feature point and a center of the first image and a second distance between the second feature point and a center of the second image includes:

and screening out the feature points corresponding to the longer distance from the first distance corresponding to the first feature point and the second distance corresponding to the second feature point.

8. The visual positioning method of claim 4, further comprising:

and determining the feature points obtained after the first screening operation and the second screening operation as target feature points.

9. The method of visual localization according to claim 4, wherein said acquiring first feature points on said first image and second feature points on said second image within said common view region comprises:

and carrying out matching operation on the characteristic points on the first image and the second image in the common view area to obtain the first characteristic points and the second characteristic points with similar relations.

10. A visual positioning device, comprising:

the acquisition module is used for acquiring a reference image acquired based on a camera;

the detection module is used for carrying out corner detection on the reference image and obtaining a plurality of corner points and scores corresponding to each corner point;

the screening module is used for carrying out first screening operation on the plurality of corner points according to the scores and the position information of the plurality of corner points on the reference image, and determining target feature points, wherein the distance between every two adjacent target feature points is not smaller than a preset distance;

And the positioning module is used for performing visual positioning according to the target characteristic points.

11. A computer readable storage medium comprising instructions which, when run on a terminal device, cause the terminal device to perform the visual positioning method of any of claims 1 to 9.

12. A computer device, characterized in that it comprises a processor and a memory, in which a computer program is stored, the processor being adapted to perform the steps of the visual positioning method according to any of claims 1 to 9 by calling the computer program stored in the memory.

13. A computer program product comprising a computer program, characterized in that the computer program, when being executed by a processor, realizes the steps of the method according to claims 1 to 9.