WO2019113859A1

WO2019113859A1 - Machine vision-based virtual wall construction method and device, map construction method, and portable electronic device

Info

Publication number: WO2019113859A1
Application number: PCT/CN2017/116015
Authority: WO
Inventors: 李北辰
Original assignee: 广州艾若博机器人科技有限公司
Priority date: 2017-12-13
Filing date: 2017-12-13
Publication date: 2019-06-20
Also published as: CN109313822A; CN109313822B

Abstract

Disclosed are a virtual wall construction method and device based on machine vision, a map construction method, and a portable electronic device, wherein the virtual wall construction method based on machine vision performs a matching operation on images acquired in real time; when matching is successful, selecting specific characteristic points on the images to act as key points, calculating the position of a virtual wall relative to the portable electronic device according to the key points, and automatically constructing the virtual wall, such that a boundary line dividing an accessible region and an entry-prohibited region may be accurately constructed, and an obvious boundary line such as the aforementioned may completely prohibit the portable electronic device from entering the entry-prohibited region; thus the present method has simple, practical and reliable advantages. Furthermore, the solution in the present embodiment configures the virtual wall without the need for additional interactive devices and also without the need for setting location tags and the like in specific locations, as the degree of intelligence is higher.

Description

Virtual wall construction method and device based on machine vision, map construction method, mobile electronic device

Technical field

The invention relates to the field of real-time positioning and map construction, in particular to a virtual wall construction method and device based on machine vision, a map construction method and a movable electronic device.

Background technique

The positioning and map construction of mobile devices is a hot research topic in the field of robotics. There has been a practical solution for map creation of mobile devices in a known environment with autonomous positioning and known robot positions. However, in many environments mobile devices cannot be located using a global positioning system, and it is difficult or even impossible to obtain a map of the working environment of the mobile device in advance. At this time, the mobile device needs to construct a map in a completely unknown environment under the condition that its position is uncertain, and at the same time utilize the map for autonomous positioning and navigation. This is called so-called simultaneous localization and mapping (SLAM).

In real-time location and map construction, the mobile device uses the sensors carried by itself to identify feature marks (such as RFID tags and color block tags) in the unknown environment, and then recognizes the accessible area and the forbidden entry area according to the information carried on the feature mark. Thereby, the mobile device is guided to enter the designated area according to the personalized needs of the user. The existing guidance method has the following defects: the inaccessible area and the forbidden entry area cannot be accurately identified, and misidentification and inaccurate identification are likely to occur, and the mobile device enters the forbidden entry area, which may cause damage to the mobile device.

Summary of the invention

The object of the embodiments of the present invention is to provide a virtual wall construction method based on machine vision, a map construction method, and a mobile electronic device, which can effectively solve the problem that the prior art is prone to misidentification and inaccurate recognition, and the mobile device enters the forbidden entry area. The problem.

Embodiments of the present invention provide a virtual wall construction method based on machine vision, including the steps of:

During the traversing of the to-be-positioned area by the mobile electronic device, the image of the surrounding environment is collected in real time by a camera disposed on the movable electronic device, and the image captured at each moment is projected. Forming a target image to a photosensitive surface of an image sensor provided in the movable electronic device;

Obtaining x key points of the target image on the photosensitive surface when the target image acquired at any time matches any of the marking patterns in the marking pattern library according to a preset image matching algorithm; The key points of the target image on the photosensitive surface are all located on a characteristic line; x≥2;

Calculating a distance between the virtual wall and the movable electronic device based on a key point of the target image on the photosensitive surface, according to a distance between the virtual wall and the movable electronic device, Forming the virtual wall on an angle that is perpendicular to the photosensitive surface; wherein, when the predetermined angle is equal to 0° or 180°, then parallel to the characteristic line and the photosensitive surface The virtual wall is constructed on a vertical plane.

As a modification of the above embodiment, the method further comprises the steps of:

And acquiring an image directly above the movable electronic device at the current moment in response to the calibration instruction, and storing the image in the mark pattern library as a new mark pattern;

And acquiring, by the image matching algorithm, a plurality of feature points of the mark pattern and a feature descriptor corresponding to each of the feature points.

As a modification of the above embodiment, according to a preset image matching algorithm, when the target image acquired at any time matches any of the mark patterns in the mark pattern library, the target image is acquired in the light-sensitive The x key points on the surface are:

Obtaining, by the image matching algorithm, a plurality of feature points of the target image at the current moment and a feature descriptor corresponding to each of the feature points;

Obtaining a feature point of the matching relationship between the target image and the mark pattern by calculating a Euclidean distance of each feature descriptor of the target image and each feature descriptor of the mark pattern, when acquiring When the number of feature points having a matching relationship is greater than a preset threshold, determining that the target image matches any of the mark patterns in the mark pattern library;

According to the positional relationship of the feature points having the matching relationship on the target image, the feature points in which x are on the same straight line are taken as key points.

As a modification of the above embodiment, the image matching algorithm is a scale invariant feature transform algorithm or an accelerated robust feature algorithm, and each feature descriptor of the mark pattern/target image is obtained by the following steps:

The scale space of the marker image/target image is established by Gaussian blur, and the extreme point in the scale space of the marker image/target image is identified by a Gaussian differential function, and the extreme point in the scale space of the marker image/target image Perform verification to remove the marker image/target image An unstable extreme point in the scale space, thereby obtaining a feature point of the mark image/target image and a scale and a position of the feature point;

And assigning a direction to each of the feature points according to a gradient direction distribution characteristic of a neighboring pixel of the feature point;

According to the scale, position and direction of the feature point, the intra-block gradient histogram is calculated by performing area segmentation on the surrounding image of the feature point, thereby generating a feature descriptor of the feature point.

As a modification of the above embodiment, the image acquired by the camera is projected to the photosensitive surface of the image sensor through the imaging lens; the distance between the virtual wall and the movable electronic device is calculated by a triangulation method.

As a modification of the above embodiment, the calculating the distance between the virtual wall and the movable electronic device based on the key point of the target image on the photosensitive surface is specifically:

The distance between the virtual wall and the removable electronic device is calculated by the following formula:

D=a/b*S*|cosθ|

Where a is the distance from the upper border of the virtual wall to the photosensitive surface, b is the distance between the imaging lens and the photosensitive surface, and S is the distance between the characteristic straight line and the center point of the photosensitive surface, D is a distance between the virtual wall and a center point of the photosensitive surface, and θ is the preset angle.

As a modification of the above embodiment, the image sensor includes a PSD sensor, a CCD sensor, or a CMOS sensor.

As a modification of the above embodiment, the width of the virtual wall is calculated by the following formula:

Where W is the width of the virtual wall, a is the distance from the upper border of the virtual wall to the photosensitive surface, and λ is the wide angle of the camera.

When the distance between the virtual wall and the mobile electronic device is less than a preset distance, moving the mobile electronic device by a preset avoidance strategy to make the distance between the movable electronic device and the virtual wall Increase.

After constructing the virtual wall, the movable electronic device is controlled to pass through the virtual wall in a preset path.

As an improvement of the above embodiment, after the captured image is projected onto the photosensitive surface of the image sensor provided in the movable electronic device to form a target image, the method further includes the following steps:

Correction of lens deformation is performed on the target image.

The embodiment of the invention further provides a map construction method, which comprises the steps of:

Constructing a coordinate system with any position or specific position in the to-be-positioned area as a coordinate origin, and calculating a displacement of the movable electronic device relative to the coordinate origin in a real-time process in which the movable electronic device traverses the to-be-positioned area And a direction to acquire coordinate values of the movable electronic device in the coordinate system in real time;

Constructing a map using a machine vision-based virtual wall construction method as described in any of the above;

Real-time map construction is performed on the to-be-positioned area according to coordinate values of the movable electronic device in the coordinate system and a coordinate plane of the virtual wall.

As a modification of the above embodiment, the real-time map construction of the to-be-positioned area according to the coordinate value of the movable electronic device in the coordinate system and the position of the virtual wall includes:

Calculating and recording a coordinate value of an obstacle position when the movable electronic device detects an obstacle each time based on a moving direction and a moving distance of the movable electronic device with respect to the starting point;

Real-time map construction is performed on the to-be-positioned area based on a coordinate plane of the virtual wall and coordinate values of each of the obstacle positions.

As a modification of the above embodiment, at least two positioning tags are disposed on the to-be-positioned area, and each positioning tag is correspondingly disposed at a specific position of the to-be-positioned area, and each of the positioning tag information includes The uniquely encoded information of the absolute position; the real-time map construction of the to-be-positioned area according to the coordinate value of the movable electronic device in the coordinate system and the position of the virtual wall further includes:

Calculating, according to the moving direction and the moving distance of the movable electronic device with respect to the starting point, a coordinate value of a position of the positioning tag when the positioning tag information acquired by the movable electronic device is acquired, and Recording positioning tag information and corresponding coordinate values;

And based on a coordinate plane of the virtual wall, coordinate values of each of the obstacle positions, and information of each of the positioning tags and coordinate values thereof, The to-be-positioned area performs real-time map construction.

Calculating, according to a perspective deformation of the target image, a distance of the movable electronic device from a center line of the virtual wall;

The movable electronic device is returned to a center line of the virtual wall along a trajectory parallel to the virtual wall according to a distance of the movable electronic device from a center line of the virtual wall.

As a modification of the above embodiment, the removable electronic device includes a driving wheel and a driven wheel, and the method further includes the steps of:

During the traveling of the movable electronic device along an arbitrary straight line, when the speed of the driving wheel of the movable electronic device is detected to be inconsistent with the speed of the driven wheel at any time, the speed of the driving wheel and the driven wheel The smaller of the speeds is used as the reference speed, and the displacement and direction of the movable electronic device relative to the coordinate origin are calculated according to the reference speed;

During the turning of the movable electronic device at an arbitrary center point, when the speed of the driven wheel of the movable electronic device is detected to be less than the theoretical speed at any time, the speed of the driven wheel is used as a reference speed, according to Calculating, by the reference speed, a displacement and a direction of the movable electronic device relative to the coordinate origin;

In the process of turning the movable electronic device at an arbitrary center point, when the speed of the driven wheel of the movable electronic device is detected to be greater than the theoretical speed at any time, the speed of the driving wheel is used as a reference speed And calculating, according to the reference speed, a displacement and a direction of the movable electronic device relative to the coordinate origin; the theoretical speed is calculated according to a speed of the driving wheel.

The embodiment of the present invention further provides a virtual machine building device based on machine vision. The virtual machine building device based on machine vision is disposed on the mobile electronic device, and includes:

a camera for collecting an image of the surrounding environment in real time at a preset frequency during the traversing of the area to be located by the movable electronic device;

An image sensor for receiving an image acquired at each moment on a photosensitive surface of the image sensor to form a target image;

a storage device for pre-storing a plurality of mark patterns;

a controller, configured to acquire, according to a preset image matching algorithm, an image of the target image on the photosensitive surface when the target image acquired at any time matches any of the marking patterns in the mark pattern library a key point; calculating a distance between the virtual wall and the movable electronic device based on a key point of the target image on the photosensitive surface, according to a distance between the virtual wall and the movable electronic device, Constructing the virtual wall with a characteristic straight line at a predetermined angle and perpendicular to the photosensitive surface; wherein, when the predetermined angle is equal to 0° or 180°, then parallel with the characteristic line and The virtual wall is constructed on a surface perpendicular to the photosensitive surface; the key points of the target image on the photosensitive surface are all located on a characteristic line; x≥2.

As a modification of the above embodiment, the controller is further configured to acquire, by the camera, an image directly above the movable electronic device by the camera in response to the calibration instruction, and store the image in the mark pattern library as a new mark. a pattern; acquiring, by the image matching algorithm, a plurality of feature points of the mark pattern and a feature descriptor corresponding to each of the feature points.

As a modification of the above embodiment, according to a preset image matching algorithm, when the target image acquired at any time matches any of the mark patterns in the mark pattern library, the controller acquires the target image in the The x key points on the photosensitive surface are as follows:

Obtaining, by the image matching algorithm, a plurality of feature points of the target image and a feature descriptor corresponding to each of the feature points;

According to the positional relationship of the feature points having the matching relationship on the target image, the feature points in which a x are located on the same straight line are taken as key points.

As an improvement of the above embodiment, the image matching algorithm is a scale-invariant feature transform algorithm or an accelerated robust feature algorithm, and each feature descriptor of the mark pattern/target image is obtained by the controller by using the following steps:

The scale space of the marker image/target image is established by Gaussian blur, and the extreme point in the scale space of the marker image/target image is identified by a Gaussian differential function, and the extreme point in the scale space of the marker image/target image Performing a check to remove an unstable extreme point in the scale space of the mark image/target image, thereby obtaining a feature point of the mark image/target image and a scale and a position of the feature point;

As a modification of the above embodiment, the camera further includes an imaging lens, and an image captured by the camera is projected to the photosensitive surface of the image sensor through the imaging lens; the distance between the virtual wall and the movable electronic device is triangulated Calculated by the distance method.

As a modification of the above embodiment, the controller calculates the distance between the virtual wall and the movable electronic device based on the key point of the target image on the photosensitive surface:

D=a/b*S*|cosθ|

As a modification of the above embodiment, the controller calculates the width of the virtual wall based on the following formula:

As a modification of the foregoing embodiment, when the distance between the virtual wall and the mobile electronic device is less than a preset distance, the controller is further configured to move the mobile electronic device by using a preset avoidance policy. The distance of the mobile electronic device from the virtual wall is increased.

As a modification of the above embodiment, the controller is further configured to control the movable electronic device to pass through the virtual wall in a preset path after constructing the virtual wall.

As an improvement of the above embodiment, after the captured image is projected onto the photosensitive surface of the image sensor provided in the movable electronic device to form a target image, the controller is further configured to perform perspective distortion on the target image. Correction.

The embodiment of the invention further provides a removable electronic device, including:

A machine vision-based virtual wall building device according to any of the preceding claims, for constructing a virtual wall;

The controller is further configured to construct a coordinate system by using any position or a specific position in the to-be-positioned area as a coordinate origin;

An encoder, configured to calculate, in real time, a displacement and a direction of the movable electronic device relative to the coordinate origin in a process in which the movable electronic device traverses the to-be-positioned area;

The controller is further configured to receive a displacement and a direction of the movable electronic device sent by the encoder with respect to the coordinate origin, and acquire coordinate values of the movable electronic device in the coordinate system at any time;

The controller is further configured to perform real-time map construction on the to-be-positioned area according to coordinate values of the movable electronic device in the coordinate system and a coordinate plane of the virtual wall.

As a modification of the above embodiment, further comprising a collision sensor, a laser sensor or an infrared sensor, the controller is based on a moving direction and movement of the movable electronic device relative to the starting point when an obstacle is sensed by the collision sensor a distance, a coordinate value of a current position of the movable electronic device as a coordinate value of the obstacle position;

When the obstacle is detected by the laser sensor/infrared sensor, the controller calculates the position of the obstacle relative to the currently movable electronic device according to the laser/infrared distance calculation principle, so that the movable electronic device according to the current moment Calculating a coordinate value of the obstacle at the current time relative to a moving direction and a moving distance of the starting point;

The controller is configured to perform real-time map construction on the to-be-positioned area based on a coordinate plane of the virtual wall and coordinate values of each of the obstacle positions.

As a modification of the above embodiment, at least two positioning tags are disposed on the to-be-positioned area, and each positioning tag is correspondingly disposed at a specific position of the to-be-positioned area, and each of the positioning tag information includes The unique coding information of the absolute position; the real-time map construction of the to-be-positioned area by the controller according to the coordinate value of the movable electronic device in the coordinate system and the position of the virtual wall includes:

Real-time map construction is performed on the to-be-positioned area based on a coordinate plane of the virtual wall, coordinate values of each of the obstacle positions, and information of each of the positioning tags and coordinate values thereof.

As a modification of the above embodiment, the controller is further configured to calculate, according to the perspective deformation of the target image, a distance of the movable electronic device from the center line of the virtual wall; according to the movable electronic device deviating from the virtual The distance from the centerline of the wall causes the moveable electronic device to return to the centerline of the virtual wall along a trajectory parallel to the virtual wall.

As a modification of the above embodiment, the movable electronic device includes a driving wheel and a driven wheel, and the controller is further configured to detect the detachable time when the movable electronic device travels along any straight line When the speed of the driving wheel of the mobile electronic device is inconsistent with the speed of the driven wheel, the smaller value of the speed of the driving wheel and the speed of the driven wheel is used as a reference speed, and the movable electronic is calculated according to the reference speed. The displacement and direction of the device relative to the origin of the coordinate;

Compared with the prior art, the embodiment of the present invention provides a virtual wall construction method and device based on machine vision, a map construction method, and a mobile electronic device. When a matching operation is performed on an image acquired in real time, when the matching is successful, Selecting a specific feature point on the image as a key point, calculating a position of the virtual wall relative to the movable electronic device according to the key point, and automatically constructing a virtual wall, thereby accurately constructing a dividing line dividing the accessible area and the forbidden entering area, The obvious dividing line can completely prohibit the movable electronic device from entering the forbidden entry area, and has the advantages of simple practicality and high reliability; in addition, the solution in this embodiment does not require additional interactive devices for setting the virtual wall, and does not need to be specific. Position setting tabs, etc., more intelligent.

DRAWINGS

1 is a schematic flow chart of a method for constructing a virtual wall based on machine vision according to Embodiment 1 of the present invention;

2 is a schematic view showing a distance between a virtual wall and a center of a photosensitive surface in Embodiment 1 of the present invention;

3 is a schematic diagram showing a positional relationship between another virtual wall and a mobile electronic device of FIG. 2 according to Embodiment 1 of the present invention;

4 is a schematic diagram showing another distance between a virtual wall and a center of a photosensitive surface in Embodiment 1 of the present invention;

Figure 5 is a plan view of the center of the photosensitive surface corresponding to Figure 4 and the virtual wall;

6 is a schematic diagram of calculating a distance between a virtual wall and a removable electronic device in Embodiment 1 of the present invention;

FIG. 7 is a schematic diagram of calculating the width of the virtual wall according to Embodiment 1 of the present invention; FIG.

8 is a schematic flow chart of a method for constructing a virtual wall based on machine vision according to Embodiment 2 of the present invention;

Figure 9 is a schematic view showing the calibration of the marking pattern in the second embodiment of the present invention;

10 is a schematic flow chart of calculating each feature descriptor of a mark pattern/target image in Embodiment 2 of the present invention;

11 is a schematic flow chart of a method for constructing a virtual wall based on machine vision according to Embodiment 3 of the present invention;

12 is a flowchart of a method for constructing a virtual wall based on machine vision according to Embodiment 4 of the present invention;

FIG. 13 is a schematic flowchart diagram of a map construction method according to Embodiment 5 of the present invention; FIG.

14 is a schematic flowchart diagram of a map construction method according to Embodiment 6 of the present invention;

15 is a schematic flowchart of a map construction method according to Embodiment 7 of the present invention;

16 is a schematic diagram of turning of a mobile electronic device in Embodiment 7 of the present invention;

17 is a schematic structural diagram of a virtual machine building device based on machine vision according to Embodiment 8 of the present invention;

18 is a schematic structural diagram of a mobile electronic device according to Embodiment 9 of the present invention;

FIG. 19 is a schematic structural diagram of a mobile electronic device according to Embodiment 10 of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are only a part of the embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

1 is a schematic flowchart of a method for constructing a virtual wall based on machine vision according to Embodiment 1 of the present invention, including the steps:

S11. During the traversing of the to-be-positioned area, the camera disposed on the movable electronic device captures an image of the surrounding environment in a real-time manner at a preset frequency, and images the image captured at each moment. Forming a target image to a photosensitive surface of an image sensor provided in the movable electronic device;

In this step, diffuse reflection is generated on the surface of the object by natural illumination, and the reflected light is focused by an imaging lens to form a projected target image on the photosensitive surface of the image sensor. Compared with the traditional laser triangulation, it is not necessary to use the laser to first emit laser light and then enter the target object. There is no need to construct a datum surface, and only the diffuse reflection of an object under natural light (such as a door frame) can be used for ranging, thereby improving the accuracy of ranging and reducing the cost.

Wherein, the image sensor comprises a PSD sensor, a CCD sensor or a CMOS sensor.

S12. Acquire, according to a preset image matching algorithm, when the target image acquired at any time matches any of the marking patterns in the mark pattern library, obtain x key points of the target image on the photosensitive surface. Wherein the key points of the target image on the photosensitive surface are all located on a characteristic line; x≥2;

The mark pattern in the mark pattern library needs to be pre-stored in the memory of the removable electronic device, wherein one way is specifically, when the pre-stored mark instruction is received, the input picture (such as a vector diagram) is read and stored. It is used in the mark pattern library as a subsequent virtual wall recognition. In addition, each feature pattern needs to define a characteristic line or a key point of the virtual wall, which can be implemented in two ways: one is to add characteristic line information or key point information to the pre-stored picture, and then import the movable electronic device. The other is to perform the operation of defining the feature line or key point on the imported picture in response to the defined feature line or key point instruction according to the definition operation of the user input.

Preferably, the embodiment of the present invention may import the mark pattern by: acquiring an image directly above the movable electronic device at the current time in response to the calibration instruction, and storing the image in the mark pattern library as a new mark. pattern.

It should be noted that the matching of the images is performed by matching the feature vectors of the feature points. When a new mark pattern is acquired, the mark pattern is pre-processed and the feature points are extracted to obtain a plurality of feature points of the mark pattern and a feature descriptor corresponding to each of the feature points. Image feature extraction is the premise of image analysis and image recognition. It is the most effective way to simplify the expression of high-dimensional image data.

Based on the above calibration process, the corresponding key points may be preferably determined by the following two methods: one is to display a plurality of the feature points generated on the mark pattern, and is completed according to a user's selection instruction for a plurality of the feature points. The key point definition of the mark pattern; another specifically, after generating a plurality of feature points of the mark pattern, the system selects a plurality of the feature points by a preset algorithm to complete the mark The definition of the key points of the pattern.

S13. Calculate, according to a key point of the target image on the photosensitive surface, a distance between the virtual wall and the movable electronic device, according to a distance between the virtual wall and the movable electronic device, and a straight line with the feature Constructing the virtual wall at a predetermined angle and perpendicular to the photosensitive surface; wherein, when the predetermined angle is equal to 0° or 180°, then parallel with the feature line and The virtual wall is constructed on a surface perpendicular to the photosensitive surface.

In step S3, the image matching algorithm is used to perform feature extraction on the target image to obtain feature points, and then the specific feature points are selected as key points for mapping the virtual wall. It should be noted that the preset angle is an angle between the virtual wall to be constructed and the characteristic straight line, which is closely related to the selection of the key points.

It should be noted that the distance between the virtual wall to be constructed and the movable electronic device may be calculated by using laser focusing or phase focusing, and phase laser ranging, pulse laser ranging, and triangulation laser ranging may also be used. Preferably, the scheme adopts a triangulation method, and compared with the traditional triangulation laser ranging, the laser does not need to first transmit laser light to the target object and the reference surface, and then the target object is calculated by the reflection of the target object and the reflection image of the reference surface. The distance from the reference plane, so there is no need to construct a reference plane, and only the diffuse reflection of the object under natural light can be used for ranging, which has the advantages of simple structure, high precision, high speed and flexible use, and further reduces the production cost.

In step S4, if the preset angle is equal to 0° or 180°, then the virtual wall to be constructed is in parallel with the feature line by default.

According to the principle of triangulation, the distance between the virtual wall and the movable electronic device is calculated by the following formula:

D=a/b*S*|cosθ|

As shown in FIG. 2, when θ=0° or 180°, |cos 0°|=|cos 180°|=1, that is, when the virtual wall to be constructed and the characteristic line are parallel, the following formula can be calculated. The distance between the virtual wall and the removable electronic device:

D=a/b*S

Where a is the distance from the upper frame 400 of the virtual wall 100 to the photosensitive surface 301, b is the distance between the imaging lens 303 and the photosensitive surface 301, and S is the characteristic line 304 and the photosensitive surface The distance from the center point 302 of 301, D is the distance between the virtual wall 100 and the center point 302 of the photosensitive surface 301. 2 is a case where the movable electronic device is facing the virtual wall, and FIG. 3 is a case where the movable electronic device is shifted to the right with respect to the virtual wall. It can be understood that the movable electronic device is relatively In any direction of the virtual wall (including front, back, left and right offset), as long as the camera detects a certain number of matching feature points, it can accurately construct a reasonable key based on the position of the feature points. The virtual wall of the location.

As shown in FIG. 4, when the preset angle is not equal to 0° or 180°, the distance between the virtual wall and the movable electronic device is calculated by the following formula:

D=a/b*S*|cosθ|

Where a is the distance from the upper frame 400 of the virtual wall 100 to the photosensitive surface 301, b is the distance between the imaging lens 303 and the photosensitive surface 301, and S is the characteristic line 304 and the photosensitive surface The distance from the center point 302 of 301, D is the distance between the virtual wall 100 and the center point 302 of the photosensitive surface 301, and θ is the preset angle; as shown in FIG. 4, there is a ceiling on the ceiling 500 of the room. A mapping line 402 is projected onto the photosensitive surface 301 by the imaging lens 303 to generate the characteristic line 304; it is understood that h=a/b*S is the center point 302 of the photosensitive surface 301 and the mapping line 402 is located The distance of the plane 401, as shown in Fig. 5, is D = h * | cos θ |.

It should be noted that, in the embodiment of the present invention, the distance D between the virtual wall 100 and the center point 302 of the photosensitive surface 301 is the distance between the virtual wall 100 and the movable electronic device. In another preferred embodiment, as shown in FIG. 7, assuming that the removable electronic device 300 is a regular circle, the distance between the virtual wall 100 and the removable electronic device 300 can be defined as The distance from the center point 305 of the mobile electronic 300 to the virtual wall 100; specifically, when the removable electronic device 300 faces the virtual wall 100, the virtual wall 100 and the removable electronic device 300 The distance is the sum of the distance between the virtual wall 100 and the center point 302 of the photosensitive surface 301 and the distance between the center point 305 of the movable electronic device 300 and the center point 302 of the photosensitive surface 301, that is, L=D +D', wherein L is the distance between the virtual wall 100 and the movable electronic device 300, D is the distance between the virtual wall 100 and the center point 302 of the photosensitive surface 301, and D' is the movable The distance between the center point 305 of the electronic device 300 and the center point 302 of the photosensitive surface 301. It is to be understood that the above formula is applicable to the case where the movable electronic device is in a regular shape, and is not described herein.

In the embodiment of the present invention, the distance from the upper border of the virtual wall to the photosensitive surface refers to the distance from the ceiling of the room to the photosensitive surface, or the distance from the upper border of the door to the photosensitive surface. The specific value can be preset in the system.

In addition to calculating the position information of the virtual wall, the width information of the virtual wall can be calculated by the following steps, specifically calculating the width of the virtual wall by using the following formula:

Where W is the width of the virtual wall 100, a is the distance from the upper border of the virtual wall 100 to the photosensitive surface 301, and λ is the wide angle of the camera.

In this embodiment, the location image of the virtual wall needs to be directly acquired, and the image is pre-processed and extracted by using a preset image matching algorithm, and then the matching image is performed on the real-time acquired image. When the matching is successful, the image is selected. The specific feature point is used as a key point, and the position of the virtual wall relative to the movable electronic device is calculated according to the key point, and the virtual wall is automatically constructed, so that the boundary line between the partitionable entry area and the forbidden entry area can be accurately constructed. The dividing line can completely prohibit the movable electronic device from entering the forbidden entry area, and has the advantages of simple practicality and high reliability; in addition, the solution in this embodiment does not require additional interactive devices for setting the virtual wall, and there is no need to set the positioning at a specific location. Labels, etc., are more intelligent.

FIG. 8 is a flowchart of a method for constructing a virtual wall based on machine vision according to Embodiment 2 of the present invention, including the steps of:

S21, in the process of traversing the to-be-positioned area of the mobile electronic device, capturing, by using a camera disposed on the movable electronic device, an image of a surrounding environment in real time at a preset frequency, and projecting an image captured at each moment Forming a target image to a photosensitive surface of an image sensor provided in the movable electronic device;

S22. Obtain, by the image matching algorithm, a plurality of feature points of the target image at the current moment and a feature descriptor corresponding to each of the feature points;

S23. Obtain a feature point of each feature descriptor of the target image and each feature descriptor of the mark pattern, and acquire a feature point that has a matching relationship between the target image and the mark pattern. When the obtained number of feature points having a matching relationship is greater than a preset threshold, determining that the target image matches any of the mark patterns in the mark pattern library;

It should be noted that the matching of the images is performed by matching the feature vectors of the feature points. When a new mark pattern is acquired, the mark pattern is pre-processed and the feature points are extracted to obtain a plurality of feature points of the mark pattern and a feature descriptor corresponding to each of the feature points.

S24, according to the positional relationship of the feature points having the matching relationship on the target image, where the feature points on the same straight line are taken as the key points; x≥2;

According to the biological characteristics of image recognition, people's line of sight is always concentrated on the main features of the image, that is, where the contour of the image is the largest or the direction of the outline changes abruptly, and the amount of information in these places is the largest. It can be seen that in the image recognition process, the redundant information input must be excluded, and the key information is extracted, which involves the feature extraction problem. For example, taking a door frame image directly above the removable electronic device, the extracted feature points may be located at the edge of the door frame.

For example, in the calibration process of the marking pattern, the movable electronic device is placed directly below the upper frame of the door, and the calibration button can be pressed (the calibration button can be set on the mobile electronic device or on the remote controller). Or click on the calibration option on the third interactive terminal (such as mobile phone, tablet, PC, etc.), and then take the picture of the current moment through the camera, as shown in FIG. 9, the picture 200 includes the left border 201 of the door, the door Right border 202, door upper border 203 and room top 204. The feature extraction is performed on the image by a preset image matching algorithm, and several feature points (x1 to x6, y1 to y6) are obtained. In the traversing process of the mobile electronic device, if the image including the door frame is obtained, and the number of matching feature points is calculated by the preset image matching algorithm to be greater than a preset threshold, the condition of constructing the virtual wall may be considered as met. . It can be understood that when x2 and x3 are the key points, the constructed virtual wall is parallel with the features formed by x2 and x3; when y1 and y2 are the key points, the constructed virtual wall is composed of y1 and y2. The characteristic line is at a right angle of 90 degrees.

It should be noted that when the number of feature points for extracting the mark pattern is less than a certain preset number threshold, subsequent recognition may be unsuccessful, and the virtual wall may not be constructed at a specific position. Therefore, when the number of feature points is too small, the calibration may be unsuccessful to remind the user to recalibrate or construct the virtual wall in other ways.

S25. Calculate, according to a key point of the target image on the photosensitive surface, a distance between the virtual wall and the movable electronic device, according to a distance between the virtual wall and the movable electronic device, and a straight line with the feature Constructing the virtual wall at a predetermined angle and perpendicular to the photosensitive surface; wherein, when the predetermined angle is equal to 0° or 180°, then parallel with the feature line and The virtual wall is constructed on a surface perpendicular to the photosensitive surface.

As shown in FIG. 10, the image matching algorithm is a scale-invariant feature transform algorithm or an accelerated robust feature algorithm, and each feature descriptor of the mark pattern/target image is obtained by the following steps:

S26, establishing a scale space of the marker image/target image by Gaussian blur, and identifying an extreme point in the scale space of the marker image/target image by a Gaussian differential function, and a pole in the scale space of the marker image/target image The value point is verified to remove the unstable extreme point in the scale space of the mark image/target image, thereby obtaining the feature point of the mark image/target image and the scale and position of the feature point;

In step S26, since the local extremum points may not be extreme points in the true sense in the discrete space, the true pole points may fall in the gaps of the discrete points, so the position of the slots can be The interpolation check is performed, and then the coordinate position of the extreme point is obtained, thereby obtaining the coordinate position of the feature point.

S27: assign a direction to each of the feature points according to a gradient direction distribution characteristic of a neighboring pixel of the feature point;

In step S27, by performing histogram statistics on the gradient direction of the points in the neighborhood of the feature points, the histogram statistics are selected in the direction with the larger specific gravity in the histogram, and the direction with the largest specific gravity in the histogram is selected as the main direction of the feature points.

S28. Calculate a gradient histogram within the block by performing area segmentation on the surrounding image of the feature point according to the scale, position, and direction of the feature point, thereby generating a feature descriptor of the feature point.

In this embodiment, the Scale-invariant feature transform (SIFT) uses the convolution of the original image and the Gaussian kernel to establish the scale space, and extracts the feature of the scale invariance on the Gaussian difference space pyramid. point. The algorithm has certain affine invariance, visual invariance, rotation invariance and illumination invariance, so it has a wide range of applications in image feature improvement. In addition, the scale-invariant feature transform algorithm uses a first-order Gaussian difference to only a Gaussian Laplacian kernel, which greatly reduces the amount of computation.

The Speeded-Up Robust Features (SURF) is a scale-invariant feature transform algorithm. The matching process is basically the same. The difference is that the accelerated robust feature algorithm uses the approximate Harr wavelet method to extract feature points. A speckle feature detection method based on the Hessian determinant. By using the integral image on different scales, the approximate Ha rr wavelet value can be effectively calculated, which simplifies the construction of the second-order differential template and improves the efficiency of feature detection in scale space.

FIG. 11 is a flowchart of a method for constructing a virtual wall based on machine vision according to Embodiment 3 of the present invention, including the steps of:

S31. During the traversing of the to-be-positioned area, the camera disposed on the movable electronic device collects an image of the surrounding environment in a real-time manner at a preset frequency, and images the image captured at each moment. Forming a target image to a photosensitive surface of an image sensor provided in the movable electronic device;

S32. Obtain, by the image matching algorithm, a plurality of feature points of the target image at the current moment and a feature descriptor corresponding to each of the feature points;

S33, obtaining a feature point of the target image and the mark pattern by calculating a Euclidean distance of each feature descriptor of the target image and each feature descriptor of the mark pattern, when When the obtained number of feature points having a matching relationship is greater than a preset threshold, determining that the target image matches any of the mark patterns in the mark pattern library;

S34, according to the positional relationship of the feature points having the matching relationship on the target image, where the feature points on the same straight line are taken as the key points; x≥2;

S35. Calculate, according to a key point of the target image on the photosensitive surface, a distance between the virtual wall and the movable electronic device, according to a distance between the virtual wall and the movable electronic device, and a straight line with the feature Constructing the virtual wall at a predetermined angle and perpendicular to the photosensitive surface; wherein, when the predetermined angle is equal to 0° or 180°, then parallel with the feature line and Constructing the virtual wall on a surface perpendicular to the photosensitive surface;

S36. When the distance between the virtual wall and the mobile electronic device is less than a preset distance, move the mobile electronic device by using a preset avoidance policy to cause the mobile electronic device to be connected to the virtual wall. The distance increases.

The steps S31 to S35 of the embodiment are substantially the same as the steps S21 to S25 shown in FIG. 8. For details, refer to the detailed description of the steps S21 to S25, and details are not described herein again.

On the basis of Embodiment 2, the embodiment of the present invention adds a step of causing the removable electronic device to travel away from the virtual wall. In the actual application scenario, for the protection of the machine or other reasons, it is necessary to set a forbidden entry area, for example, prohibiting the sweeping robot from entering the toilet to prevent the ground area water or excessive water vapor from entering the machine, causing a short circuit, so it is necessary to construct a virtual wall and Set the appropriate avoidance strategy to prevent the robot from entering the forbidden entry area by mistake. Preferably, the avoidance strategy is specifically:

Adjusting a direction of travel of the mobile electronic device to move the movable electronic device in a direction away from the virtual wall.

It can be understood that, in addition to the above-mentioned avoidance strategy, the avoidance strategy of this embodiment may also adopt other manners, and details are not described herein again.

Preferably, since there is a change in the relative proportion of the near-far feature during the imaging of the photosensitive surface, bending or deformation occurs, so that the captured image is projected onto the image sensor provided in the movable electronic device. After the photosensitive surface forms the target image, it is necessary to correct the perspective distortion of the target image before comparing the target image with the pre-stored marking pattern in the marking pattern library, thereby constructing a more accurate virtual wall.

In the case of mobile electronic devices recognizing virtual walls, the traditional method mainly uses the following methods:

Build a map through the process of traversing the entire room area by the mobile electronic device, rasterize the constructed map, upload the rasterized map file to the computer, set a virtual wall on the map file through the computer, and then Upload the raster map file of the virtual wall to the removable electronic device. The downside of this approach is that each time a map is built in a new environment, it is cumbersome to re-export map files, upload map files, draw virtual walls, and import map files. On the other hand, additional work is required. The setting of interactive devices is not conducive to the development of intelligence.

In this embodiment, the mobile electronic device is directly placed directly under the pre-built virtual wall (directly under the door frame), that is, when the movable electronic device is at an angle of 90 degrees with the virtual wall, the camera obtains positive The upper picture is used as a mark pattern, and feature mark extraction is performed on the mark pattern to obtain feature descriptors of each feature point, and the movable electronic device acquires features of the mark pattern in a wide angle range of the camera during traversing the room. When the matching picture is matched, if the matching feature point is greater than the preset threshold, the key points on the same line in the feature point may be selected, the position information of the virtual wall is calculated, and the virtual wall is automatically constructed as a subsequent allowed entry. The boundary between the area and the forbidden area, or as a map for constructing the entire room, does not require a cumbersome import and export process, the construction process is more flexible, and has the advantages of simplicity and practicality; moreover, in addition, the solution in this embodiment does not require additional interactive equipment. To set up the virtual wall, there is no need to set the positioning label in a specific location, etc. High, when you need to cancel the virtual wall in a specific location, you only need to delete the mark pattern pre-stored into the removable electronic device, which is convenient and quick.

12 is a flowchart of a method for constructing a virtual wall based on machine vision according to Embodiment 4 of the present invention, including the steps of:

S41. In the process of traversing the to-be-positioned area, the camera disposed on the movable electronic device collects an image of the surrounding environment in a real-time manner at a preset frequency, and images the image captured at each moment. Forming a target image to a photosensitive surface of an image sensor provided in the movable electronic device;

S42. Obtain, by the image matching algorithm, a plurality of feature points of the target image at the current moment and a feature descriptor corresponding to each of the feature points;

S43, obtaining a feature point of the target image and the mark pattern by calculating a Euclidean distance of each feature descriptor of the target image and each feature descriptor of the mark pattern, when When the obtained number of feature points having a matching relationship is greater than a preset threshold, determining that the target image matches any of the mark patterns in the mark pattern library;

S44, according to the positional relationship of the feature points having the matching relationship on the target image, wherein the feature points on the same straight line are taken as the key points; x≥2;

S45. Calculate, according to a key point of the target image on the photosensitive surface, a distance between the virtual wall and the movable electronic device, according to a distance between the virtual wall and the movable electronic device, and a straight line with the feature. Constructing the virtual wall at a predetermined angle and perpendicular to the photosensitive surface; wherein, when the predetermined angle is equal to 0° or 180°, then parallel with the feature line and Constructing the virtual wall on a surface perpendicular to the photosensitive surface;

S46. After constructing the virtual wall, control the mobile electronic device to pass through the virtual wall in a preset path.

The steps S41 to S45 of the present embodiment are substantially the same as the steps S21 to S25 shown in FIG. 8. For details, refer to the detailed description of the steps S21 to S25, and details are not described herein again.

On the basis of Embodiment 2, the embodiment of the present invention adds the step of moving the mobile electronic device through the virtual wall. In an actual application scenario, after completing work such as cleaning a specific area, and continuing to work in another area, after constructing the virtual wall, the movable electronic device may be controlled to pass through the preset path. Describe the virtual wall to enter another area. Preferably, the path may be set as a straight path passing through the virtual wall and perpendicular to the virtual wall, or may be set as any curved path through the virtual wall.

It can be understood that the path of the embodiment may be set to other forms in addition to the paths disclosed in the foregoing, and details are not described herein again.

FIG. 13 is a schematic flowchart of a method for constructing a map according to Embodiment 5 of the present invention, including the steps of:

S51. Construct a coordinate system with an arbitrary position or a specific position in the to-be-positioned area as a coordinate origin, and calculate a displacement of the movable electronic device relative to the coordinate origin in real time during the process of the movable electronic device traversing the to-be-positioned area. And a direction to acquire coordinate values of the movable electronic device in the coordinate system in real time;

S52, in the process of traversing the to-be-positioned area of the mobile electronic device, collecting, by using a camera disposed on the movable electronic device, an image of a surrounding environment in real time at a preset frequency, and collecting each time Projecting an image onto a photosensitive surface of an image sensor provided in the movable electronic device to form a target image;

S53. Acquire, according to a preset image matching algorithm, when the target image acquired at any time matches any of the mark patterns in the mark pattern library, obtain x key points of the target image on the photosensitive surface. Wherein the key points of the target image on the photosensitive surface are all located on a characteristic line; x≥2;

S54. Calculate, according to a key point of the target image on the photosensitive surface, a distance between the virtual wall and the movable electronic device, according to a distance between the virtual wall and the movable electronic device, and a straight line with the feature Constructing the virtual wall at a predetermined angle and perpendicular to the photosensitive surface; wherein, when the predetermined angle is equal to 0° or 180°, then parallel with the feature line and Constructing the virtual wall on a surface perpendicular to the photosensitive surface;

S55. Perform real-time map construction on the to-be-positioned area according to coordinate values of the movable electronic device in the coordinate system and a coordinate plane of the virtual wall.

The steps S52 to S54 in this embodiment are substantially the same as the steps S11 to S13 in FIG. 1 . For details, refer to the detailed description of the steps S11 to S13, and details are not described herein again.

In this embodiment, the coordinate system is constructed by using an arbitrary position or a specific position in the area to be constructed as a reference point (coordinate origin), and then calculating the movable electronic device relative to the encoder by using an encoder disposed on the movable electronic device. The distance and direction of the origin of the coordinate to obtain the coordinate value of the movable electronic device in the coordinate system. Through the above-mentioned machine vision-based virtual wall construction method, when the position information of the virtual wall relative to the movable electronic device is calculated, the coordinate plane of the virtual wall in the coordinate system can be obtained. The mobile electronic device can establish a simple structure of the room according to the coordinate plane of each virtual wall in the coordinate system, and form a 3D three-dimensional map for navigation and use of the movable electronic device, which has the advantages of simple and practical.

Preferably, in addition to constructing the map using the virtual wall, the map can be constructed using obstacles and positioning tags encountered by the mobile electronic device during the traversal process. FIG. 14 is a schematic flowchart of a method for constructing a map according to Embodiment 6 of the present invention, where the at least two positioning tags are disposed on the to-be-positioned area, and each positioning tag is correspondingly disposed in the to-be-positioned area. At a specific location, each of the positioning tag information includes unique encoding information for distinguishing its absolute position, including the steps of:

S61. Construct a coordinate system with any position or a specific position in the to-be-positioned area as a coordinate origin, and calculate the movable electronic device relative to the coordinate origin in real time during the process of the movable electronic device traversing the to-be-positioned area. Displacement and direction to obtain coordinate values of the movable electronic device in the coordinate system in real time;

S62. In the process of traversing the to-be-positioned area, the camera disposed on the movable electronic device collects an image of the surrounding environment in real time at a preset frequency, and collects each time. Projecting an image onto a photosensitive surface of an image sensor provided in the movable electronic device to form a target image;

S63. Obtain, by the image matching algorithm, a plurality of feature points of the target image at a current time and a feature descriptor corresponding to each of the feature points;

S64. Obtain a feature point of each feature descriptor of the target image and each feature descriptor of the mark pattern, and obtain a feature point that has a matching relationship between the target image and the mark pattern. When the obtained number of feature points having a matching relationship is greater than a preset threshold, determining that the target image matches any of the mark patterns in the mark pattern library;

S65, according to the positional relationship of the feature points having the matching relationship on the target image, wherein the feature points on the same straight line are taken as key points; x≥2;

S66. Calculate, according to a key point of the target image on the photosensitive surface, a distance between the virtual wall and the movable electronic device, according to a distance between the virtual wall and the movable electronic device, and a straight line with the feature Constructing the virtual wall at a predetermined angle and perpendicular to the photosensitive surface; wherein, when the predetermined angle is equal to 0° or 180°, then parallel with the feature line and Constructing the virtual wall on a surface perpendicular to the photosensitive surface;

S67. Calculate and record a coordinate value of an obstacle position when the movable electronic device detects an obstacle each time the movable electronic device detects an obstacle based on a moving direction and a moving distance of the movable electronic device with respect to the starting point;

S68. Calculating, according to the moving direction and the moving distance of the movable electronic device with respect to the starting point, a coordinate value of a position of the positioning tag when the positioning tag information acquired by the movable electronic device is acquired in the traversing process. And recording the positioning label information and the corresponding coordinate values;

S69: Perform real-time map construction on the to-be-positioned area based on a coordinate plane of the virtual wall, coordinate values of each of the obstacle positions, and information of each of the positioning tags and coordinate values thereof.

The steps S62-66 of the embodiment are basically the same as the steps S21 to S25 shown in FIG. 8. For details, refer to the detailed description of the steps S21 to S25, and details are not described herein again.

Compared with the embodiment 5, the embodiment also constructs a map by using the position of the obstacle and the positioning tag, specifically detecting the obstacle by the obstacle sensor, the laser sensor or the infrared sensor, and when the obstacle is detected, according to the current moment The position of the movable electronic device acquires the coordinate value of the obstacle; in addition, different sensors are set according to the type of the positioning tag to obtain the position information of the positioning tag, for example, when the positioning tag is a color block label, the color sensor is set. Sensing, and when the positioning tag is an RFID tag, an RFID sensor is set for sensing. In this embodiment, by locating the coordinates of the label, the obstacle, and the coordinate plane of the virtual wall, a complete and detailed map can be constructed, which facilitates accurate navigation of the mobile electronic device, thereby facilitating the execution of subsequent work.

For robots, a common application requirement is that when the robot deviates from the centerline of the door, it is necessary to guide the robot back to the centerline position of the door before proceeding. In another preferred embodiment, based on Embodiment 6, as shown in FIG. 15, the map construction method further includes the steps of:

S71. Calculate, according to a perspective deformation of the target image, a distance of the movable electronic device from a center line of the virtual wall;

The perspective distortion is due to the relative proportion change of the near-far feature, and bending or deformation occurs, so that the distance of the movable electronic device from the center line of the virtual wall can be calculated according to the proportion of the number of pixels projected on the photosensitive surface.

S72. Return the movable electronic device to a center line of the virtual wall along a trajectory parallel to the virtual wall according to a distance of the movable electronic device from a center line of the virtual wall.

Through this step, the position of the movable electronic device to return to the center line of the virtual wall can be guided, thereby calibrating the position of the movable electronic device, facilitating subsequent rapid positioning and reforming the forward path.

In addition, due to the accuracy of the encoder and the like, there is an inevitable error in the relative distance and direction of the movable electronic device recorded by the encoder, resulting in inaccurate map construction. Therefore, after constructing the map, the embodiment obtains the coordinate values of the positioning label, the obstacle or the virtual wall multiple times by the traversal of the movable device, and then uses a recursive algorithm to locate each positioning label, obstacle or virtual. The coordinate values of the wall are corrected. The more the number of times the movable device traverses, the more accurate the calculated coordinate values of the positioning label, obstacle or virtual wall will be, until the error is reduced to negligible until the end.

It should be further noted that there is a very common phenomenon of slippage in mobile electronic devices that use wheels to travel. For example, when an obstacle is encountered, the driven wheel at the front is no longer rotated, and the driving wheel at the rear is still in a rotating state, at which time the encoder still records that the movable electronic device is in motion and according to the driving wheel Rotating the relative displacement and relative distance of the movable electronic device in real time, which will cause serious errors in the travel path, resulting in errors in the coordinate values of the subsequently detected positioning tags, obstacles or virtual walls, and cannot be accurately constructed. Maps, so you can't navigate precisely. Preferably, the movable electronic device can be detected in a slipping state by specifying effective corrective measures to avoid subsequent errors:

During the turning of the movable electronic device at an arbitrary center point, the driven wheel of the movable electronic device is detected at any time When the speed is greater than the theoretical speed, the speed of the driving wheel is used as a reference speed, and the displacement and direction of the movable electronic device relative to the coordinate origin are calculated according to the reference speed; the theoretical speed is according to the active The speed of the wheel is calculated. For the straight-through state, the line speeds of the points on the movable electronic device are equal, and when the unequal state is detected at any time, the movable electronic device can be judged to be in a slip state at the current time; and a more complicated situation is The mobile electronic device makes a turn along a central point, and the speeds at various points thereon are inconsistent. Taking the movable electronic device as a circular object as an example, as shown in FIG. 16, when the movable electronic device 300 makes a left turn at 0 o'clock at any time, it is assumed that the speed of the left driving wheel K1 is 50 cm/s, The speed of the right driving wheel K2 is 100 cm/s, and according to the distance proportional relationship between each point and 0 point, for example, the distance s1 of the front driven wheel K3 to 0 point is 80 cm, and the distance s3 of the right driving wheel K2 to 0 point is 100 cm. It can be seen that the theoretical speed of the driven wheel K3 and the speed ratio of the right driving wheel K2 are 80/100=4/5; therefore, in the normal running state, the theoretical speed of the driven wheel K3 at the front end should be 80 cm/s, then When the actual speed of the driven wheel K3 does not match the theoretical speed of 80 cm/s at the current time, it can be determined that the mobile electronic device 300 is in the slip state at the current time.

17 is a schematic structural diagram of a virtual machine building device based on machine vision according to Embodiment 8 of the present invention. The virtual wall building device is disposed on a mobile electronic device, and includes:

The camera 81 is configured to collect an image of the surrounding environment in real time at a preset frequency during the process of traversing the area to be located by the movable electronic device;

The image sensor 82 is configured to receive an image acquired at each moment on the photosensitive surface of the image sensor 72 to form a target image;

a storage device 83, configured to pre-store a plurality of mark patterns;

a controller 84, configured to acquire, according to a preset image matching algorithm, when the target image acquired at any time matches any of the marking patterns in the storage device 83, acquiring the target image on the photosensitive surface x key points; calculating a distance between the virtual wall and the movable electronic device based on the key points of the target image on the photosensitive surface, according to the distance between the virtual wall and the movable electronic device, Constructing the virtual wall with a predetermined straight line at a predetermined angle and perpendicular to the photosensitive surface; wherein, when the predetermined angle is equal to 0° or 180°, then the line is parallel to the characteristic Constructing the virtual wall on a surface perpendicular to the photosensitive surface; the key points of the target image on the photosensitive surface are all located on a characteristic line; x≥2.

Preferably, the controller 84 is configured to: first acquire, by using the image matching algorithm, a plurality of feature points of the target image at a current time and a feature descriptor corresponding to each of the feature points, and then calculate the target by calculating a feature point of each feature descriptor of the image and each feature descriptor of the mark pattern, and obtaining a feature point having a matching relationship between the target image and the mark pattern, when the acquired matching relationship is obtained When the number of the feature points is greater than the preset threshold, determining that the target image matches any of the mark patterns in the mark pattern library, and then according to the positional relationship of the feature points having the matching relationship on the target image, x feature points on the same line are used as key points; x≥2.

Wherein, the controller 84 imports the mark pattern by:

In response to the calibration instruction, an image directly above the removable electronic device at the current time is acquired and stored in the mark pattern library as a new mark pattern.

The image matching algorithm is a scale invariant feature transform algorithm or an accelerated robust feature algorithm, and the controller 84 obtains each feature descriptor of the mark pattern/target image by the following steps:

For the working principle and process of the virtual wall building device of this embodiment, reference may be made to the descriptions of Embodiment 1 and Embodiment 2, and details are not described herein again.

Wherein, the camera 81 is provided with an imaging lens (or imaging lens) for imaging focusing, so that the reflective object forms a target image on the photosensitive surface of the image sensor 82.

In this embodiment, the distance between the virtual wall and the mobile electronic device is calculated by a triangulation method.

D=a/b*S*|cosθ|

D=a/b*S

D=a/b*S*|cosθ|

It should be noted that, in the embodiment of the present invention, the distance D between the virtual wall 100 and the center point 302 of the photosensitive surface 301 is the distance between the virtual wall 100 and the movable electronic device. In another preferred embodiment, as shown in FIG. 7, assuming that the removable electronic device 300 is a regular circle, the distance between the virtual wall 100 and the removable electronic device 300 can be defined as The distance from the center point 305 of the mobile electronic 300 to the virtual wall 100; specifically, when the removable electronic device 300 faces the virtual wall 100, as shown in FIG. 7, the portable electronic device 300 is assumed For a regular circle, the distance between the virtual wall 100 and the removable electronic device 300 may be defined as the distance between the virtual wall 100 and the center point 302 of the photosensitive surface 301 and the movable electronic device 300. The sum of the distances between the center point 305 and the center point 302 of the photosensitive surface 301, that is, L=D+D', where L is the distance between the virtual wall 100 and the removable electronic device 300, and D is the virtual The distance of the wall 100 from the center point 302 of the photosensitive surface 301, D', is the distance between the center point 305 of the movable electronic device 300 and the center point 302 of the photosensitive surface 301. It is to be understood that the above formula is applicable to the case where the movable electronic device is in a regular shape, and is not described herein.

Where W is the width of the virtual wall 100, a is the distance from the upper border of the virtual wall 100 to the photosensitive surface 301, and λ is the wide angle of the camera 81.

The image sensor 82 includes a PSD sensor, a CCD sensor, or a CMOS sensor.

In another preferred embodiment, the controller 84 is further configured to: when the distance between the virtual wall and the mobile electronic device is less than a preset distance, the controller 84 is further configured to bypass the preset The policy moves the mobile electronic device such that the distance of the mobile electronic device from the virtual wall increases.

In another preferred embodiment, the controller 84 is further configured to control the movable electronic device to pass through the virtual wall in a preset path after constructing the virtual wall.

Preferably, after the ingested picture is projected onto the photosensitive surface of the image sensor 82 provided in the movable electronic device to form a target image, the target image is compared with the pre-stored mark pattern in the mark pattern library. The controller 84 is further configured to perform a correction of the perspective deformation of the target image.

FIG. 18 is a schematic structural diagram of a mobile electronic device according to Embodiment 9 of the present invention, including:

The virtual wall construction device 91 according to any one of the preceding claims, configured to construct a virtual wall;

The controller 84 is further configured to construct a coordinate system by using any position or a specific position in the to-be-positioned area as a coordinate origin;

The encoder 92 is configured to calculate, in real time, the displacement and direction of the movable electronic device relative to the coordinate origin in the process of traversing the to-be-positioned area by the mobile electronic device;

The controller 84 is configured to receive a displacement and a direction of the movable electronic device sent by the encoder 92 with respect to the coordinate origin, and acquire coordinate values of the mobile electronic device in the coordinate system at an arbitrary time;

The controller 84 is further configured to perform real-time map construction on the to-be-positioned area according to coordinate values of the movable electronic device in the coordinate system and a coordinate plane of the virtual wall.

The working principle and process of the real-time map construction of the mobile electronic device in this embodiment can be referred to the description of the foregoing embodiment 5, and details are not described herein again.

It should be noted that the coordinate plane of the virtual wall is calculated according to the coordinate value of the movable electronic device in the coordinate system and the distance between the virtual wall and the movable electronic device.

In this embodiment, the mobile electronic device includes a driving wheel and a driven wheel, and the controller 84 is further configured to detect the movable time at any time during the traveling of the movable electronic device along an arbitrary straight line. When the speed of the driving wheel of the electronic device does not coincide with the speed of the driven wheel, the smaller value of the speed of the driving wheel and the speed of the driven wheel is used as a reference speed, and the movable electronic device is calculated according to the reference speed. The displacement and direction relative to the origin of the coordinate;

19 is a schematic structural diagram of a mobile electronic device according to Embodiment 10 of the present invention, including:

The virtual wall construction device 101 according to any one of the preceding claims, configured to construct a map;

The encoder 102 is configured to calculate, in real time, the displacement and direction of the movable electronic device relative to the coordinate origin in the process of traversing the to-be-positioned area by the mobile electronic device;

The controller 84 is configured to receive a displacement and a direction of the movable electronic device sent by the encoder 102 with respect to the coordinate origin, and acquire coordinate values of the mobile electronic device in the coordinate system at an arbitrary time;

Collision sensor / laser sensor / infrared sensor 103 for detecting obstacles;

The controller 84 is further configured to perform real-time map construction on the to-be-positioned area based on a coordinate plane of the virtual wall and coordinate values of each of the obstacle positions.

For the working principle and process of the real-time map construction of the mobile electronic device of the present embodiment, reference may be made to the description of the foregoing embodiment 6, and details are not described herein again.

In this embodiment, when the obstacle is sensed by the collision sensor 103, the controller 84 sets the current position of the movable electronic device based on the moving direction and the moving distance of the movable electronic device with respect to the starting point. The coordinate value is used as the coordinate value of the obstacle position; when the obstacle is detected by the laser sensor/infrared sensor 103, the controller 84 calculates the obstacle according to the laser/infrared distance calculation principle. The position of the mobile electronic device is based on the coordinate value of the obstacle.

In another preferred embodiment, at least two positioning tags are disposed on the to-be-positioned area, and each positioning tag is correspondingly disposed at a specific position of the to-be-positioned area, and each of the positioning tag information includes The unique encoding information of the absolute position of the controller; the controller 84 performs real-time map construction on the to-be-positioned area according to the coordinate value of the movable electronic device in the coordinate system and the position of the virtual wall, including:

Preferably, the controller 84 is further configured to calculate, according to the perspective deformation of the target image, a distance of the mobile electronic device from the center line of the virtual wall; according to a distance of the mobile electronic device from a center line of the virtual wall; And causing the mobile electronic device to return to a center line of the virtual wall along a trajectory parallel to the virtual wall.

It should be noted that, in this specification, the terms "including", "comprising", or any other variations thereof are intended to encompass a non-exclusive inclusion, such that a process, method, article, or device that comprises a And also includes other elements not explicitly listed, or elements that are inherent to such a process, method, item, or device. An element that is defined by the phrase "comprising", without limiting the invention, does not exclude the presence of additional elements in the process, method, article, or device.

Finally, it should also be noted that the series of processes described above include not only processes that are performed in time series in the order described herein, but also processes that are performed in parallel or separately, rather than in chronological order. Through the description of the above embodiments, those skilled in the art can clearly understand that the present invention can be implemented by means of software plus a necessary hardware platform, and of course, all can be implemented by software. Based on such understanding, all or part of the technical solution of the present invention contributing to the background art may be embodied in the form of a software product, which may be stored in a storage medium such as a ROM/RAM, a magnetic disk, an optical disk, or the like. A number of instructions are included to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform the methods described in various embodiments of the present invention or in some portions of the embodiments.

The above is a preferred embodiment of the present invention, and it should be noted that those skilled in the art can also make several improvements and retouchings without departing from the principles of the present invention. It is the scope of protection of the present invention.

Claims

A virtual wall construction method based on machine vision, comprising the steps of:

In the process of traversing the area to be located, the camera disposed on the movable electronic device captures an image of the surrounding environment in real time at a preset frequency, and projects the image acquired at each moment to the device. Forming a target image on a photosensitive surface of the image sensor in the movable electronic device;

Obtaining x key points of the target image on the photosensitive surface when the target image acquired at any time matches any of the marking patterns in the marking pattern library according to a preset image matching algorithm; The key points of the target image on the photosensitive surface are all located on a characteristic line; x≥2;

Calculating a distance between the virtual wall and the movable electronic device based on a key point of the target image on the photosensitive surface, according to a distance between the virtual wall and the movable electronic device, Forming the virtual wall on an angle that is perpendicular to the photosensitive surface; wherein, when the predetermined angle is equal to 0° or 180°, then parallel to the characteristic line and the photosensitive surface The virtual wall is constructed on a vertical plane.
The method according to claim 1, wherein the method further comprises the steps of:

And acquiring an image directly above the movable electronic device at the current moment in response to the calibration instruction, and storing the image in the mark pattern library as a new mark pattern;

And acquiring, by the image matching algorithm, a plurality of feature points of the mark pattern and a feature descriptor corresponding to each of the feature points.
The machine vision-based virtual wall construction method according to claim 2, wherein the target image acquired at any time and any of the mark patterns in the mark pattern library according to a preset image matching algorithm When matching, the x key points of the target image on the photosensitive surface are specifically:

Obtaining, by the image matching algorithm, a plurality of feature points of the target image at the current moment and a feature descriptor corresponding to each of the feature points;

Obtaining a feature point of the matching relationship between the target image and the mark pattern by calculating a Euclidean distance of each feature descriptor of the target image and each feature descriptor of the mark pattern, when acquiring When the number of feature points having a matching relationship is greater than a preset threshold, determining that the target image matches any of the mark patterns in the mark pattern library;

According to the positional relationship of the feature points having the matching relationship on the target image, the feature points in which x are on the same straight line are taken as key points.
The machine vision-based virtual wall construction method according to claim 3, wherein the image matching algorithm is a scale-invariant feature transform algorithm or an accelerated robust feature algorithm, and each feature description of the mark pattern/target image is The child gets it by the following steps:

The scale space of the marker image/target image is established by Gaussian blur, and the extreme point in the scale space of the marker image/target image is identified by a Gaussian differential function, and the extreme point in the scale space of the marker image/target image Performing a check to remove an unstable extreme point in the scale space of the mark image/target image, thereby obtaining a feature point of the mark image/target image and a scale and a position of the feature point;

And assigning a direction to each of the feature points according to a gradient direction distribution characteristic of a neighboring pixel of each of the feature points in the mark image/target image;

Calculating an intra-block gradient histogram by performing area segmentation on a surrounding image of the feature point according to a scale, a position, and a direction of each of the feature points in the mark image/target image, thereby generating the feature point Feature descriptor.
The method of constructing a virtual machine based on a machine vision according to claim 1, wherein an image captured by the camera is projected through an imaging lens to a photosensitive surface of the image sensor; the virtual wall and the movable electronic device The distance is calculated by the triangulation method.
The machine vision-based virtual wall construction method according to claim 5, wherein the calculating the distance between the virtual wall and the movable electronic device based on the key point of the target image on the photosensitive surface is specifically :

The distance between the virtual wall and the removable electronic device is calculated by the following formula:

D=a/b*S*|cosθ|

Where a is the distance from the upper border of the virtual wall to the photosensitive surface, b is the distance between the imaging lens and the photosensitive surface, and S is the distance between the characteristic straight line and the center point of the photosensitive surface, D is a distance between the virtual wall and a center point of the photosensitive surface, and θ is the preset angle.
The machine vision-based virtual wall construction method according to claim 1, wherein the image sensor comprises a PSD sensor, a CCD sensor or a CMOS sensor.
The machine vision-based virtual wall construction method according to claim 1, wherein the width of the virtual wall is calculated by the following formula:

Where W is the width of the virtual wall, a is the distance from the upper border of the virtual wall to the photosensitive surface, and λ is the wide angle of the camera.
The method according to claim 1, wherein the method further comprises the steps of:

When the distance between the virtual wall and the mobile electronic device is less than a preset distance, moving the mobile electronic device by a preset avoidance strategy to make the distance between the movable electronic device and the virtual wall Increase.
The method according to claim 1, wherein the method further comprises the steps of:

After constructing the virtual wall, the movable electronic device is controlled to pass through the virtual wall in a preset path.
The method for constructing a virtual machine based on a machine vision according to claim 1, wherein the step of projecting the captured image onto the photosensitive surface of the image sensor provided in the movable electronic device to form the target image further comprises the steps of:

Correction of lens deformation is performed on the target image.
A map construction method, comprising the steps

Constructing a coordinate system with any position or specific position in the to-be-positioned area as a coordinate origin, and calculating a displacement of the movable electronic device relative to the coordinate origin in a real-time process in which the movable electronic device traverses the to-be-positioned area And a direction to acquire coordinate values of the movable electronic device in the coordinate system in real time;

Constructing a map using the machine vision-based virtual wall construction method according to any one of claims 1 to 10;

Real-time map construction is performed on the to-be-positioned area according to coordinate values of the movable electronic device in the coordinate system and a coordinate plane of the virtual wall.
The map construction method according to claim 12, wherein the real-time map is performed on the to-be-positioned area according to coordinate values of the movable electronic device in the coordinate system and a position of the virtual wall The build includes:

Calculating and recording a coordinate value of an obstacle position when the movable electronic device detects an obstacle each time based on a moving direction and a moving distance of the movable electronic device with respect to the starting point;

Real-time map construction is performed on the to-be-positioned area based on a coordinate plane of the virtual wall and coordinate values of each of the obstacle positions.
The map construction method according to claim 13, wherein at least two positioning tags are disposed on the to-be-positioned area, and each of the positioning tags is correspondingly disposed at a specific position of the to-be-positioned area, each of the The positioning tag information includes unique encoding information for distinguishing its absolute position; then the real-time operation of the to-be-positioned area according to the coordinate value of the movable electronic device in the coordinate system and the position of the virtual wall Map construction also includes:

Calculating, according to the moving direction and the moving distance of the movable electronic device with respect to the starting point, a coordinate value of a position of the positioning tag when the positioning tag information acquired by the movable electronic device is acquired, and Recording positioning tag information and corresponding coordinate values;

Determining a coordinate plane based on the virtual wall, coordinate values of each of the obstacle positions, and information about each of the positioning tags A real-time map is constructed for the to-be-positioned area.
The map construction method according to claim 12, wherein the method further comprises the steps of:

Calculating, according to a perspective deformation of the target image, a distance of the movable electronic device from a center line of the virtual wall;

The movable electronic device is returned to a center line of the virtual wall along a trajectory parallel to the virtual wall according to a distance of the movable electronic device from a center line of the virtual wall.
The map construction method according to claim 12, wherein the removable electronic device comprises a driving wheel and a driven wheel, and the method further comprises the steps of:

During the traveling of the movable electronic device along an arbitrary straight line, when the speed of the driving wheel of the movable electronic device is detected to be inconsistent with the speed of the driven wheel at any time, the speed of the driving wheel and the driven wheel The smaller of the speeds is used as the reference speed, and the displacement and direction of the movable electronic device relative to the coordinate origin are calculated according to the reference speed;

During the turning of the movable electronic device at an arbitrary center point, when the speed of the driven wheel of the movable electronic device is detected to be less than the theoretical speed at any time, the speed of the driven wheel is used as a reference speed, according to Calculating, by the reference speed, a displacement and a direction of the movable electronic device relative to the coordinate origin;

In the process of turning the movable electronic device at an arbitrary center point, when the speed of the driven wheel of the movable electronic device is detected to be greater than the theoretical speed at any time, the speed of the driving wheel is used as a reference speed And calculating, according to the reference speed, a displacement and a direction of the movable electronic device relative to the coordinate origin; the theoretical speed is calculated according to a speed of the driving wheel.
A machine-based virtual wall building device based on machine vision is characterized in that the machine vision-based virtual wall building device is disposed on the mobile electronic device, and includes:

a camera for collecting an image of the surrounding environment in real time at a preset frequency during the traversing of the area to be located by the movable electronic device;

An image sensor for receiving an image acquired at each moment on a photosensitive surface of the image sensor to form a target image;

a storage device for pre-storing a plurality of mark patterns;

a controller, configured to acquire, according to a preset image matching algorithm, the x key of the target image on the photosensitive surface when the target image acquired at any time matches any of the marking patterns in the storage device Pointing a distance between the virtual wall and the movable electronic device based on a key point of the target image on the photosensitive surface, according to a distance between the virtual wall and the movable electronic device, in a straight line with the feature Constructing the virtual wall at a predetermined angle and perpendicular to the photosensitive surface; wherein, when the predetermined angle is equal to 0° or 180°, then parallel with the feature line and The virtual wall is constructed on a surface perpendicular to the photosensitive surface; the key points of the target image on the photosensitive surface are all located on a characteristic line; x≥2.
The machine vision-based virtual wall construction apparatus according to claim 17, wherein the controller is further configured to: in response to the calibration instruction, control the camera to acquire an image directly above the movable electronic device at a current time, And storing in the storage device as a new mark pattern; acquiring, by the image matching algorithm, a plurality of feature points of the mark pattern and a feature descriptor corresponding to each of the feature points.
The machine vision-based virtual wall construction apparatus according to claim 18, wherein the target image acquired at any time matches any of the mark patterns in the mark pattern library according to a preset image matching algorithm The controller obtains x key points of the target image on the photosensitive surface, specifically:

Obtaining, by the image matching algorithm, a plurality of feature points of the target image and a feature descriptor corresponding to each of the feature points;

Obtaining a feature point of the matching relationship between the target image and the mark pattern by calculating a Euclidean distance of each feature descriptor of the target image and each feature descriptor of the mark pattern, when acquiring When the number of feature points having a matching relationship is greater than a preset threshold, determining that the target image matches any of the mark patterns in the mark pattern library;

According to the positional relationship of the feature points having the matching relationship on the target image, the feature points in which x are on the same straight line are taken as key points.
A machine vision-based virtual wall construction apparatus according to claim 19, wherein said image matching algorithm is The scale invariant feature transform algorithm or the accelerated robust feature algorithm, each feature descriptor of the mark pattern/target image is obtained by the controller by using the following steps:

The scale space of the marker image/target image is established by Gaussian blur, and the extreme point in the scale space of the marker image/target image is identified by a Gaussian differential function, and the extreme point in the scale space of the marker image/target image Performing a check to remove an unstable extreme point in the scale space of the mark image/target image, thereby obtaining a feature point of the mark image/target image and a scale and a position of the feature point;

And assigning a direction to each of the feature points according to a gradient direction distribution characteristic of a neighboring pixel of each of the feature points in the mark image/target image;

Calculating an intra-block gradient histogram by performing area segmentation on a surrounding image of the feature point according to a scale, a position, and a direction of each of the feature points in the mark image/target image, thereby generating the feature point Feature descriptor.
The machine vision-based virtual wall construction apparatus according to claim 17, wherein the camera further comprises an imaging lens, and an image captured by the camera is projected through an imaging lens to a photosensitive surface of the image sensor; The distance between the wall and the movable electronic device is calculated by a triangulation method.
The machine vision-based virtual wall construction apparatus according to claim 21, wherein the controller calculates a distance between the virtual wall and the movable electronic device based on a key point of the target image on the photosensitive surface Specifically:

The distance between the virtual wall and the removable electronic device is calculated by the following formula:

D=a/b*S*|cosθ|

Where a is the distance from the upper border of the virtual wall to the photosensitive surface, b is the distance between the imaging lens and the photosensitive surface, and S is the distance between the characteristic straight line and the center point of the photosensitive surface, D is a distance between the virtual wall and a center point of the photosensitive surface, and θ is the preset angle.
A machine vision-based virtual wall construction apparatus according to claim 22, wherein said controller calculates the width of said virtual wall based on the following formula:

Where W is the width of the virtual wall, a is the distance from the upper border of the virtual wall to the photosensitive surface, and λ is the wide angle of the camera.
A machine vision-based virtual wall construction apparatus according to claim 17, wherein said image sensor comprises a PSD sensor, a CCD sensor or a CMOS sensor.
The machine vision-based virtual wall construction apparatus according to claim 17, wherein when the distance between the virtual wall and the movable electronic device is less than a preset distance, the controller is further configured to pass the pre- The avoidance strategy moves the mobile electronic device such that the distance of the removable electronic device from the virtual wall increases.
The machine vision-based virtual wall construction method according to claim 17, wherein the controller is further configured to control the movable electronic device to pass through the preset path after constructing the virtual wall. Describe the virtual wall.
The machine vision-based virtual wall construction device according to claim 17, wherein after the captured image is projected onto a photosensitive surface of an image sensor provided in the movable electronic device to form a target image, the control The device is also used to correct the perspective distortion of the target image.
A mobile electronic device, comprising:

A machine vision-based virtual wall building device according to any one of claims 17 to 27, for constructing a virtual wall;

The controller is further configured to construct a coordinate system by using any position or a specific position in the to-be-positioned area as a coordinate origin;

An encoder, configured to calculate, in real time, a displacement and a direction of the movable electronic device relative to the coordinate origin in a process in which the movable electronic device traverses the to-be-positioned area;

The controller is further configured to receive a displacement and a direction of the movable electronic device sent by the encoder with respect to the coordinate origin, and acquire coordinate values of the movable electronic device in the coordinate system at any time;

The controller is further configured to perform real-time map construction on the to-be-positioned area according to coordinate values of the movable electronic device in the coordinate system and a coordinate plane of the virtual wall.
A mobile electronic device according to claim 28, further comprising a collision sensor, a laser sensor or an infrared sensor, said controller being based on said movable electron when an obstacle is sensed by the collision sensor a moving direction and a moving distance of the device relative to the starting point, and a coordinate value of a current position of the movable electronic device as a coordinate value of the obstacle position;

When the obstacle is detected by the laser sensor/infrared sensor, the controller calculates the position of the obstacle relative to the currently movable electronic device according to the laser/infrared distance calculation principle, so that the movable electronic device according to the current moment Calculating a coordinate value of the obstacle at the current time relative to a moving direction and a moving distance of the starting point;

The controller is configured to perform real-time map construction on the to-be-positioned area based on a coordinate plane of the virtual wall and coordinate values of each of the obstacle positions.
The mobile electronic device according to claim 29, wherein at least two positioning tags are disposed on the to-be-positioned area, and each of the positioning tags is correspondingly disposed at a specific position of the to-be-positioned area, and each The positioning tag information includes unique encoding information for distinguishing its absolute position; the controller determines the to-be-determined according to the coordinate value of the movable electronic device in the coordinate system and the position of the virtual wall Real-time map construction for bit areas includes:

Calculating, according to the moving direction and the moving distance of the movable electronic device with respect to the starting point, a coordinate value of a position of the positioning tag when the positioning tag information acquired by the movable electronic device is acquired, and Recording positioning tag information and corresponding coordinate values;

A map is constructed based on a coordinate plane of the virtual wall, coordinate values of each of the obstacle positions, and information of each of the positioning tags and coordinate values thereof.
The mobile electronic device according to claim 28, wherein the controller is further configured to calculate a distance of the movable electronic device from the center line of the virtual wall according to a perspective deformation of the target image; The movable electronic device is returned to a center line of the virtual wall along a trajectory parallel to the virtual wall according to a distance of the movable electronic device from a center line of the virtual wall.
A removable electronic device according to claim 28, wherein said movable electronic device comprises a driving wheel and a driven wheel, and said controller is further for traveling along said straight line at said movable electronic device During the process, when the speed of the driving wheel of the movable electronic device is detected to be inconsistent with the speed of the driven wheel at any time, the smaller of the speed of the driving wheel and the speed of the driven wheel is used as the reference speed. Calculating a displacement and a direction of the movable electronic device relative to the coordinate origin according to the reference speed;

During the turning of the movable electronic device at an arbitrary center point, when the speed of the driven wheel of the movable electronic device is detected to be less than the theoretical speed at any time, the speed of the driven wheel is used as a reference speed, according to Calculating, by the reference speed, a displacement and a direction of the movable electronic device relative to the coordinate origin;

In the process of turning the movable electronic device at an arbitrary center point, when the speed of the driven wheel of the movable electronic device is detected to be greater than the theoretical speed at any time, the speed of the driving wheel is used as a reference speed And calculating, according to the reference speed, a displacement and a direction of the movable electronic device relative to the coordinate origin; the theoretical speed is calculated according to a speed of the driving wheel.