WO2020052667A1

WO2020052667A1 - Visual repositioning method and apparatus for intelligent device

Info

Publication number: WO2020052667A1
Application number: PCT/CN2019/105786
Authority: WO
Inventors: 张胜美; 李冠毅; 李庚�; 马健; 林焘
Original assignee: 北京猎户星空科技有限公司
Priority date: 2018-09-14
Filing date: 2019-09-12
Publication date: 2020-03-19
Also published as: CN109255817A; TW202030699A

Abstract

Disclosed are a visual repositioning method and apparatus for an intelligent device. The method comprises: acquiring an environment image collected by a visual sensor of an intelligent device, and performing in-situ visual repositioning according to the acquired environment image (101); if the in-situ visual repositioning fails, controlling the intelligent device to move according to a pre-set motion rule so as to change the environment image collected by the visual sensor (102); and reacquiring an environment image collected by the visual sensor, and performing visual repositioning again according to the reacquired environment image (103). When an intelligent device moves, the features of an environment image collected by a visual sensor change, the success rate of visual repositioning for the intelligent device is improved, and the problem of how to improve the success rate of positioning after the intelligent device fails to perform visual repositioning in situ is solved.

Description

Method and device for visual repositioning of intelligent equipment

Cross-reference to related applications

This application claims the priority of a Chinese patent application filed on September 14, 2018 with the Chinese Patent Office, with the application number 201811075083.5, and the application name "A method and device for visual relocation of a smart device", the entire contents of which are incorporated by reference In this application.

Technical field

The present application relates to the field of positioning technology, and in particular, to a method and a device for visual repositioning of a smart device.

Background technique

The relocation of autonomously moving artificial intelligence devices, such as robots, re-determines the location of smart devices based on known map environments and sensor data. After the robot is successfully relocated, it can realize autonomous navigation and provide other services. There are two main types of current robot relocation technologies:

1) Laser relocation technology

The user first manually clicks the approximate position of the robot on the map, and then the robot continuously emits laser light. Through laser distance matching, the robot is calibrated to the correct position with small errors.

However, this relocation technology requires human participation and has a high error rate.

2) Visual relocation technology

Vision relocation technology collects vision sensing images through vision sensors and implements relocation based on known map environments. The amount of information obtained by the vision sensor is rich, and no human involvement is required in the relocation process, which improves the efficiency of positioning.

In visual relocation, when moving to some more special location, such as facing a white wall, the amount of information collected is relatively small, and the positioning may fail, or when the positioning fails for other reasons, human intervention is still required. The user experience is bad.

At present, there is a lack of a solution for how to improve the success rate of intelligent devices that can move autonomously after failure of visual relocation in situ.

Summary of the Invention

The present application provides a method and device for visual relocation of a smart device, which solves the problem of how to improve the success rate of positioning after the failure of visual relocation using a smart device in place.

According to a first aspect of the present application, a method for visual repositioning of a smart device is provided. The method includes:

Obtain the environmental image collected by the visual sensor of the smart device, and perform in-situ visual relocation according to the acquired environmental image;

If the in-situ visual relocation fails, controlling the smart device to move according to a preset motion rule to change an environmental image collected by the vision sensor;

The environment image acquired by the vision sensor is acquired again, and the visual relocation is performed again according to the environment image acquired again.

According to a second aspect of the present application, a visual repositioning device applied to a smart device is provided. The device includes:

In-situ relocation unit, which is used to obtain the environment image collected by the visual sensor of the smart device, and perform in-situ visual relocation according to the obtained environment image;

A control motion unit, configured to visually fail in situ, and control the smart device to move according to a preset motion rule to change an environmental image collected by the vision sensor;

The re-repositioning unit is configured to obtain the environmental image collected by the vision sensor again, and perform visual re-positioning according to the acquired environmental image again.

According to a third aspect of the present application, an electronic device is provided, including a processor and a memory, where:

The memory is configured to store an executable program;

The processor is configured to implement the foregoing visual relocation method when the executable program is executed.

According to a fourth aspect of the present application, a computer storage medium is provided. The computer storage medium stores a computer program. When the computer program is executed by an electronic device, the electronic device can execute the visual repositioning method.

According to a fifth aspect of the present application, a computer program product is provided, and when the computer program product is called and executed by an electronic device, the electronic device can execute the foregoing visual relocation method.

The method and device for visual repositioning of a smart device provided in this application have the following beneficial effects:

1. After the in-situ visual relocation of the smart device fails, the visual relocation can be automatically carried out through the above visual relocation method without human involvement;

2. By controlling the intelligent device to move according to a preset motion rule, the image characteristics of the environmental image collected by the vision sensor are changed, and the success rate of repositioning is greatly improved.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flowchart of a visual repositioning method provided in Embodiment 1 of the present application;

FIG. 2 is a flowchart of a method for re-visualizing relocation provided in Embodiment 2 of the present application; FIG.

3 is a flowchart of a visual relocation method provided in Embodiment 3 of the present application;

FIG. 4 is a diagram of a visual relocation device for a smart device provided in Embodiment 4 of the present application.

detailed description

In order to make the purpose, technical solution, and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

Embodiment one:

The present application provides a method for visual repositioning of a smart device. The method is executed by an electronic device. The electronic device may be the smart device itself. The smart device may be, but is not limited to, a robot. The electronic device may also be an external device that communicates with the smart device. A device, such as a server, is shown in FIG. 1. The electronic device in this embodiment is a smart device. The method includes:

Step 101: Obtain an environment image collected by a visual sensor of a smart device, and perform in-situ visual relocation according to the obtained environment image.

When the smart device enables the visual relocation function, in order to support the smart device navigation function, the smart device is required to perform in-situ visual relocation when the in-situ visual relocation conditions are met.

Satisfying the in-situ visual relocation conditions may be the first time the smart device is started or the smart device starts the navigation function, or an instruction to re-situ is received.

In implementation, the smart device uses the environmental image collected by the visual sensor to perform in-situ visual relocation according to the image characteristics of the environmental image. The specific method can be the existing method, which will not be described in detail here.

Step 102: If the in-situ visual relocation fails, control the smart device to move according to a preset motion rule to change the environmental image collected by the vision sensor.

The above preset motion rules can be flexibly set according to the scene where the smart device is located. Optionally, different preset motion rules corresponding to different scene settings can be set, and uniform preset motion rules can be set for different scenes. The specific preset motion rule can be flexibly set, and the manner and time of the motion can be specified. The motion rule that can change the image characteristics of the environmental image collected by the vision sensor can be used as the preset motion rule.

In step 103, the environment image acquired by the vision sensor is acquired again, and visual relocation is performed again according to the acquired environment image again.

In the implementation, it is possible to flexibly specify when to acquire the environmental image collected by the above-mentioned vision sensor again according to the needs. The environmental image collected by the above-mentioned vision sensor may be continuously obtained again at a certain sampling frequency during the movement, or it may be satisfied during the movement When a certain image acquisition condition is obtained, the environment image acquired by the vision sensor is acquired again, and specific image acquisition conditions can be flexibly set.

In this embodiment, after the in-situ visual relocation fails, the above-mentioned smart device is controlled to move according to a preset rule, thereby changing the visual field range or angle range of the visual sensor, changing the characteristics of the environmental image collected by the visual sensor, and improving the relocation success rate.

The embodiments of the present application perform in-situ visual relocation or re-visual relocation according to an environmental image collected by a vision sensor, and an existing manner may be adopted. Specifically, it is possible, but not limited to, to adopt the following manner: before performing visual relocation, control the smart device to move once in most locations corresponding to the pre-relocated environment map in advance, and collect environmental images at various locations during the movement, combining The location of the location point on the environment map establishes the mapping relationship between the characteristics of the collected environment image and the location point, and saves the characteristics of all the collected environment images and their corresponding location points in the positioning database. When the smart device performs the in situ / When the visual relocation is performed again, the characteristics of the environment image collected are compared with the characteristics of the environment image in the positioning database, and matched according to certain rules. If the characteristics of an environment image are successfully matched, the corresponding feature of the environment image is found. Position, the in-situ / re-relocation is successful, otherwise the in-situ / re-relocation fails.

Optionally, in this embodiment, at least one exercise mode is specified in the preset exercise rule. The specific preset exercise rule may be, but is not limited to, any one of the following exercise modes or a combination of multiple exercise modes:

Rotate in place at a set angular velocity;

The displacement is performed at the set displacement speed.

The above-mentioned set angular velocity and set displacement velocity may be suitable for the speed at which the vision sensor collects the image, so that the vision sensor can easily acquire the appropriate image and the angular velocity and displacement velocity of its characteristics, and those skilled in the art can set it according to actual conditions.

When the combination of the above two kinds of exercise modes is included, any one exercise mode can be implemented only once or repeatedly. When repeatedly performed multiple times, the sequence and implementation of multiple exercise modes in the preset exercise rules The number of times is not specifically limited, and those skilled in the art may flexibly set according to actual needs; it may be rotated at a set angular velocity in situ, then displaced at a set displacement velocity, and finally rotated at a set angular velocity in situ. Of course, other combinations and repetitions are also possible.

Optionally, in this embodiment, each of the above two exercise modes is implemented only once, and the preset exercise rule is:

Rotate in situ at a set angular velocity and then move at a set displacement speed; or

Displace at the set displacement speed first, and then rotate in place at the set angular velocity.

As described above, the environment image collected by the vision sensor may be flexibly specified when needed. When the preset motion rule includes only any one of the foregoing movement modes, the environment image acquired by the vision sensor in this embodiment may be adopted as follows. Either way:

Way 1

While controlling the above-mentioned smart device to move according to this kind of movement mode, the environment image acquired by the visual sensor is acquired again.

Specifically, when the preset motion rule includes a motion mode that rotates in situ at a set angular velocity, controlling the motion of the smart device includes controlling the smart device to rotate in situ at a set angular velocity, and obtaining the environmental image collected by the vision sensor again. The specific method is: While the smart device rotates in place at a set angular velocity, the environmental image acquired by the visual sensor is acquired again, and the environmental image acquired by the visual sensor may be acquired again at a certain sampling frequency.

When the preset motion rule includes a motion method of performing a displacement at a set displacement speed, controlling the movement of the smart device includes controlling the smart device to perform displacement at a set displacement speed, and obtaining the environmental image collected by the vision sensor again. The specific method is to control the smart device to While setting the displacement speed for displacement, the environment image acquired by the vision sensor is acquired again, and the environment image acquired by the vision sensor may be acquired again at a certain sampling frequency.

Way 2

At the end of controlling the smart device to perform the movement in this manner, the environment image acquired by the vision sensor is acquired again.

Specifically, when the preset motion rule includes a motion mode that rotates in situ at a set angular velocity, controlling the motion of the smart device includes controlling the smart device to rotate in situ at a set angular velocity, and obtaining the environmental image collected by the vision sensor again. The specific method is: At the end of the in-situ rotation at a set angular velocity, the smart device acquires the environmental image collected by the vision sensor again.

When the preset motion rule includes a motion method of performing a displacement at a set displacement speed, controlling the movement of the smart device includes controlling the smart device to perform displacement at a set displacement speed, and obtaining the environmental image collected by the vision sensor again. The specific method is to control the smart device to When the displacement speed is set to complete the displacement, the environment image acquired by the vision sensor is acquired again.

As described above, the environment image collected by the vision sensor can be flexibly specified when needed again. When the preset motion rule includes only the two movement modes, this embodiment obtains the environment image collected by the vision sensor again, including:

While controlling the smart device to move in any one or more of a variety of ways, the environment image acquired by the vision sensor is acquired again; or

At the end of controlling the smart device to move in any one or more of a variety of ways, the environment image acquired by the vision sensor is acquired again.

In the implementation, according to the number of times each motion mode is implemented in the preset motion rule, it is defined whether to control whether the smart device acquires the environmental image collected by the vision sensor again when performing any one of the motions in each motion mode in the preset motion rule. And if the environment image collected by the vision sensor is acquired again, specifically, whether the environment image collected by the vision sensor is acquired again while moving, or the environment image collected by the vision sensor is acquired again at the end of the movement.

Specifically, the preset motion rule includes both the above-mentioned two movement modes of rotating at a set angular velocity in situ and performing displacement at a set displacement speed, and each movement mode is implemented at least once, including at least the following implementation modes:

method one

While controlling the smart device to perform each movement in each of a plurality of movement modes, the environment image acquired by the vision sensor is acquired again.

Way two

When the smart device is controlled to perform each movement in each of a plurality of movement modes, the environment image acquired by the vision sensor is acquired again.

Way three

When the smart device is controlled to perform each movement in one of the multiple motion modes, the environmental image collected by the vision sensor is acquired again. When the smart device is controlled to perform in each other of the multiple motion modes at the end of each exercise, Obtain the environment image collected by the vision sensor again.

Way four

While controlling the intelligent device to perform part of the movement in one of a variety of movement modes, the environment image acquired by the vision sensor is acquired again. The other times of this movement mode are simultaneously and ended without acquiring the environment image collected by the vision sensor. When the smart device finishes part of the exercise according to the other exercise mode of the multiple exercise modes, the environment image collected by the visual sensor is not acquired again at the same time and at the end of the other exercise modes of the exercise mode.

Way five

While controlling the smart device to move according to one of the multiple movement modes, the environment image collected by the visual sensor is not acquired again at the same time and at the end, and the smart device is controlled to perform at least one movement in the other movement mode among the multiple movement modes. At the end, the environment image acquired by the vision sensor is acquired again.

Way six

While controlling the smart device to move in at least one of the multiple motion modes, the environmental image acquired by the vision sensor is acquired again, and the smart device is controlled to not perform the motion again and at the end when it is in another of the multiple motion modes. Obtain the image of the environment collected by the vision sensor.

The above are only the six optional implementation situations. Specifically, you can combine other implementation situations according to the definition of whether each movement mode is implemented each time and if it is acquired, and other implementation situations are not listed here in detail. .

In implementation, after controlling the smart device to move according to a preset motion rule and re-acquire the environment image acquired by the vision sensor at least once, and perform the visual relocation according to the acquired environmental image again, the method further includes:

Step 104: When it is determined that the end exercise condition is satisfied, the end control of the smart device is performed according to a preset exercise rule.

In implementation, the end exercise condition may be based on the result of repositioning again, or according to the control of the smart device to perform a movement according to a preset movement rule, or a combination of the two may be used for judgment.

Optionally, it may be determined that the end motion condition is satisfied only based on the result of the relocation, which specifically includes: if the visual relocation is successful again, determining that the end motion condition is satisfied, and ending the control of the smart device to move according to a preset motion rule.

Optionally, the smart device may only be controlled according to a preset motion rule. In implementation, the preset motion rule includes a motion end parameter. When the smart device ’s motion according to the preset motion rule meets the motion end parameter, Make sure the end exercise conditions are met.

Optionally, the judgment may be made based on the results of the relocation again, in combination with controlling the smart device to perform the exercise according to the preset motion rule. In implementation, before controlling the smart device to perform the motion according to the preset motion rule before the end-of-motion parameter is satisfied, Determine whether the visual relocation is successful again; if the visual relocation is successful before the end-of-motion parameters are not satisfied, determine that the end-of-motion condition is satisfied; otherwise, when the end-of-motion parameters are satisfied, determine that the end-of-motion condition is satisfied.

The end-of-exercise parameter may be specifically defined according to the exercise modes included in the preset exercise rule and the number and / or time of exercise implementation of each exercise mode. In an optional specific implementation mode, each time each exercise mode is implemented may be defined. End of exercise parameters. When there are multiple exercise modes and each exercise mode is implemented at least once, each exercise is performed in the defined order. After one exercise meets the end of exercise parameters, the next exercise is performed. When all the movements meet the corresponding exercise end When the parameter is set, it is determined that the end condition of the motion is satisfied.

Specifically, for the preset motion rule that includes a motion method of rotating in place at a set angular velocity: the end of motion parameter can be a preset angle, and the preset angle can be 90 degrees, 180 degrees, 270 degrees, etc. Technical personnel can set according to actual needs. The preset motion rules including the end-of-motion parameters are: when controlling the smart device to rotate at the set angular velocity to a preset angle, determine the end-of-motion parameters that meet this type of motion. .

For the scheme in which the preset motion rule includes a motion method of performing displacement at a set displacement speed, the end-of-motion parameter may be a preset displacement, and those skilled in the art may set according to actual needs, which includes a preset motion of the end-of-motion parameter. The rule is: when controlling the intelligent device to move to a preset displacement at a set displacement speed, determine the end-of-movement parameters that meet this type of movement.

In an optional implementation manner, the preset motion rule only includes any one of the motion modes. The environment image collected by the vision sensor is acquired again, and when it is determined that the end motion condition is met, the intelligent device is ended to be controlled according to the preset motion rule. Sports, including:

Control the above-mentioned smart device to rotate at the set angular velocity on the spot, acquire the environmental image collected by the vision sensor while moving, and perform visual repositioning again; control the above-mentioned smart device to rotate at the set angular velocity to the preset angle to stop the movement, that is, if In the process of rotating in place, the visual repositioning of the smart device is successful again at a certain time, and the smart device is still controlled to rotate in place to a preset angle, and then the movement is stopped.

The preset angle can be selected as 360 degrees, that is, the smart device is controlled to rotate once in place.

In another optional implementation manner, the above-mentioned smart device is controlled to perform displacement at a set displacement speed, and the environment image collected by the vision sensor is acquired while moving, and the visual repositioning is performed again. At a moment, when the visual relocation is successful again, the above-mentioned smart device is controlled to stop the current displacement movement; otherwise, the above-mentioned smart device is controlled to stop moving after the preset displacement.

The above are just three alternative implementations. The specific implementation can also be combined into multiple implementations based on the preset rules and the way to obtain the environment image again and determine whether it is possible to end the movement, which will not be detailed here one by one. .

Embodiment two:

The present application provides a method for visual repositioning of a smart device. The method is executed by an electronic device. The electronic device may be the smart device itself. The smart device may be, but is not limited to, a robot. The electronic device may also be an external device that communicates with the smart device. Devices such as servers, etc. Based on the first embodiment described above, in this embodiment, after controlling the above-mentioned smart device to move according to a preset motion rule, the visual relocation fails again, and the method in this embodiment can be used for visual relocation. The specific content is as follows: confirm that the above-mentioned smart device fails to relocate visually again, and output an instruction to move the smart device to a preset position;

If an instruction is received that the smart device has moved to a preset position, the environment image acquired by the vision sensor is acquired again, and visual relocation is performed again according to the acquired environment image again.

As an optional implementation manner, if an indication that the smart device has moved to a preset position is received, the environment image acquired by the vision sensor is acquired again, and when the visual relocation fails according to the acquired environment image again, the current The result of the visual relocation is the above-mentioned preset position.

The preset position is not specifically limited, and may be the charging pile position of the smart device, a fixed position point where the device has been positioned in advance, etc. A person skilled in the art can flexibly set according to the scene in which the smart device is located. The following is a specific implementation case:

When a robot for shopping mall guidance fails after the in-situ visual relocation fails, the visual relocation still fails after the movement is completed according to a preset rule, as shown in FIG. 2, then the following steps are performed:

In step 201, an instruction to move the robot to a preset position is issued.

Instructions are displayed on the robot's display screen, prompting the user to move the robot to a preset position, such as the position of the robot charging post.

You can also use other custom instructions, such as playing voice.

In step 202, when the robot determines that an instruction has been received that the user has moved to a preset position, the robot reacquires the environmental image collected by the visual sensor, and performs visual repositioning again according to the characteristics of the environmental image collected by the newly acquired visual sensor.

In step 203, it is determined whether the visual relocation is successful again. If not, step 204 is performed, and if it is successful, step 205 is performed.

Step 204: Set the current visual relocation result to the preset position.

Step 205: The relocation ends.

In the embodiment of the present application, when the re-acquisition of the robot is controlled and the image relocation is still unsuccessful, the cooperation with the user ensures that the final successful relocation result is accurate.

Embodiment three:

In the embodiments of the present application, the smart device is a robot.

Take the scene where the robot is located at the airport as an example. When the robot at the airport helps passengers to guide the direction, they need to know where they are at the moment. At this time, after receiving the command to direct the direction, the robot enters the visual relocation process. The flowchart is shown in Figure 3, including:

Step 310: Perform in-situ visual repositioning.

Obtain the environmental image collected by the vision sensor in situ, perform in-situ visual relocation according to the characteristics of the collected environmental image, and determine whether the visual relocation is successful. If successful, go to step 380, otherwise, go to step 320.

Step 320: The control robot rotates in place at a set angular velocity, acquires an environmental image collected by the vision sensor while rotating, and performs visual repositioning according to the acquired environmental image.

The robot receives the instruction "rotate 360 degrees at a speed of 30 degrees / second", executes the instruction to rotate in place, acquires the visual sensor image while rotating, and performs visual repositioning again.

It is determined whether the visual relocation is successful again. If successful, step 380 is performed; otherwise, step 330 is performed.

Step 330: The robot is controlled to perform displacement at a set displacement speed, and the environment image acquired by the vision sensor is acquired while the displacement is performed, and visual repositioning is performed again according to the acquired environment image.

The robot executes the instruction of "moving 2m forward at a speed of 0.2m / s" to acquire the vision sensor image while performing the displacement, and performs visual repositioning again, and judges whether the visual repositioning is successful again. If it is successful, the control The robot stops at the current position and executes step 380, otherwise, it is judged whether the robot has been moved straight ahead to a preset displacement of 2m. If the robot has been moved straight ahead to 2m, the displacement movement is stopped, and step 340 is performed, otherwise, the execution step is returned. 330.

Turn on the automatic obstacle avoidance function during the displacement process. If the robot encounters an obstacle during the displacement process, control the robot to execute the instruction "Stop at a distance of 20cm from the obstacle", stop at 20cm from the obstacle, and perform step 340. .

In step 340, the control robot is rotated at the set angular velocity to the preset angle in situ, and the environment image acquired by the vision sensor is acquired after the rotation is completed, and the visual repositioning is performed again according to the acquired environment image.

The robot receives the instruction "rotate 90 degrees at a speed of 30 degrees / second". After executing the instruction 90 degrees in situ, the robot acquires the environment image collected by the vision sensor, and performs visual repositioning based on the obtained environment image.

It is determined whether the visual relocation is successful again. If successful, step 380 is performed; otherwise, step 350 is performed.

In step 350, an instruction is given to move the robot to a preset position for repositioning.

The robot displays the message "Please move me to the charging station and try to locate again" on its display, or announces the request message by voice, and issues a command requesting user intervention; After the user hears the prompt, the robot moves the above-mentioned robot to the preset position. , And click on the "Charging Station" button.

Step 360: The robot determines that it has received the instruction that it has moved to the preset position, and re-opens the visual relocation again, acquires the environmental image collected by the vision sensor, and performs visual relocation according to the acquired environmental image; determines whether the visual relocation is successful. If the visual relocation is successful, step 380 is performed; otherwise, step 370 is performed.

In step 370, the current visual relocation result is set to a preset position, and step 380 is performed.

If the relocation fails after moving to the set position, execute the “Force Positioning” command to force the robot to the current position of the charging pile, and then turn on the guidance function to guide the passengers to a fixed position.

In step 380, the relocation ends, and the robot is controlled to enable the navigation function according to the positioning result.

Embodiment 4:

An embodiment of the present application provides a visual repositioning device applied to a smart device. As shown in FIG. 4, the device includes:

The in-situ relocation unit 401 is configured to obtain an environment image collected by a visual sensor of a smart device, and perform in-situ visual relocation according to the obtained environment image;

A control movement unit 402, configured to control the in-situ visual relocation failure, and control the smart device to move according to a preset motion rule to change an environmental image collected by the vision sensor;

The relocation unit 403 is configured to obtain the environment image collected by the vision sensor again, and perform visual relocation according to the environment image obtained again.

Optionally, the preset motion rule on which the motion control unit 402 is based includes any one of the following motion modes or a combination of any of the following motion modes:

Rotate in place at a set angular velocity;

The displacement is performed at the set displacement speed.

Optionally, the preset motion rule on which the motion control unit 402 is based is:

Optionally, if the preset motion rule includes only one motion mode, the above-mentioned relocation unit 403 is specifically configured to:

While controlling the above-mentioned smart device to move according to this kind of movement mode, acquiring the environment image collected by the above-mentioned vision sensor again; or

When the smart device is controlled to end in the movement mode, the environment image acquired by the vision sensor is acquired again.

Optionally, if the preset motion rule includes a combination of multiple motion modes, the above-mentioned relocation unit 403 is specifically configured to:

While controlling the above-mentioned smart device to move in any one or more of a variety of motion modes, acquiring the environmental image collected by the vision sensor again; or

When the smart device is controlled to end in any one or more of the multiple movement modes, the environment image collected by the vision sensor is acquired again.

Optionally, the control motion unit 402 is further configured to:

When it is determined that the end exercise condition is satisfied, the above-mentioned smart device is ended to be controlled to move according to a preset exercise rule.

Optionally, the preset motion rule includes a motion end parameter, and the control motion unit 402 is specifically configured to:

When the motion of the smart device according to the preset motion rule satisfies the motion end parameter, it is determined that the end motion condition is satisfied.

Before controlling the movement of the above-mentioned smart device according to the preset motion rule to meet the end-of-motion parameter, determine whether the visual relocation is successful again;

If the visual relocation is successful before the end-of-motion parameter is not satisfied, it is determined that the end-of-motion condition is satisfied; otherwise, when the end-of-motion parameter is satisfied, it is determined that the end-of-motion condition is satisfied.

Optionally, the visual relocation device further includes:

An output instruction unit 404, configured to output an instruction to move the smart device to a preset position after the visual relocation fails again;

The preset position relocation unit 405 is configured to receive an instruction that the smart device has moved to a preset position, acquire the environmental image collected by the vision sensor again, and perform visual relocation according to the acquired environmental image again.

Optionally, the visual relocation device further includes:

The forced relocation unit 406 is configured to receive an indication that the smart device has moved to a preset position, obtain the environmental image collected by the vision sensor again, and determine the current visual relocation when the visual relocation fails according to the acquired environmental image again. The positioning result is the above-mentioned preset position.

Embodiment 5:

An electronic device includes a processor and a memory, wherein:

The memory is configured to store an executable program;

The processor is configured to implement any of the visual relocation methods of the first embodiment to the third embodiment when the executable program is executed.

The electronic device may be a smart device itself, and the smart device may be, but is not limited to, a robot; the electronic device may also be an external device that communicates with the smart device, such as a server.

A computer storage medium stores a computer program, and the computer program, when executed, implements any of the visual relocation methods of the first embodiment to the third embodiment.

A computer program product that, when called and executed, implements any of the visual relocation methods of Embodiments 1 to 3 described above.

Those skilled in the art should understand that the embodiments of the present application may be provided as a method, a system, or a computer program product. Therefore, this application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) containing computer-usable program code.

This application is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present application. It should be understood that each process and / or block in the flowcharts and / or block diagrams, and combinations of processes and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, so that the instructions generated by the processor of the computer or other programmable data processing device are used to generate instructions Means for implementing the functions specified in one or more flowcharts and / or one or more blocks of the block diagrams.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to work in a particular manner such that the instructions stored in the computer-readable memory produce a manufactured article including an instruction device, the instructions The device implements the functions specified in one or more flowcharts and / or one or more blocks of the block diagram.

These computer program instructions can also be loaded on a computer or other programmable data processing device, so that a series of steps can be performed on the computer or other programmable device to produce a computer-implemented process, which can be executed on the computer or other programmable device. The instructions provide steps for implementing the functions specified in one or more flowcharts and / or one or more blocks of the block diagrams.

Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. In this way, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application also intends to include these changes and variations.

Claims

A method for visual repositioning of a smart device, which is characterized in that the method includes:

Obtain the environmental image collected by the visual sensor of the smart device, and perform in-situ visual relocation according to the acquired environmental image;

If the in-situ visual relocation fails, controlling the smart device to move according to a preset motion rule to change an environmental image collected by the vision sensor;

The environment image acquired by the vision sensor is acquired again, and the visual relocation is performed again according to the environment image acquired again.
The method according to claim 1, wherein the preset exercise rule comprises any one of the following exercise modes or a combination of any plurality of exercise modes:

Rotate in place at a set angular velocity;

The displacement is performed at the set displacement speed.
The method according to claim 2, wherein the preset motion rule is:

Rotate in situ at a set angular velocity and then move at a set displacement speed; or

Displace at the set displacement speed first, and then rotate in place at the set angular velocity.
The method according to claim 2, wherein if the preset motion rule includes only one motion mode, reacquiring the environment image collected by the vision sensor includes:

While controlling the smart device to move according to this kind of movement mode, acquiring the environment image collected by the vision sensor again; or

When controlling the end of the smart device's movement in this kind of movement mode, the environment image acquired by the vision sensor is acquired again.
The method according to claim 2, wherein if the preset motion rule includes a combination of multiple motion modes, reacquiring the environment image collected by the vision sensor includes:

While controlling the smart device to move in any one or more of a variety of motion modes, acquiring the environmental image collected by the vision sensor again; or

At the end of controlling the smart device to move in any one or more of a variety of ways, the environment image acquired by the vision sensor is acquired again.
The method according to any one of claims 1 to 5, further comprising:

When it is determined that the end exercise condition is satisfied, the smart device ends control to move according to a preset exercise rule.
The method according to claim 6, wherein the preset motion rule includes a motion end parameter, and determining that the end motion condition is satisfied includes:

When the motion of the smart device according to the preset motion rule satisfies the motion end parameter, it is determined that the end motion condition is satisfied.
The method according to claim 6, wherein the preset motion rule includes a motion end parameter, and determining that the end motion condition is satisfied includes:

Determining whether the visual relocation is successful again before controlling the movement of the smart device according to a preset motion rule to meet the end-of-motion parameter;

If the visual relocation is successful before the end-of-motion parameter is not satisfied, it is determined that the end-of-motion condition is satisfied; otherwise, when the end-of-motion parameter is satisfied, it is determined that the end-of-motion condition is satisfied.
The method according to claim 1, wherein if the visual relocation fails again, the method further comprises:

Output an instruction to move the smart device to a preset position;

If an instruction is received that the smart device has moved to the preset position, the environment image acquired by the vision sensor is acquired again, and the visual repositioning is performed again according to the environment image acquired again.
The method according to claim 9, further comprising:

If an indication that the smart device has been moved to the preset position is received, the environment image acquired by the vision sensor is acquired again, and the visual relocation fails according to the acquired environmental image again, and the current visual relocation result is determined to be The preset position.
A visual relocation device for a smart device, characterized in that the device includes:

In-situ relocation unit, which is used to obtain the environment image collected by the visual sensor of the smart device, and perform in-situ visual relocation according to the obtained environment image;

A control motion unit, configured to control the smart device to move according to a preset motion rule to change an environmental image collected by the vision sensor if the in-situ visual relocation fails;

The re-repositioning unit is configured to obtain the environmental image collected by the vision sensor again, and perform visual re-positioning according to the acquired environmental image again.
An electronic device, comprising a processor and a memory, wherein:

The memory is configured to store an executable program;

The processor is configured to implement the visual repositioning method according to any one of claims 1 to 10 when the executable program is executed.
A computer storage medium, characterized in that the computer storage medium stores a computer program, and when the computer program is executed by an electronic device, the electronic device can execute the visual repositioning method according to any one of claims 1 to 10.
A computer program product, characterized in that, when the computer program product is called and executed by an electronic device, the electronic device can execute the visual repositioning method according to any one of claims 1 to 10.