CN111538321A - Control method and device of self-moving equipment, equipment and computer readable storage medium - Google Patents
Control method and device of self-moving equipment, equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN111538321A CN111538321A CN201910059083.4A CN201910059083A CN111538321A CN 111538321 A CN111538321 A CN 111538321A CN 201910059083 A CN201910059083 A CN 201910059083A CN 111538321 A CN111538321 A CN 111538321A
- Authority
- CN
- China
- Prior art keywords
- self
- image information
- geomagnetic
- moving
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 52
- 238000003860 storage Methods 0.000 title claims abstract description 16
- 230000003068 static effect Effects 0.000 claims abstract description 18
- 238000012545 processing Methods 0.000 claims description 18
- 230000008447 perception Effects 0.000 claims description 6
- 238000004891 communication Methods 0.000 claims description 5
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 4
- 238000011426 transformation method Methods 0.000 claims description 4
- 238000010408 sweeping Methods 0.000 description 17
- 238000010586 diagram Methods 0.000 description 15
- 238000004590 computer program Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 238000001514 detection method Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 238000013461 design Methods 0.000 description 4
- 238000013507 mapping Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000004140 cleaning Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000005389 magnetism Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
- G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0259—Control of position or course in two dimensions specially adapted to land vehicles using magnetic or electromagnetic means
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Electromagnetism (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
Abstract
The embodiment of the application provides a control method, a control device, control equipment and a computer readable storage medium of self-moving equipment; the method is applied to a control device, a magnetometer is arranged in the control device, and the method comprises the following steps: acquiring image information, wherein the image information comprises self-moving equipment in a static state; determining a first direction from the mobile device in the image information; acquiring a first geomagnetic direction of a control device through a magnetometer; determining a second geomagnetic direction from the mobile device based on the first direction and the first geomagnetic direction; and controlling the self-moving equipment according to the second geomagnetic direction. According to the embodiment, the first direction of the mobile equipment in the image information is determined, the first geomagnetic direction of the control device is obtained through the magnetometer, the second geomagnetic direction of the mobile equipment is determined based on the first direction and the first geomagnetic direction, and the mobile equipment is controlled according to the second geomagnetic direction, so that the mobile equipment can be effectively guaranteed to work in a vertical or parallel mode in a working environment, and the quality and the efficiency of the work on the mobile equipment are improved.
Description
Technical Field
The present application relates to the field of self-moving devices, and in particular, to a method, an apparatus, a device, and a computer-readable storage medium for controlling a self-moving device.
Background
With the rapid development of science and technology, robots gradually enter the lives of people. For example, some stores are equipped with vending robots that assist in selling goods; the sweeping robot may assist the user in cleaning the floor, etc.
At present, the existing robot generally obtains the position of the robot through a preset sensing module, wherein the sensing module may be a module formed based on a laser direct structuring technology LDS, or may also be a visual navigation positioning system VSLAM. However, after the position information of the robot is acquired through the LDS or the VSLAM, the robot may construct a map based on the acquired position information, and due to the influence of the detection accuracy of the sensing module, a certain line-lifting error is likely to exist between a map coordinate system in the constructed map and a working environment where the map coordinate system is located.
Disclosure of Invention
In view of the above, the present application is directed to a method, an apparatus, a device and a computer readable storage medium for controlling a self-moving device that solve the above problems or at least partially solve the above problems.
Accordingly, in an embodiment of the present application, there is provided a control method of a mobile device, applied to a control apparatus, in which a magnetometer is disposed, the control method including:
acquiring image information, wherein the image information comprises self-moving equipment in a static state;
determining a first direction of the self-moving device in the image information;
acquiring a first geomagnetic direction of the control device through the magnetometer;
determining a second geomagnetic direction of the self-mobile device based on the first direction and the first geomagnetic direction;
and controlling the self-moving equipment according to the second geomagnetic direction.
In another embodiment of the present application, there is provided a control apparatus from a mobile device, including:
the image acquisition module is used for acquiring image information, wherein the image information comprises self-moving equipment in a static state;
a processing module for determining a first direction of the self-moving device in the image information;
the magnetic force acquisition module is used for acquiring a first geomagnetic direction of the control device;
the processing module is further configured to determine a second geomagnetic direction of the mobile device based on the first direction and the first geomagnetic direction;
and the control module is used for controlling the self-moving equipment according to the second geomagnetic direction.
In another embodiment of the present application, there is provided a control device from a mobile device, including: the system comprises a processor, an image collector and a magnetometer, wherein the image collector and the magnetometer are in communication connection with the processor;
the image collector is used for obtaining image information, and the image information comprises self-moving equipment in a static state;
the magnetometer is used for acquiring a first geomagnetic direction of the control device;
the processor is used for acquiring image information through the image collector and determining a first direction of the self-moving equipment in the image information; acquiring a first geomagnetic direction of a control device through the magnetometer; determining a second geomagnetic direction of the self-mobile device based on the first direction and the first geomagnetic direction; and controlling the self-moving equipment according to the second geomagnetic direction.
In yet another embodiment of the present application, a computer readable storage medium storing computer instructions is provided for use in a control device having a magnetometer disposed therein, which when executed by a processor, causes the processor to perform acts comprising:
acquiring image information, wherein the image information comprises self-moving equipment in a static state;
determining a first direction of the self-moving device in the image information;
acquiring a first geomagnetic direction of the control device through the magnetometer;
determining a second geomagnetic direction of the self-mobile device based on the first direction and the first geomagnetic direction;
and controlling the self-moving equipment according to the second geomagnetic direction.
According to the technical scheme provided by the embodiment of the application, the geomagnetic data in the control device comes from the mobile equipment without a built-in geomagnetic meter, so that the problem that the geomagnetic direction of the mobile equipment cannot be acquired is solved; specifically, by acquiring the image information, a first direction of the mobile equipment in the image information is determined, a first geomagnetic direction of the control device is acquired through the magnetometer, a second geomagnetic direction of the mobile equipment is determined based on the first direction and the first geomagnetic direction, and the mobile equipment is controlled according to the second geomagnetic direction, so that the influence of a line-lifting error on the operation of the mobile equipment is effectively avoided, the mobile equipment can operate in a vertical or parallel mode in a working environment, and the quality and the efficiency of the operation of the mobile equipment are improved.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative efforts.
Fig. 1 is a flowchart of a control method of a self-moving device according to an embodiment of the present disclosure;
fig. 2 is a first flowchart for determining a first direction of the self-moving device in the image information according to an embodiment of the present application;
fig. 3 is a second flowchart of determining a first direction of the self-moving device in the image information according to the embodiment of the present application;
FIG. 4 is a flow chart illustrating identifying a first circle of the fuselage and a second circle of the perception module according to an embodiment of the present disclosure;
fig. 5 is a flowchart illustrating determining a second geomagnetic direction of the mobile device based on the first direction and the first geomagnetic direction according to an embodiment of the present application;
fig. 6 is a flowchart of another control method of a self-moving device according to an embodiment of the present application;
fig. 7 is a flowchart of a control method of a self-moving device according to an embodiment of the present application;
fig. 8 is a flowchart illustrating controlling the self-moving device according to the second geomagnetic direction according to an embodiment of the present application;
fig. 9 is a first schematic diagram illustrating a map created by a mobile device according to an embodiment of the present application;
fig. 10 is a first schematic diagram illustrating a map created by a mobile device according to an embodiment of the present application;
fig. 11 is a first functional block diagram implemented by a control apparatus of a self-moving device according to an embodiment of the present disclosure;
fig. 12 is a second schematic block diagram of an implementation of a control apparatus of a self-moving device according to an embodiment of the present application;
fig. 13 is a functional block diagram of an implementation of a control device of a self-moving device according to an embodiment of the present disclosure.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The terminology used in the embodiments of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the examples of the present invention and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, and "a" and "an" generally include at least two, but do not exclude at least one, unless the context clearly dictates otherwise.
It should be understood that the term "and/or" as used herein is merely one type of association that describes an associated object, meaning that three relationships may exist, e.g., a and/or B may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship.
The words "if", as used herein, may be interpreted as "at … …" or "at … …" or "in response to a determination" or "in response to a detection", depending on the context. Similarly, the phrases "if determined" or "if detected (a stated condition or event)" may be interpreted as "when determined" or "in response to a determination" or "when detected (a stated condition or event)" or "in response to a detection (a stated condition or event)", depending on the context.
It is also noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a good or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such good or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a commodity or system that includes the element.
In addition, the sequence of steps in each method embodiment described below is only an example and is not strictly limited.
Fig. 1 is a flowchart of a control method of a self-moving device according to an embodiment of the present disclosure; referring to fig. 1, the present embodiment provides a control method for a mobile device, which is applied to a control apparatus, where a magnetometer is disposed in the control apparatus, where the magnetometer can be used to measure the strength and direction of the earth magnetic field; when the device is applied specifically, the control device can be a device with a magnetometer, such as a mobile phone, a tablet personal computer and an intelligent watch; the control device is used for controlling the self-moving equipment, and the controlled self-moving equipment can comprise any one of the following components: a sweeping robot, a vending robot, a mobile robot, etc. Specifically, the control method may include:
s101: and acquiring image information, wherein the image information comprises the self-moving equipment in a static state.
In the acquisition of the image information, the image information may be acquired by an image acquisition device, wherein the image acquisition device may be integrated in the control device or may be separately provided with respect to the control device, and the separately provided image acquisition device may be communicatively connected to the control device. Specifically, the image capturing device may be a camera, a video camera, or the like having an image capturing function. When image information is acquired by the image acquisition equipment, the self-moving equipment can be in a static state, or when the self-moving equipment is in a moving state, the self-moving equipment can be controlled to pause firstly, so that the self-moving equipment is changed from the moving state to the static state, the image information of the self-moving equipment in the static state can be acquired, the accuracy of identifying the operation direction of the self-moving equipment can be ensured by acquiring the image information of the self-moving equipment in the static state, and the accuracy of controlling the self-moving equipment is further improved.
It can be understood that, after the image information is acquired, in order to improve the accuracy and reliability of the image information identification, the method in this embodiment may further include:
s1010: the image information is converted into a corresponding grayscale image.
Specifically, the image information may be converted into the corresponding grayscale image by using an image processing method in the prior art, as long as the grayscale image corresponding to the image information can be stably and clearly obtained, which is not described herein again.
S102: a first direction in the image information from the mobile device is determined.
After the image information is acquired, a first direction of the self-moving device in the image information can be determined by performing analysis processing on the self-moving device in the image information, wherein the first direction is generally a forward direction of the self-moving device. Specifically, referring to fig. 2, one way in which this may be accomplished is that determining a first direction in the image information from the mobile device may include:
s1021: and determining a target standard picture matched with the image information in a plurality of preset standard pictures, wherein each standard picture comprises the self-moving equipment in a preset direction.
S1022: and determining the direction information corresponding to the self-mobile equipment in the target standard picture as the first direction of the self-mobile equipment in the image information.
For example, the existing standard pictures are as follows: a first standard picture, a second standard picture and a third standard picture; the first standard picture comprises a self-moving device located in the direction of D1, the second standard picture comprises a self-moving device located in the direction of D2, and the third standard picture comprises a self-moving device located in the direction of D3. Then, comparing the acquired image information with the standard pictures one by one, and obtaining: the first standard picture is not matched with the image information, the second standard picture is not matched with the image information, and the third standard picture is matched with the image information, wherein the matching can mean that the similarity between the image information and the standard image information is greater than or equal to a preset threshold value. Thus, the target standard picture matching the image information can be determined to be the third standard picture, and further, the direction D3 of the self-moving device in the third standard picture can be determined to be the first direction of the self-moving device in the image information.
In specific application, for the self-moving device, the self-moving device may include a main body and a sensing module disposed on the main body, and for the main body and the sensing module, the preferable shape structure is: the machine body and the sensing module are both of circular structures; at this time, as for the determination manner of the first direction, referring to fig. 3, determining the first direction in the image information from the mobile device may include:
s1023: a first circle of the body and a second circle of the sensing module are identified.
Specifically, referring to fig. 4, identifying a first circle of the body and a second circle of the sensing module may include:
s10231: and adjusting the self-moving equipment in the image information by using an affine transformation method so that the adjusted self-moving equipment meets a preset standard state.
The preset standard state may refer to a preset state in which the self-moving device is in a correct position, and at this time, as for the shape structure of the self-moving device, the shape structure of the self-moving device in the preset standard state in the image information is the same as the actual shape structure of the self-moving device. By adjusting the self-moving device, the situation that the self-moving device in the image information is different from the actual self-moving device can be avoided, for example, when the control device is used for acquiring the image information, the control device has an image acquisition function, a certain shooting angle exists between the control device and the self-moving device, and the existing shooting angle may cause the difference between the shape structure of the self-moving device in the acquired image information and the actual shape structure of the self-moving device, for example, the self-moving device with a circular actual shape structure may become an ellipse in the image information. At this time, in order to accurately and reliably identify the self-moving device in the image information, the self-moving device in an elliptical shape in the image information may be adjusted into a circular shape by using an affine transformation method, so that the adjusted self-moving device satisfies a preset standard state, thereby ensuring that the shape structure of the self-moving device in the image information is the same as the actual shape structure of the self-moving device.
S10232: and identifying the adjusted image information by using a Hough transform method to obtain a first circle of the machine body and a second circle of the perception module.
S1024: and determining the direction of a connecting line of the circle center of the first circle and the circle center of the second circle as a first direction.
After the first circle and the second circle are identified, the circle center of the first circle and the circle center of the second circle may be obtained, and a connection line between the two circle centers is determined as a first direction from the mobile device, where the first direction is generally a forward direction from the mobile device.
S103: a first geomagnetic direction of the control device is obtained through the magnetometer.
S104: a second geomagnetic direction from the mobile device is determined based on the first direction and the first geomagnetic direction.
After the first direction and the first geomagnetic direction are acquired, coordinate system conversion may be performed on a coordinate system in which the first geomagnetic direction is located and a coordinate system in which the first direction is located, so that a second geomagnetic direction of the mobile device may be determined. Specifically, referring to fig. 5, determining the second geomagnetic direction from the mobile device based on the first geomagnetic direction and the first geomagnetic direction may include:
s1041: and acquiring a direction angle formed between the first geomagnetic direction and the first direction.
S1042: and controlling the first direction to rotate according to the direction angle in a preset plane to obtain a second geomagnetic direction from the mobile equipment.
After acquiring the first geomagnetic direction of the control device and the first direction of the mobile device in the image information, it can be understood that the first geomagnetic direction is located in a geomagnetic coordinate system, and the first direction is located in a preset coordinate system, at this time, the first direction located in the preset coordinate system needs to be converted into the geomagnetic direction located under the geomagnetic coordinate system, and therefore, a direction angle formed between the first geomagnetic direction and the first direction needs to be acquired, and the direction angle is a coordinate system deviation angle existing between the geomagnetic coordinate system and the preset coordinate system. After the direction angle is obtained, namely the angle difference between the first direction in the preset coordinate system and the geomagnetic direction in the geomagnetic coordinate system is obtained, the first direction can be controlled to rotate according to the direction angle in a preset plane, and the direction after the rotation is finished can be determined as the second geomagnetic direction of the mobile device, so that the accuracy and reliability of obtaining the second geomagnetic direction are effectively guaranteed.
S105: and controlling the self-moving equipment according to the second geomagnetic direction.
After the second geomagnetic direction is acquired, the self-moving device can be controlled according to the second geomagnetic direction. In this embodiment, a specific implementation manner of controlling the self-moving device according to the second geomagnetic direction is not limited, and a person skilled in the art may arbitrarily set according to specific design requirements and application requirements, for example: the mobile device may be controlled to directly perform the operation according to the second terrestrial magnetic direction, and the like, and preferably, as shown in fig. 8, the controlling of the mobile device according to the second terrestrial magnetic direction may include:
s1051: and correcting the map pre-established by the mobile equipment according to the second terrestrial magnetism direction.
S1052: and controlling the mobile equipment to work according to the corrected map.
Specifically, in order to facilitate understanding of the process of controlling the self-moving device, a floor sweeping robot is used as an example for explanation, before the floor sweeping robot performs a cleaning task, a map may be established based on a working environment where the floor sweeping robot is located, and generally, the map established by the floor sweeping robot corresponds to the working environment where the floor sweeping robot is located, that is, an X axis of the map may be a longer straight line in the working environment, and a Y axis of the map may be another line perpendicular to the longer straight line in the working environment. When the sweeping robot establishes a map, a certain error is easy to exist between a map coordinate system in the map and a preset coordinate system in a working environment, and the error is a line lifting error. When there is a line-lifting error, the sweeping robot does not work in a vertical or parallel mode in a working environment but works in an inclined mode when working according to the established map, so that not only is the working quality influenced, but also the difficulty in controlling the sweeping robot is improved.
At this moment, in order to avoid the situation that a line-lifting error exists when a map is constructed, and it cannot be guaranteed that the sweeping robot performs operation in a vertical or parallel manner in a working environment, the map can be corrected by using the second ground magnetic direction, the specific implementation manner of correction is not limited in this embodiment, and a skilled person in the art can set the map according to specific design requirements, for example: the map can be established by combining the first direction, the second direction and the surrounding working environment, so that the line-lifting error is avoided when the map is established. Or adjusting a map coordinate system in the established map according to the second terrestrial magnetic direction, so that the map coordinate system in the map is consistent with the second terrestrial magnetic direction, and the working environment where the sweeping robot is located at this time is preferably a working environment facing north and south (for example, a home environment facing north and south). After the map is corrected, the sweeping robot can be controlled to operate according to the corrected map, and the sweeping robot can be ensured to operate in a vertical or parallel mode in a working environment, so that the quality and the efficiency of operation of the sweeping robot are improved.
In the control method for the mobile device provided by the embodiment, the geomagnetic data in the control device comes from the mobile device without a built-in geomagnetic meter, so that the geomagnetic direction of the mobile device cannot be acquired; specifically, by acquiring the image information, a first direction of the mobile equipment in the image information is determined, a first geomagnetic direction of the control device is acquired through the magnetometer, a second geomagnetic direction of the mobile equipment is determined based on the first direction and the first geomagnetic direction, and the mobile equipment is controlled according to the second geomagnetic direction, so that the influence of a line-lifting error on the operation of the mobile equipment is effectively avoided, the mobile equipment can operate in a vertical or parallel mode in a working environment, and the quality and the efficiency of the operation of the mobile equipment are improved.
Fig. 6 is a flowchart of another control method of a self-moving device according to an embodiment of the present application; on the basis of the foregoing embodiment, with continuing reference to fig. 6, in order to further improve the practicability of the method, the method in this embodiment may further include:
s201: a second direction from when the mobile device constructed the map is obtained.
Wherein, when the map is constructed from the mobile device, a second direction from the mobile device can be obtained, and the second direction is used for identifying the direction in the constructed map. The embodiment does not limit the specific implementation manner of the second direction acquisition, and a person skilled in the art may select different implementation manners according to specific design requirements, for example: when the map is built by the self-moving equipment, image acquisition is carried out on the self-moving equipment, and a second direction when the map is built by the self-moving equipment is obtained by analyzing and identifying the acquired image; alternatively, the second direction when the map is built by the mobile device is acquired by a direction detection device which is preset on the mobile device, and the direction detection device can be an acceleration sensor, a gyroscope or the like. Of course, those skilled in the art may also use other methods to obtain the second direction, as long as the accuracy and reliability of obtaining the second direction can be ensured, which is not described herein again.
S202: and determining a third geomagnetic direction of the map established by the mobile device based on the second geomagnetic direction and the second direction.
After the second geomagnetic direction and the second direction are acquired, a third geomagnetic direction of the map may be determined based on the acquired second geomagnetic direction and the second direction. Specifically, the second direction when the map is constructed by the mobile device is in a preset coordinate system, the second geomagnetic direction of the mobile device is in a geomagnetic coordinate system, and the third geomagnetic direction of the map established by the mobile device can be determined by converting the preset coordinate system and the geomagnetic coordinate system, where the specific conversion manner is similar to the above implementation manner of determining the second geomagnetic direction of the mobile device based on the first direction and the first geomagnetic direction, and is not described herein again.
Further, the method in this embodiment may further include:
s301 displays the third geomagnetic direction on the map.
As shown in fig. 9, not only the current location of the self-moving device may be displayed on the constructed map, but also the location of the self-moving device may be marked and displayed in a preset marking manner. And a third geomagnetic direction for identifying the north-south direction can be displayed on the constructed map, and at this time, the third geomagnetic direction is displayed in the upper right corner direction of the map. Of course, the display position of the third geomagnetic direction in this embodiment may also be at other positions, for example: the upper middle position of the map, the upper left corner position of the map, the lower right corner position of the map, the lower left corner position of the map, and the like, and those skilled in the art can arbitrarily set the positions according to specific use habits and requirements, and details are not described herein. For example: when the self-moving equipment is a sweeping robot, the due north and/or due south directions (or compass directions) can be marked in a sweeping map displayed in the control device, so that the established map is closer to the real working environment where the self-moving equipment is located.
The second direction when the map is constructed by the mobile equipment is obtained, and the third geomagnetic direction of the map constructed by the mobile equipment is determined based on the second geomagnetic direction and the second direction, so that the accuracy and reliability of obtaining the geomagnetic direction in the map are effectively guaranteed, the third geomagnetic direction of the map can be displayed in the control device, and the mobile equipment can be conveniently controlled.
Fig. 7 is a flowchart of a control method of a self-moving device according to an embodiment of the present application; on the basis of the foregoing embodiment, with continuing reference to fig. 7, in order to further improve the flexible reliability of the method, the method in this embodiment may further include:
s401: the moving direction of the control device is acquired.
The specific obtaining manner of the moving direction of the control device is similar to the specific obtaining manner of the second direction, and the above statements may be specifically referred to, and are not repeated herein.
S402: and adjusting the displayed map based on the moving direction so that the right front of the control device is consistent with the third geomagnetic direction on the map.
As shown in fig. 9, after the moving direction of the control device is acquired, since the third geomagnetic direction may be displayed in the map, the map may be adjusted based on the acquired moving direction, so that the right front of the control device, which is the first geomagnetic direction of the control device, coincides with the third geomagnetic direction on the map, that is: so that the first geomagnetic direction of the control device is consistent with the third geomagnetic direction inside the map. For example, when the control device is a mobile phone, the user can hold the mobile phone to move freely. When a user holds the mobile phone to move, the map constructed by the mobile device and displayed on the mobile phone can correspondingly rotate according to the moving direction of the mobile phone, so that the map can correspondingly rotate when the mobile phone points to the mobile device. For example: the constructed map is adjusted from the display direction in the figure 9 to the display direction in the figure 10 according to the moving direction of the user, the map is kept to follow the front of the user, the convenience in interaction of a UI (user interface) is highlighted, the user can be helped to be better familiar with the map and be familiar with the operation environment, so that the user can conveniently use and control the mobile device, and more real map using feeling can be provided.
Fig. 11 is a first functional block diagram implemented by a control apparatus of a self-moving device according to an embodiment of the present disclosure; referring to fig. 11, the present embodiment provides a control apparatus for controlling a self-moving device, and specifically, the control apparatus may include:
the image acquisition module 11 is configured to acquire image information, where the image information includes self-moving equipment in a static state;
a processing module 12 for determining a first direction in the image information from the mobile device;
the magnetic force acquisition module 13 is used for acquiring a first geomagnetic direction of the control device;
the processing module 12 is further configured to determine a second geomagnetic direction from the mobile device based on the first direction and the first geomagnetic direction;
and the control module 14 is used for controlling the self-moving equipment according to the second terrestrial magnetic direction.
Optionally, the processing module 12 in this embodiment may also be configured to convert the image information into a corresponding grayscale image after acquiring the image information.
In one implementation, when the processing module 12 determines the first direction of the self-moving device in the image information, the processing module 12 may be configured to perform: determining a target standard picture matched with the image information in a plurality of preset standard pictures, wherein each standard picture comprises a self-moving device in a preset direction; and determining the direction information corresponding to the self-mobile equipment in the target standard picture as the first direction of the self-mobile equipment in the image information.
In addition, the self-moving equipment can comprise a machine body and a sensing module arranged on the machine body, wherein the machine body and the sensing module are both of a circular structure; at this time, when the processing module 12 determines the first direction in the image information from the mobile device, the processing module 12 may be configured to perform: identifying a first circle of the body and a second circle of the sensing module; and determining the direction of a connecting line of the circle center of the first circle and the circle center of the second circle as a first direction.
In particular, when the processing module 12 identifies a first circle of the body and a second circle of the sensing module, the processing module 12 may be configured to perform: adjusting the self-moving equipment in the image information by using an affine transformation method so that the adjusted self-moving equipment meets a preset standard state; and identifying the adjusted image information by using a Hough transform method to obtain a first circle of the machine body and a second circle of the perception module.
Optionally, when the processing module 12 determines the second geomagnetic direction of the mobile device based on the first geomagnetic direction and the first geomagnetic direction, the processing module 12 may be configured to perform: acquiring a direction angle formed between a first geomagnetic direction and a first direction; and controlling the first direction to rotate according to the direction angle in a preset plane to obtain a second geomagnetic direction from the mobile equipment.
Optionally, the processing module 12 in this embodiment may be further configured to perform: obtaining a second direction when the map is constructed by the mobile device; and determining a third geomagnetic direction of the map established by the mobile device based on the second geomagnetic direction and the second direction.
Optionally, the control device in this embodiment may further include:
and a display module 15, configured to display the third geomagnetic direction on a map.
Optionally, the processing module 12 in this embodiment may be further configured to perform: acquiring the moving direction of the control device; the displayed map is adjusted based on the moving direction so that the direction directly in front of the control device coincides with the third geomagnetic direction on the map.
Optionally, when the control module 14 controls the self-moving device according to the second ground magnetic direction, the control module 14 may be configured to perform: correcting a map pre-established by the mobile device according to the second terrestrial magnetic direction; and controlling the mobile equipment to work according to the corrected map.
The apparatus shown in fig. 11 can perform the method of the embodiment shown in fig. 1-10, and the detailed description of this embodiment can refer to the related description of the embodiment shown in fig. 1-10. The implementation process and technical effect of the technical solution refer to the descriptions in the embodiments shown in fig. 1 to fig. 10, and are not described herein again.
In one possible design, the structure of the control apparatus of the self-moving device shown in fig. 11 may be implemented as an electronic device, which may be a mobile phone, a tablet computer, a server, or other devices. As shown in fig. 12, the electronic device may include: a processor 21 and a memory 22. Wherein the memory 22 is used for storing programs that support the electronic device to execute the control method of the self-moving device provided in the embodiments shown in fig. 1-10, and the processor 21 is configured to execute the programs stored in the memory 22.
The program comprises one or more computer instructions which, when executed by the processor 21, are capable of performing the steps of:
the program comprises one or more computer instructions which, when executed by the processor 21, are capable of performing the steps of:
acquiring image information, wherein the image information comprises self-moving equipment in a static state;
determining a first direction from the mobile device in the image information;
acquiring a first geomagnetic direction of a control device through a magnetometer;
determining a second geomagnetic direction from the mobile device based on the first direction and the first geomagnetic direction;
and controlling the self-moving equipment according to the second geomagnetic direction.
Optionally, the processor 21 is further configured to perform all or part of the steps in the embodiments of fig. 1-10 described above.
The electronic device may further include a communication interface 23 for communicating with other devices or a communication network.
In addition, the embodiment of the present invention provides a computer readable storage medium storing computer instructions for storing computer software instructions for an electronic device, wherein the computer instructions, when executed by a processor, cause the processor to execute the program related to the control method of the self-moving device in the method embodiments shown in fig. 1 to 10.
Fig. 13 is a schematic block diagram of an implementation of a control device of a self-moving device according to an embodiment of the present disclosure; referring to fig. 13, the present embodiment provides a control device for a self-moving device, the control device being configured to control the self-moving device, and specifically, the control device includes: a processor 31, an image collector 32 and a magnetometer 33 which are in communication connection with the processor 31;
the image collector 32 is configured to obtain image information, where the image information includes a self-moving device in a static state;
a magnetometer 33 for acquiring a first geomagnetic direction of the control apparatus;
the processor 31 is used for acquiring image information through the image acquirer 32 and determining a first direction of the self-moving device in the image information; acquiring a first geomagnetic direction of the control device through the magnetometer 33; determining a second geomagnetic direction of the self-mobile device based on the first direction and the first geomagnetic direction; and controlling the self-moving equipment according to the second geomagnetic direction.
The apparatus shown in fig. 13 can perform the method of the embodiment shown in fig. 1-10, and the detailed description of this embodiment can refer to the related description of the embodiment shown in fig. 1-10. The implementation process and technical effect of the technical solution refer to the descriptions in the embodiments shown in fig. 1 to fig. 10, and are not described herein again.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by adding a necessary general hardware platform, and of course, can also be implemented by a combination of hardware and software. With this understanding in mind, the above-described aspects and portions of the present technology which contribute substantially or in part to the prior art may be embodied in the form of a computer program product, which may be embodied on one or more computer-usable storage media having computer-usable program code embodied therein, including without limitation disk storage, CD-ROM, optical storage, and the like.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable mapping apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable mapping apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable mapping apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable mapping apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (13)
1. A control method of a self-moving device is applied to a control device, a magnetometer is arranged in the control device, and the control method comprises the following steps:
acquiring image information, wherein the image information comprises self-moving equipment in a static state;
determining a first direction of the self-moving device in the image information;
acquiring a first geomagnetic direction of the control device through the magnetometer;
determining a second geomagnetic direction of the self-mobile device based on the first direction and the first geomagnetic direction;
and controlling the self-moving equipment according to the second geomagnetic direction.
2. The method of claim 1, wherein after acquiring the image information, the method further comprises:
and converting the image information into a corresponding gray-scale image.
3. The method of claim 1, wherein determining the first direction of the self-moving device in the image information comprises:
determining a target standard picture matched with the image information in a plurality of preset standard pictures, wherein each standard picture comprises self-moving equipment in a preset direction;
and determining direction information corresponding to the self-moving equipment in the target standard picture as a first direction of the self-moving equipment in the image information.
4. The method of claim 1, wherein the self-moving device comprises a body and a sensing module disposed on the body, wherein the body and the sensing module are both circular in structure; determining a first direction of the self-moving device in the image information, comprising:
identifying a first circle of the body and a second circle of the perception module;
and determining the direction of a connecting line of the circle center of the first circle and the circle center of the second circle as the first direction.
5. The method of claim 4, wherein identifying a first circle of the fuselage and a second circle of the perception module comprises:
adjusting the self-moving equipment in the image information by using an affine transformation method so that the adjusted self-moving equipment meets a preset standard state;
and identifying the adjusted image information by using a Hough transform method to obtain a first circle of the machine body and a second circle of the perception module.
6. The method of claim 1, wherein determining the second geomagnetic direction of the self-mobile device based on the first direction and the first geomagnetic direction comprises:
acquiring a direction angle formed between the first geomagnetic direction and the first direction;
and controlling the first direction to rotate according to the direction angle in a preset plane to obtain a second terrestrial magnetic direction of the mobile device.
7. The method according to any one of claims 1-6, further comprising:
acquiring a second direction when the map is built by the self-moving equipment;
and determining a third geomagnetic direction of the map established by the self-mobile device based on the second geomagnetic direction and the second direction.
8. The method of claim 7, further comprising:
displaying the third geomagnetic direction on the map.
9. The method of claim 7, further comprising:
acquiring the moving direction of the control device;
and adjusting the displayed map based on the moving direction so that the right front of the control device is consistent with the third geomagnetic direction on the map.
10. The method according to any one of claims 1-6, wherein controlling the self-moving device according to the second terrestrial magnetic direction comprises:
correcting the map pre-established by the self-moving equipment according to the second terrestrial magnetic direction;
and controlling the self-moving equipment to operate according to the corrected map.
11. A control apparatus for a self-moving device, comprising:
the image acquisition module is used for acquiring image information, wherein the image information comprises self-moving equipment in a static state;
a processing module for determining a first direction of the self-moving device in the image information;
the magnetic force acquisition module is used for acquiring a first geomagnetic direction of the control device;
the processing module is further configured to determine a second geomagnetic direction of the mobile device based on the first direction and the first geomagnetic direction;
and the control module is used for controlling the self-moving equipment according to the second geomagnetic direction.
12. A control device from a mobile device, comprising: the system comprises a processor, an image collector and a magnetometer, wherein the image collector and the magnetometer are in communication connection with the processor;
the image collector is used for obtaining image information, and the image information comprises self-moving equipment in a static state;
the magnetometer is used for acquiring a first geomagnetic direction of the control device;
the processor is used for acquiring image information through the image collector and determining a first direction of the self-moving equipment in the image information; acquiring a first geomagnetic direction of a control device through the magnetometer; determining a second geomagnetic direction of the self-mobile device based on the first direction and the first geomagnetic direction; and controlling the self-moving equipment according to the second geomagnetic direction.
13. A computer readable storage medium storing computer instructions for use in a control device having a magnetometer disposed therein, the computer instructions when executed by a processor cause the processor to perform acts comprising:
acquiring image information, wherein the image information comprises self-moving equipment in a static state;
determining a first direction of the self-moving device in the image information;
acquiring a first geomagnetic direction of the control device through the magnetometer;
determining a second geomagnetic direction of the self-mobile device based on the first direction and the first geomagnetic direction;
and controlling the self-moving equipment according to the second geomagnetic direction.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910059083.4A CN111538321B (en) | 2019-01-22 | 2019-01-22 | Control method and device of self-moving equipment, equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910059083.4A CN111538321B (en) | 2019-01-22 | 2019-01-22 | Control method and device of self-moving equipment, equipment and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111538321A true CN111538321A (en) | 2020-08-14 |
| CN111538321B CN111538321B (en) | 2022-05-17 |
Family
ID=71976646
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910059083.4A Active CN111538321B (en) | 2019-01-22 | 2019-01-22 | Control method and device of self-moving equipment, equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111538321B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115290068A (en) * | 2022-07-20 | 2022-11-04 | 尚科宁家(中国)科技有限公司 | Map rectification method based on geomagnetic direction and cleaning robot |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120083961A1 (en) * | 2010-09-30 | 2012-04-05 | Honda Motor Co., Ltd. | Control apparatus for autonomous operating vehicle |
| CN102460069A (en) * | 2009-06-05 | 2012-05-16 | 苹果公司 | Calibration techniques for an electronic compass in a portable device |
| CN107504971A (en) * | 2017-07-05 | 2017-12-22 | 桂林电子科技大学 | A kind of indoor orientation method and system based on PDR and earth magnetism |
| CN107635046A (en) * | 2016-07-19 | 2018-01-26 | 中兴通讯股份有限公司 | A kind of earth's magnetic field method of adjustment and device |
| CN107713915A (en) * | 2017-11-27 | 2018-02-23 | 深圳市沃特沃德股份有限公司 | The localization method of sweeping robot and sweeping robot |
| CN108801275A (en) * | 2018-06-11 | 2018-11-13 | 杭州晶智能科技有限公司 | Indoor mobile robot fingerprint map method for building up based on wireless network and Geomagnetic signal |
-
2019
- 2019-01-22 CN CN201910059083.4A patent/CN111538321B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102460069A (en) * | 2009-06-05 | 2012-05-16 | 苹果公司 | Calibration techniques for an electronic compass in a portable device |
| US20120083961A1 (en) * | 2010-09-30 | 2012-04-05 | Honda Motor Co., Ltd. | Control apparatus for autonomous operating vehicle |
| CN107635046A (en) * | 2016-07-19 | 2018-01-26 | 中兴通讯股份有限公司 | A kind of earth's magnetic field method of adjustment and device |
| CN107504971A (en) * | 2017-07-05 | 2017-12-22 | 桂林电子科技大学 | A kind of indoor orientation method and system based on PDR and earth magnetism |
| CN107713915A (en) * | 2017-11-27 | 2018-02-23 | 深圳市沃特沃德股份有限公司 | The localization method of sweeping robot and sweeping robot |
| CN108801275A (en) * | 2018-06-11 | 2018-11-13 | 杭州晶智能科技有限公司 | Indoor mobile robot fingerprint map method for building up based on wireless network and Geomagnetic signal |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115290068A (en) * | 2022-07-20 | 2022-11-04 | 尚科宁家(中国)科技有限公司 | Map rectification method based on geomagnetic direction and cleaning robot |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111538321B (en) | 2022-05-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20210118238A1 (en) | Registration between actual mobile device position and environmental model | |
| AU2023200411B2 (en) | Automated control of image acquisition via acquisition location determination | |
| US10482659B2 (en) | System and method for superimposing spatially correlated data over live real-world images | |
| US9679414B2 (en) | Federated mobile device positioning | |
| CN110095752B (en) | Positioning method, apparatus, device and medium | |
| JP6273400B2 (en) | High-speed template matching using peripheral information | |
| JP5688793B2 (en) | Hand-held geodetic device, computer-implemented method and computer-readable storage medium for determining the location of a point of interest | |
| EP2458405A1 (en) | Graphics-aided remote position measurement with handheld geodesic device | |
| US11933604B2 (en) | Detection method and apparatus for automatic driving sensor, and electronic device | |
| JP2015535980A (en) | Image processing method used for vision-based positioning, particularly for apparatus | |
| CN110686704A (en) | Pose calibration method, system and medium for laser radar and combined inertial navigation | |
| Tomažič et al. | Fusion of visual odometry and inertial navigation system on a smartphone | |
| CN111680596B (en) | Positioning true value verification method, device, equipment and medium based on deep learning | |
| CN112560769B (en) | Method for detecting obstacle, electronic device, road side device and cloud control platform | |
| CN111127584A (en) | Method and device for establishing visual map, electronic equipment and storage medium | |
| CN112964276A (en) | Online calibration method based on laser and vision fusion | |
| US20210004978A1 (en) | Method for acquiring depth information of target object and movable platform | |
| CN115205128A (en) | Depth camera temperature drift correction method, system, equipment and medium based on structured light | |
| CN111538321B (en) | Control method and device of self-moving equipment, equipment and computer readable storage medium | |
| CN117241142A (en) | Dynamic correction method and device for pitch angle of pan-tilt camera, equipment and storage medium | |
| KR102173286B1 (en) | Electronic device, method, and computer readable medium for correction of spacing coordinate information in mixed reality | |
| US9842402B1 (en) | Detecting foreground regions in panoramic video frames | |
| US9824455B1 (en) | Detecting foreground regions in video frames | |
| KR20150089366A (en) | Indoor location tracking system using earth magnetic field sensor | |
| KR20180106178A (en) | Unmanned aerial vehicle, electronic device and control method thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |