WO2019174484A1 - Charging base identification method and mobile robot - Google Patents

Abstract

A charging base identification method and a mobile robot. The mobile robot comprises an infrared photographing module (112). The method comprises: obtaining an image captured by the infrared photographing module (S201); determining an image region of a charging base to be identified from the image according to a preset charging base image feature (S202); and determining position information of the charging base to be identified according to the determined image region (S203), wherein the charging base to be identified can emit infrared light. The method can improve the success rate of identifying a charging base.

Description

Charging base recognition method and mobile robot

The present application claims priority to Chinese Patent Application No. 201,810,020, 018, filed on March 12, 2018, entitled "A Charge Holder Identification Method and Mobile Robot", the entire contents of which are incorporated herein by reference. in.

Technical field

The present application relates to the field of mobile robot control technologies, and in particular, to a charging stand recognition method and a mobile robot.

Background technique

A mobile robot is a machine that can perform work in accordance with a predetermined program. The mobile robot has a mobile function. Mobile robots are capable of performing many types of tasks during the move. For example, the cleaning robot can clean the road surface during the movement, and the care robot can transport the medical device or the patient during the movement.

In the field of mobile robots, automatic refill technology can improve the intelligence of mobile robots. The automatic recharging process of the mobile robot specifically includes: when the battery power of the mobile robot is lower than the threshold, the mobile robot can move to the charging base according to the program, complete the charging task, and continue to perform the task after the charging is completed. In order to achieve automatic recharging of the mobile robot, the mobile robot is required to recognize the position of the charging stand.

In the related art, the mobile robot can scan the mark of the charging stand by a laser radar mounted on the mobile robot to identify the position of the charging stand. For example, the laser radar on the mobile robot in FIG. 1a can emit a laser scanning line. When the laser scanning line is irradiated to the black and white mark on the charging stand, the laser radar can receive the laser signal reflected by the charging stand and determine according to the reflected laser signal. The location of the charging stand.

The position of the charging stand can be identified in the above manner. However, since the laser scanning line is swept in a plane, when the ground on which the mobile robot is located is tilted, the laser scanning line may not be irradiated onto the mark of the charging stand, and the position of the charging stand may not be recognized. Referring to Fig. 1b, the ground on which the mobile robot is located is tilted downward, and the laser scanning line cannot be irradiated onto the mark of the charging stand, which causes the mobile robot to not recognize the position of the charging stand. Therefore, when the position of the charging stand is recognized in the above manner, the recognition success rate is not high enough.

Summary of the invention

The purpose of the embodiment of the present application is to provide a charging stand identification method and a mobile robot to improve the recognition success rate of the charging stand.

In order to achieve the above object, an embodiment of the present application provides a charging stand identification method, which is applied to a mobile robot, and the mobile robot includes: an infrared camera module; the method includes:

Obtaining an image acquired by the infrared camera module;

Determining an image area of the charging stand to be identified from the image according to a preset charging stand image feature; wherein the charging stand to be identified can emit infrared light;

Determining the location information of the charging stand to be identified according to the determined image area.

Optionally, the step of determining an image area of the charging stand to be identified from the image according to the preset charging stand image feature includes:

The image area of the charging stand to be identified is determined from the image according to a preset charging stand pixel feature and/or a preset charging stand size feature.

Optionally, the step of determining location information of the charging stand to be identified according to the determined image area includes:

Determining an image position of the first preset number of identification points from the image area according to the first preset number of identification points on the to-be-identified charging stand acquired in advance;

Determining the to-be-identified charging according to the spatial position of the first preset number of identification points on the charging stand to be identified and the image position of the first preset number of identification points, and a first preset formula Position information relative to the mobile robot.

Optionally, the spatial position of the first preset number of identification points on the charging stand to be identified and the image position of the first preset number of identification points, and the first preset formula, Determining the location information of the charging stand to be identified relative to the mobile robot includes:

Determining, according to the following first preset formula, the rotation matrix R and the translation matrix t of the charging stand to be identified relative to the mobile robot, and obtaining position information of the charging stand to be identified relative to the mobile robot:

Wherein (X _i , Y _i , Z _i ) represents a spatial position of the i-th identification point of the first preset number of identification points on the charging stand to be identified, (u _i , v _i ) representing an image position of the i-th identification point of the first preset number of identification points, the K represents a preset internal parameter matrix of the infrared camera module, and the argmin represents a minimum projection error function, The n represents the first preset number, and the (X _i ', Y _i ', Z _i ') represents coordinates obtained by coordinate transformation (X _i , Y _i , Z _i ), (u _i ' , v _i ') represents the projected coordinates of (X _i , Y _i , Z _i ) on the imaging plane of the image.

Optionally, the step of determining location information of the charging stand to be identified according to the determined image area includes:

Obtaining depth information of a second preset number of identification points in the image area;

Determining, according to the depth information and the image position of the second preset number of the identification points in the image area, a spatial position of the second preset number of identification points in the image area according to the second preset formula;

And determining location information of the charging stand to be identified according to a spatial position of the second preset number of identification points in the image area.

Optionally, the step of acquiring the depth information of the second preset number of identifier points in the image area includes:

When the infrared camera module further has a depth sensing function, acquiring a depth image corresponding to the image collected by the infrared camera module, and acquiring a second preset number of the image regions from the depth image Identifying depth information of the point; wherein the depth image includes depth information of each identification point; or

When the infrared camera module includes a left camera module and a right camera module, the image includes a first image captured by the left camera module and a second image captured by the right camera module, the image The region is a first image region determined from the first image or a second image region determined from the second image, according to different images of corresponding pixel points in the first image region and the second image region Position, determining depth information of a second preset number of identification points in the image area; or

When the mobile robot further includes an Inertial Measurement Unit (IMU), acquiring a previous image acquired by the infrared camera module before acquiring the image, acquiring a feature according to a preset charging stand image Obtaining a previous image region of the to-be-identified charging cradle determined in an image, acquiring a motion parameter when the IMU is collected from the first position to the second position, according to the motion parameter and the previous image region Determining depth information of a second predetermined number of identification points in the image area; wherein the first position is a spatial position of the mobile robot when the previous image is collected, and the second location is a collection location The spatial position of the mobile robot when the image is described.

Optionally, the second predetermined number of identification points in the image area are determined according to the second preset formula according to the depth information and the image position of the second preset number of identification points in the image area. The steps of the spatial location include:

Determining a spatial location of the second predetermined number of identification points in the image area according to the following second preset formula:

Wherein, (X _i , Y _i , Z _i ) represents a spatial position of an i-th identification point in the image area, and the Z _i is depth information of an i-th identification point in the image area, K represents a preset internal reference matrix of the infrared camera module, and the (u _i , v _i ) represents an image position of the i-th identification point in the image region.

Optionally, the step of determining location information of the charging stand to be identified according to the determined image area includes:

Determining, according to the determined image area, a spatial position and a spatial orientation of the charging stand to be identified with respect to the mobile robot.

Optionally, the mobile robot further includes: an infrared emitter capable of emitting infrared light; and the infrared light emitted by the infrared emitter can be irradiated on the charging stand to be identified.

Optionally, the charging stand to be identified includes a reflective material, and the reflective material is capable of returning the reflected light along the optical path of the incident light.

Optionally, the reflective material is included on each side of the charging stand to be identified, and the pattern features of the reflective materials on each side are different.

In order to achieve the above object, an embodiment of the present application further provides a mobile robot, including: a processor, a memory, and an infrared camera module;

The infrared camera module is configured to collect an image and store the image to the memory;

The processor is configured to acquire the image in the memory, determine an image area of the charging stand to be identified from the image according to a preset charging stand image feature, and determine, according to the determined image area, Determining the location information of the charging cradle; wherein the charging cradle to be identified can emit infrared light.

Optionally, the processor is specifically configured to:

The image area of the charging stand to be identified is determined from the image according to a preset charging stand pixel feature and/or a preset charging stand size feature.

Optionally, the processor is specifically configured to:

Determining an image position of the first preset number of identification points from the image area according to the first preset number of identification points on the to-be-identified charging cradle obtained in advance; according to the first preset quantity Determining a spatial position of the identification point on the charging stand to be identified and an image position of the first preset number of identification points, and a first preset formula determining the charging stand to be identified relative to the mobile robot location information.

Optionally, the processor is specifically configured to:

Determining, according to the following first preset formula, the rotation matrix R and the translation matrix t of the charging stand to be identified relative to the mobile robot, and obtaining position information of the charging stand to be identified relative to the mobile robot:

Wherein (X _i , Y _i , Z _i ) represents a spatial position of the i-th identification point of the first preset number of identification points on the charging stand to be identified, (u _i , v _i ) representing an image position of the i-th identification point of the first preset number of identification points, the K represents a preset internal parameter matrix of the infrared camera module, and the argmin represents a minimum projection error function, The n represents the first preset number, and the (X _i ', Y _i ', Z _i ') represents coordinates obtained by coordinate transformation (X _i , Y _i , Z _i ), (u _i ' , v _i ') represents the projected coordinates of (X _i , Y _i , Z _i ) on the imaging plane of the image.

Optionally, the processor is specifically configured to:

Obtaining depth information of a second preset number of identification points in the image area;

Determining, according to the depth information and the image position of the second preset number of the identification points in the image area, a spatial position of the second preset number of identification points in the image area according to the second preset formula;

And determining location information of the charging stand to be identified according to a spatial position of the second preset number of identification points in the image area.

Optionally, when the infrared camera module further has a depth sensing function, the infrared camera module is further configured to collect a depth image corresponding to the image, and store the image to the memory; the processor, Specifically, the depth image is obtained from the memory, and the depth information of the second preset number of identifier points in the image area is obtained from the depth image, where the depth image includes depths of each identifier point. Information; or,

The processor is specifically configured to: when the infrared camera module includes a left camera module and a right camera module, the image includes the first image captured by the left camera module and the right camera module. a second image, the image area being a first image area determined from the first image or a second image area determined from the second image, according to the first image area and the second image area Determining depth information of a second preset number of identification points in the image area; or

The processor is specifically configured to acquire, when the mobile robot further includes an IMU, a previous image acquired by the infrared camera module before acquiring the image, and acquire an image according to a preset charging stand image from the upper image Obtaining, in an image, a previous image region of the charging stand to be identified, acquiring a motion parameter when the IMU is collected from the first position to the second position, and determining, according to the motion parameter and the previous image region, a depth information of a second predetermined number of identifier points in the image area; the IMU is configured to collect motion parameters when the first position is to the second position; wherein the first position is an acquisition The spatial position of the mobile robot when the previous image is, and the second position is a spatial position of the mobile robot when the image is acquired.

Optionally, the processor is specifically configured to:

Determining a spatial location of the second predetermined number of identification points in the image area according to the following second preset formula:

Wherein, (X _i , Y _i , Z _i ) represents a spatial position of an i-th identification point in the image area, and the Z _i is depth information of an i-th identification point in the image area, K represents a preset internal reference matrix of the infrared camera module, and the (u _i , v _i ) represents an image position of the i-th identification point in the image region.

Optionally, the processor is specifically configured to:

Determining, according to the determined image area, a spatial position and a spatial orientation of the charging stand to be identified with respect to the mobile robot.

Optionally, the mobile robot further includes: an infrared emitter capable of emitting infrared light; and the infrared light emitted by the infrared emitter can be irradiated on the charging stand to be identified.

Optionally, the charging stand to be identified includes a reflective material, and the reflective material is capable of returning the reflected light along the optical path of the incident light.

Optionally, the reflective material is included on each side of the charging stand to be identified, and the pattern features of the reflective materials on each side are different.

In order to achieve the above object, an embodiment of the present application further provides a computer readable storage medium, where the computer readable storage medium stores a computer program, and when the computer program is executed by the processor, the charging stand provided by the embodiment of the present application is implemented. recognition methods. The method includes:

Obtaining an image acquired by the infrared camera module;

Determining an image area of the charging stand to be identified from the image according to a preset charging stand image feature; wherein the charging stand to be identified can emit infrared light;

Determining the location information of the charging stand to be identified according to the determined image area.

In order to achieve the above object, the embodiment of the present application further provides a computer program, which is implemented by the processor to implement the charging stand identification method provided by the embodiment of the present application. The method includes:

Obtaining an image acquired by the infrared camera module;

Determining an image area of the charging stand to be identified from the image according to a preset charging stand image feature; wherein the charging stand to be identified can emit infrared light;

Determining the location information of the charging stand to be identified according to the determined image area.

The charging stand identification method and the mobile robot provided by the embodiment of the present application can obtain an image of the charging stand to be recognized that can be emitted by the infrared camera module, and determine the charging to be identified from the image according to the preset charging stand image feature. The image area of the seat determines the position information of the charging stand to be identified based on the image area. Since the image acquisition range of the infrared camera module is a conical range with the apex of the infrared camera module, when the location of the mobile robot is tilted, the charging stand to be identified can also be within the image acquisition range of the infrared camera module, and further The position information recognition of the charging stand to be identified can be realized. Moreover, the charging stand to be identified can emit infrared light, and the charging stand to be identified in the image collected by the infrared camera module can have a relatively obvious image feature, so when the position information of the charging stand to be identified is identified from the image, Can improve the accuracy of recognition. Of course, implementing any of the products or methods of the present application does not necessarily require that all of the advantages described above be achieved at the same time.

DRAWINGS

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the embodiments or the prior art description will be briefly described below. Obviously, the drawings in the following description are only some of the embodiments of the present application, and those skilled in the art can obtain other drawings according to the drawings without any creative work.

Figure 1a and Figure 1b are several reference diagrams of a mobile robot using a lidar to identify a charging stand;

2 is a schematic flow chart of a charging stand identification method according to an embodiment of the present application;

FIG. 3 is a reference diagram of a marker on a charging stand to be identified according to an embodiment of the present application; FIG.

Figure 3b is a reference diagram of the infrared camera captured containing the marker of Figure 3a;

Figure 3c is a schematic view of the relative position between the mobile robot and the charging stand;

Figure 3d is a schematic diagram of a refill path determined by the mobile robot;

4 is a schematic diagram of a mounting position of an infrared emitter and an infrared camera module according to an embodiment of the present application;

Figure 5a is a schematic structural view of a reflective sticker coated with glass beads;

Figure 5b is a schematic diagram of a principle of crystal reflection of light;

Figure 5c is a schematic diagram of a principle of reflecting light by a prism;

6 is a schematic flow chart of step S203 in FIG. 2;

FIG. 7 is a schematic diagram of locations of respective identification points on a charging stand to be identified according to an embodiment of the present application; FIG.

FIG. 8 is another schematic flowchart of step S203 in FIG. 2;

Figure 9a and Figure 9b are schematic diagrams of the imaging principle and depth calculation of the binocular camera, respectively;

FIG. 9c is a schematic diagram of a principle for calculating depth information in a monocular camera + IMU embodiment;

10a is a schematic diagram of a relative position of an infrared camera module and a charging stand to be identified;

FIG. 10b is a schematic cross-sectional view of a charging stand to be identified according to an embodiment of the present application; FIG.

FIG. 11 is a schematic structural diagram of a mobile robot according to an embodiment of the present application.

detailed description

The technical solutions in the embodiments of the present application are clearly and completely described in the following with reference to the accompanying drawings in the embodiments. It is apparent that the described embodiments are only a part of the embodiments of the present application, and not all of them. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present application without departing from the inventive scope are the scope of the present application.

In order to improve the recognition success rate of the charging stand, the embodiment of the present application provides a charging stand identification method and a mobile robot. The charging stand identification method provided by the embodiment of the present application will be described in detail below through specific embodiments.

FIG. 2 is a schematic flow chart of a charging stand identification method according to an embodiment of the present application. The method embodiment is applied to a mobile robot comprising: an infrared camera module. The infrared camera module can be mounted at the front of the mobile robot or near the front. The infrared camera module can be an infrared camera, an infrared camera, and the like. The infrared camera module is a camera module based on near-infrared light imaging. Generally, light having a wavelength of 0.76 μm to 1.5 μm is referred to as near-infrared light. Generally, an optical sensor in an ordinary camera can sense light in a near-infrared light region and a visible light region, and thus the infrared camera module can be obtained by adding a filter that blocks visible light to an ordinary camera.

The charging stand identification method provided in this embodiment includes the following steps S201 to S203.

Step S201: Acquire an image acquired by the infrared camera module.

The infrared camera module can collect images according to a preset period, and the mobile robot can acquire images collected by the infrared camera module according to a preset period. When the infrared camera module is located at the front of the mobile robot, the mobile robot can acquire an image of the moving direction of the mobile robot collected by the infrared camera module.

The image captured by the infrared camera module can be understood as an image of an environmental object surrounding the mobile robot. Since the mobile robot is movable, the mobile robot may be farther away from the charging stand to be identified, or may be closer; the charging stand to be identified may be within the image capturing range of the infrared camera module, or may not be in the infrared camera module. The image is captured within range. Therefore, the image may include the charging stand to be identified, or may not include the charging stand to be identified; when the image contains the charging stand to be identified, the charging stand to be identified may be located at any position in the image.

In this embodiment, the infrared camera module can be a monocular camera or a binocular camera.

Step S202: Determine an image area of the charging stand to be identified from the image according to the preset charging stand image feature.

Wherein, the charging stand to be identified can emit infrared light. When the charging stand to be recognized emits infrared light, the image area of the image to be recognized in the image is displayed as a highlighted area, which makes the feature of the charging stand to be identified in the image more obvious and easier to be recognized.

The area where the infrared light is to be emitted on the charging stand may be the entire area of the charging stand to be identified, or may be the mark area on the charging stand to be identified. By setting the marker to a specific shape or pattern, the charging stand can be made recognizable. For example, the marker may be a rectangular block of four shape rules arranged in a predetermined order or the like.

In one embodiment, the infrared light emitted by the charging stand to be identified is: infrared light emitted by the charging stand itself to be identified. For example, the inside of the charging stand to be identified may have an infrared light emitter, so that the charging stand to be identified emits infrared light outward, so that the charging stand to be identified has obvious highlight features in the image. In another embodiment, the infrared light emitted by the charging stand to be identified is: infrared light emitted by other external devices reflected by the charging stand. For example, the charging stand to be identified can reflect the infrared light emitted by the external infrared light emitter, so that the charging stand to be identified can emit infrared light. In order to enable the charging stand to be recognized to reflect the infrared light, it can also present a highlight feature in the image, which can be realized by setting some special materials on the charging stand to be identified.

When the charging stand to be recognized can emit infrared light, the image area of the image to be recognized in the charging stand is a highlighted area. The shape of the marker on the charging stand may be a preset shape. For example, FIG. 3a is a reference diagram of a marker on a charging stand to be identified according to an embodiment of the present application, wherein the black to-be-identified charging stand has four white rectangular markers, and the markers can emit infrared light. FIG. 3b is a reference diagram of the marker included in FIG. 3a collected by the infrared camera module, wherein four rectangular highlight areas are visible as markers on the charging stand to be identified.

If the shape of the marker on the charging stand is a preset shape, the area of the charging stand in the image is a highlighted area of the preset shape. In order to identify the image area of the charging stand to be identified, the preset charging stand image feature may be the preset shape.

When the marker of the charging stand to be identified is detected from the image, the area of the outer frame of each marker can be used as the image area of the charging stand to be identified. For example, when the charging stand to be identified includes four rectangular markers, the determined image area includes image areas of four rectangular markers.

In an embodiment, the step S202 may specifically include: determining, according to a preset charging stand image feature, whether there is a charging stand to be identified in the image; if present, determining an image area of the charging stand to be identified from the image; If it does not exist, it can be left unprocessed or the image can be discarded.

The charging stand image feature may include a charging stand pixel feature. For example, the charging stand pixel feature may include: the pixel value is greater than a preset pixel threshold. The charging stand image feature may also include a charging stand size feature. For example, the charging stand pixel feature may include at least one of an aspect ratio feature, a length range feature, and a width range feature of the image area. Wherein, the preset pixel threshold may be determined in advance according to the pixel value of the highlight area portion of the sample charging stand in the image, for example, the preset pixel threshold may be 200 or other values.

The above pixel feature is a feature determined based on the size of the pixel value. The cradle pixel feature is a feature determined based on the size of the pixel value of the cradle in the image.

Correspondingly, in this step S202, the image area of the charging stand to be identified is determined from the image according to the preset charging stand pixel feature and/or the preset charging stand size feature. Specifically, the image may be scanned to detect an area having a charging stand pixel feature and/or a charging stand size feature, and the area is used as an image area of the charging stand to be identified.

Step S203: Determine location information of the charging stand to be identified according to the determined image area.

In an embodiment, the location information of the charging stand to be identified may include: a spatial location and a spatial orientation of the charging stand to be identified. Spatial position, ie space coordinates. The spatial orientation can be represented by the normal vector of the plane in which the charging dock is to be identified. The spatial position and spatial orientation of the charging stand to be identified may be the spatial position and spatial orientation of the charging stand to be identified relative to the mobile robot, that is, the spatial position and spatial orientation of the charging stand in the mobile robot coordinate system to be identified. The mobile robot coordinate system can be understood as the coordinate system whose coordinate origin is located on the mobile robot. For example, the coordinate origin of the mobile robot coordinate system is the center position of the mobile robot. At this time, in this step S203, specifically, according to the determined image area, the spatial position and spatial orientation of the charging stand to be identified relative to the mobile robot are determined.

In an embodiment, the step S203 may include: determining a spatial position of each pixel in the image region relative to the mobile robot, and using an average value of spatial positions of the respective pixel points in the image region as the charging stand to be identified relative to the mobile The spatial position of the robot; the target plane is determined according to the spatial position of each pixel in the image region, and the normal vector of the target plane is taken as the spatial orientation of the charging stand to be identified relative to the mobile robot.

The spatial position of each pixel in the image region relative to the mobile robot is the position of each pixel in the image region in the mobile robot coordinate system.

After determining the spatial position and spatial orientation of the charging stand to be identified relative to the mobile robot, the mobile robot may further determine a refill path from the mobile robot to the charging stand to be identified according to the determined spatial position and spatial orientation, by controlling the mobile robot. The driving component drives the mobile robot to move along the recharging path to the charging stand to be identified. The above refilling path can enable the mobile robot to treat the charging stand when the charging stand is to be recognized, and realize automatic charging of the mobile robot.

For example, Figure 3c shows the spatial position and spatial orientation of the charging stand relative to the mobile robot, wherein the front of the mobile robot is not facing the charging stand. Depending on the spatial position and spatial orientation, a refill path from the mobile robot to the charging cradle can be planned. Figure 3d shows a schematic diagram of a refill path from the mobile robot to the charging cradle. When the mobile robot moves from point A to point B along the recharging path, the mobile robot can rotate itself to face the charging cradle. And continue straight from point B to the charging stand for charging. 3c and 3d above are both plan views. In this example, the charging component on the mobile robot is located at the front of the mobile robot. When the mobile robot obtains the spatial orientation of the charging stand relative to the mobile robot, the recharging path can be better planned, so that the mobile robot faces the charging stand when reaching the charging stand, thereby completing the charging operation more accurately.

Wherein, the mobile robot is treating the charging stand, that is, the line connecting the front part of the mobile robot and the charging stand to be identified is perpendicular to the target plane.

It can be seen from the above that the image obtained by the infrared camera module and containing the infrared charging light to be recognized can be obtained, and the image area of the charging stand to be identified is determined from the image according to the preset charging seat image feature, according to The image area determines location information of the charging stand to be identified. Since the image acquisition range of the infrared camera module is a conical range with the apex of the infrared camera module, when the location of the mobile robot is tilted, the charging stand to be identified can also be within the image acquisition range of the infrared camera module, and further The position information recognition of the charging stand to be identified can be realized. Moreover, the charging stand to be identified can emit infrared light, and the charging stand to be identified in the image collected by the infrared camera module can have a relatively obvious image feature, so when the position information of the charging stand to be identified is identified from the image, Can improve the accuracy of recognition.

In order to improve the imaging quality, in another embodiment of the present application, in the embodiment shown in FIG. 2, the mobile robot may further include: an infrared emitter capable of emitting infrared light. The infrared light emitted by the infrared emitter can be illuminated on the charging stand to be identified. The infrared emitter can be mounted close to the infrared camera module.

In an embodiment of the present application, the marker on the charging stand to be identified may be a mirror material. When both the infrared emitter and the infrared camera module are to identify the charging stand, the charging stand to be identified may be emitted by the infrared emitter. The infrared light is reflected into the lens of the infrared camera module, and the reflection is specular reflection. In this way, the infrared camera module can collect an image containing the highlighted charging stand to be recognized, and the charging stand to be identified is more easily recognized.

The infrared emitter acts as a fill light to provide illumination to the surrounding environment of the mobile robot. When the charging stand to be recognized is illuminated by infrared light, the charging stand to be identified in the image captured by the infrared camera module can be made clearer.

In another embodiment of the present application, in order to make the mobile robot at any position relative to the charging stand to be identified, the charging stand to be identified can display a highlight feature in the image, and the reflective seat can be used for reflection. The material, for example, the charging stand to be identified is covered with a reflective material, or the marker of the charging stand to be identified uses a reflective material. Among them, the reflective material can return the reflected light along the optical path of the incident light.

For example, FIG. 4 is a schematic diagram of a mounting position of an infrared emitter and an infrared camera module in a mobile robot according to an embodiment of the present application. When the infrared emitter emits infrared light, an object in the field of view (FOV) of the infrared emitter reflects a certain amount of infrared light into the lens of the infrared camera module, thereby generating an infrared image in the lens. When the charging stand to be identified appears in the overlapping area of the infrared emitter FOV and the infrared camera lens FOV, since the reflective material can almost completely reflect the infrared light, a bright area appears in the image, that is, the charging stand to be identified It can be highlighted in the image, and the charging stand to be identified can have higher recognition in the image than the object around the charging stand to be identified.

In this embodiment, the reflective material can be a reflective sticker, and the reflective sticker is a high reflectivity material. The surface of the reflective material is coated with a high refractive index layer. Wherein, the high refractive index layer may comprise high refractive index glass beads, crystals or prisms. These high reflectivity layers are capable of reflecting light from different directions and reflecting the light back in the direction of incidence. For example, Figure 5a is a schematic view of a structure of a reflective sticker coated with high refractive index glass beads. The reflective sticker comprises a surface resin layer, a high refractive index glass bead, an adhesive layer, a reflective layer and a sticker layer, and the incident light is projected on the high refractive index glass microsphere through the surface resin layer, and after being reflected by the reflective layer, It is reflected back from the surface resin layer. The glass beads in the reflective sticker of Figure 5a can also be crystals or lenses. Figure 5b is a schematic diagram of the crystal reflecting light, and Figure 5c is a schematic diagram of the prism reflecting light. It can be seen that after the incident light is reflected and refracted after being projected onto the crystal or the lens, the light can be emitted in the opposite direction of the incident light.

When the infrared light is illuminated on the reflective sticker of an object, the infrared light will almost completely return, causing a highlight area to appear in the pixel area of the object in the image. A reflective sticker having a special shape is pasted on the charging stand to be identified, and the special shape can be used to identify the charging stand.

In this embodiment, no matter where the mobile robot is located, the charging stand to be recognized can reflect the infrared light emitted by the infrared emitter of the mobile robot back to the infrared camera module, so that the charging stand to be recognized can present a high-brightness pattern in the image. .

In another embodiment of the present application, in the embodiment shown in FIG. 2, in step S203, the step of determining the location information of the charging stand to be identified according to the determined image area may be performed according to the flow diagram shown in FIG. The following steps S203a and S203b may be included.

Step S203a: Determine an image position of the first preset number of identification points from the image area according to the first preset number of identification points on the charging stand to be identified that are acquired in advance.

The first preset quantity may be a preset quantity value. The first predetermined number may be a value greater than three. The first preset number of identification points are preset points, and the relative positions between the points are fixed.

Obtaining, in advance, the first preset number of identification points on the charging stand to be identified, which may be understood as: pre-acquiring the spatial position of the first preset number of identification points on the charging stand to be identified, that is, the first preset number of identification points The spatial position on the charging stand to be identified. The spatial position of the first preset number of identification points may be a space coordinate in a coordinate system established by using one of the first preset number of identification points as a coordinate origin, or may be any space The space coordinate in the coordinate system established when the fixed point is the origin of the coordinate.

Determining the image position of the first preset number of identification points may be understood as determining the coordinates of the first preset number of identification points in the image. Specifically, for each identification point, when determining the image position of the identification point, the pixel point corresponding to the identification point may be determined from the image area, and the image position of the identification point is determined according to the coordinates of the pixel point. The pixel corresponding to the identifier point may be one or more. When the pixel point corresponding to the identifier point is one, the image coordinate of the pixel point may be directly determined as the image position of the identifier point; when the pixel point corresponding to the identifier point is multiple, the plurality of pixel points may be The average of the image coordinates is determined as the image position of the marker point. Among them, the image coordinates are coordinates in the image coordinate system.

For example, referring to FIG. 7 , the four rectangular frames are the markers on the charging stand to be identified, and the center points of the four rectangular frames are used as preset marking points, and the first preset number is 4. Taking the marker point in the lower left corner as the coordinate origin and establishing the coordinate system shown in Fig. 7, the space coordinates of each marker point are determined in the counterclockwise direction to be (0, 0, 0), (L ₂ , 0, 0), ( L ₂ , -L ₁ , 0) and (0, -L ₁ , 0). According to the above content, it can be determined that the central pixel point of each rectangular area in the image area is the first preset number of identification points in the image area, and the image coordinates of the identification points are (u ₁ , v ₁ ), respectively (u ₂ , v ₂ ), (u ₃ , v ₃ ) and (u ₄ , v ₄ ).

In this embodiment, the infrared camera module can be a monocular camera.

Step S203b: determining, according to the spatial position of the first preset number of identification points on the charging stand to be identified and the image position of the first preset number of identification points, and the first preset formula, determining the charging stand to be identified relative to the mobile robot Location information.

In an embodiment, the step S203b may specifically include: determining, according to the following first preset formula (1), a rotation matrix R and a translation matrix t of the charging stand to be identified relative to the mobile robot, the rotation matrix R and the translation matrix t That is, the position information of the charging stand to be identified relative to the mobile robot:

Wherein, (X _i , Y _i , Z _i ) represents a spatial position of the i-th identification point of the first preset number of identification points on the charging stand to be identified, and (u _i , v _i ) represents the first preset quantity. The image position of the i-th identification point in the identification point, K represents the internal parameter matrix of the preset infrared camera module, argmin represents the minimum projection error function, and n is the first preset number, (X _i ', Y _i ' , Z _i ') represents the coordinates obtained by coordinate transformation (X _i , Y _i , Z _i ), and (u _i ', v _i ') represents (X _i , Y _i , Z _i ) on the imaging plane of the image The image coordinates of the projection.

K can be

Where f _u and f _v are the equivalent focal lengths of the lens of the infrared camera module in the u-axis and the v-axis direction of the image, respectively, and c _u and c _v are the image coordinates of the projection center of the optical axis of the lens in the image. .

By solving

The optimized solution can obtain the above rotation matrix R and translation matrix t.

For example, when n is 4, the spatial coordinates of each identification point are (0, 0, 0), (L ₂ , 0, 0), (L ₂ , -L ₁ , 0) and (0, -L ₁ , 0); the image coordinates of each marker point are (u ₁ , v ₁ ), (u ₂ , v ₂ ), (u ₃ , v ₃ ) and (u ₄ , v ₄ ), respectively, according to these known quantities, And the first preset formula described above, the rotation matrix R and the translation matrix t can be obtained.

In another embodiment, the spatial orientation and spatial position of the charging stand to be identified relative to the mobile robot may be determined according to the rotation matrix R and the translation matrix t as the position information of the charging stand to be identified.

In summary, in this embodiment, the spatial position of the identification point on the charging stand to be identified and the image position of the identification point in the image area are determined in advance, and the spatial position of the charging stand to be identified relative to the mobile robot can be determined. The position information of the charging stand to be identified is accurately determined.

In another embodiment of the present application, in the embodiment shown in FIG. 2, in step S203, the step of determining the location information of the charging stand to be identified according to the determined image area may be performed according to the flow diagram shown in FIG. Specifically, the following steps S203A, S203B, and S203C may be included.

Step S203A: Acquire depth information of a second preset number of identification points in the image area.

The second preset quantity may be a preset quantity value, and may be a quantity value greater than 3. The second preset number may be the same as the first preset number or may be different.

The second preset number of identification points in the image area may be pixels determined according to a preset rule, or may be preset pixels. The foregoing preset rule may be randomly selecting a pixel point, or may be selecting a pixel point at a preset position. For example, when the charging stand to be identified includes four rectangular markers, the central pixel of the rectangular region corresponding to each rectangular marker in the image region may be used as the second predetermined number of identification points. Of course, it is also possible to select a plurality of identification points from the image area. A larger number of identification points can improve the accuracy of the determined position of the charging stand to be identified.

The depth information may include at least one of a distance value, a distance error range, and the like. The depth information can be understood as the distance between the point on the object corresponding to each identifier point and the infrared camera module, that is, the distance between the point on the object corresponding to each identifier point and the mobile robot.

Step S203B: Determine, according to the depth information and the image position of the second preset number of identification points in the image region, the spatial position of the second preset number of identification points in the image region according to the second preset formula.

The spatial position of the second preset number of identification points in the image area can be understood as the spatial coordinate of the point on the object corresponding to the second preset number of identification points in the image area.

In an embodiment, the step S203B may specifically include:

The spatial position of the second preset number of identification points in the image area is determined according to the following second preset formula (2):

Wherein, (X _i , Y _i , Z _i ) represents the spatial position of the i-th identification point in the image region, i may take a value between 1 and m, and m represents a second preset number. The origin of the coordinate system corresponding to (X _i , Y _i , Z _i ) can be established on the mobile robot, that is, the origin of the coordinate system corresponding to (X _i , Y _i , Z _i ) can be the center position of the mobile robot. Z _i represents the depth information of the i-th identification point in the image area, (X _i , Y _i ) represents the plane coordinate of the i-th identification point in the image area, and K represents the internal reference matrix of the preset infrared camera module. (u _i , v _i ) represents the image position of the i-th identification point in the image area.

K can be

Where f _u and f _v are the equivalent focal lengths of the lens of the infrared camera module in the u-axis and the v-axis direction of the image, respectively, and c _u and c _v are the image coordinates of the projection center of the optical axis of the lens in the image. .

For example, the image positions of the second predetermined number of identification points are {(u ₁ , v ₁ ), (u ₂ , v ₂ ), . . . , (u _m , v _m )}, respectively, the depth of each identification point. The information may be {Z ₁ , Z ₂ , . . . , Z _m }, respectively, where m is a second preset number, and according to the known quantities, and the second preset formula described above, a second image area may be obtained. The spatial coordinates of a preset number of identification points.

Step S203C: Determine location information of the charging stand to be identified according to the spatial position of the second preset number of identification points in the image area.

In an embodiment, the step S203C may specifically include: using an average value of a spatial position of the second preset number of identification points in the image area as a spatial position of the charging station to be identified relative to the mobile robot; determining the image area The plane where the spatial position corresponding to the preset number of identification points is located, and the normal vector of the plane is used as the spatial orientation of the charging stand to be identified relative to the mobile robot.

In summary, the present embodiment can determine the spatial position of the identification point according to the depth information of the pixel in the image area, and determine the position information of the charging stand to be identified according to the spatial position, thereby improving the accuracy of determining the position information of the charging stand to be identified.

In the embodiment of the present application, the depth information in step S203A of the foregoing embodiment may be obtained by using various embodiments.

In an embodiment, when the infrared camera module further has a depth sensing function, the step of acquiring the depth information of the second preset number of the identifier points in the image region may include:

Obtaining a depth image corresponding to the image acquired by the infrared camera module, and acquiring depth information of the second preset number of identifier points in the image region from the depth image. The depth image includes depth information of each identification point.

In this embodiment, the infrared camera module may include a depth sensor and an infrared emitter. The depth sensor can be a Time Of Flight (TOF) sensor. The TOF sensor can calculate the depth information between the object and the lens by using the time difference between the infrared light emitted by the infrared emitter and the infrared light received by the infrared camera module lens to generate a depth image. When generating the depth image, the TOF sensor can also modulate the infrared light to a certain frequency to obtain modulated light, emit the modulated light, and calculate the object to the lens by calculating the phase difference between the received modulated light and the emitted modulated light. The depth value between.

In this embodiment, the infrared camera module also obtains a depth image corresponding to the image when the image is acquired, and the depth image includes depth information of each pixel in the image, and the identifier point is determined from the pixel of the image. The depth image thus includes depth information for each of the identified points.

In another embodiment, when the infrared camera module is an infrared binocular camera, that is, when the infrared camera module includes a left camera module and a right camera module, the image includes a first image captured by the left camera module, and a second image acquired by the right camera module, wherein the image area is a first image area determined from the first image or a second image area determined from the second image. The first image area is an image area of the charging stand to be identified determined from the first image according to the preset charging stand image feature, and the second image area is to be identified from the second image according to the preset charging stand image feature. The image area of the charging stand. Step S203A: The step of acquiring the depth information of the second preset number of identifier points in the image area may include:

Determining depth information of a second predetermined number of identification points in the image area according to different image positions of the corresponding identification points in the first image area and the second image area.

The corresponding identifier points in the first image area and the second image area are: the first identifier point in the first image area and the second identifier point in the second image area correspond to the same point in the space; that is, The same point in space is the imaged point in the first image area and the second image area, respectively. The first identification point is any identification point in the first image area, and the second identification point is any identification point in the second image area. Here, the first identification point and the second identification point are taken as an example for illustration, and are not limiting.

For example, Figure 9a is an imaging schematic of a binocular camera. The infrared binocular camera includes a left-eye camera and a right-eye camera, and the line between the center points of the two cameras is a baseline. Point P is the left eye pixel in the left eye camera and the right eye pixel in the right eye camera. The left eye pixel and the right eye pixel are corresponding identification points.

In an embodiment, the depth information of the second preset number of identification points in the image area may be determined according to the following third preset formula (3):

Where Z is the depth information of an identification point in the image area. Here, the identification point S _{1 is taken} as an example. f is the focal length of the left camera module lens or the focal length of the right camera module lens. In one example, the left camera module or the right camera module has the same focal length. b is a baseline length between the center of the lens and the right lens center of the camera module of the left camera module, u _L and u _R are the identifier identifying the corresponding point S and point _{S. 1. 1} identifies image coordinate point S _2. For example, when the image area is the first image area, the identification point S ₁ is a pixel point in the first image area, the corresponding identification point S ₂ is a pixel point in the second image area; and when the image area is the second image area The identification point S ₁ is a pixel point in the second image area, and the corresponding identification point S ₂ is a pixel point in the first image area.

For each of the second preset number of identification points in the image area, the third preset formula may be used to determine the depth information of the identification point. The center of the lens is the center of the aperture.

Referring to Figure 9b, Figure 9b is a schematic diagram of the depth calculation of the binocular camera. In Figure 9b, _OL is the lens center of the left camera module, O _R is the lens center of the right camera module, and b is the baseline length between the lens center of the left camera module and the lens center of the right camera module. P _L is the imaging pixel point (ie, the identification point) of the P point on the imaging plane of the left camera module, and P _R is the imaging pixel point (ie, the corresponding identification point) of the P point on the imaging plane of the right camera module. Z is the distance between the P point and the baseline, that is, the depth information of the identification point. u _L is the coordinate of P _L and u _R is the coordinate of P _R .

In another embodiment, when the mobile robot further includes an Inertial Measurement Unit (IMU), the step of acquiring the depth information of the second preset number of identification points in the image region may include the following steps. 1 to 4.

Step 1: Obtain the previous image acquired by the infrared camera module before acquiring the above image.

The infrared camera module can collect images according to a preset period. The previous image is an image captured by the infrared camera module at a previous acquisition time before the current acquisition time, and the last acquisition time is an acquisition time adjacent to the current collection time. That is, the last acquisition time is the time at which the image is acquired by the previous infrared camera module. The acquisition time is the time at which the infrared camera module collects images. For ease of understanding, the image acquired at the above acquisition time is image A, that is, the previous image is image A, and the image acquired at the current acquisition time is image B as an example.

When the infrared camera module collects the previous image A, the position of the mobile robot is the first position, and the position of the mobile robot when the infrared camera module acquires the image B is the second position. When the infrared camera module collects images during the movement of the mobile robot, the first position and the second position are different.

In this embodiment, the infrared camera module can be a monocular camera.

Step 2: Acquire a previous image area of the charging stand to be identified determined from the previous image according to the preset charging stand image feature.

For the step, refer to step S202 in the embodiment shown in FIG. 2, and the specific content is not described in detail herein.

Step 3: Acquire the motion parameters collected by the IMU from the first position to the second position.

The above motion parameters may include the amount of rotation and the amount of translation. During the movement of the mobile robot, the IMU can capture the amount of rotation and the amount of translation between any two positions during the movement of the mobile robot.

Among them, the amount of rotation can be understood as a rotation matrix. The amount of translation can be understood as a translation matrix. The first position is the spatial position of the mobile robot when the previous image is acquired, and the second position is the spatial position of the mobile robot when the image is acquired.

Step 4: Determine depth information of the second preset number of identification points in the image area according to the motion parameter and the previous image area.

In an embodiment, the step 4 may specifically include the following embodiments: for the target identification point in the image area, according to the image position of the corresponding identification point of the target identification point in the previous image area and the motion parameter, according to the following The fourth preset formula (4) determines the depth information of the target identification point; wherein the target identification point is any one of the second preset number of identification points in the image area, and the corresponding identification point of the target identification point is Identify points in the previous image area.

s _A x _A =s _B x _B R+t,x _A =K ^-1 p' _A ,x _B =K ^-1 p' _B (4)

Where p' _A = (u _A , v _A , 1) ^T , p' _B = (u _B , v _B , 1) ^T , T is a matrix transpose symbol. (u _B , v _B ) is the image position of the target marker point, (u _A , v _A ) is the image position of the corresponding marker point of the target marker point in the previous image region, and p′ _A is (u _A , v _A The normalized coordinates of p) _B are the normalized coordinates of (u _B , v _B ). R and t are respectively the amount of rotation and the amount of translation in the above-mentioned motion parameters, that is, R is a rotation matrix in the above-described motion parameters, and t is a translation matrix in the above-described motion parameters. K is the internal reference matrix of the preset infrared camera module. s _A is the depth information of the corresponding identification point of the target identification point, and s _B is the depth information of the target identification point. x _A represents the normalized plane coordinate of the corresponding identification point of the target identification point, and x _B represents the normalized plane coordinate of the target identification point. According to the fourth preset formula, depth information of each of the second preset number of identification points in the image area may be obtained.

K can be

Where f _u and f _v are the equivalent focal lengths of the lens of the infrared camera module in the u-axis and the v-axis direction of the image, respectively, and c _u and c _v are the image coordinates of the projection center of the optical axis of the lens in the image. .

The corresponding identifier points of the target identifier point and the target identifier point are: the target identifier point of the image area and the corresponding identifier point of the target identifier point in the previous image area correspond to the same point in the space; that is, the same point in the space Imaging points in the previous image area and image area, respectively.

Referring to FIG. 9c, FIG. 9c is a schematic diagram of a principle for calculating depth information in a monocular camera+IMU embodiment. Where A is the first position and B is the second position. The image point at which the P point is detected at the A position is p _A (u _A , v _A ), and the image point at which the point P is detected when the mobile robot moves to the B position is p _B (u _B , v _B ), O _A It is the lens center of the infrared camera module at the A position, and O _B is the lens center of the infrared camera module at the B position. According to the schematic diagram of Fig. 9c, the depth information of the imaged point can be acquired by triangulation.

The amount of rotation and the amount of translation from position A to position B are R and t, respectively. According to the camera model, x _A = K ^-1 p' _A , x _B = K ^-1 p' _B is the normalization of the two imaging points. For the plane coordinates, s _A x _A = s _B x _B R+t can be obtained, where s _A and s _B are the depth information of the two image points, respectively. The above s _A and s _B are unknowns. To solve s _A x _A = s _B x _B R+t, we can simultaneously multiply an antisymmetric matrix of x _A on both sides of s _A x _A = s _B x _B R+t

due to

Therefore, it is available

By solving

Equation, the depth information s _B can be calculated.

After the reflective material is marked on a certain plane of the charging stand, the marker of the charging stand is used to use the reflective material, and when the charging stand to be identified appears in the FOV of the infrared camera module, it is easier to obtain the mobile robot. Relative positional relationship with the charging stand to be identified. However, when the mobile robot moves outside the FOV of the infrared camera module during the movement, the marker of the charging stand to be identified does not appear in the FOV of the infrared camera module, and it is difficult for the mobile robot to recognize the charging stand to be identified.

In addition, the marker on a single plane has a corresponding identifiable angle, ie, the FOV is identified, as shown in Figure 10a, the FOV of the cradle to be identified. When the infrared camera module moves to the position shown in FIG. 10a, that is, when the mobile robot moves to the position shown in FIG. 10a during the movement, the infrared camera module moves outside the FOV of the marker to be identified by the charging stand. It is difficult for the robot to recognize the charging stand to be identified.

In order to be able to identify the charging stand to be identified in the above case, in another embodiment of the present application, the reflective side of each of the sides of the charging stand to be identified may include a reflective material, and the pattern features of the reflective material on each side are different. Specifically, when the charging stand is identified, the orientation of the charging stand to be identified may be determined according to the preset pattern features of the respective sides.

In this embodiment, the charging stand to be identified may be divided into a plurality of sides, and the side faces may also be referred to as a cross section, and each side is marked with a different pattern of reflective material. In this way, the FOV of the entire charging stand can be increased. Referring to FIG. 10b, FIG. 10b is a schematic cross-sectional view of the charging stand to be identified in the embodiment. The upper side view of FIG. 10b is a schematic view of each side of the charging stand, including three sides, and the lower side view of FIG. 10b is a top view of the charging stand. Each side of the charging stand is marked with a different pattern using a reflective material. In the case of a charging stand against the wall, the charging stand can be identified within a range of 180 degrees. In the lower side view of Fig. 10b, the trapezoid is shown as a charging stand. The longest side of the trapezoid is the side of the charging stand against the wall, the other side of the trapezoid represents the other side of the charging stand, and the reflective materials of the other sides are marked with different patterns. .

FIG. 11 is a schematic structural diagram of a mobile robot according to an embodiment of the present application. This embodiment corresponds to the method embodiment shown in FIG. 2. The mobile robot includes a processor 110, a memory 111, and an infrared camera module 112. The infrared camera module 112 can be mounted at the front of the mobile robot or near the front. The infrared camera module 112 can be an infrared camera, an infrared camera, or the like. The infrared camera module 112 is a camera module that is imaged according to near-infrared light. Generally, light having a wavelength of 0.76 μm to 1.5 μm is referred to as near-infrared light. Generally, an optical sensor in an ordinary camera can sense light in a near-infrared light region and a visible light region, and thus the infrared camera module 112 can be obtained by adding a filter that blocks visible light on an ordinary camera.

The infrared camera module 112 is configured to collect images and store the images in the memory 111;

The processor 110 is configured to acquire an image in the memory 111, determine an image area of the charging stand to be identified from the image according to the preset charging stand image feature, and determine position information of the charging stand to be identified according to the determined image area; The charging stand to be identified can emit infrared light.

The above memory 111 may include a random access memory (RAM), and may also include a non-volatile memory (NVM), such as at least one disk storage. Optionally, the memory 111 may also be at least one storage device located away from the foregoing processor.

The processor 110 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; or a digital signal processing (DSP), dedicated integration. Application Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component.

In another embodiment of the present application, in the embodiment shown in FIG. 11, the processor 110 may be specifically configured to:

The image area of the charging stand to be identified is determined from the image according to a preset charging stand pixel feature and/or a preset charging stand size feature.

In another embodiment of the present application, in the embodiment shown in FIG. 11, the processor 110 may be specifically configured to:

Determining an image position of the first preset number of identification points from the image area according to the first preset number of identification points on the pre-acquired charging stand; and determining, according to the first preset number of identification points, the charging stand to be identified The spatial position on the upper surface and the image position of the first preset number of identification points, and the first preset formula determine the position information of the charging stand to be identified relative to the mobile robot.

In another embodiment of the present application, in the embodiment shown in FIG. 11, the processor 110 may be specifically configured to:

Determining the rotation matrix R and the translation matrix t of the charging stand to be identified relative to the mobile robot according to the following first preset formula:

Wherein, (X _i , Y _i , Z _i ) represents a spatial position of the i-th identification point of the first preset number of identification points on the charging stand to be identified, and (u _i , v _i ) represents the first preset quantity. The image position of the i-th identification point in the identification point, K represents the internal parameter matrix of the preset infrared camera module, argmin represents the minimum projection error function, and n represents the first preset number, (X _i ', Y _i ' , Z _i ') represents the coordinates obtained by coordinate transformation (X _i , Y _i , Z _i ), and (u _i ', v _i ') represents (X _i , Y _i , Z _i ) on the imaging plane of the image Projection coordinates.

In another embodiment of the present application, in the embodiment shown in FIG. 11, the processor 110 may be specifically configured to:

Obtaining depth information of a second preset number of identification points in the image area;

Determining, according to the depth information and the image position of the second preset number of the identification points in the image area, the spatial position of the second preset number of identification points in the image area according to the second preset formula;

Determining the location information of the charging stand to be identified according to the spatial position of the second preset number of identification points in the image area.

In another embodiment of the present application, in the embodiment shown in FIG. 11 , when the infrared camera module 112 further has a depth sensing function, the infrared camera module 112 is further configured to collect a depth image corresponding to the image, and store the image to Memory 111. The processor 110 is specifically configured to acquire a depth image from the memory, and obtain depth information of a second preset number of identifier points in the image region from the depth image, where the depth image includes depth information of each identifier point.

In an embodiment of the present application, the infrared camera module 112 may include a depth sensor (not shown), which may be a TOF sensor. The depth sensor is used to acquire depth information of each pixel in the depth image. The infrared camera module 112 can also be used to collect a depth image corresponding to the image and store it in the memory 111. The processor 110 is specifically configured to obtain a depth image from the memory 111, and obtain depth information of a second preset number of identifier points in the image region from the depth image, where the depth image includes depth information of each identifier point.

In another embodiment of the present application, in the embodiment shown in FIG. 11, the processor 110 may be specifically configured to: when the infrared camera module includes a left camera module and a right camera module (not shown), The image includes a first image acquired by the left camera module and a second image respectively acquired by the right camera module, and the image region is a first image region determined from the first image or a second image region determined from the second image. Determining depth information of a second predetermined number of identification points in the image area according to different image positions of the corresponding identification points in the first image area and the second image area.

In another embodiment of the present application, in the embodiment shown in FIG. 11, the processor 110 may be specifically configured to: when the mobile robot further includes an IMU (not shown), acquire the infrared camera module 112 to acquire an image. The previously acquired previous image acquires a previous image region of the charging stand to be identified determined from the previous image according to the preset charging stand image feature, and acquires a motion parameter when the IMU collects from the first position to the second position, Determining depth information of a second predetermined number of identification points in the image area according to the motion parameter and the previous image area. The IMU is configured to collect motion parameters when moving from the first position to the second position; wherein the first position is a spatial position of the mobile robot when the previous image is acquired, and the second position is a spatial position of the mobile robot when the image is acquired.

In another embodiment of the present application, in the embodiment shown in FIG. 11, the processor 110 is specifically configured to:

Determining the spatial location of the second predetermined number of identification points in the image area according to the following second preset formula:

Wherein, (X _i , Y _i , Z _i ) represents the spatial position of the i-th identification point in the image region, Z _i represents the depth information of the i-th identification point in the image region, and K represents the preset infrared camera module. The internal reference matrix, (u _i , v _i ), represents the image position of the i-th identified point in the image region.

In another embodiment of the present application, in the embodiment shown in FIG. 11, the processor 110 is specifically configured to:

Based on the determined image area, the spatial position and spatial orientation of the charging stand to be identified relative to the mobile robot are determined.

In another embodiment of the present application, in the embodiment shown in FIG. 11, the mobile robot may further include: an infrared emitter (not shown) capable of emitting infrared light. The infrared light emitted by the infrared emitter can be illuminated on the charging stand to be identified.

In another embodiment of the present application, in the embodiment shown in FIG. 11, the charging stand to be identified may include a reflective material, and the reflective material can return the reflected light along the optical path of the incident light.

In another embodiment of the present application, in the embodiment shown in FIG. 11, each side of the charging stand to be identified may include a reflective material, and the pattern features of the reflective materials on the respective sides are different.

Since the above-described mobile robot embodiment is obtained based on the method embodiment, and has the same technical effect as the method, the technical effects of the mobile robot embodiment are not described herein again. For the mobile robot embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment.

The embodiment of the present application further provides a computer readable storage medium. The computer readable storage medium stores a computer program. When the computer program is executed by the processor, the charging stand identification method provided by the embodiment of the present application is implemented. The method includes:

Obtaining an image acquired by an infrared camera module;

Determining an image area of the charging stand to be identified from the image according to a preset charging stand image feature; wherein the charging stand to be identified can emit infrared light;

The position information of the charging stand to be identified is determined according to the determined image area.

In this embodiment, the mobile robot can acquire an image of the charging stand to be recognized that can be emitted by the infrared camera module, and determine an image area of the charging stand to be identified from the image according to the preset charging stand image feature. The image area determines the position information of the charging stand to be identified. Since the image acquisition range of the infrared camera module is a conical range with the apex of the infrared camera module, when the location of the mobile robot is tilted, the charging stand to be identified can also be within the image acquisition range of the infrared camera module, and further The position information recognition of the charging stand to be identified can be realized. Moreover, the charging stand to be identified can emit infrared light, and the charging stand to be identified in the image collected by the infrared camera module can have a relatively obvious image feature, so when the position information of the charging stand to be identified is identified from the image, Can improve the accuracy of recognition.

The embodiment of the present application further provides a computer program, which is implemented by the processor to implement the charging stand identification method provided by the embodiment of the present application. The method includes:

Obtaining an image acquired by an infrared camera module;

Determining an image area of the charging stand to be identified from the image according to a preset charging stand image feature; wherein the charging stand to be identified can emit infrared light;

The position information of the charging stand to be identified is determined according to the determined image area.

In this embodiment, the mobile robot can acquire an image of the charging stand to be recognized that can be emitted by the infrared camera module, and determine an image area of the charging stand to be identified from the image according to the preset charging stand image feature. The image area determines the position information of the charging stand to be identified. Since the image acquisition range of the infrared camera module is a conical range with the apex of the infrared camera module, when the location of the mobile robot is tilted, the charging stand to be identified can also be within the image acquisition range of the infrared camera module, and further The position information recognition of the charging stand to be identified can be realized. Moreover, the charging stand to be identified can emit infrared light, and the charging stand to be identified in the image collected by the infrared camera module can have a relatively obvious image feature, so when the position information of the charging stand to be identified is identified from the image, Can improve the accuracy of recognition.

It should be noted that, in this context, relational terms such as first and second are used merely to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply such entities or operations. There is any such actual relationship or order between them. Furthermore, the terms "comprises," "comprising," or "includes" or "includes" or "includes" or "includes" or "includes" or "includes" Other elements, or elements that are inherent to such a process, method, item, or device. In the absence of further limitation, an element defined by the phrase "comprising a ..." does not exclude the presence of additional equivalent elements in the process, method, article, or device that comprises the element.

The various embodiments in the present specification are described in a related manner, and the same or similar parts between the various embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments.

The above description is only the preferred embodiment of the present application, and is not intended to limit the scope of the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application are included in the scope of the present application.

Claims (24)

1. A charging stand recognition method is characterized in that it is applied to a mobile robot, and the mobile robot includes: an infrared camera module; the method includes:

   Obtaining an image acquired by the infrared camera module;

   Determining an image area of the charging stand to be identified from the image according to a preset charging stand image feature; wherein the charging stand to be identified can emit infrared light;

   Determining the location information of the charging stand to be identified according to the determined image area.
2. The method according to claim 1, wherein the step of determining an image area of the charging stand to be identified from the image according to a preset charging stand image feature comprises:

   The image area of the charging stand to be identified is determined from the image according to a preset charging stand pixel feature and/or a preset charging stand size feature.
3. The method according to claim 1, wherein the determining the location information of the charging stand to be identified according to the determined image area comprises:

   Determining an image position of the first preset number of identification points from the image area according to the first preset number of identification points on the to-be-identified charging stand acquired in advance;

   Determining the to-be-identified charging according to the spatial position of the first preset number of identification points on the charging stand to be identified and the image position of the first preset number of identification points, and a first preset formula Position information relative to the mobile robot.
4. The method according to claim 3, wherein the image according to the first preset number of identification points on the charging stand to be identified and the image of the first preset number of identification points a position, and a first preset formula, determining a position information of the charging stand to be identified relative to the mobile robot, comprising:

   Determining, according to the following first preset formula, the rotation matrix R and the translation matrix t of the charging stand to be identified relative to the mobile robot, and obtaining position information of the charging stand to be identified relative to the mobile robot:

   

   Wherein (X _i , Y _i , Z _i ) represents a spatial position of the i-th identification point of the first preset number of identification points on the charging stand to be identified, (u _i , v _i ) representing an image position of the i-th identification point of the first preset number of identification points, the K represents a preset internal parameter matrix of the infrared camera module, and the argmin represents a minimum projection error function, The n represents the first preset number, and the (X _i ', Y _i ', Z _i ') represents coordinates obtained by coordinate transformation (X _i , Y _i , Z _i ), (u _i ' , v _i ') represents the projected coordinates of (X _i , Y _i , Z _i ) on the imaging plane of the image.
5. The method according to claim 1, wherein the determining the location information of the charging stand to be identified according to the determined image area comprises:

   Obtaining depth information of a second preset number of identification points in the image area;

   Determining, according to the depth information and the image position of the second preset number of the identification points in the image area, a spatial position of the second preset number of identification points in the image area according to the second preset formula;

   And determining location information of the charging stand to be identified according to a spatial position of the second preset number of identification points in the image area.
6. The method according to claim 5, wherein the step of acquiring the depth information of the second predetermined number of identifier points in the image area comprises:

   When the infrared camera module further has a depth sensing function, acquiring a depth image corresponding to the image collected by the infrared camera module, and acquiring a second preset number of the image regions from the depth image Identifying depth information of the point; wherein the depth image includes depth information of each identification point; or

   When the infrared camera module includes a left camera module and a right camera module, the image includes a first image captured by the left camera module and a second image captured by the right camera module, the image The region is a first image region determined from the first image or a second image region determined from the second image, according to different images of corresponding identification points in the first image region and the second image region Position, determining depth information of a second preset number of identification points in the image area; or

   When the mobile robot further includes an inertial sensing unit IMU, acquiring a previous image acquired by the infrared camera module before acquiring the image, and obtaining, according to a preset charging seat image feature, determining from the previous image And obtaining a motion parameter when the IMU collects the first position to the second position, and determining the image area according to the motion parameter and the previous image area. a depth information of the second predetermined number of identification points; wherein the first position is a spatial position of the mobile robot when the previous image is acquired, and the second position is the movement when the image is collected The spatial position of the robot.
7. The method according to claim 5, wherein the determining the image region according to the second preset formula according to the depth information and an image position of a second predetermined number of identification points in the image region The step of the second preset number of spatial locations of the identification points includes:

   Determining a spatial location of the second predetermined number of identification points in the image area according to the following second preset formula:

   

   Wherein, (X _i , Y _i , Z _i ) represents a spatial position of an i-th identification point in the image area, and the Z _i is depth information of an i-th identification point in the image area, K represents a preset internal reference matrix of the infrared camera module, and the (u _i , v _i ) represents an image position of the i-th identification point in the image region.
8. The method according to claim 1, wherein the determining the location information of the charging stand to be identified according to the determined image area comprises:

   Determining, according to the determined image area, a spatial position and a spatial orientation of the charging stand to be identified with respect to the mobile robot.
9. The method according to any one of claims 1 to 8, wherein the mobile robot further comprises: an infrared emitter capable of emitting infrared light.
10. The method according to claim 9, wherein the charging stand to be identified comprises a reflective material, and the reflective material is capable of returning the reflected light along the optical path of the incident light.
11. The method according to claim 10, wherein the reflective material is included on each side of the charging stand to be identified, and the pattern characteristics of the reflective material on each side are different.
12. A mobile robot, comprising: a processor, a memory, and an infrared camera module;

    The infrared camera module is configured to collect an image and store the image to the memory;

    The processor is configured to acquire the image in the memory, determine an image area of the charging stand to be identified from the image according to a preset charging stand image feature, and determine, according to the determined image area, Determining the location information of the charging cradle; wherein the charging cradle to be identified can emit infrared light.
13. The robot according to claim 12, wherein the processor is specifically configured to:

    The image area of the charging stand to be identified is determined from the image according to a preset charging stand pixel feature and/or a preset charging stand size feature.
14. The robot according to claim 12, wherein the processor is specifically configured to:

    Determining an image position of the first preset number of identification points from the image area according to the first preset number of identification points on the to-be-identified charging cradle obtained in advance; according to the first preset quantity Determining a spatial position of the identification point on the charging stand to be identified and an image position of the first preset number of identification points, and a first preset formula determining the charging stand to be identified relative to the mobile robot location information.
15. The robot according to claim 14, wherein the processor is specifically configured to:

    Determining, according to the following first preset formula, the rotation matrix R and the translation matrix t of the charging stand to be identified relative to the mobile robot, and obtaining position information of the charging stand to be identified relative to the mobile robot:

    

    Wherein (X _i , Y _i , Z _i ) represents a spatial position of the i-th identification point of the first preset number of identification points on the charging stand to be identified, (u _i , v _i ) representing an image position of the i-th identification point of the first preset number of identification points, the K represents a preset internal parameter matrix of the infrared camera module, and the argmin represents a minimum projection error function, The n represents the first preset number, and the (X _i ', Y _i ', Z _i ') represents coordinates obtained by coordinate transformation (X _i , Y _i , Z _i ), (u _i ' , v _i ') represents the projected coordinates of (X _i , Y _i , Z _i ) on the imaging plane of the image.
16. The robot according to claim 12, wherein the processor is specifically configured to:

    Obtaining depth information of a second preset number of identification points in the image area;

    Determining, according to the depth information and the image position of the second preset number of the identification points in the image area, a spatial position of the second preset number of identification points in the image area according to the second preset formula;

    And determining location information of the charging stand to be identified according to a spatial position of the second preset number of identification points in the image area.
17. The robot according to claim 16, wherein

    When the infrared camera module further has a depth sensing function, the infrared camera module is further configured to collect a depth image corresponding to the image and store the image to the memory; the processor is specifically configured to Acquiring the depth image in the memory, and acquiring depth information of the second preset number of identifier points in the image area from the depth image; wherein the depth image includes depth information of each identifier point; or

    The processor is specifically configured to: when the infrared camera module includes a left camera module and a right camera module, the image includes the first image captured by the left camera module and the right camera module. a second image, the image area being a first image area determined from the first image or a second image area determined from the second image, according to the first image area and the second image area Determining depth information of a second preset number of identification points in the image area; or

    The processor is specifically configured to acquire, when the mobile robot further includes an inertial sensing unit IMU, acquire a previous image acquired by the infrared camera module before acquiring the image, and acquire image characteristics according to a preset charging stand Acquiring, from the previous image area of the to-be-identified charging stand determined in the previous image, a motion parameter when the IMU collects from the first position to the second position, according to the motion parameter and the previous one An image area, where depth information of a second predetermined number of identification points in the image area is determined; the IMU is configured to collect motion parameters from the first position to the second position; A position is a spatial position of the mobile robot when the previous image is acquired, and the second position is a spatial position of the mobile robot when the image is acquired.
18. The robot according to claim 16, wherein the processor is specifically configured to:

    Determining a spatial location of the second predetermined number of identification points in the image area according to the following second preset formula:

    

    Wherein, (X _i , Y _i , Z _i ) represents a spatial position of an i-th identification point in the image area, and the Z _i is depth information of an i-th identification point in the image area, K represents a preset internal reference matrix of the infrared camera module, and the (u _i , v _i ) represents an image position of the i-th identification point in the image region.
19. The robot according to claim 12, wherein the processor is specifically configured to:

    Determining, according to the determined image area, a spatial position and a spatial orientation of the charging stand to be identified with respect to the mobile robot.
20. The robot according to any one of claims 12 to 19, wherein the mobile robot further comprises: an infrared emitter capable of emitting infrared light.
21. The robot according to claim 20, wherein said charging stand to be identified comprises a light reflecting material, and said light reflecting material is capable of returning the reflected light along the optical path of the incident light.
22. The robot according to claim 21, wherein each of the side faces of the charging stand to be identified includes the reflective material, and the pattern features of the reflective materials on the respective sides are different.
23. A computer readable storage medium, wherein the computer readable storage medium stores a computer program, the computer program being executed by a processor to implement the method steps of any of claims 1-11.
24. A computer program, characterized in that the computer program, when executed by a processor, implements the method steps of any of claims 1-11.

Images