CN114259188A

CN114259188A - Cleaning device, image processing method and apparatus, readable storage medium

Info

Publication number: CN114259188A
Application number: CN202210015395.7A
Authority: CN
Inventors: 周技锋; 孙涛; 鞠汶奇
Original assignee: Midea Robozone Technology Co Ltd
Current assignee: Midea Robozone Technology Co Ltd
Priority date: 2022-01-07
Filing date: 2022-01-07
Publication date: 2022-04-01

Abstract

The invention provides a cleaning device, an image processing method and device and a readable storage medium, wherein the cleaning device comprises: a housing; the first image acquisition device is arranged on the shell and used for acquiring a first type of image outside the shell; the second image acquisition device is arranged on the shell and used for acquiring a second type of image outside the shell; and pulse width modulation means for outputting a pulse signal; the first image acquisition device and the second image acquisition device work simultaneously when acquiring the preset signal in the pulse signal. The calculation result of the real scene obtained by reduction is more accurate. Because the first-class image and the second-class image are shot at the same time, the influence of the two images caused by different shooting times does not need to be considered when the real scene is restored, so that the calculation workload when the real scene is restored can be effectively reduced, and the real scene is restored more timely and accurately.

Description

Cleaning device, image processing method and apparatus, readable storage medium

Technical Field

The invention belongs to the technical field of cleaning equipment, and particularly relates to cleaning equipment, an image processing method and device and a readable storage medium.

Background

The floor sweeping robot can automatically clean indoor floor, and is widely used at present.

The sweeping robot comprises a depth camera and an RGB camera, the depth camera shoots to obtain a depth image, the RGB camera shoots to obtain an RGB image, and the depth image and the RGB image are fused to restore real scenes around the sweeping robot.

However, at present, the calculation process of fusing the depth image and the RGB image is complex, it is difficult to quickly obtain the real scene around the sweeping robot, and the accuracy of the method for determining the real scene around the sweeping robot by fusing the depth image and the RGB image is low.

Disclosure of Invention

The present invention is directed to solving one of the technical problems of the prior art or the related art.

In view of this, the present invention proposes, in a first aspect, a cleaning device comprising: a housing; the first image acquisition device is arranged on the shell and used for acquiring a first type of image outside the shell; the second image acquisition device is arranged on the shell and used for acquiring a second type of image outside the shell; and pulse width modulation means for outputting a pulse signal; the first image acquisition device and the second image acquisition device work simultaneously when acquiring the preset signal in the pulse signal.

According to the cleaning equipment provided by the invention, two groups of image acquisition devices are arranged on the shell, and the two groups of image acquisition devices can respectively acquire the first type of images and the second type of images outside the shell. The cleaning device can move and clean the ground, so that in order to avoid collision between the cleaning device and an obstacle, the distance between the cleaning device and the obstacle and the shape of the obstacle need to be determined, and the cleaning device can restore a real scene around the cleaning device according to the first type of image and the second type of image.

In the related art, a first image capturing device and a second image capturing device in a cleaning apparatus independently capture captured images, and the images captured by the first image capturing device and the second image capturing device are not captured synchronously. Since the cleaning device is in a state of continuous movement during operation, the images captured by the first image capturing device and the second image capturing device may not be images at the same viewing angle. Therefore, for images at different viewing angles, a complex calculation process is required to restore the real scene around the cleaning device, and the timeliness of the calculation result is difficult to ensure. Moreover, since the images captured by the first image capturing device and the second image capturing device may not be images from the same viewing angle, it is also difficult to ensure the accuracy of restoring the real scene.

In the invention, the pulse width modulation device can output a pulse signal, the first image acquisition device and the second image acquisition device can acquire the pulse signal output by the pulse width modulation device, and when the first image acquisition device and the second image acquisition device acquire a preset signal in the pulse signal, the first image acquisition device and the second image acquisition device work simultaneously, so that the first image acquisition device and the second image acquisition device acquire the first type image and the second type image at the same time. Therefore, when the preset signal is not received, the first image capturing device and the second image capturing device do not operate.

Since the first type of image and the second type of image are taken at the same time, the first type of image and the second type of image are taken at the same viewing angle, and the function of simultaneously taking the first type of image and the second type of image can be realized even if the cleaning device is in the process of continuous movement.

And the real scene around the cleaning equipment is restored according to the first type of image and the second type of image, and the first type of image and the second type of image are shot at the same moment, so that the influence of the two images caused by different shooting moments is not required to be considered when the real scene is restored, the calculation workload when the real scene is restored can be effectively reduced, the timeliness of the real scene around the cleaning equipment can be ensured, and the cleaning equipment is prevented from colliding with obstacles in the moving process.

In addition, the first type of image and the second type of image are shot at the same time, so that the calculation result of the real scene obtained by restoration is more accurate, and the collision between the cleaning equipment and the obstacle is effectively avoided.

The preset signal in the pulse signal may be a preset signal, and for example, the preset signal may be a rising edge signal or a falling edge signal, and the preset signal is set as the rising edge signal or the falling edge signal, which is beneficial for the first image capturing device and the second image capturing device to work simultaneously.

In addition, according to the cleaning device in the above technical solution provided by the present invention, the following additional technical features may also be provided:

in one possible design, the cleaning device further comprises: and a processing device for performing synthesis processing on the first type image and the second type image to generate a synthetic image.

In the design, after a first type image and a second type image are acquired, the first type image and the second type image are synthesized, so that the first type image and the second type image are fused, the first type image and the second type image are synthesized into one image, and a real scene around the cleaning equipment is restored according to the synthesized image. The synthetic image has the characteristics of the first type of image and the second type of image, the condition of the obstacles around the cleaning equipment can be accurately determined according to the synthetic image, and collision between the cleaning equipment and the obstacles is avoided.

In one possible design, the processing device is further configured to: the output frequency of the pulse signal is adjusted according to the synthesized image.

In this design, the output frequency of the pulse signal of the pulse width modulation device is associated with the composite image. The composite image is capable of restoring a real scene around the cleaning device, including the distance between the cleaning device and the obstacle.

The distance between the housing and the obstacle may be set to be inversely related to the output frequency of the pulse signal. The larger the distance between the housing and the obstacle, the slower the frequency of the pulse width modulation device outputting the pulse signal, and the smaller the distance between the housing and the obstacle, the faster the frequency of the pulse width modulation device outputting the pulse signal.

Specifically, when the distance between the shell and the barrier is large, the cleaning equipment is not easy to collide with the barrier, the frequency of pulse signals output by the pulse width modulation device can be reduced, the times of shooting images by the first image acquisition device and the second image acquisition device are reduced, electric energy can be effectively saved, the service life of the cleaning equipment is prolonged, and the charging frequency of the cleaning equipment is reduced. When the distance between the shell and the obstacle is small, the cleaning equipment is easy to collide with the obstacle, at the moment, the frequency of the pulse signal output by the pulse width modulation device needs to be increased, the distance between the shell and the obstacle needs to be refreshed quickly, the collision between the cleaning equipment and the obstacle is avoided, and the damage rate of the cleaning equipment is reduced.

In one possible application, the processing device may be a processor.

In one possible design, the cleaning device further comprises: the moving device is arranged on the shell and used for driving the shell to move; the processing device is further configured to: and adjusting the output rotating speed of the mobile device according to the synthesized image.

In this design, the output rotational speed of the mobile device is associated with the composite image. The composite image is capable of restoring a real scene around the cleaning device, including the distance between the cleaning device and the obstacle.

The distance between the housing and the obstacle may be set to be positively correlated with the output rotation speed of the mobile device. The greater the distance between the housing and the obstacle, the faster the output rotational speed of the moving means and the faster the moving speed of the cleaning apparatus. The smaller the distance between the housing and the obstacle, the slower the output rotational speed of the moving means and the slower the moving speed of the cleaning apparatus.

Specifically, when the distance between casing and the barrier is great, cleaning device is difficult for bumping with the barrier, can increase mobile device's output rotational speed this moment for cleaning device can the quick travel, improves clean efficiency. When the distance between the shell and the obstacle is small, the cleaning equipment is easy to collide with the obstacle, and the output rotating speed of the moving device needs to be reduced at the moment, so that the moving speed of the cleaning equipment is reduced, the collision between the cleaning equipment and the obstacle is avoided, and the damage rate of the cleaning equipment is reduced.

For example, the moving device may include a motor, a transmission device and a wheel body, the motor may drive the transmission device to move, and the transmission device drives the wheel body to rotate, so as to implement a moving function of the cleaning device. The output rotation speed of the motor may be adjusted according to the synthesized image.

In one possible design, the cleaning device further comprises: and the storage device is arranged on the shell and used for storing the composite image.

In this design, the composite image can recreate the real scene surrounding the cleaning device. Since the cleaning device is continuously moving during operation, multiple composite images can be acquired, so that the real scene in the user's room can be restored. After the synthetic image is stored, the cleaning equipment is equivalent to a model stored in a user room, the user can control the cleaning equipment to work in a memory mode, the cleaning equipment can work through the stored synthetic image and avoid obstacles, at the moment, the shooting functions of the first image acquisition device and the second image acquisition device can be stopped, and the electric energy saving is facilitated.

When the cleaning equipment collides when working in the memory mode, the partial layout of the interior of the user room is changed, at the moment, the cleaning equipment can automatically exit the memory mode, and the first image acquisition device and the second image acquisition device capture images and continue to acquire a composite image, so that the model in the user room is refreshed.

In one possible application, the storage device may be a memory.

In one possible design, the first type of image includes a depth image and the second type of image includes an RGB image and/or a grayscale image.

In this design, the first type of image comprises a depth image, so the first type of image may be taken by a depth camera, i.e. the first image acquisition device may be a depth camera, e.g. a TOF camera. The depth camera may acquire a distance between the determined housing and the obstacle.

The second type of image comprises an RGB image and/or a greyscale image, so the second type of image can be captured by an RGB camera and/or a greyscale camera, i.e. the second image capturing means comprises an RGB camera and/or a greyscale camera, the RGB image and/or the greyscale image being capable of determining two-dimensional spatial coordinates of points in a plane.

According to the distance between the shell and the obstacle and the coordinates of each point in the plane, the three-dimensional space coordinates of each point around the cleaning equipment can be determined, the real scene around the cleaning equipment can be restored according to the three-dimensional space coordinates, and the function of scene modeling is achieved.

In a second aspect, the present invention provides an image processing method for a cleaning apparatus, comprising: receiving a pulse signal; and controlling the first image acquisition device and the second image acquisition device to work simultaneously based on a preset signal in the received pulse signal. The first image acquisition device is used for acquiring a first type of image outside a shell of the cleaning equipment; and the second image acquisition device is used for acquiring a second type of image outside the shell of the cleaning equipment.

According to the image processing method of the cleaning equipment, the two groups of image acquisition devices can respectively acquire the first type of image and the second type of image outside the shell. The cleaning device can move and clean the ground, so that in order to avoid collision between the cleaning device and an obstacle, the distance between the cleaning device and the obstacle and the shape of the obstacle need to be determined, and the cleaning device can restore a real scene around the cleaning device according to the first type of image and the second type of image.

In one possible design, the image processing method further includes: further comprising: and synthesizing the first type of image and the second type of image to generate a synthesized image.

In one possible design, generating the composite image further includes: the output frequency of the pulse signal is adjusted according to the synthesized image.

In this design, the output frequency of the pulse signal is associated with the composite image. The composite image is capable of restoring a real scene around the cleaning device, including the distance between the cleaning device and the obstacle.

The distance between the housing and the obstacle may be set to be inversely related to the output frequency of the pulse signal. The larger the distance between the housing and the obstacle, the slower the frequency of the output pulse signal, and the smaller the distance between the housing and the obstacle, the faster the frequency of the output pulse signal.

Specifically, when the distance between the shell and the barrier is large, the cleaning equipment is not prone to colliding with the barrier, the frequency of output pulse signals can be reduced, the number of times of shooting images by the first image acquisition device and the second image acquisition device is reduced, electric energy can be effectively saved, the service life of the cleaning equipment is prolonged, and the charging frequency of the cleaning equipment is reduced. When the distance between the shell and the obstacle is small, the cleaning equipment is easy to collide with the obstacle, the frequency of the output pulse signal needs to be increased at the moment, the distance between the shell and the obstacle needs to be refreshed quickly, collision between the cleaning equipment and the obstacle is avoided, and the damage rate of the cleaning equipment is reduced.

In one possible design, the cleaning device comprises: the mobile device is arranged on the shell and used for driving the shell to move; after generating the composite image, the method further comprises the following steps: and adjusting the output rotating speed of the mobile device according to the synthesized image.

In one possible design, generating the composite image further includes: the composite image is stored.

In one possible design, the first type of image includes a depth image and the second type of image includes an RGB image.

In this design, the first type of image comprises a depth image, so the first type of image may be captured by a depth camera, i.e. the first image acquisition device may be a depth camera. The depth camera may acquire a distance between the determined housing and the obstacle.

In a third aspect, the present invention provides an image processing apparatus for a cleaning device, the image processing apparatus comprising: the receiving module is used for receiving the pulse signal; the control module is used for controlling the first image acquisition device and the second image acquisition device to work simultaneously based on a preset signal in the received pulse signal; the first image acquisition device is used for acquiring a first type of image outside a shell of the cleaning equipment; and the second image acquisition device is used for acquiring a second type of image outside the shell of the cleaning equipment.

According to the image processing device provided by the invention, the two groups of image acquisition devices can respectively acquire the first type of images and the second type of images outside the shell. The cleaning device can move and clean the ground, so that in order to avoid collision between the cleaning device and an obstacle, the distance between the cleaning device and the obstacle and the shape of the obstacle need to be determined, and the cleaning device can restore a real scene around the cleaning device according to the first type of image and the second type of image.

In one possible design, the image processing apparatus further includes: and the processing module is used for carrying out synthesis processing on the first-class images and the second-class images to generate synthetic images.

In one possible design, the processing module is further to: the output frequency of the pulse signal is adjusted according to the synthesized image.

In one possible design, the cleaning device further comprises: the mobile device is arranged on the shell and used for driving the shell to move; the processing module is further configured to: and adjusting the output rotating speed of the mobile device according to the synthesized image.

In one possible design, the cleaning device further comprises: and the storage module is arranged on the shell and used for storing the composite image.

In a fourth aspect, the present invention provides a cleaning apparatus comprising: the image processing device in any design, therefore, the cleaning equipment provided by the invention has the beneficial effects of the image processing device.

In a fifth aspect, the present invention provides a cleaning apparatus comprising: a memory storing programs or instructions; the controller, when the controller executes the program or the instruction, implements the image processing method in any one of the above possible designs, so the cleaning device provided by the invention has the beneficial effects of the image processing method in any one of the above possible designs.

In a sixth aspect, the present invention provides a readable storage medium, on which a program or instructions are stored, and the program or instructions, when executed by a processor, implement the steps of the image processing method in any one of the above possible designs, so that the readable storage medium provided by the present invention has the beneficial effects of the image processing method in any one of the above possible designs.

Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.

Drawings

The above and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:

FIG. 1 shows one of the block diagrams of the structure of a cleaning device in an embodiment of the invention;

FIG. 2 shows a second block diagram of the cleaning apparatus in an embodiment of the invention;

FIG. 3 shows a schematic flow chart of an image processing method in an embodiment of the invention;

FIG. 4 is a block diagram showing the configuration of an image processing apparatus in the embodiment of the present invention;

FIG. 5 shows a third block diagram of a cleaning apparatus according to an embodiment of the present invention;

fig. 6 shows a fourth block diagram of the structure of the cleaning apparatus in the embodiment of the present invention.

Detailed Description

In order that the above objects, features and advantages of the present invention can be more clearly understood, a more particular description of the invention will be rendered by reference to the appended drawings. It should be noted that the embodiments and features of the embodiments of the present application may be combined with each other without conflict.

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, however, the present invention may be practiced in other ways than those specifically described herein, and therefore the scope of the present invention is not limited by the specific embodiments disclosed below.

A cleaning device, an image processing method for a cleaning device, an image processing apparatus for a cleaning device, and a readable storage medium provided according to some embodiments of the present invention are described below with reference to fig. 1 to 6.

As shown in fig. 1, in some embodiments of the invention, a cleaning apparatus 100 is presented, comprising: the image acquisition device comprises a shell 110, a first image acquisition device 120, a second image acquisition device 130 and a pulse width modulation device 140, wherein the first image acquisition device 120, the second image acquisition device 130 and the pulse width modulation device 140 are arranged on the shell 110, and the first image acquisition device 120 is used for acquiring a first type of image outside the shell 110; the second image acquisition device 130 is used for acquiring a second type of image outside the shell 110; the pulse width modulation device 140 is used for outputting pulse signals; the first image capturing device 120 and the second image capturing device operate simultaneously when capturing the preset signal in the pulse signal.

In the cleaning apparatus 100 provided in the present embodiment, two sets of image capturing devices are disposed on the housing 110, and the two sets of image capturing devices can respectively capture the first type image and the second type image outside the housing 110. The cleaning device 100 can move and clean the floor, in order to avoid collision between the cleaning device 100 and an obstacle, the distance between the cleaning device 100 and the obstacle and the shape of the obstacle need to be determined, and the cleaning device 100 can restore the real scene around the cleaning device 100 according to the first type image and the second type image.

In the related art, the first image capturing device and the second image capturing device in the cleaning apparatus independently capture the captured images, and the images captured by the first image capturing device and the second image capturing device are not captured synchronously. Since the cleaning device is in a state of continuous movement during operation, the images captured by the first image capturing device and the second image capturing device may not be images from the same viewing angle. Therefore, for images at different viewing angles, a complex calculation process is required to restore the real scene around the cleaning device, and the timeliness of the calculation result is difficult to ensure. Moreover, since the images captured by the first image capturing device and the second image capturing device may not be images from the same viewing angle, it is difficult to ensure the accuracy of restoring the real scene.

In this embodiment, the pulse width modulation device 140 can output a pulse signal, the first image capturing device 120 and the second image capturing device 130 can simultaneously capture the pulse signal, and when the first image capturing device 120 and the second image capturing device 130 capture the preset signal, the first image capturing device 120 and the second image capturing device 130 simultaneously operate, so that the first image capturing device 120 and the second image capturing device 130 capture the first type image and the second type image at the same time. Therefore, when the preset signal is not received, the first image capturing device 120 and the second image capturing device 130 are not operated.

Since the first type image and the second type image are captured at the same time, the first type image and the second type image are captured at the same viewing angle, and the function of capturing the first type image and the second type image simultaneously can be realized even if the cleaning device 100 is in the process of continuous movement.

Moreover, the real scene around the cleaning device 100 is restored according to the first type of image and the second type of image, and the first type of image and the second type of image are shot at the same time, so that the influence of the two images caused by different shooting times is not required to be considered when the real scene is restored, the calculation workload when the real scene is restored can be effectively reduced, the timeliness of the real scene around the cleaning device 100 can be ensured, and the cleaning device 100 is prevented from colliding with obstacles in the moving process.

In addition, the first-class image and the second-class image are shot at the same time, so that the calculation result of the real scene obtained by restoration is more accurate, and the collision between the cleaning equipment 100 and the obstacle is effectively avoided.

The preset signal in the pulse signal may be a preset signal, and for example, the preset signal may be a rising edge signal or a falling edge signal, and the preset signal is set as the rising edge signal or the falling edge signal, which is beneficial for the first image capturing device 120 and the second image capturing device 130 to operate simultaneously.

In one possible application, the housing 110 includes a casing and a chassis disposed at the bottom of the casing, and the first image capturing device 120, the second image capturing device 130 and the pulse width modulation device 140 are disposed on the chassis.

In a possible embodiment, the cleaning device further comprises: and the processing device 150, wherein the processing device 150 is used for synthesizing the first type image and the second type image into a synthetic image.

In this embodiment, after the first-type image and the second-type image are acquired, the first-type image and the second-type image are synthesized so that the first-type image and the second-type image are fused, the first-type image and the second-type image are synthesized into one image, and a real scene around the cleaning device 100 is restored according to the synthesized image. The synthetic image has the characteristics of the first type of image and the second type of image, and the condition of the obstacles around the cleaning device 100 can be accurately determined according to the synthetic image, so that the cleaning device 100 is prevented from colliding with the obstacles.

Because the first-class images and the second-class images are shot at the same time, the capture of the images at the same time is ensured, and the success rate of fusion of the first-class images and the second-class images is improved.

As shown in fig. 2, in one possible embodiment, the processing device 150 is further configured to: the output frequency of the pulse signal is adjusted based on the composite image.

In this embodiment, the output frequency of the pulse signal of the pulse width modulation device 140 is associated with the synthesized image. The composite image is capable of restoring a real scene around the cleaning device 100, including the distance between the cleaning device 100 and the obstacle.

The distance between the housing 110 and the obstacle may be set to be inversely related to the output frequency of the pulse signal. The larger the distance between the housing 110 and the obstacle, the slower the frequency of the pulse width modulation device 140 outputting the pulse signal, and the smaller the distance between the housing 110 and the obstacle, the faster the frequency of the pulse width modulation device 140 outputting the pulse signal.

Specifically, when the distance between the housing 110 and the obstacle is large, the cleaning device 100 is not likely to collide with the obstacle, and at this time, the frequency of the pulse width modulation device 140 outputting the pulse signal may be reduced, the number of times of capturing images by the first image capturing device 120 and the second image capturing device 130 may be reduced, the electric energy may be effectively saved, the use duration of the cleaning device 100 may be improved, and the charging frequency of the cleaning device 100 may be reduced. When the distance between the housing 110 and the obstacle is small, the cleaning device 100 is likely to collide with the obstacle, and at this time, it is necessary to increase the frequency of the pulse signal output by the pulse width modulation device 140, to quickly refresh the distance between the housing 110 and the obstacle, to avoid collision between the cleaning device 100 and the obstacle, and to reduce the damage rate of the cleaning device 100.

In one possible application, the processing device 150 may be a processor.

In a possible application, the processing device 150 is connected to the first image obtaining device 120 and the second image obtaining device 130 through a data collecting interface, which may be an mipi (mobile Industry Processor interface) interface or a dvp (digital Video port) interface. The pulse width modulation device 140 is connected to the first image capturing device 120 and the second image capturing device 130 through the IO port.

In a possible embodiment, the cleaning device 100 further comprises: the mobile device is arranged on the shell 110 and can drive the shell 110 to move; the processing device is further configured to: the output speed of the mobile device is adjusted based on the composite image.

In this embodiment, the output rotation speed of the mobile device is associated with the composite image. The composite image is capable of restoring a real scene around the cleaning device 100, including the distance between the cleaning device 100 and the obstacle.

The distance between the housing 110 and the obstacle may be set to be positively correlated with the output rotation speed of the mobile device. The greater the distance between the housing 110 and the obstacle, the faster the output rotational speed of the moving means and the faster the moving speed of the cleaning apparatus 100. The smaller the distance between the housing 110 and the obstacle, the slower the output rotational speed of the moving means and the slower the moving speed of the cleaning apparatus 100.

Specifically, when the distance between the housing 110 and the obstacle is large, the cleaning apparatus 100 is not likely to collide with the obstacle, and at this time, the output rotation speed of the moving device may be increased, so that the cleaning apparatus 100 may be moved quickly, improving the cleaning efficiency. When the distance between the housing 110 and the obstacle is small, the cleaning apparatus 100 is likely to collide with the obstacle, and at this time, the output rotation speed of the moving device needs to be reduced, so that the moving speed of the cleaning apparatus 100 is reduced, the cleaning apparatus 100 is prevented from colliding with the obstacle, and the damage rate of the cleaning apparatus 100 is reduced.

For example, the moving device may include a motor, a transmission device, and a wheel, wherein the motor may drive the transmission device to move, and the transmission device drives the wheel to rotate, so as to implement the moving function of the cleaning apparatus 100. The output rotation speed of the motor may be adjusted according to the synthesized image.

In a possible embodiment, the cleaning device 100 further comprises: and a storage device disposed in the housing 110, the storage device being configured to store the composite image.

In this embodiment, the composite image is capable of restoring a real scene around the cleaning device 100. Since the cleaning device 100 is continuously moving during operation, multiple composite images can be acquired, so that the real scene in the user's room can be restored. After the composite image is stored, the cleaning apparatus 100 is equivalent to a model stored in a user room, the user can control the cleaning apparatus 100 to operate in a memory mode, the cleaning apparatus 100 can operate through the stored composite image and avoid obstacles, and at this time, the shooting functions of the first image capturing device 120 and the second image capturing device 130 can be stopped, which is beneficial to saving electric energy.

When the cleaning device 100 collides when operating in the memory mode, which indicates that the layout of the part inside the user's room has changed, the cleaning device 100 may automatically exit the memory mode, and the first image capturing device 120 and the second image capturing device 130 capture images and continue to capture the composite image, thereby refreshing the model in the user's room.

In one possible application, the storage device may be a memory.

In a possible embodiment, the first type of image comprises a depth image and the second type of image comprises a grayscale image and/or an RGB image.

In this embodiment, the first type of image may be a depth image, so the first type of image may be captured by a depth camera, that is, the first image obtaining device 120 may be a depth camera, such as a tof (time of flight) camera. The depth camera may acquire a distance that determines the distance between the housing 110 and the obstacle.

The second type of image comprises a greyscale image and/or an RGB image, so the second type of image may be captured by an RGB camera and/or a greyscale camera, i.e. the second image capturing means 130 comprises an RGB camera and/or a greyscale camera, the greyscale image and/or the RGB image being capable of determining two-dimensional spatial coordinates of points in a plane.

According to the distance between the shell 110 and the obstacle and the coordinates of each point in the plane, the three-dimensional space coordinates of each point around the cleaning device 100 can be determined, the real scene around the cleaning device 100 can be restored according to the three-dimensional space coordinates, and the scene modeling function is achieved.

The cleaning device in this embodiment includes: any one of a sweeper, a mopping machine and a sweeping and mopping all-in-one machine.

As shown in fig. 3, in some embodiments of the present invention, there is provided an image processing method for a cleaning apparatus, the image processing method including:

step 302, receiving a pulse signal;

and step 304, under the condition that a preset signal in the pulse signal is received, controlling the first image acquisition device and the second image acquisition device to work simultaneously.

The first image acquisition device is used for acquiring a first type of image outside a shell of the cleaning equipment; the second image acquisition device is used for acquiring a second type of image outside the shell of the cleaning device.

In the image processing method provided by this embodiment, the two sets of image capturing devices can respectively capture the first type image and the second type image outside the housing. The cleaning device can move and clean the ground, so that in order to avoid collision between the cleaning device and an obstacle, the distance between the cleaning device and the obstacle and the shape of the obstacle need to be determined, and the cleaning device can restore a real scene around the cleaning device according to the first type of image and the second type of image.

In the related art, a first image capturing device and a second image capturing device in the cleaning apparatus capture a captured image independently, and the images captured by the first image capturing device and the second image capturing device are not captured synchronously. Since the cleaning device is in a state of continuous movement during operation, the images captured by the first image capturing device and the second image capturing device may not be images at the same viewing angle. Therefore, for images at different viewing angles, a complex calculation process is required to restore the real scene around the cleaning device, and the timeliness of the calculation result is difficult to ensure. Moreover, since the images captured by the first image capturing device and the second image capturing device may not be images at the same viewing angle, it is also difficult to ensure the accuracy of restoring the real scene.

In this embodiment, the pulse width modulation device can output a pulse signal, the first image capturing device and the second image capturing device can capture the pulse signal output by the pulse width modulation device, and when the first image capturing device and the second image capturing device capture a preset signal in the pulse signal, the first image capturing device and the second image capturing device operate simultaneously, so that the first image capturing device and the second image capturing device capture the first type image and the second type image at the same time. Therefore, when the preset signal is not received, the first image acquisition device and the second image acquisition device do not work.

Since the first type image and the second type image are shot at the same time, the first type image and the second type image are shot at the same visual angle, and the function of simultaneously shooting the first type image and the second type image can be realized even if the cleaning equipment is in the process of continuous movement.

In one possible embodiment, the image processing method further includes: further comprising: and synthesizing the first type of image and the second type of image into a synthesized image.

In the embodiment, after the first type of image and the second type of image are acquired, the first type of image and the second type of image are synthesized to be fused, the first type of image and the second type of image are synthesized into one image, and a real scene around the cleaning equipment is restored according to the synthesized image. The synthetic image has the characteristics of the first type of image and the second type of image, the condition of the obstacles around the cleaning equipment can be accurately determined according to the synthetic image, and collision between the cleaning equipment and the obstacles is avoided.

In a possible embodiment, the step of generating the composite image further comprises: the output frequency of the pulse signal is adjusted based on the composite image.

In this embodiment, the output frequency of the pulse signal is associated with the synthesized image. The composite image is capable of restoring a real scene around the cleaning device, including the distance between the cleaning device and the obstacle.

In one possible embodiment, the cleaning apparatus comprises: the mobile device is arranged on the shell and used for driving the shell to move; the step after generating the composite image further comprises: the output speed of the mobile device is adjusted based on the composite image.

In this embodiment, the output rotation speed of the mobile device is associated with the composite image. The composite image is capable of restoring a real scene around the cleaning device, including the distance between the cleaning device and the obstacle.

In a possible embodiment, the step of generating the composite image further comprises: the composite image is stored.

In this embodiment, the composite image is capable of restoring a real scene around the cleaning device. Since the cleaning device is continuously moving during operation, multiple composite images can be acquired, so that the real scene in the user's room can be restored. After the synthetic image is stored, the cleaning equipment is equivalent to a model stored in a user room, the user can control the cleaning equipment to work in a memory mode, the cleaning equipment can work through the stored synthetic image and avoid obstacles, at the moment, the shooting functions of the first image acquisition device and the second image acquisition device can be stopped, and the electric energy saving is facilitated.

When the cleaning equipment collides when working in the memory mode, the partial layout of the interior of the user room is changed, at the moment, the cleaning equipment can automatically exit the memory mode, and the first image acquisition device and the second image acquisition device shoot images and continue to acquire a composite image, so that the model in the user room is refreshed.

In this embodiment, the first type of image may be a depth image, so the first type of image may be captured by a depth camera, i.e. the first image acquisition device may be a depth camera. The depth camera may acquire a distance between the determined housing and the obstacle.

The second type of image comprises a greyscale image and/or an RGB image, so the second type of image can be captured by an RGB camera and/or a greyscale camera, i.e. the second image capturing means comprises an RGB camera and/or a greyscale camera, the greyscale image and/or the RGB image being capable of determining two-dimensional spatial coordinates of points in a plane.

As shown in fig. 4, in some embodiments of the present invention, an image processing apparatus 400 is provided for a cleaning device, the image processing apparatus 400 including:

a receiving module 410, configured to receive a pulse signal;

the control module 420 controls the first image capturing device and the second image capturing device to work simultaneously when receiving the preset signal in the pulse signal.

In the image processing apparatus provided in this embodiment, the two sets of image capturing devices can respectively capture the first type of image and the second type of image outside the housing. The cleaning device can move and clean the ground, so that in order to avoid collision between the cleaning device and an obstacle, the distance between the cleaning device and the obstacle and the shape of the obstacle need to be determined, and the cleaning device can restore a real scene around the cleaning device according to the first type of image and the second type of image.

In the related art, a first image capturing device and a second image capturing device in a cleaning apparatus capture a captured image independently, and the images captured by the first image capturing device and the second image capturing device are not captured synchronously. Since the cleaning device is in a state of continuous movement during operation, the images captured by the first image capturing device and the second image capturing device may not be images from the same viewing angle. Therefore, for images at different viewing angles, a complex calculation process is required to restore the real scene around the cleaning device, and the timeliness of the calculation result is difficult to ensure. Moreover, since the images captured by the first image capturing device and the second image capturing device may not be images at the same viewing angle, it is also difficult to ensure the accuracy of restoring the real scene.

In this embodiment, the pulse width modulation device can output a pulse signal, the first image capturing device and the second image capturing device can capture the pulse signal output by the pulse width modulation device, and when the first image capturing device and the second image capturing device capture a preset signal in the pulse signal, the first image capturing device and the second image capturing device operate simultaneously, so that the first image capturing device and the second image capturing device capture the first type image and the second type image at the same time. Therefore, when the preset signal is not received, the first image capturing device and the second image capturing device do not operate.

In a possible embodiment, the image processing apparatus further comprises: and the processing module is used for synthesizing the first type of image and the second type of image into a synthetic image.

In the embodiment, after the first-class image and the second-class image are acquired, the first-class image and the second-class image are synthesized to be fused with each other, the first-class image and the second-class image are synthesized into one image, and a real scene around the cleaning device is restored according to the synthesized image. The synthetic image has the characteristics of the first type of image and the second type of image, the condition of the obstacles around the cleaning equipment can be accurately determined according to the synthetic image, and collision between the cleaning equipment and the obstacles is avoided.

In a possible embodiment, the processing module is further configured to: the output frequency of the pulse signal is adjusted based on the composite image.

In a possible embodiment, the cleaning device further comprises: the mobile device is arranged on the shell and used for driving the shell to move; the processing module is further configured to: the output speed of the mobile device is adjusted based on the composite image.

In a possible embodiment, the cleaning device further comprises: and the storage module is arranged on the shell and used for storing the synthetic image.

When the cleaning equipment collides when working in the memory mode, the partial layout of the interior of the user room is changed, at the moment, the cleaning equipment can automatically exit the memory mode, and the first image acquisition device and the second image acquisition device capture images and continue to acquire the composite image, so that the model in the user room is refreshed.

As shown in fig. 5, an embodiment of the present invention proposes a cleaning apparatus 500 including: the image processing apparatus 400 in any of the above embodiments, therefore, the cleaning device 500 provided by the present invention has the beneficial effects of the above image processing apparatus 400.

As shown in fig. 6, an embodiment of the present invention proposes a cleaning apparatus 600 including: a memory 610 and a controller 620, the memory 610 storing programs or instructions; the controller 620 implements the image processing method in any of the above possible embodiments when executing the program or the instructions, and thus the cleaning apparatus 600 provided by the present invention has the beneficial effects of the image processing method in any of the above possible embodiments.

In a sixth aspect, the present invention provides a readable storage medium, on which a program or instructions are stored, and the program or instructions, when executed by a processor, implement the steps of the image processing method in any of the above possible embodiments, so that the readable storage medium provided by the present invention has the beneficial effects of the image processing method in any of the above possible embodiments.

In the present invention, the term "plurality" means two or more unless explicitly defined otherwise. The terms "mounted," "connected," "fixed," and the like are to be construed broadly, and for example, "connected" may be a fixed connection, a removable connection, or an integral connection; "coupled" may be direct or indirect through an intermediary. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.

In the description herein, the description of the terms "one embodiment," "some embodiments," "specific embodiments," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A cleaning apparatus, comprising:

a housing;

the first image acquisition device is arranged on the shell and used for acquiring a first type of image outside the shell;

the second image acquisition device is arranged on the shell and used for acquiring a second type of image outside the shell; and

pulse width modulation means for outputting a pulse signal;

the first image acquisition device and the second image acquisition device work simultaneously when acquiring preset signals in the pulse signals.

2. The cleaning apparatus as recited in claim 1, further comprising:

and the processing device is used for carrying out synthesis processing on the first type of image and the second type of image to generate a synthetic image.

3. The cleaning apparatus of claim 2,

the processing device is further configured to: and adjusting the output frequency of the pulse signal according to the composite image.

4. The cleaning apparatus of claim 2, further comprising:

the moving device is arranged on the shell and used for driving the shell to move;

the processing device is further configured to: and adjusting the output rotating speed of the mobile device according to the synthetic image.

5. The cleaning apparatus of claim 2, further comprising:

and the storage device is arranged on the shell and used for storing the composite image.

6. The cleaning apparatus according to any one of claims 1 to 5,

the first type of image comprises a depth image and the second type of image comprises an RGB image and/or a grayscale image.

7. An image processing method for a cleaning apparatus, comprising:

receiving a pulse signal;

controlling a first image acquisition device and a second image acquisition device to work simultaneously based on a preset signal in the received pulse signal;

the first image acquisition device is used for acquiring a first type of image outside the shell of the cleaning equipment; the second image acquisition device is used for acquiring a second type of image outside the shell of the cleaning equipment.

8. The image processing method according to claim 7, further comprising:

and synthesizing the first type of image and the second type of image to generate a synthesized image.

9. The image processing method according to claim 8, further comprising, after the generating the composite image:

and adjusting the output frequency of the pulse signal according to the composite image.

10. The image processing method according to claim 8, wherein the cleaning device comprises: the mobile device is arranged on the shell and used for driving the shell to move;

the generating of the composite image further comprises:

and adjusting the output rotating speed of the mobile device according to the synthetic image.

11. The image processing method according to claim 8, further comprising, after the generating the composite image:

storing the composite image.

12. The image processing method according to any one of claims 7 to 11,

the first type of image comprises a depth image and the second type of image comprises an RGB image.

13. An image processing apparatus for a cleaning device, characterized by comprising:

the receiving module is used for receiving the pulse signal;

the control module is used for controlling the first image acquisition device and the second image acquisition device to work simultaneously based on a preset signal in the received pulse signal;

14. A cleaning apparatus, comprising: the image processing apparatus according to claim 13.

15. A cleaning apparatus, comprising:

a memory storing programs or instructions;

a controller which, when executing the program or instructions, implements the image processing method of any of claims 7 to 12.

16. A readable storage medium on which a program or instructions are stored, which program or instructions, when executed by a processor, carry out the steps of the image processing method according to any one of claims 7 to 12.