WO2021115185A1

WO2021115185A1 - Warehousing system and related method

Info

Publication number: WO2021115185A1
Application number: PCT/CN2020/133514
Authority: WO
Inventors: 边铁栋
Original assignee: 灵动科技（北京）有限公司
Priority date: 2019-12-13
Filing date: 2020-12-03
Publication date: 2021-06-17
Also published as: CN111105455A; CN111105455B

Abstract

A warehousing system (100), comprising: a first autonomous mobile device (110), a server (130), and an image sensor (140). The first autonomous mobile device (110) is configured to move in a warehouse; the server (130) is electrically connected to the first autonomous mobile device (110) and is configured to monitor the first autonomous mobile device (110); the image sensor (140) is electrically connected to the server (130) and is configured to capture images of the first autonomous mobile device (110), wherein when the first autonomous mobile device (110) is disconnected from the server (130), the server (130) determines the position of the first autonomous mobile device (110) by means of the images of the first autonomous mobile device (110) captured by the image sensor (140). The warehousing system (100) provided in the present solution can locate a failed autonomous mobile device, so that personnel can quickly come to the scene to fix errors, thus preventing lowering work efficiency.

Description

Warehousing system and related methods

Technical field

The present invention relates to a system, in detail, it relates to a storage system and related methods.

Background technique

When an existing autonomous mobile device (such as an unmanned logistics vehicle) moves in a warehouse, it usually relies on the sensor of the autonomous mobile device itself for positioning, and returns the positioning result to the server. However, when the autonomous mobile device fails, disconnects, the battery is too low, and other unexpected conditions (such as sensor interference), the server loses the connection with the autonomous mobile device and cannot know the status of the autonomous mobile device. The positioning of the autonomous mobile device cannot eliminate the situation immediately, resulting in a decrease in work efficiency.

Summary of the invention

One of the objectives of the present invention is to provide a storage system and a method applied to the storage system to solve the above-mentioned problems, for example, when an autonomous mobile device fails, disconnects, power is too low, and other unexpected conditions (such as sensor interference with each other) ), the autonomous mobile device can still be positioned smoothly.

According to an embodiment of the present invention, a storage system is disclosed, including: a first autonomous mobile device, a server, and an image sensor. The first autonomous mobile device is used to move in the warehouse; the server is electrically connected to the first autonomous mobile device and is used to monitor the first autonomous mobile device; the image sensor is electrically connected to the The server is used to capture an image of the first autonomous mobile device; wherein when the connection between the first autonomous mobile device and the server is disconnected, the server captures the image by the image sensor The image of the first autonomous mobile device to determine the location of the first autonomous mobile device.

According to an embodiment of the present invention, the image sensor is integrated on the security camera.

According to an embodiment of the present invention, the server determines the position of the first autonomous mobile device based on the size and position of the feature point in the image and the shooting angle and position of the security camera.

According to an embodiment of the present invention, the characteristic point is the whole or one of the components of the first autonomous mobile device or an identification mark on the body of the first autonomous mobile device.

According to an embodiment of the present invention, the image sensor is further used to capture the image of the second autonomous mobile device, and the server is also used to capture the image of the second autonomous mobile device and the image captured by the image sensor. The location of the second autonomous mobile device and the location of the security camera are used to optimize the accuracy of determining the location of the first autonomous mobile device.

According to an embodiment of the present invention, the image sensor is disposed on a second autonomous mobile device, wherein the second autonomous mobile device is electrically connected to the server.

According to an embodiment of the present invention, the second autonomous mobile device senses the distance between the first autonomous mobile device and the second autonomous mobile device through a sensing device, and the server uses the first autonomous mobile device. The distance between the mobile device and the second autonomous mobile device and the location of the second autonomous mobile device are used to determine the location of the first autonomous mobile device.

According to an embodiment of the present invention, the sensing device includes at least one of an optical radar, a depth camera, an ultrasonic sensor, and an infrared sensor.

According to an embodiment of the present invention, the server is further configured to transmit information of a plurality of autonomous mobile devices electrically connected to the second autonomous mobile device, when the second autonomous mobile device passes through the image sensor When the first autonomous mobile device is detected and the first autonomous mobile device is not among the plurality of autonomous mobile devices, the second autonomous mobile device notifies the server.

According to an embodiment of the present invention, when the connection between the first autonomous mobile device and the server is disconnected, the server is further configured to identify the person appearing in the image through the image And notify the person to return the location of the first autonomous mobile device.

According to an embodiment of the present invention, the server determines the person's identity based on the person's appearance characteristics or the identification mark worn by the person.

According to an embodiment of the present invention, a method applied to a warehouse system is disclosed. The method includes: monitoring the movement of an autonomous mobile device in the warehouse through a server; When the connection is disconnected, the image of the autonomous mobile device is captured by an image sensor; and the location of the autonomous mobile device is determined by the image of the autonomous mobile device captured by the image sensor.

According to an embodiment of the present invention, a storage system is disclosed. The storage system includes a first autonomous mobile device and a second autonomous mobile device, a server, and an image sensor. The first autonomous mobile device and the second autonomous mobile device are used for moving in a warehouse. The server is electrically connected to the first autonomous mobile device and the second autonomous mobile device, and is used to monitor the first autonomous mobile device and the second autonomous mobile device. The image sensor is integrated in the second autonomous mobile device, and is used to capture an image of the first autonomous mobile device. When the connection between the first autonomous mobile device and the server is disconnected and the image sensor captures the image of the first autonomous mobile device, the second autonomous mobile device determines that the first autonomous mobile device The location of the mobile device, and the location of the first autonomous mobile device is transmitted to the server.

The storage system proposed by the present invention can locate the fault of the autonomous mobile device, so that personnel can quickly come to the scene to remove obstacles and avoid lowering work efficiency.

Description of the drawings

Fig. 1 is a schematic diagram of a storage system according to an embodiment of the present invention.

FIG. 2 is a schematic diagram of a display screen of a server display according to an embodiment of the present invention.

Fig. 3 is a schematic diagram of an autonomous mobile device according to an embodiment of the present invention.

4 is a schematic diagram of a display screen of a server display according to another embodiment of the present invention.

FIG. 5 is a schematic diagram of positioning a first autonomous mobile device through a security camera according to an embodiment of the present invention.

Fig. 6 is a schematic diagram of an autonomous mobile device for positioning according to an embodiment of the present invention.

FIG. 7 is a schematic diagram of positioning a first autonomous mobile device through a second autonomous mobile device according to an embodiment of the present invention.

Fig. 8 is a schematic diagram of an identification person according to an embodiment of the present invention.

Fig. 9 is a system block diagram of a storage system according to an embodiment of the present invention.

Fig. 10 is a flowchart of a method applied to a storage system according to an embodiment of the present invention.

Detailed ways

The following disclosure provides multiple implementations or examples, which can be used to realize different features of the disclosure. The specific examples of components and configurations described below are used to simplify the present disclosure. When it is conceivable, these narratives are only examples, and they are not intended to limit the content of this disclosure. For example, in the following description, forming a first feature on or on a second feature may include some embodiments where the first and second features are in direct contact with each other; and may also include In some embodiments, additional components are formed between the above-mentioned first and second features, so that the first and second features may not be in direct contact. In addition, the present disclosure may reuse component symbols and/or labels in multiple embodiments. Such repeated use is based on the purpose of brevity and clarity, and does not in itself represent the relationship between the different embodiments and/or configurations discussed.

Furthermore, the use of spatially relative terms here, such as "below", "below", "below", "above", "above" and similar, may be used to facilitate the description of the drawing in the figure The relationship between one component or feature relative to another component or feature is shown. The original meaning of these spatially-relative vocabulary covers not only the orientation shown in the figure, but also the various orientations of the device in use or operation. The device may be placed in other orientations (for example, rotated 90 degrees or in other orientations), and these spatially-relative description vocabulary should be explained accordingly.

Although the numerical ranges and parameters used to define the broader scope of the present application are approximate numerical values, the relevant numerical values in the specific embodiments have been presented here as accurately as possible. However, any value inevitably contains standard deviations due to individual test methods. Here, "about" usually means that the actual value is within plus or minus 10%, 5%, 1%, or 0.5% of a specific value or range. Or, the term "about" means that the actual value falls within the acceptable standard error of the average value, depending on the consideration of a person with ordinary knowledge in the technical field to which this application belongs. It should be understood that all ranges, quantities, values and percentages used herein (for example, used to describe the amount of material, time length, temperature, operating conditions, quantity ratio and other Similar ones) have been modified by "about". Therefore, unless otherwise stated to the contrary, the numerical parameters disclosed in this specification and the accompanying patent scope are approximate values and can be changed according to requirements. At least these numerical parameters should be understood as the indicated effective number of digits and the value obtained by applying the general carry method. Here, the numerical range is expressed from one end point to the other end point or between the two end points; unless otherwise specified, the numerical range described here includes the end points.

At present, autonomous mobile devices on the market (such as unmanned logistics vehicles) are often used in warehousing for cargo transportation. Autonomous mobile devices rely on their own sensors, such as distance sensors, optical radar (Light Laser Detection and Ranging, LiDAR), and depth sensors. Camera (or RGBD camera), etc., to achieve obstacle avoidance, movement and other functions. In addition, autonomous mobile devices can also locate themselves based on the above-mentioned sensors, and rely on communication devices to transmit location information to the server for control. However, when the autonomous mobile device fails, disconnects, the battery is too low, and other unexpected conditions (such as sensor interference), the server cannot know the status of the autonomous mobile device, and because the warehouse occupies a large area, once the autonomous mobile device If you lose the connection, you can only rely on manual methods to slowly search for faulty autonomous mobile devices, resulting in reduced work efficiency. Therefore, the present invention provides a storage system and a method applied to the storage system to solve the above-mentioned problems.

FIG. 1 is a schematic diagram of a storage system 100 according to an embodiment of the present invention. The storage system 100 includes autonomous mobile devices for moving in the warehouse, for example, the first autonomous mobile device 110 and the second autonomous mobile device 120 in FIG. 1. In this embodiment, the first autonomous mobile device 110 and the second autonomous mobile device 120 are unmanned transport vehicles, which are used to move goods between storage shelves, such as the shelves H1 to H4 in the figure. Handling. In order to move autonomously and smoothly carry goods, the first autonomous mobile device 110 and the second autonomous mobile device 120 have sensors, such as distance sensors, optical radars, depth cameras, etc., to implement obstacle avoidance and movement functions. In addition, the first autonomous mobile device 110 and the second autonomous mobile device 120 can also be positioned using the above-mentioned sensors.

The storage system 100 further includes a server 130 and an image sensor 140. The server 130 is electrically connected to the first autonomous mobile device 110 and the second autonomous mobile device 120, wherein the first autonomous mobile device 110 and the second autonomous mobile device 120 can communicate with the server 130 through the communication device provided therein , And return positioning information to the server 130, so that the server 130 can manage and monitor autonomous mobile devices, control cargo handling status, order status, and so on. In addition, when an autonomous mobile device (such as the first autonomous mobile device 110 or the second autonomous mobile device 120) fails or has a connection problem, the server 130 can find and solve the problem in time. In this embodiment, the server 130 is not limited to be installed in the warehouse, and the server 130 can also be connected to the autonomous mobile device and the image sensor 140 remotely. The image sensor 140 is installed in the warehouse and is electrically connected to the server 130. The image sensor 140 is used to capture images in the warehouse. In this embodiment, the image sensor 140 is mainly used to capture images of autonomous mobile devices. In detail, when an autonomous mobile device (such as the first autonomous mobile device 110 or the second autonomous mobile device 120) fails, the server 130 When the connection with the failed autonomous mobile device is disconnected, the server 130 can capture the image of the failed autonomous mobile device through the image sensor 140 installed in the warehouse, and determine the failed autonomous mobile device based on the image captured by the image sensor 140 Location in order to troubleshoot in time.

However, the present invention is not limited to be applied only when the connection between the autonomous mobile device and the server 130 is disconnected. In other embodiments, when the server 130 determines that the data returned by the autonomous mobile device is incorrect, when the server 130 The server 130 determines whether an unreasonable change in the position of the autonomous mobile device in a short period of time or other abnormal conditions determined by the server 130 can apply the present invention. For example, when the server 130 determines that the position returned by the autonomous mobile device (such as the first autonomous mobile device 110 or the second autonomous mobile device 120) is obviously not a position that the autonomous mobile device can reach, the server 130 can set The image sensor 140 in the warehouse captures the image of the autonomous mobile device to determine the location of the autonomous mobile device. For example, from the information returned by the autonomous mobile device, the server 130 observes that the power of the autonomous mobile device is unreasonably reduced. The server 130 can capture the autonomous mobile device through the image sensor 140 installed in the warehouse. Image to determine the location of the autonomous mobile device and determine the state of the autonomous mobile device. When necessary, personnel can be notified immediately to be present.

In general, the present invention uses the server 130 to capture the image of the autonomous mobile device through the image sensor 140 arranged in the warehouse, so that the abnormal condition of the image sensor 140 can be quickly eliminated, thereby avoiding the reduction of work efficiency.

It should be noted that the above-mentioned "electrical connection" is not limited to the physical connection through a physical circuit. If the two communicate wirelessly to transmit signals, it can also belong to the category of electrical connection. For example, the image sensor 140 may transmit the captured image to the server 130 via a cable. In another example, the image sensor 140 can transmit the captured image to the server 130 through the access point, so as to realize wireless communication.

FIG. 2 shows a screen observed from the server 130 when the server 130 is normally connected to the autonomous mobile device. FIG. 2 takes the first autonomous mobile device 110 as an example. The server 130 can display the status of the first autonomous mobile device 110 on the display, for example, the model of the first autonomous mobile device 110, whether it is normally connected to the server 130, the latest connection time and task information, the current power and Firmware version and so on. The server 130 may also display the positioning information returned by the first autonomous mobile device 110 on the display, for example, display that the first autonomous mobile device 110 is currently between the shelf H2 and the shelf H3. In addition, the server 130 can also display real-time live images captured by the sensor of the first autonomous mobile device 110 on the display, so as to locate the first autonomous mobile device 110 more accurately.

It should be noted that the present invention does not limit the information displayed on the display of the server 130, and the information displayed on the display of the server 130 may be different according to actual application requirements.

Fig. 3 is a schematic diagram of an autonomous mobile device according to an embodiment of the present invention. FIG. 3 uses the first autonomous mobile device 110 as an example for illustration. As described in the embodiment in FIG. 1, the first autonomous mobile device 110 is an unmanned logistics vehicle for carrying goods. The first autonomous mobile device 110 includes a sensing device 301, a processor 302, a communication device 303, a driving component 304, a column 305, a display device 306, and a body 307. The sensing device 301 includes an image sensor 3011 arranged on the pillar 305 and a distance sensor 3012 arranged in front of the body 307. The image sensor 3011 is used to capture an image of an object. The distance sensor 3012 is used to sense the distance between the object and the first autonomous mobile device 110. In this embodiment, the image sensor 3011 can be realized by a general camera, and the distance sensor 3012 can be realized by one of a depth camera, a lidar, an ultrasonic sensor, and an infrared sensor, or a combination of multiple of them. The present invention does not Limited by this.

The processor 302 is disposed in the first autonomous mobile device 110, and is used to receive the information sensed by the sensing device 301, and make a judgment accordingly. For example, the processor 302 determines the travel route of the first autonomous mobile device 110 after receiving the image captured by the image sensor 3011, and controls the first autonomous mobile device 110 to proceed toward the travel route to complete cargo transportation. For another example, in order to effectively avoid obstacles, after the processor 302 receives the distance information between the obstacle sensed by the distance sensor 3012 and the first autonomous mobile device 110, it determines that the first autonomous mobile device 110 should avoid the obstacle. The time and distance of the obstacle, and control the first autonomous mobile device 110 to bypass the obstacle to avoid the obstacle.

The communication device 303 is used for wireless communication with the server 130. In detail, the first autonomous mobile device 110 can transmit its own positioning information to the first autonomous mobile device 110 through the communication device 303. In addition, the first autonomous mobile device 110 can receive instructions sent by the server 130 through the communication device 303. It should be noted that the type of communication between the first autonomous mobile device 110 and the server 130 is not limited in the present invention. For example, the communication between the first autonomous mobile device 110 and the server 130 may be wireless communication technologies such as Bluetooth, Wi-Fi, and ZIG-BEE. The driving component 304 is used to drive the first autonomous mobile device 110 to move. In this embodiment, the driving assembly 304 includes a motor and a power wheel, and the motor provides kinetic energy to the power wheel to drive the first autonomous moving device 110.

The display device 306 is arranged at the top of the column 305, and the display device 306 can display the task list of the first autonomous mobile device 110, so that the warehouse management personnel can check the information of the goods being carried through the display device 306. In other embodiments, the display device 306 may also display a warning slogan to remind passing people to avoid colliding with the first autonomous mobile device 110. In addition, when the first autonomous mobile device 110 fails, the display device 306 may also display failure information to remind passing personnel to assist in repairs. In some embodiments, when the first autonomous mobile device 110 fails, the display device 306 may display the failure information by means of fixed display or carousel conversion. During the carousel conversion, the display device 306 can alternately display the fault information, the model of the first autonomous mobile device 110, the two-dimensional identification code, and other information.

In the embodiment of FIG. 3, there are multiple identification marks on the body 307. For example, the body 307 may have a two-dimensional identification code corresponding to the first autonomous mobile device 110 affixed on the top surface and the side surface. In addition, the body 307 may also have a model identification (such as the word 110) corresponding to the first autonomous mobile device 110 affixed on the top surface and the side surface. The identification mark on the fuselage 307 can be used by the warehouse staff to identify the model of the first autonomous mobile device 110. In addition to the identification mark on the fuselage 307, the column 305 may also have an identification mark (such as a two-dimensional identification code) for the warehouse staff to identify the model of the first autonomous mobile device 110.

It should be noted that the first autonomous mobile device 110 may also include other components and elements to implement other functions of the first autonomous mobile device 110. For example, the first autonomous mobile device 110 further includes a storage device for storing information, a battery for providing power, and a power distribution module for distributing power to various components. It should be noted that the first autonomous mobile device 110 shown in FIG. 3 is only an example, and the present invention does not limit the detailed structure of the first autonomous mobile device 110. Similarly, the present invention does not limit other autonomous mobile devices in the storage system 100. The detailed architecture of the mobile device.

When the connection between the autonomous mobile device and the server 130 is disconnected due to factors such as failure, disconnection, low battery power, and other unexpected conditions (such as mutual interference of sensors), the server 130 passes through the image sensor set in the warehouse. 140 to capture the image of the failed autonomous mobile device, and determine the location of the failed autonomous mobile device based on the image captured by the image sensor 140. Referring to FIG. 4, FIG. 4 shows a screen viewed from the server 130 when the server 130 is disconnected from the autonomous mobile device. FIG. 4 takes the first autonomous mobile device 110 as an example. When the connection between the first autonomous mobile device 110 and the server 130 is disconnected, the server 130 may display a prompt message on the display to notify the administrator that the first autonomous mobile device 110 has an error. In addition, the server 130 determines the position of the first autonomous mobile device 110 through the image of the first autonomous mobile device 110 captured by the image sensor 140, and displays the determined position on the display. At the same time, the server 130 also displays on-site real-time images captured by the image sensor 140 on the display.

In this way, even if the first autonomous mobile device 110 is unable to actively provide positioning information to the server 130 due to various possible factors, the server 130 can still locate the position of the first autonomous mobile device 110 to avoid low work efficiency.

In an embodiment of the present invention, the image sensor 140 is integrated in the security camera. As a result, when the connection between the autonomous mobile device and the server 130 is disconnected due to factors such as failure, disconnection, low battery power, and other unexpected conditions (such as mutual interference of sensors), the server 130 is installed in the warehouse. The security camera in the computer captures the image of the malfunctioning autonomous mobile device and determines the location of the malfunctioning autonomous mobile device. Referring to FIG. 5, FIG. 5 is a schematic diagram of positioning the first autonomous mobile device 110 through a security camera according to an embodiment of the present invention. In the embodiment of FIG. 5, the storage system 100 further includes a plurality of security cameras arranged in the storage. For example,

security cameras

510, 520, 530, 540, and 550 are installed in a warehouse, and the image sensor 140 in the above-mentioned embodiment is integrated into each of the

security cameras

510, 520, 530, 540, and 550.

Under normal circumstances, the

security cameras

510, 520, 530, 540, and 550 are used as monitors to monitor the storage to achieve security. When the connection between the first autonomous mobile device 110 and the server 130 is disconnected due to factors such as failure, disconnection, low battery, and other unexpected conditions (such as mutual interference of sensors), the server 130 starts to browse the security camera 510, Each of 520, 530, 540, and 550 searches for the first autonomous mobile device 110. For example, the server 130 captures the image of the first autonomous mobile device 110 in the surveillance image of the security camera 530, and the security camera 530 is set to shoot the image between the shelf H2 and the shelf H3. Therefore, the server 130 determines that the first autonomous mobile device 110 is located between the shelf H2 and the shelf H3.

Then, the server 130 determines the accurate position of the first autonomous mobile device 110 based on the size and position of the feature points of the first autonomous mobile device 110 in the image and the shooting angle of the security camera 530. Fig. 6 is a schematic diagram of an autonomous mobile device for positioning according to an embodiment of the present invention. In the embodiment of FIG. 6, the characteristic point is the identification mark on the body 307 or the column 305, and the server 130 determines the first autonomous mobile device according to the size and position of the identification mark in the image and the shooting angle of the security camera 530 The exact location of 110.

In detail, the server 130 determines the actual distance between the first autonomous mobile device 110 and the security camera 530 according to the pixel size of the feature point in the image. For example, if the width of the two-dimensional identification code of the fuselage 307 occupies 55 to 60 pixels in the image, it can be inferred that the two-dimensional identification code is about 6 meters away from the security camera 530; the width of the two-dimensional identification code of the fuselage 307 is The image occupies 65 to 70 pixels in size, it can be inferred that the two-dimensional identification code is about 5 meters away from the security camera 530. Different cameras, different hardware and on-site environment configurations may affect the actual distance judgment between the main mobile device 110 and the security camera 530. The example here only means that the server 130 can make judgments based on feature points. The size of the pixels occupied is inversely proportional to the actual distance between the main mobile device 110 and the security camera 530. In this embodiment, the actual distance corresponding to the pixel size occupied by the two-dimensional identification code in the image can be entered in the server 130 in advance, so that the server 130 can pass through when detecting the image captured by the security camera 530 The actual distance between the two-dimensional identification code and the security camera 530 is obtained by means of interpolation. It should be noted that the size of the pixels occupied by the two-dimensional identification code in the image and the corresponding actual distance are only examples. The actual correspondence depends on the resolution of the security camera 530 (or more accurately the image sensor 140) and other reasons.

Then, the server 130 determines the relative position of the first autonomous mobile device 110 according to the position of the feature point in the image and the shooting angle of the security camera 530. For example, the shooting angle of the security camera 530 is downward deflection θ, and the server 130 determines that the first autonomous mobile device 110 is located between the aisle between the shelf H2 and the shelf H3, and the two-dimensional recognition of the body 307 at this time The code position is in the lower right corner of the image. Therefore, the server 130 can determine that the first autonomous mobile device 110 is located relatively close to the shelf H2.

In summary, the server 130 determines the accurate position of the first autonomous mobile device 110 according to the size and position of the feature points of the first autonomous mobile device 110 in the image and the shooting angle and position of the security camera 530.

It should be noted that the feature points are not limited to the identification marks on the fuselage 307 or the pillar 305. In other embodiments of the present invention, the characteristic point may be the whole of the first autonomous mobile device 110 or one of its components. For example, one of the pillar 305, the display device 306, or the body 307 of the first autonomous mobile device 110 is used as a feature point. The server 130 can infer the distance between the first autonomous mobile device 110 and the security camera 530 according to the pixel size occupied by the whole of the first autonomous mobile device 110 or one of its components in the image. Those with ordinary knowledge in the art should be able to easily understand the details of judging the distance between the first autonomous mobile device 110 and the security camera 530 according to different feature points, and the detailed description is omitted here to save space.

As described in the embodiment of FIG. 3, when the first autonomous mobile device 110 fails, the display device 306 may also display the failure information of the first autonomous mobile device 110 to remind passing personnel to assist in repairs. Therefore, the server 130 can not only obtain the position of the first autonomous mobile device 110 from the size and position of the feature point in the image captured by the security camera 530, but can also directly observe the fault information displayed by the display device 306 from the image. In this way, the cause of the failure of the first autonomous mobile device 110 is known. In this way, the server 130 can more effectively eliminate the failure of the first autonomous mobile device 110, reduce the time required for personnel to be present for diagnosis, and avoid a decrease in work efficiency. For example, if the first autonomous mobile device 110 cannot maintain the connection with the server 130 because the battery is too low, and then stops traveling, the display device 306 can permanently display the information that the battery is too low. In this way, after observing the screen displayed by the server 130, the management personnel can directly dispatch the personnel to carry the battery or charging device to the first autonomous mobile device 110 for troubleshooting.

Those with ordinary knowledge in the field should be able to easily understand that if more than two security cameras capture the images of the malfunctioning autonomous mobile device at the same time, the server 130 can refer to the images captured by multiple security cameras to determine the autonomy of the malfunction. The location of the mobile device can improve the accuracy of the judgment.

In order to optimize the accuracy of the judgment, the server 130 can perform the calibration through a normally operating autonomous mobile device. For example, when the second autonomous mobile device 120 is operating normally, the second autonomous mobile device 120 may transmit the positioning information to the server 130 through its own sensing device and communication device. In this way, when the second autonomous mobile device 120 moves in the warehouse and the image is captured by the security camera 530, according to the method mentioned in the above-mentioned embodiment, the server 130 first uses the feature points of the second autonomous mobile device 120 in the image The size and position in and the shooting angle and position of the security camera 530 are used to determine the position of the second autonomous mobile device 120. At the same time, the positioning information sent to the server 130 by the second autonomous mobile device 120 is compared with the determined position to correct the determination of the server 130. The correction process can be simply summarized into the following formula:

P _AMR (x,y)=f(α·P _camera ,β·Angle,γ·Depth,κ)

Where f() is a certain function, P _AMR is the positioning information sent by the second autonomous mobile device 120 itself to the server 130, P _camera is the setting position of the security camera 530, Angle is the shooting angle of the security camera 530, and Depth is the servo The distance between the security camera 530 and the second autonomous mobile device 120 determined by the device 130, α, β, γ, and κ are constants. The server 130 adjusts the constants α, β, γ, and κ to make the function value of f() close to the positioning information _PAMR sent to the server 130 by the second autonomous mobile device 120 itself, thereby achieving a correction effect. In this way, when the second autonomous mobile device 120 autonomously moves in the warehouse, the server 130 can perform corrections by capturing each security camera of the second autonomous mobile device 120.

In another embodiment of the present invention, the image sensor 140 is integrated in the autonomous mobile device. In other words, the image sensor 140 is an image sensor in the sensing device of the autonomous mobile device. In this way, when the connection between the autonomous mobile device and the server 130 is disconnected due to factors such as failure, disconnection, low battery power, and other unexpected conditions (such as mutual interference of sensors), the server 130 is set to another The image sensor on the autonomous mobile device captures the image of the failed autonomous mobile device and determines the location of the failed autonomous mobile device.

FIG. 7 is a schematic diagram of positioning the first autonomous mobile device 110 through the second autonomous mobile device 120 according to an embodiment of the present invention. The detailed architecture of the second autonomous mobile device 120 can be referred to but not limited to the architecture shown in FIG. 3. When the second autonomous mobile device 120 is operating normally, the second autonomous mobile device 120 can transmit positioning information to the server 130 through its own sensing device and communication device, and at the same time move between shelves to perform the task of transporting goods . When the malfunctioning first autonomous mobile device 110 appears in the sensing range of the sensing device of the second autonomous mobile device 120, the second autonomous mobile device 120 senses the first autonomous movement through a sensing device (such as a distance sensor) The distance between the device 110 and the second autonomous mobile device 120, and the distance between the first autonomous mobile device 110 and the second autonomous mobile device 120 are transmitted to the server 130. In this way, the server 130 can determine the location of the first autonomous mobile device 110 according to the distance between the first autonomous mobile device 110 and the second autonomous mobile device 120 and the positioning information transmitted by the second autonomous mobile device 120.

However, this is not a limitation of the present invention. In other embodiments, when the faulty first autonomous mobile device 110 appears in the sensing range of the sensing device of the second autonomous mobile device 120, the second autonomous mobile device 120 passes through the sensing device (such as a distance sensor). After sensing the distance between the first autonomous mobile device 110 and the second autonomous mobile device 120, the second autonomous mobile device 120 uses its own processor to determine the location of the failed first autonomous mobile device 110, and calculates the location of the failed first autonomous mobile device 110. The location of the mobile device 110 is returned to the server 130. After the server 130 receives the location of the failed first autonomous mobile device 110, it can immediately dispatch personnel to the scene to eliminate the abnormal situation.

In this embodiment, the server 130 can transmit the information of all autonomous mobile devices that are currently electrically connected to each autonomous mobile device. In this way, when the sensing device of the second autonomous mobile device 120 senses that the first autonomous mobile device 110 is disconnected from the server 130 due to a failure, the first autonomous mobile device 110 is no longer in contact with the server 130. Therefore, the second autonomous mobile device 120 can determine that the first autonomous mobile device 110 is faulty and notify the server 130 accordingly. For example, when the sensing device of the second autonomous mobile device 120 senses the first autonomous mobile device 110, it can learn the first autonomous mobile device by scanning the identification mark (such as a two-dimensional identification code) of the first autonomous mobile device 110. The model of the device 110. Then, after comparing the model of the first autonomous mobile device 110 with the information list sent by the server 130, it is found that the first autonomous mobile device 110 is not in the information list, and the second autonomous mobile device 120 can determine that the first autonomous mobile device 110 is already Disconnect from the server 130, and then notify the server 130.

In order to more effectively eliminate the cause of the failure of the first autonomous mobile device 110, if the image of the first autonomous mobile device 110 captured by the image sensor 140 simultaneously captures a worker, the server 130 can identify the work by means of image recognition The identity of the personnel and instruct the personnel to eliminate the cause of the failure of the first autonomous mobile device 110, thereby reducing the time for dispatching personnel to arrive and avoiding a decrease in work efficiency.

Fig. 8 is a schematic diagram of an identification person according to an embodiment of the present invention. When the first autonomous mobile device 110 is disconnected from the server 130, the server 130 uses the image of the first autonomous mobile device 110 captured by the image sensor 140 to determine the location of the first autonomous mobile device 110. In this embodiment, in the image captured by the image sensor 140, a worker is also captured near the first autonomous mobile device 110 at the same time. The server 130 recognizes the identity of the worker through the appearance (such as skeleton or face) of the person registered in the server 130 in advance, and instructs the worker to eliminate the cause of the failure of the first autonomous mobile device 110. In another embodiment, the server 130 may identify the worker's identity in other ways. For example, if the worker wears an identification mark (for example, the employee number B24 is embroidered on the clothes), the server 130 can identify the worker's identity through the identification mark.

The storage system of the present invention is summarized in Fig. 9 to facilitate the understanding of the present invention. 9 is a system block diagram of a storage system according to an embodiment of the present invention. Take the storage system 100 as an example. The storage system 100 includes a plurality of autonomous mobile devices that move in the warehouse, such as a first autonomous mobile device 110 and a second autonomous mobile device.装置120. The autonomous mobile device includes a sensing device 301, a processor 302, a communication device 303, a driving component 304, and a display device 306. The sensing device 301 is used to sense the distance of objects and capture images to realize the functions of moving and avoiding obstacles; the processor 302 is used to process the information sensed by the sensing device 301; the driving component 304 is used to drive the autonomous mobile device to move; The device 306 is used to display task information of the autonomous mobile device. The storage system 100 further includes a server 130 and an image sensor 140. The server 130 is electrically connected to the plurality of autonomous mobile devices, and the plurality of autonomous mobile devices can communicate with the server 130 through a communication device (such as the communication device 303) provided therein, and return positioning information to The server 130 enables the server 130 to manage and monitor autonomous mobile devices, control cargo handling status, order status, and so on. In addition, when the autonomous mobile device fails or has a connection problem, the server 130 can find and solve the problem in time. The image sensor 140 is electrically connected to the server 130, and the image sensor 140 is disposed in the warehouse, and is used to capture images in the warehouse. In this embodiment, the image sensor 140 is mainly used to capture the image of the autonomous mobile device. Specifically, when the connection between the server 130 and the autonomous mobile device (such as the first autonomous mobile device 110) is disconnected, the server 130 can capture the image of the first autonomous mobile device 110 through the image sensor 140 installed in the warehouse, and thereby determine the location of the first autonomous mobile device 110, so as to eliminate the situation in time.

FIG. 10 is a flowchart of a method 400 applied to a storage system according to an embodiment of the present invention. Provided that substantially the same result can be obtained, the present invention is not limited to be implemented completely in accordance with the process steps shown in FIG. 10. Method 400 can be summarized as follows:

Step 410: Monitor the movement of the autonomous mobile device in the warehouse through the server.

Step 420: When the connection between the autonomous mobile device and the server is disconnected, capture an image of the autonomous mobile device through an image sensor.

Step 430: Determine the position of the autonomous mobile device through the image of the autonomous mobile device captured by the image sensor.

Those skilled in the art should be able to easily understand the operation flow of the method 400 after reading the above-mentioned embodiments, and the detailed description is omitted here to save space.

The features of several embodiments are summarized above, so that those skilled in the art can better understand the aspects of the present application. Those skilled in the art should understand that they can easily use this application as a basis for designing or modifying other processes and structures for carrying out the same purpose and/or achieving the same advantages of the embodiments introduced herein. Those skilled in the art should also realize that these equivalent structures do not depart from the spirit and scope of the disclosure and that various changes, substitutions and alterations can be made in this text without departing from the spirit and scope of the disclosure.

Furthermore, the scope of the present application is not intended to be limited to the specific embodiments of the processes, machines, manufacturing, material composition, components, methods, and steps described in this specification. Those of ordinary skill in the art will easily understand based on the disclosure of the present disclosure, and can use the current existing or subsequent developments that substantially perform the same functions as the corresponding embodiments described herein or substantially achieve the same results according to the present disclosure. Process, machine, manufacturing, material composition, component, method or step. Therefore, the appended claims intend to include these processes, machines, manufacturing, material compositions, components, methods, or steps within their scope.

Claims

A storage system, characterized in that it comprises:

The first autonomous mobile device, which is used to move in the warehouse;

A server electrically connected to the first autonomous mobile device and used for monitoring the first autonomous mobile device; and

An image sensor, electrically connected to the server, for capturing an image of the first autonomous mobile device;

Wherein when the connection between the first autonomous mobile device and the server is disconnected, the server determines the first autonomous mobile device based on the image of the first autonomous mobile device captured by the image sensor The location of the mobile device.
The storage system of claim 1, wherein the image sensor is integrated on a security camera.
The storage system of claim 2, wherein the server determines the size and position of the feature point in the image and the shooting angle and position of the security camera to determine the status of the first autonomous mobile device position.
The storage system according to claim 3, wherein the characteristic point is the whole or one of the components of the first autonomous mobile device or an identification mark on the body of the first autonomous mobile device.
The storage system of claim 2, wherein the image sensor is also used to capture an image of a second autonomous mobile device, and the server is also used to capture the second autonomous mobile device captured by the image sensor. The image of the mobile device, the position of the second autonomous mobile device, and the shooting angle and position of the security camera are used to optimize the accuracy of determining the position of the first autonomous mobile device.
The storage system of claim 1, wherein the image sensor is disposed on a second autonomous mobile device, wherein the second autonomous mobile device is electrically connected to the server.
The storage system of claim 6, wherein the second autonomous mobile device senses the distance between the first autonomous mobile device and the second autonomous mobile device through a sensing device, and the server The location of the first autonomous mobile device is determined by the distance between the first autonomous mobile device and the second autonomous mobile device and the location of the second autonomous mobile device.
8. The storage system of claim 7, wherein the sensing device includes at least one of an optical radar, a depth camera, an ultrasonic sensor, and an infrared sensor.
The storage system according to claim 6, wherein the server is further configured to transmit information of a plurality of autonomous mobile devices electrically connected to the second autonomous mobile device, and when the second autonomous mobile device When the device detects the first autonomous mobile device through the image sensor and the first autonomous mobile device is not among the plurality of autonomous mobile devices, the second autonomous mobile device notifies the server.
The storage system according to claim 1, wherein when the connection between the first autonomous mobile device and the server is disconnected, the server is also used to identify the presence of The identity of the person in the image is notified, and the person is notified to return the location of the first autonomous mobile device.
The storage system according to claim 10, wherein the server determines the identity of the person based on the appearance characteristics of the person or the identification mark worn by the person.
A method applied to a storage system, characterized in that it comprises:

Monitor the movement of autonomous mobile devices in the warehouse through the server;

When the connection between the autonomous mobile device and the server is disconnected, capturing an image of the autonomous mobile device through an image sensor; and

The location of the autonomous mobile device is determined by the image of the autonomous mobile device captured by the image sensor.
A storage system, characterized in that it comprises:

The first autonomous mobile device and the second autonomous mobile device are used to move in the warehouse;

A server electrically connected to the first autonomous mobile device and the second autonomous mobile device, and is used to monitor the first autonomous mobile device and the second autonomous mobile device; and

An image sensor integrated in the second autonomous mobile device and used to capture an image of the first autonomous mobile device;

Wherein when the connection between the first autonomous mobile device and the server is disconnected and the image sensor captures the image of the first autonomous mobile device, the second autonomous mobile device determines that the first autonomous mobile device The location of the autonomous mobile device is transmitted, and the location of the first autonomous mobile device is transmitted to the server.