CN113382203A - AGV goods falling detection method and system based on artificial intelligence - Google Patents
AGV goods falling detection method and system based on artificial intelligence Download PDFInfo
- Publication number
- CN113382203A CN113382203A CN202110549331.0A CN202110549331A CN113382203A CN 113382203 A CN113382203 A CN 113382203A CN 202110549331 A CN202110549331 A CN 202110549331A CN 113382203 A CN113382203 A CN 113382203A
- Authority
- CN
- China
- Prior art keywords
- goods
- falling
- agv
- distance
- probability
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 31
- 238000013473 artificial intelligence Methods 0.000 title claims abstract description 22
- 230000008859 change Effects 0.000 claims abstract description 38
- 238000000034 method Methods 0.000 claims abstract description 30
- 238000012886 linear function Methods 0.000 claims abstract description 17
- 238000004458 analytical method Methods 0.000 claims description 13
- 238000012545 processing Methods 0.000 claims description 7
- 230000001788 irregular Effects 0.000 claims description 6
- 230000006870 function Effects 0.000 description 38
- 230000008569 process Effects 0.000 description 12
- 238000013507 mapping Methods 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 4
- 238000012549 training Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000004888 barrier function Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012856 packing Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
- G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the technical field of artificial intelligence, in particular to an AGV goods falling detection method and system based on artificial intelligence. According to the method, when the goods fall, the falling probability of the goods is obtained according to the stability degree and the position deviation degree of the goods relative to an AGV in each frame of image; obtaining a change curve function changing along with time according to the falling probability in continuous multi-frame images; and establishing a linear function related to the stable falling trend of the goods, and judging the reason for falling the goods according to the deviation degree of the maximum falling probability in the corresponding time period in the change curve function and the linear function. The reason that the goods dropped is analyzed according to the change degree of the goods probability of dropping, and corresponding measures can be taken in time according to the reason that the goods dropped so as to prevent the possibility that the goods dropped subsequently, thereby reducing the probability that the goods dropped and improving the transport efficiency of the AGV.
Description
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to an AGV goods falling detection method and system based on artificial intelligence.
Background
The AGV is also called an unmanned transport vehicle, an automatic navigation vehicle and a laser navigation vehicle, and is remarkably characterized by being unmanned. AGVs are equipped with automatic guidance systems that ensure that they can automatically follow a predetermined route and automatically transport goods or materials from a starting point to a destination without the need for manual navigation. Another characteristic of AGVs is that the flexibility is good, degree of automation is high and the level of intellectuality is high, and the route of travel of AGVs can change according to the change of storage goods position requirement, production technology flow etc. and the expense that the route of travel changed compares very cheaply with traditional conveyer belt, rigid transmission line.
In practice, the inventors found that the above prior art has the following disadvantages: when the AGV dolly is transporting goods, because factors such as regularity and transportation route environment when the goods loads influence, the condition that the goods dropped easily takes place, especially carton type goods easily cause the damage of incasement article, the goods drops and also can become the route barrier, consequently, through the condition analysis goods that the goods dropped reason, and then takes corresponding measure according to the reason that drops and be the problem that awaits a urgent solution at present.
Disclosure of Invention
In order to solve the technical problems, the invention aims to provide an AGV cargo drop detection method and system based on artificial intelligence, and the adopted technical scheme is as follows:
in a first aspect, an embodiment of the present invention provides an AGV cargo drop detection method based on artificial intelligence, which includes the following specific steps:
when the goods fall, obtaining the falling probability of the goods according to the stability degree and the position deviation degree of the goods relative to the AGV trolley in each frame of image;
obtaining a change curve function changing along with time according to the falling probability in continuous multi-frame images; the continuous multi-frame image is a video image before the collected goods fall;
and establishing a linear function related to the stable falling trend of the goods according to the two corresponding maximum falling probabilities in the starting time period and the ending time period in the maximum falling probability sequence in the time periods in the change curve function, and judging the reason for falling the goods according to the deviation degree of the maximum falling probabilities in the time periods in the change curve function and the linear function.
Further, the method for judging the falling of the goods comprises the following steps:
according to the received change of the sound frequency around the AGV, carrying out target detection on the received continuous multi-frame images so as to identify the AGV, the goods and the color bands;
and determining the suspected dropping distance between the AGV and the goods according to the maximum distance between the coordinate of the AGV and the coordinate of the goods in the video image, and further determining whether the goods drop or not according to the suspected dropping distance.
Further, the determining the reason for the falling of the cargo according to the deviation degree of the maximum falling probability in the corresponding time period between the change curve function and the straight line function includes:
when the deviation degree is smaller than a deviation degree threshold value, judging that the goods fall due to the fact that the goods are loaded in an irregular mode; otherwise, it is determined that the cargo is dropped due to a traveling road surface problem.
Further, when the goods falls, the method further comprises the following steps:
determining a linear equation of the color band according to the pixel points of the color band;
and calculating the distance between the dropped goods and the color band through the coordinates of the central point of the dropped goods and the linear equation, and judging whether the dropped goods block the road traffic according to the distance.
Further, the degree of smoothness is obtained according to the distance between the goods and the center point of the AGV trolley.
Further, the position deviation degree is obtained according to the difference between the distance between the adjacent cargos and the distance between the standard adjacent cargos.
In a second aspect, another embodiment of the present invention provides an AGV cargo drop detection system based on artificial intelligence, which specifically includes:
the probability obtaining unit is used for obtaining the falling probability of the goods according to the stability degree and the position deviation degree of the goods relative to the AGV trolley in each frame of image when the goods fall;
the function establishing unit is used for obtaining a change curve function changing along with time according to the falling probability in continuous multi-frame images; the continuous multi-frame image is a video image before the collected goods fall;
and the reason analysis unit is used for establishing a linear function related to the stable falling trend of the goods according to the corresponding two maximum falling probabilities in the starting time period and the ending time period in the maximum falling probability sequence in a plurality of time periods in the change curve function, and judging the reason for falling the goods according to the deviation degree of the maximum falling probabilities in the corresponding time periods in the change curve function and the linear function.
Further, the probability obtaining unit includes:
the image processing unit is used for carrying out target detection on the received continuous multi-frame images according to the received change of the sound frequency around the AGV trolley so as to identify the AGV trolley, the goods and the color bands;
and the falling detection unit is used for determining the suspected falling distance between the AGV and the goods according to the maximum distance between the coordinates of the AGV and the coordinates of the goods in the video image, and further confirming whether the goods fall or not according to the suspected falling distance.
Further, the determining, by the cause analysis unit, the cause of the falling of the cargo according to the deviation degree of the maximum falling probability in the corresponding time period between the change curve function and the straight line function includes:
when the deviation degree is smaller than a deviation degree threshold value, judging that the goods fall due to the fact that the goods are loaded in an irregular mode; otherwise, it is determined that the cargo is dropped due to a traveling road surface problem.
Further, the probability obtaining unit includes:
the equation acquisition unit is used for determining a linear equation of the color band according to the pixel points of the color band;
and the distance analysis unit is used for calculating the distance between the dropped goods and the color band through the coordinates of the central point of the dropped goods and the linear equation, and further judging whether the dropped goods block the road traffic according to the distance.
The invention has at least the following beneficial effects: the falling probability of the goods is obtained according to the stable degree of the goods relative to the AGV trolley and the position deviation degree of the goods, the falling reasons of the goods are analyzed according to the change degree of the falling probability of the goods, corresponding measures can be taken in time according to the falling reasons of the goods, the possibility that the goods fall in the follow-up process is prevented, the falling probability of the goods is reduced, and the transportation efficiency of the AGV trolley is also improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions and advantages of the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a flowchart illustrating an AGV cargo drop detection method based on artificial intelligence according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating the steps of a method for AGV drop detection based on artificial intelligence according to an embodiment of the present invention;
FIG. 3 is a block diagram of an artificial intelligence based AGV fall detection system according to another embodiment of the present invention;
fig. 4 is a block diagram of a probability obtaining unit according to an embodiment of the present invention.
Detailed Description
To further illustrate the technical means and effects of the present invention for achieving the predetermined objects, the following detailed description of the present invention will be provided with reference to the accompanying drawings and preferred embodiments for an AGV cargo drop detection method and system based on artificial intelligence, and the detailed implementation, structure, features and functions thereof. In the following description, different "one embodiment" or "another embodiment" refers to not necessarily the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
The embodiment of the present invention is described by taking the following application scenario as an example:
the application scene is as follows: the AGV with the goods travels along the guiding direction of the color ribbon, and a plurality of cameras are arranged in the extending direction of the color ribbon and are used for acquiring images of a transport line formed by the color ribbon and the peripheral area of the transport line in the visual field range.
Because the AGV travelling bogie that utilizes the typewriter ribbon to carry out the guide, its goods usually are the article of carton packing, and this goods can be because goods self puts and the phenomenon that drops that the external reason leads to at the in-process of transportation. Therefore, the embodiment of the invention provides an AGV goods falling detection method based on artificial intelligence, which is used for judging the falling reasons of goods according to the acquired image information and the stable trend of the goods in the moving process of the analysis trolley.
The specific scheme of the AGV cargo drop detection method and system based on artificial intelligence provided by the invention is specifically described below with reference to the accompanying drawings.
Referring to fig. 1 and fig. 2, an embodiment of the present invention provides an AGV cargo drop detection method based on artificial intelligence, including:
and S001, when the goods fall, obtaining the falling probability of the goods according to the stability degree and the position deviation degree of the goods in each frame of image relative to the AGV.
Specifically, the embodiment of the invention confirms whether goods fall or not through sound detection and image processing.
In consideration of the fact that abnormal sound is generated if goods fall when the AGV runs along the color band path, the sound sensor is arranged outside the AGV, and therefore sound information of the surrounding environment of the AGV can be collected constantly.
According to the change of the sound frequency around the received AGV, carrying out target detection on the received continuous multi-frame images so as to identify the AGV, goods and color bands, and specifically comprising the following steps: setting a threshold f of sound frequencymaxWhen the sound frequency f of the surrounding environment acquired by the sound sensor is less than the sound frequency threshold fmaxWhen the goods fall, the goods are not considered to fall; otherwise, when the sound frequency f received by the sound sensor is greater than the sound frequency threshold fmaxWhen the AGV is used, the situation that the cargoes fall off possibly exists in the current AGV.
When the sound frequency f detected by the sound sensor in the running process of the AGV trolley is greater than the sound frequency threshold value fmaxWhen the AGV falls, the current AGV sends self position information to the server, triggers the RGB camera covering the visual field range of the position, transmits the video images shot by the RGB camera to the server, and then analyzes continuous multi-frame images in the video images to determine whether goods fall. Wherein, the RGB camera is deployed in the travel path of the AGV trolley to record the transportation state information of the AGV trolley.
It should be noted that (1) the position information of the AGV cart can be obtained by a positioning sensor such as a GPS or a balun sensor.
(2) The starting of the RGB camera is mainly set according to the position information of the AGV, namely when the AGV is located in the visual field range of the current RGB camera, the RGB camera starts to start and shoots and records the transportation state of the current AGV.
3) The RGB camera is deployed according to the current freight scene, the freight transportation environment is divided into different areas, and the RGB camera is deployed at the center positions of the different areas, so that the visual field of the RGB camera can better cover the area to which the RGB camera belongs.
The embodiment of the invention identifies the AGV, the goods and the color band in the received continuous multi-frame images in a semantic segmentation mode.
Preferably, in the embodiment of the present invention, a DNN network with an encoder-decoder structure is used for target detection, and the specific training content is as follows:
1) and marking the data set by taking the collected image containing the AGV trolley, the goods and the color bands as a training data set, wherein the pixel area belonging to the AGV trolley is marked with 1, the pixel area belonging to the goods is marked with 2, the area belonging to the guiding color bands of the trolley is marked with 3, and the rest is marked with 0, wherein 80% of the data set is randomly selected as the training set, and the rest 20% is taken as a verification set.
2) Inputting image data and label data into a DNN network, extracting image characteristics by using an encoder, and converting the number of channels into the number of categories; the height and width of the feature map are then transformed into the size of the input image by a decoder, thereby outputting a class of each pixel.
3) The loss function in the DNN network uses a cross-entropy loss function.
Further, according to the maximum distance between the coordinates of the AGV trolley and the coordinates of the goods in the video image, the suspected falling distance between the AGV trolley and the goods is determined, and then whether the goods fall or not is determined according to the suspected falling distance.
Specifically, the specific method for judging whether the AGV has the goods falling according to the suspected falling distance in the embodiment of the invention comprises the following steps:
1) and extracting the last frame of image in the video image, and detecting the goods, the AGV and the color band by using the DNN.
2) For the cargo pixel point region obtained by the DNN network, a connected domain analysis method is utilized to obtain the minimum external rectangle of each cargo and the central point coordinate (x) corresponding to the minimum external rectanglei，yi) Wherein x isiThe abscissa is the coordinate of the center point of the ith cargo; y isiThe ordinate of the coordinate of the center point of the ith cargo.
3) A connected domain analysis method is also adopted for the pixel point region of the AGV trolley, and then the minimum external rectangle of the AGV trolley and the central point coordinate (x) corresponding to the minimum external rectangle are obtainedagv，yagv)。
4) According to the coordinate (x) of the central point of the minimum circumscribed rectangle of the AGVagv，yagv) And coordinates (x) of the center point of the smallest circumscribed rectangle of the cargoi，yi) Calculating the suspected dropping distance between the AGV trolley and the goodsThe formula for calculating the distance is as follows:
wherein D isiThe suspected dropping distance between the AGV trolley and the ith goods is obtained; and delta is a scaling coefficient of a pixel point in the image relative to the real distance, and is obtained according to the calibration of the RGB camera.
5) Setting a distance threshold value H, and setting a suspected dropping distance D between each cargo and the AGViAll are compared with a distance threshold H, if suspected to fall distance DiIf the distance is smaller than the distance threshold value H, the goods belong to the goods currently carried by the AGV and do not fall; otherwise, the goods are the falling goods, namely the goods fall off from the AGV currently running.
Further, when the goods fall, the falling probability of the goods is obtained according to the stability degree and the position deviation degree of the goods in each frame of image relative to the AGV.
Specifically, the reasons that goods fall off when the AGV trolley runs are mainly divided into external factors and internal factors, wherein the external factors refer to the running road surface problems of the AGV trolley, such as the problems that the AGV trolley collides with other objects in the running process, slips occur, and the ground is concave-convex to cause vibration of a trolley body; the internal factors mean that the goods loading is not standard, namely the goods are not placed stably or regularly during loading, and the instability of the goods is gradually accumulated due to the speed problem of the AGV during the running process until the stability requirement of the goods placement is exceeded, and the goods fall off.
Based on the analysis of the reason for the falling of the goods, the embodiment of the invention analyzes the goods state of the AGV trolley in the running process, and obtains the falling probability of the goods according to the stable degree of the goods in each frame of image relative to the AGV trolley and the position deviation degree of the goods, and the specific process of obtaining the falling probability of the goods is as follows:
1) when the goods fall, video images before the goods fall are collected to obtain continuous multi-frame images.
2) Acquiring each frameThe steady degree of goods for the AGV dolly in the image, this steady degree obtains according to the central point distance between goods and the AGV dolly, specifically is: for the goods on the current AGV trolley, the coordinates of the central point of the minimum external rectangle of all the goods are obtained, and then a central point set A { (x)1，y1)，(x2，y2)......(xi，yi) ... }, according to the coordinates (x) of the center point of each cargoi，yi) And center point coordinates (x) of AGVagv，yagv) Calculating the distance d between each load and the AGViWherein, the calculation formula of the distance is as follows:
further, according to the distance from each goods to the AGV, the stability degree k of the goods relative to the AGV is calculated, and the calculation method comprises the following steps:
wherein d isjThe distance between the jth goods and the AGV is taken as the distance; dj+1The distance between the j +1 th goods and the AGV trolley is set; m is the total number of cargo.
3) And acquiring the position deviation degree of the cargos in each frame of image, wherein the position deviation degree is obtained according to the difference between the distance between the adjacent cargos and the distance between the standard adjacent cargos. In order to better express respective motion conditions of the current AGV trolley and the goods, the embodiment of the invention analyzes according to the motion conditions of the goods to obtain the position change conditions between the adjacent goods, and specifically comprises the following steps: firstly, the distance W between adjacent cargos is obtained according to the coordinates of the central point of the cargosiAnd then obtaining the position deviation degree r of the goods according to the distance between all the goods, wherein the formula for obtaining the position deviation degree r of the goods is as follows:
wherein, Wf_maxIs the distance between standard adjacent goods.
4) The smoothness degree k of the goods relative to the AGV trolley and the position deviation degree r of the goods in each frame of image are mapped through a mapping function f (x) 1-e-xMapping the data to the same interval (0, 1), and further obtaining the dropping probability s of the goods by adopting a weighted summation mode for the mapping values, wherein the calculation method comprises the following steps:
s＝α*f(k)+β*f(r)
wherein alpha is an influence coefficient of the stationary degree; β is an influence coefficient of the degree of positional deviation.
Preferably, in the embodiment of the present invention, α is 0.6, and β is 0.4.
S002, obtaining a change curve function changing along with time according to the falling probability in continuous multi-frame images; the continuous multiframe images are video images of the collected goods before falling.
Specifically, the embodiment of the invention respectively acquires the timestamp corresponding to the current frame image of each frame image, obtains the falling probability of the goods corresponding to the continuous multi-frame image by using the falling probability acquiring process of the goods, then constructs a two-dimensional function by using the time as the abscissa and the falling probability of the goods as the ordinate, and establishes a state curve of the AGV transporting the goods in the current time period, namely a change curve function f of the falling probability by using a data fitting method1(t) the function reflects the variation of the drop probability of the goods increasing with time.
And S003, establishing a linear function related to the stable falling trend of the goods according to the two corresponding maximum falling probabilities in the starting time period and the ending time period in the maximum falling probability sequence in the time periods in the change curve function, and judging the reason for falling the goods according to the deviation degree of the maximum falling probabilities in the time periods in the change curve function and the linear function.
Specifically, according to the embodiment of the invention, the function f of the change curve is subjected to the empirical value of the time period1(t) during this timeSegmenting on the intermediate shaft to obtain a plurality of time periods T1，T2，...，TnAnd obtaining the maximum falling probability of the goods in each time period to form a maximum falling probability sequence(s)1，s2，...，sn}。
Based on a function f of the variation curve1(t) the maximum drop probability obtained, and the embodiment of the invention uses two corresponding maximum drop probability points (t) in the starting time period and the ending time period in the maximum drop probability sequence1，s1) Point (t)n，sn) Reestablishing the straight-line function f relating to the steady tendency of the fall of the goods2(t), the stable falling trend of the goods reflects the probability variation trend of falling of the goods caused by internal factors, and then the maximum falling probability sequence { s }is obtained1，s2，...，snThe mapping probability of the corresponding time on the linear function is calculated, and the change curve function f at the same time is calculated1(t) maximum drop probability and straight function f2(t) to obtain a deviation degree M of the falling probability of the cargo, wherein the deviation degree is calculated by the following formula:
wherein f is2(ti) Is tiMapping probability of time; siIs tiMaximum probability of falling at the moment.
If the falling probability of the AGV transporting goods is gradually increased along with the increase of the time and is a process of steadily increasing, the obtained change curve function f1(t) and the straight-line function f2(t) the deviation degree of the falling probability at the corresponding moment is very small, and the reason for falling the goods at the moment belongs to internal factors; otherwise it belongs to an external factor. Therefore, the embodiment of the present invention sets a deviation threshold MmidWhen the deviation degree M < deviation degree threshold MmidWhen the AGV falls, the falling of the cargos of the current AGV is caused by the fact that the cargos are loaded in an irregular mode, namely caused by internal factorsReminding the goods loader of the AGV car to carry out standard loading; otherwise, it is that the traveling road surface problem caused that the goods dropped that takes place for current AGV dolly, that is to say that external factor causes, need inform the staff to carry out in time inspection to the road surface information in current region to when preventing that follow-up AGV dolly from passing this position region, also take place the goods condition of dropping.
Further, when the AGV is detected to have goods falling, the embodiment of the invention judges whether the current falling goods can influence the operation of the follow-up AGV or not by analyzing the relation between the falling position of the goods and the position of the color band in the image, and takes corresponding measures to carry out relevant processing.
Specifically, the embodiment of the invention judges whether the current dropped goods influence the operation of the subsequent AGV according to the distance between the dropping position and the color belt position, and the specific judgment method comprises the following steps:
1) and performing linear fitting according to the pixel points of the color band to obtain a linear equation of the color band.
And carrying out binarization on the image of the color band area to obtain a binarized image, wherein the pixel value of the pixel point belonging to the color band area is 1, and the pixel values of other pixel points are 0. And thinning the binary image to obtain color band line information only containing single pixels.
Preferably, the refinement method in the embodiment of the present invention adopts Zhang fast parallel refinement, and in other embodiments, a Hilditch refinement algorithm, a Deutch refinement algorithm, and the like may also be adopted.
Performing linear fitting on the obtained single-pixel color band information, fitting a single pixel point belonging to the color band to obtain a linear equation belonging to the color band, wherein the linear equation of the color band is expressed by the following formula:
y＝k*x+b
wherein k is the slope of the color band information in the current image; b is the intercept of the straight line.
It should be noted that, in the embodiment of the present invention, the straight line fitting is performed by using a least square method.
2) The distance between the dropped goods and the color band is calculated through the coordinates of the central point of the dropped goods and a linear equation, and whether the dropped goods influence the transportation efficiency of the AGV trolley is judged according to the distance.
According to the coordinates (x) of the central point of each falling cargoi，yi) Calculating the distance from the dropped goods to the current color band, wherein the specific calculation formula is as follows:
w is the proportion of the pixel point distance to the real distance obtained according to the camera calibration principle; l isiThe distance from the ith dropped cargo to the ribbon.
Further, according to the prior theory, when each AGV runs on the color bars, the maximum distance between the left side and the right side of each AGV and the color bars is PmaxDistance L from each dropped cargo to the ribboniIs at the maximum distance PmaxMaking a comparison if the distance L isiLess than the maximum distance PmaxIf the current dropped goods cause transportation obstacles to the AGV trolley for transporting the goods subsequently, the transportation efficiency of the AGV trolley is influenced; otherwise, the current dropped goods cannot cause any influence.
If the current AGV trolley falling goods are detected to cause line blockage to the follow-up transport trolley, the worker is reminded to recover the relevant falling goods at once according to the position information of the camera corresponding to the falling goods level, and the relevant falling goods are all falling goods seen by the worker on site.
After the working personnel finish the goods recovery, a cache clearing control instruction is generated to the camera at the current position, and the information of a plurality of video images stored in the camera at the current position is deleted, so that the condition that the memory occupancy rate of the camera is too full and the subsequent images cannot be cached is prevented.
In conclusion, the invention provides an AGV goods falling detection method based on artificial intelligence, the method preliminarily judges that goods of an AGV car fall through a sound sensor, simultaneously triggers a camera in a position area of the AGV car to transmit video images to a server, the server determines whether the AGV car has goods falling phenomenon or not by analyzing the distance between the goods in the last frame of image and the AGV car, and further analyzes the reasons of goods falling to take corresponding measures according to the falling probability of the goods obtained by the stability degree of the goods relative to the AGV car and the position deviation degree of the goods in continuous multi-frame images, and further judges whether the falling goods influence the operation of the AGV car or not according to the distance from the falling goods to a color band to take corresponding measures when the falling goods exist. The falling probability of the goods is obtained according to the stable degree of the goods relative to the AGV trolley and the position deviation degree of the goods, the falling reasons of the goods are analyzed according to the change degree of the falling probability of the goods, corresponding measures can be taken in time according to the falling reasons of the goods, the possibility that the goods fall in the follow-up process is prevented, the falling probability of the goods is reduced, and the transportation efficiency of the AGV trolley is also improved.
Based on the same inventive concept as the method, the embodiment of the invention provides an AGV goods falling detection system based on artificial intelligence.
Referring to fig. 3, an embodiment of the present invention provides an AGV cargo drop detection system based on artificial intelligence, which specifically includes: a probability acquisition unit 10, a function establishment unit 20, and a cause analysis unit 30.
The probability obtaining unit 10 is configured to obtain a dropping probability of the goods according to a stationary degree and a position deviation degree of the goods in each frame of image with respect to the AGV.
The function establishing unit 20 is configured to obtain a variation curve function varying with time according to the drop probability in the continuous multi-frame images; the continuous multiframe images are video images of the collected goods before falling.
The reason analyzing unit 30 is configured to establish a linear function about a steady trend of dropping the cargo according to two corresponding maximum dropping probabilities in the start time period and the end time period in the maximum dropping probability sequence in the plurality of time periods in the variation curve function, and determine the reason for dropping the cargo according to a deviation degree of the maximum dropping probability in the corresponding time period in the variation curve function and the linear function.
Further, referring to fig. 4, the probability acquisition unit 10 includes an image processing unit 11 and a drop detection unit 12.
The image processing unit 11 is configured to perform target detection on the received continuous multi-frame images according to the received change of the sound frequency around the AGV to identify the AGV, the goods, and the color bands.
The falling detection unit 12 is used for determining a suspected falling distance between the AGV and the goods according to the maximum distance between the coordinates of the AGV and the coordinates of the goods in the video image, and further determining whether the goods fall or not according to the suspected falling distance.
Further, the determining, in the cause analyzing unit 30, the cause of the falling of the goods according to the deviation degree of the maximum falling probability in the corresponding time period in the variation curve function and the straight line function includes:
when the deviation degree is smaller than the deviation degree threshold value, judging that the goods fall due to the fact that the goods are loaded in an irregular mode; otherwise, it is determined that the cargo is dropped due to a traveling road surface problem.
Further, referring to fig. 4, the probability obtaining unit 10 further includes an equation obtaining unit 13 and a distance analyzing unit 14.
The equation obtaining unit 13 is configured to determine a linear equation of the color bar according to the pixel points of the color bar.
The distance analysis unit 14 is used for calculating the distance between the dropped goods and the color band through the coordinates of the central point of the dropped goods and a linear equation, and further judging whether the dropped goods block the road traffic according to the distance.
In summary, the embodiment of the present invention provides an AGV cargo drop detection system based on artificial intelligence, which obtains a drop probability of a cargo according to a degree of stability and a degree of position deviation of the cargo in each frame of image relative to an AGV by using a probability obtaining unit 10 when confirming that the cargo drops; obtaining a change curve function changing along with time according to the falling probability in the continuous multi-frame images in the function establishing unit 20; and further establishing a linear function of the stable falling trend of the goods, and judging the reason of the falling of the goods according to the deviation degree of the maximum falling probability in the corresponding time period in the change curve function and the linear function. The falling probability of the goods is obtained according to the stable degree of the goods relative to the AGV trolley and the position deviation degree of the goods, the falling reasons of the goods are analyzed according to the change degree of the falling probability of the goods, corresponding measures can be taken in time according to the falling reasons of the goods, the possibility that the goods fall in the follow-up process is prevented, the falling probability of the goods is reduced, and the transportation efficiency of the AGV trolley is also improved.
It should be noted that: the precedence order of the above embodiments of the present invention is only for description, and does not represent the merits of the embodiments. And specific embodiments thereof have been described above. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (10)
1. An AGV goods falling detection method based on artificial intelligence is characterized by comprising the following steps:
when the goods fall, obtaining the falling probability of the goods according to the stability degree and the position deviation degree of the goods relative to the AGV trolley in each frame of image;
obtaining a change curve function changing along with time according to the falling probability in the continuous multi-frame images; the continuous multi-frame image is a video image before the collected goods fall;
and establishing a linear function related to the stable falling trend of the goods according to the two corresponding maximum falling probabilities in the starting time period and the ending time period in the maximum falling probability sequence in the time periods in the change curve function, and judging the reason for falling the goods according to the deviation degree of the maximum falling probabilities in the time periods in the change curve function and the linear function.
2. The method of claim 1, wherein the cargo drop determination method comprises:
according to the received change of the sound frequency around the AGV, carrying out target detection on the received continuous multi-frame images so as to identify the AGV, the goods and the color bands;
and determining the suspected dropping distance between the AGV and the goods according to the maximum distance between the coordinate of the AGV and the coordinate of the goods in the video image, and further determining whether the goods drop or not according to the suspected dropping distance.
3. The method of claim 1, wherein the determining the cause of the cargo drop according to the deviation degree of the maximum drop probability in the corresponding time period of the variation curve function and the straight line function comprises:
when the deviation degree is smaller than a deviation degree threshold value, judging that the goods fall due to the fact that the goods are loaded in an irregular mode; otherwise, it is determined that the cargo is dropped due to a traveling road surface problem.
4. The method of claim 2, wherein when the cargo is dropped, further comprising:
determining a linear equation of the color band according to the pixel points of the color band;
and calculating the distance between the dropped goods and the color band through the coordinates of the central point of the dropped goods and the linear equation, and judging whether the dropped goods block the road traffic according to the distance.
5. The method of claim 1 wherein said smoothness is based on a center point distance between said load and said AGV cart.
6. The method of claim 1, wherein the degree of positional offset is based on a difference between a distance between adjacent cargo items and a distance between standard adjacent cargo items.
7. The utility model provides a AGV goods detecting system that drops based on artificial intelligence which characterized in that, this system includes:
the probability obtaining unit is used for obtaining the falling probability of the goods according to the stability degree and the position deviation degree of the goods relative to the AGV trolley in each frame of image when the goods fall;
the function establishing unit is used for obtaining a change curve function changing along with time according to the falling probability in continuous multi-frame images; the continuous multi-frame image is a video image before the collected goods fall;
and the reason analysis unit is used for establishing a linear function related to the stable falling trend of the goods according to the corresponding two maximum falling probabilities in the starting time period and the ending time period in the maximum falling probability sequence in a plurality of time periods in the change curve function, and judging the reason for falling the goods according to the deviation degree of the maximum falling probabilities in the corresponding time periods in the change curve function and the linear function.
8. The system of claim 7, wherein the probability acquisition unit comprises:
the image processing unit is used for carrying out target detection on the received continuous multi-frame images according to the received change of the sound frequency around the AGV trolley so as to identify the AGV trolley, the goods and the color bands;
and the falling detection unit is used for determining the suspected falling distance between the AGV and the goods according to the maximum distance between the coordinates of the AGV and the coordinates of the goods in the video image, and further confirming whether the goods fall or not according to the suspected falling distance.
9. The system of claim 7, wherein the determining, in the cause analysis unit, the cause of the falling of the cargo according to the deviation degree of the maximum falling probability in the corresponding time period between the variation curve function and the straight line function includes:
when the deviation degree is smaller than a deviation degree threshold value, judging that the goods fall due to the fact that the goods are loaded in an irregular mode; otherwise, it is determined that the cargo is dropped due to a traveling road surface problem.
10. The system of claim 8, wherein the probability acquisition unit comprises:
the equation acquisition unit is used for determining a linear equation of the color band according to the pixel points of the color band;
and the distance analysis unit is used for calculating the distance between the dropped goods and the color band through the coordinates of the central point of the dropped goods and the linear equation, and further judging whether the dropped goods block the road traffic according to the distance.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110549331.0A CN113382203A (en) | 2021-05-20 | 2021-05-20 | AGV goods falling detection method and system based on artificial intelligence |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110549331.0A CN113382203A (en) | 2021-05-20 | 2021-05-20 | AGV goods falling detection method and system based on artificial intelligence |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113382203A true CN113382203A (en) | 2021-09-10 |
Family
ID=77571329
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110549331.0A Pending CN113382203A (en) | 2021-05-20 | 2021-05-20 | AGV goods falling detection method and system based on artificial intelligence |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113382203A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113821042A (en) * | 2021-11-23 | 2021-12-21 | 南京冈尔信息技术有限公司 | Cargo conveying obstacle identification system and method based on machine vision |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150154522A1 (en) * | 2013-12-04 | 2015-06-04 | Global Maritime As | Method of estimating risks caused by accidental dropped object loads to subsea pipelines or other subsea assets associated with offshore oil & gas and marine operations |
| US20180208196A1 (en) * | 2017-01-23 | 2018-07-26 | Suzuki Motor Corporation | Driving assistance device |
| WO2019015526A1 (en) * | 2017-07-18 | 2019-01-24 | 北京极智嘉科技有限公司 | Self-driven robot |
| CN111232233A (en) * | 2020-03-04 | 2020-06-05 | 北京三快在线科技有限公司 | Cargo-carrying unmanned aerial vehicle and method and device for detecting falling of cargo |
| CN112446896A (en) * | 2021-02-01 | 2021-03-05 | 苏州澳昆智能机器人技术有限公司 | Conveying material falling monitoring method, device and system based on image recognition |
| CN112560567A (en) * | 2019-09-26 | 2021-03-26 | 英特尔公司 | Vehicle risk identification and avoidance |
-
2021
- 2021-05-20 CN CN202110549331.0A patent/CN113382203A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150154522A1 (en) * | 2013-12-04 | 2015-06-04 | Global Maritime As | Method of estimating risks caused by accidental dropped object loads to subsea pipelines or other subsea assets associated with offshore oil & gas and marine operations |
| US20180208196A1 (en) * | 2017-01-23 | 2018-07-26 | Suzuki Motor Corporation | Driving assistance device |
| WO2019015526A1 (en) * | 2017-07-18 | 2019-01-24 | 北京极智嘉科技有限公司 | Self-driven robot |
| CN112560567A (en) * | 2019-09-26 | 2021-03-26 | 英特尔公司 | Vehicle risk identification and avoidance |
| CN111232233A (en) * | 2020-03-04 | 2020-06-05 | 北京三快在线科技有限公司 | Cargo-carrying unmanned aerial vehicle and method and device for detecting falling of cargo |
| CN112446896A (en) * | 2021-02-01 | 2021-03-05 | 苏州澳昆智能机器人技术有限公司 | Conveying material falling monitoring method, device and system based on image recognition |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113821042A (en) * | 2021-11-23 | 2021-12-21 | 南京冈尔信息技术有限公司 | Cargo conveying obstacle identification system and method based on machine vision |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11120280B2 (en) | Geometry-aware instance segmentation in stereo image capture processes | |
| US11544936B2 (en) | In-path obstacle detection and avoidance system | |
| CN111461107A (en) | Material handling method, apparatus and system for identifying regions of interest | |
| CN102227758A (en) | Camera device | |
| CN110619329A (en) | Carriage number and loading state identification method of railway freight open wagon based on airborne vision | |
| CN113342004B (en) | Multi-AGV (automatic guided vehicle) scheduling method and system based on artificial intelligence and visual perception | |
| CN108986148B (en) | Method for realizing multi-intelligent-trolley collaborative search, identification and tracking of specific target group | |
| KR20150076307A (en) | Traffic parameter estimation system using cctv video analysis | |
| US20200012979A1 (en) | System and method for providing service of loading and storing passenger article | |
| CN113382203A (en) | AGV goods falling detection method and system based on artificial intelligence | |
| WO2022132239A1 (en) | Method, system and apparatus for managing warehouse by detecting damaged cargo | |
| US11952823B2 (en) | Apparatus and method for door control | |
| CN115576329B (en) | Obstacle avoidance method of unmanned AGV based on computer vision | |
| CN111738109A (en) | Van-type cargo vehicle carriage door state identification method based on deep learning | |
| Park | Lane detection algorithm based on Hough transform for high-speed self driving vehicles | |
| Alsawy et al. | An Image Processing Based Classifier to Support Safe Dropping for Delivery-by-Drone | |
| CN117132872A (en) | Intelligent collision recognition system for material transport vehicle on production line | |
| CN115027506B (en) | Logistics luggage tractor driving control system and method | |
| JP2024520533A (en) | A system for tracking inventory | |
| US11772717B1 (en) | Autonomous versatile vehicle system | |
| CN112598660B (en) | Automatic detection method for pulp cargo quantity in wharf loading and unloading process | |
| CN115293606A (en) | Unmanned delivery vehicle system | |
| CN113379761B (en) | Linkage method and system of multiple AGVs and automatic doors based on artificial intelligence | |
| CN114550105B (en) | Goods transportation method and system based on Internet of vehicles | |
| CN115272351B (en) | Mine trackless rubber-tyred vehicle overrun detection method based on binocular vision and linear laser |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |