CN113086847A

CN113086847A - Method and device for determining state information of gantry crane equipment

Info

Publication number: CN113086847A
Application number: CN202110302056.2A
Authority: CN
Inventors: 刘飞; 周发明; 史永乐; 刘松; 张忠河
Original assignee: China Railway Guangzhou Group Co Ltd
Current assignee: China Railway Guangzhou Group Co Ltd
Priority date: 2021-03-22
Filing date: 2021-03-22
Publication date: 2021-07-09
Anticipated expiration: 2041-03-22

Abstract

The embodiment of the application provides a method and a device for determining state information of gantry crane equipment. In the embodiment of the application, the corresponding moving speed is determined by identifying the position corresponding to the gantry crane body and the position corresponding to the lifting appliance, so that the movement of the gantry crane equipment is controlled accurately and safely. The determination of the position and the speed of the gantry crane equipment can be automatically realized, so that the working efficiency and the operation safety of the gantry crane equipment can be improved.

Description

Method and device for determining state information of gantry crane equipment

Technical Field

The invention relates to the field of control, in particular to a method and a device for determining state information of gantry crane equipment.

Background

With the rapid development of the modern logistics industry, the competition of various transportation modes such as roads, railways, water ways and the like is intensified day by day, and the traditional transportation enterprises utilize advanced logistics modes and technologies. Railway transportation is a main transportation mode of national economy, and has certain advantages in the development of large-scale logistics. In recent years, China pays attention to container transportation, and a series of measures such as strengthening container equipment facilities, improving transportation service quality and launching transborder transportation shifts are adopted, so that the sending quantity of containers is greatly increased year by year.

Under the great trend, the production management of railway freight yards puts forward the construction requirements of a digital and visual container hoisting operation management system of gantry crane equipment, so that the working efficiency and the operation safety are improved, the remote hoisting operations of taking, putting and turning over the railway containers and the like are realized, the comprehensive competitiveness of railway freight is improved, and the railway freight yard can steadily advance to comprehensively realize the aim of modern logistics.

Disclosure of Invention

The technical problem to be solved by the invention is to overcome the defects of the prior art and provide a method and a device for determining state information of gantry crane equipment, which are used for controlling the gantry crane equipment more accurately and safely so as to improve the working efficiency and the operation safety.

In order to achieve the above technical object, in one aspect, the present invention provides a method for determining status information of a gantry crane apparatus, where the gantry crane apparatus includes: the image acquisition device comprises a gantry crane body vertically arranged on the ground and a lifting appliance sliding on the gantry crane body in parallel, wherein a first image acquisition device is arranged at the gantry crane body, and a second image acquisition device is arranged at the lifting appliance; the method comprises the following steps: acquiring a first image through the first image acquisition device, wherein the first image records the current line number corresponding to the position of the goods area on the ground of the gantry crane body; acquiring a second image through the second image acquisition device, wherein the second image records the current column number corresponding to the position of the cargo area of the lifting appliance on the ground; identifying the current line number in the first image, determining a current first position corresponding to the gantry crane body according to the current line number, and determining a first moving speed of the gantry crane body according to the current first position; and identifying the current column number in the second image, determining a current second position corresponding to the lifting appliance according to the current column number, and determining a second moving speed of the lifting appliance according to the current second position.

In another aspect, the present invention provides a device for determining status information of a gantry crane apparatus, where the gantry crane apparatus includes: the image acquisition device comprises a gantry crane body vertically arranged on the ground and a lifting appliance sliding on the gantry crane body in parallel, wherein a first image acquisition device is arranged at the gantry crane body, and a second image acquisition device is arranged at the lifting appliance; the device comprises: the first acquisition module is used for acquiring a first image through the first image acquisition device, and the first image records the current line number corresponding to the position of the goods area on the ground of the gantry crane body; the second acquisition module is used for acquiring a second image through the second image acquisition device, and the second image records the current column number corresponding to the position of the cargo area of the lifting appliance on the ground; the first identification module is used for identifying the current line number in the first image, determining a current first position corresponding to the gantry crane body according to the current line number, and determining a first moving speed of the gantry crane body according to the current first position; and the second identification module is used for identifying the current column number in the second image, determining a current second position corresponding to the lifting appliance according to the current column number, and determining a second moving speed of the lifting appliance according to the current first position.

In the embodiment of the application, the corresponding moving speed is determined by identifying the position corresponding to the gantry crane body and the position corresponding to the lifting appliance, so that the movement of the gantry crane equipment is controlled accurately and safely. The determination of the position and the speed of the gantry crane equipment can be automatically realized, so that the working efficiency and the operation safety of the gantry crane equipment can be improved.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

Fig. 1 is a schematic flow chart of a method for determining status information of a gantry crane apparatus according to an embodiment of the present application;

FIG. 2 is a block diagram of a status information determination system according to an embodiment of the present application;

fig. 3 is a schematic view of a mounting position of a camera according to an embodiment of the present application;

FIG. 4 is a schematic view of a longitudinal coordinate position of a gantry crane body according to an embodiment of the present disclosure;

fig. 5 is a schematic view of a lateral coordinate position of a spreader in an embodiment of the present application;

fig. 6 is a schematic view illustrating a movement of a gantry crane apparatus according to an embodiment of the present disclosure;

fig. 7 is a schematic frame diagram of a state information determination device of a gantry crane apparatus according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

As shown in fig. 1, the present application provides a method for determining status information of a gantry crane apparatus, where the gantry crane apparatus includes: the image acquisition device comprises a gantry crane body vertically arranged on the ground and a lifting appliance sliding on the gantry crane body in parallel, wherein a first image acquisition device is arranged at the gantry crane body, and a second image acquisition device is arranged at the lifting appliance; the method 100 includes:

101: a first image is acquired by a first image acquisition device.

The first image records the current line number corresponding to the position of the goods area of the gantry crane body on the ground.

102: and acquiring a second image through a second image acquisition device.

And the second image records the current column number corresponding to the position of the cargo area of the lifting appliance on the ground.

103: and identifying the current line number in the first image, determining a current first position corresponding to the gantry crane body according to the current line number, and determining a first moving speed of the gantry crane body according to the current first position.

104: and identifying the current column number in the second image, determining a current second position corresponding to the lifting appliance according to the current column number, and determining a second moving speed of the lifting appliance according to the current second position.

It should be noted that the execution subject of the method 100 may be a device with a computing function, such as a server, a computer, and the like. The gantry crane equipment is mainly used for loading and unloading outdoor goods yards, stockyards and bulk goods. Including the portal crane body, its metal construction sets up subaerial like portal frame is perpendicular, and this portal crane body can include portal crane bridge and bear two stabilizer blades of this bridge for can directly walk on the track on ground, be provided with the hoist on this bridge, can carry out parallel slip, thereby carry out the hoist and mount of goods.

The following is set forth in detail with respect to the above steps:

101: a first image is acquired by a first image acquisition device.

The first image records the current line number corresponding to the position of the goods area of the gantry crane body on the ground. For the number of rows and columns of goods marked in the cargo area on the ground. As can be seen from the foregoing, the gantry crane apparatus may further include: the gantry crane track is arranged on the ground; the number of rows of the above-ground marked cargo areas and the number of columns of the above-ground cargo areas can be set with the gantry crane track as a reference. It is understood that the transverse direction is a row and the longitudinal direction is a column.

For example, it can be seen from the foregoing that the first image capturing device may be disposed on a leg of the gantry crane body, or may be disposed below the bridge of the gantry crane, as long as the corresponding number of rows can be acquired.

Specifically, the first image capturing device includes: a first camera; the gantry crane body comprises a gantry crane bridge parallel to the ground, and a first camera facing the ground is arranged below the gantry crane bridge, so that the number of lines of the goods area marked on the ground can be collected by the first camera, and a first image recorded with the current number of lines can be acquired.

The first camera can be a camera of a zoom ball machine, and the number of lines of the ground marks of the storage yard, such as numbers, letters or characters, can be focused and shot in real time after fixed focal length adjustment.

For example, as shown in fig. 3, as can be seen from the foregoing, the first camera 302, i.e., the camera of the gantry crane body, may be disposed below the gantry crane bridge 307 of the gantry crane body 306, the first camera 302 shoots towards the ground, and the shooting line number 308 is obtained. And the gantry crane body moves on a gantry crane track 303 on the ground.

As shown in fig. 2, the camera 201 of the gantry crane body transmits the captured clear video image (i.e. the corresponding image) to the central system 203 through the remote communication module, and the central system may be composed of a server and/or a computer.

The remote communication module can be a WiFi wireless communication module arranged on camera equipment arranged on the gantry crane body and the lifting appliance and used for transmitting shot images to the central system and transmitting control instructions such as focusing of the camera fed back by the central system for shooting clear images to the camera.

102: and acquiring a second image through a second image acquisition device.

And the second image records the current column number corresponding to the position of the cargo area of the lifting appliance on the ground. For example, as can be seen from the foregoing, the second image capturing device may be disposed on the hanger, and may be oriented toward the ground to record the number of columns. Or, the column number may be recorded facing the lower side of the gantry crane bridge, and it should be noted that, when facing the lower side of the gantry crane bridge, the column number needs to be set below the gantry crane bridge and corresponds to the column number on the ground.

Specifically, the second image capturing device includes: a second camera; the gantry crane body comprises a gantry crane bridge parallel to the ground, the number of rows of cargo areas is marked below the gantry crane bridge, and the number of rows of cargo areas marked below the gantry crane bridge corresponds to the number of rows of cargo areas on the ground; and at least one second camera is arranged on two sides of the lifting hook of the lifting appliance, and faces the lower part of the gantry crane bridge, so that at least one second camera acquires a second image, and the second image records the number of rows of the cargo areas marked on the lower part of the gantry crane bridge correspondingly.

The second camera can be a camera of a variable-focus dome camera, and the number of columns of marks at the bottom of the gantry crane bridge, such as numbers, letters or characters, can be focused and shot in real time after fixed focal length adjustment. The number of the second cameras may be 1, two, or more. Correspondingly, the number of the first cameras may also be multiple, which is similar to the image processing manner of the second cameras, and for the multiple first cameras, the image processing manner of the multiple second cameras may be referred to subsequently.

For example, as shown in fig. 3, it can be seen from the foregoing that 2 second cameras 301, i.e., cameras of a spreader, may be provided on both sides of a hook of the spreader 305, and the first camera 302 is directed to shoot below a gantry bridge 307, and the number of shooting columns 304 is counted. And spreader 305 moves laterally across gantry bridge 307. It should be understood that the number of columns 304 corresponds to the number of columns on the ground (not shown in FIG. 3), and the number of columns 308 on the ground is also in a row-column relationship.

As shown in fig. 2, the camera 202 of the spreader transmits the captured clear video image (i.e. the corresponding image) to the central system through the remote communication module.

It should be noted that, the first image capturing device and the second image capturing device respectively capture images. However, for subsequent identification, the final position and speed of the gantry crane equipment should be determined by identifying two images at the same time, determining the position, or determining the speed.

The first moving speed is a parallel moving speed of the gantry crane body relative to the ground or a gantry crane rail, or a longitudinal moving speed when the gantry crane body is overlooked.

Specifically, the current line number in the first image is identified, and the current first position corresponding to the gantry crane body is determined according to the current line number, including: identifying a first straight line in the first image for marking a row of the cargo area on the ground based on a Hough transform algorithm, and determining an angle of the first straight line with the first image; determining the position of the first straight line in the first image based on the angle; adjusting the first image based on a coordinate conversion algorithm so that the first straight line becomes a vertical line; detecting a first marker for marking the number of lines of the cargo area on the ground in the first image based on the convolutional neural network, the first marker being disposed between two adjacent first straight lines; identifying each first identifier from the detected first identifiers based on a similarity algorithm; drawing a first marking line parallel to the first straight line from the middle point of the top boundary of the first image, intersecting the region where the row number of the cargo area on the ground is located at a first intersection point, and determining the ordinate of the first intersection point in the first image and the corresponding first identifier based on the position of the first straight line in the first image and the identification of each first identifier.

Wherein, the goods district can be the container district, namely the box district. The first line may be a separation line. The first mark may be a mark letter, which may be disposed between two adjacent separation straight lines.

For example, as can be seen from the foregoing, as shown in FIG. 4, the yard floor is drawn with identification letters 405 in order from A, B, C to Z to identify the number of bin rows, and a separation line 401 (which may be a white separation line) is drawn between the letters. The length (i.e., the column height of the box area, in fig. 4, the longitudinal height of the identification area 404) and the width of the identification area 404 where the letter is located are fixed values and preset values, and before identification, configuration can be performed through a position basic parameter configuration module, that is, necessary parameter configuration performed by a subsequent automatic detection and identification algorithm is completed. If the line of sight 402 of the first camera 302, which is parallel to the separation line 401, intersects the identification area 404 at the X point 403, the position of the X point 403 is determined as follows:

a. the server in the central system may perform detection on the separation straight lines 401 in the first image by using a hough transform algorithm for image recognition, recognize an angle of the separation straight lines 401 in the image with respect to the image, and determine coordinates of each separation straight line 401 in the image based on the angle, so as to restore the position, i.e., the coordinates, of the identification area 404 in the image. b. The image rotation is performed using a coordinate transformation algorithm such that the separation straight line 401 becomes a vertical line and the vertical line of each vertical midpoint of the identification letter 405 becomes a horizontal line. c. The convolutional neural network is used to perform an object detection algorithm to detect all objects to be identified, such as the individual identification letters 405 (indicating the number of rows). d. All the identification letters 405 are identified by text features by a similarity algorithm, so that the coordinates of the identification letters 405 are determined. e. From the middle point of the top boundary of the image, a straight line (i.e., the line of sight 402) parallel to the separation straight line 401 intersects the identification area 404 at the X point 403, so as to determine an identification letter 405 corresponding to the identification area 404 where the X point 403 is located, such as the letter B and the ordinate position in fig. 4. Wherein the area occupied by each good in the row number is the rectangular area of the area 406 in which the letter C is located.

In addition, the method 100 further comprises: based on the identified neighboring first markers, an occluded first marker is determined.

For example, as can be seen from the foregoing, the text features in step d. using the similarity algorithm recognize all identification letters 405, and infer from the adjacent identification letters 405 which identification letters 405 may be occluded.

In the first image, the minimum ordinate point in the first column may be regarded as the origin of coordinates, and the corresponding number of rows may be calculated.

The location-based parameter configuration module: the necessary parameter configuration for completing the automatic detection and identification algorithm may include the number of rows and columns of containers stacked in the yard, the row width (row spacing) and column width (column spacing), the numbers, letters or characters corresponding to the number of rows and columns, the labels for dividing the container areas, the standard layer height of the containers, the height of the gantry crane bridge, the relative coordinate position of the yard to the gantry crane rail, and other parameters, which are preset values.

Furthermore, the position of the line of sight 402 may be determined by a depth-learning image line of sight height detection algorithm, so as to identify the corresponding X point 403.

Specifically, the first moving speed of the gantry crane body is determined according to the current first position, and the method comprises the following steps: determining a difference value of vertical coordinates according to the current vertical coordinate and the vertical coordinate corresponding to the target moving position of the gantry crane body; determining a time difference value according to the current time corresponding to the current ordinate and the target time corresponding to the target moving position; and determining the longitudinal moving speed of the gantry crane body according to the difference value of the vertical coordinates and the time difference value.

For example, as can be seen from the foregoing, the movement speed can be calculated from the positions obtained by image capturing and recognition (the process described above) performed on the video image corresponding to the gantry crane body at the current time point T1 and the previous time point T0, and the calculation algorithm is as follows:

the longitudinal moving speed of the gantry crane body is (longitudinal coordinate position of the gantry crane body S1-longitudinal coordinate position of the gantry crane body S0)/(T1-T0), wherein T1 and T0 correspond to the time of position S1 and position S0, respectively. The ordinate positions at different times can be determined in the manner described above.

Therefore, the server and/or the computer in the central system can calculate the moving speed between two longitudinal coordinates and within two corresponding time periods for the gantry crane body, and can use the speed as control information, as shown in fig. 2, the central system 203 sends the control information, such as the moving speed, to the control system 204 of the gantry crane, or the control system 204 of the gantry crane collects the control information, and then controls the gantry crane body to move according to the collected control information.

The second moving speed refers to a transverse moving speed of the lifting appliance on the gantry crane bridge, and also refers to a vertical moving speed of the lifting appliance, namely a longitudinal moving speed. The second position may also be a lateral coordinate of the spreader moving in the lateral direction, and may also be a vertical coordinate of the spreader moving in the vertical direction, i.e. a vertical coordinate.

Specifically, the current column number in the second image is identified, and the current second position corresponding to the lifting appliance is determined according to the current column number, including: identifying a second straight line in the second image for marking the row of the goods area on the ground based on a Hough transform algorithm, and determining an angle between the second straight line and the second image; determining the position of the second straight line in the second image based on the angle; adjusting the second image based on a coordinate conversion algorithm so that the second straight line becomes a vertical line; detecting a second mark for marking the number of columns of the goods area on the ground in the second image based on the convolutional neural network, wherein the second mark is arranged between two adjacent second straight lines; identifying each second identifier from the detected second identifiers based on a similarity algorithm; and drawing a second marking line parallel to the second straight line from the middle point of the top boundary of the second image, intersecting the area where the column number of the cargo area on the ground is located at a second intersection point, and determining the abscissa of the second intersection point in the second image and the corresponding second identifier based on the position of the second straight line in the second image and the identification of each second identifier.

Wherein the second line may be a separation line. The second mark may be a mark letter, which may be disposed between two adjacent separation straight lines.

For example, as can be seen from the foregoing, as shown in fig. 5, the identification letters 506 drawn at the bottom of the gantry crane bridge 501 are sequentially moved from A, B, C to Z for identifying the scale, and the straight separation lines 507 are drawn between the identification letters 506. The length of the identification region 502 where the identification letter 506 is located is a fixed numerical value, that is, a preset value configured by the location-based parameter configuration module. The shooting angle of view of the second camera 301 is a fixed angle α. The sight line 503 of the second camera 301 perpendicular to the bottom plane of the gantry crane bridge 501 intersects the plane of the marked area 502 at the X point 504, and then the position of the X point 504 and the distance between the X point 504 and the camera (i.e. the height h of the spreader) are calculated as follows:

a. the server in the central system may perform line detection on the separation straight lines 507 in the second image by using a hough transform algorithm for image recognition, recognize angles of the separation straight lines 507 in the image to the second image, determine coordinates of the separation straight lines 507 in the image based on the angles, and restore the positions, i.e., the coordinates, of the identification areas 502. b. A second image rotation is performed using a coordinate transformation algorithm such that the separation straight line 507 becomes a vertical line and the vertical line of the vertical midpoint of each identification letter 506 becomes a horizontal line. c. And (3) performing an object detection algorithm by using a convolutional neural network to detect all objects to be identified, such as all identification letters 507. d. All identification letters 507 are identified by text features by a similarity algorithm. e. From the middle point of the top boundary of the second image, a straight line parallel to the separation straight line 507 is drawn, i.e. the sight line 503 intersects the identification area 502 at the X point 504, and the identification letter, such as letter B, of the identification area 502 where the X point 504 is located and the abscissa position X0 are calculated.

Note that the minimum abscissa position of the marker region may be set as the origin, and the abscissa region position may be calculated.

In addition, the method 100 further comprises: determining the shielded second identifier based on the identified adjacent second identifier, and determining the length of the area in which the number of columns of the cargo area on the ground in the second image is located; and determining the vertical distance of the lifting appliance relative to the lower part of the gantry crane bridge (or the vertical distance of the second camera relative to the lower part of the gantry crane bridge) according to the length, the abscissa and the acquisition angle of the second image acquisition device.

For example, as can be seen from the foregoing, as shown in fig. 5, d. all the identification letters 507 are recognized by text features using a similarity algorithm, and the identification letters 507 that may be occluded are deduced from the adjacent identification letters 507, and the total length l of all the identification areas 502 in the second image is calculated.

f. According to the trigonometric function, α ═ arctan (x0/h) + arctan ((l-x0)/h), h can be calculated back. As shown in fig. 5, an α angle 505 is formed by the shooting range 509 of the second camera 301.

In addition, the method 100 further comprises: when the number of the second image acquisition devices is at least two, determining the vertical distance of a lifting appliance corresponding to each second image acquisition device relative to the lower part of the gantry crane bridge; and determining the vertical distance of the final lifting appliance relative to the lower part of the gantry crane bridge according to the at least two vertical distances.

Furthermore, the position of the line of sight 503 may be determined by a depth-learning image line of sight height detection algorithm, so as to determine the corresponding X point 504.

When there are 2 or even more second image acquisition means, the final h can be determined in the following manner.

For example, as can be seen from the foregoing, in order to solve the problem of variations in the movement of the gantry crane body and the spreader due to the swinging of the spreader, 1 camera is mounted on each of the opposite corners (e.g., on both sides of the hook) of the rectangular spreader, and the average of the heights calculated by the two cameras, that is, h ═ h1+ h2)/2, is taken as the spreader height. h1 and h2 are the vertical distances of the corresponding second cameras relative to the lower part of the gantry crane bridge, namely the heights of the corresponding lifting appliances.

It should be noted that, for a plurality of second cameras, when acquiring a plurality of second images at the same time, the central system needs to stitch and combine the plurality of second images to compose one image, that is, the central system can completely acquire an image of the entire identification area, such as an image of the entire identification area below a gantry crane bridge. Detection and identification is then performed. As shown in fig. 5, for the indication area 502 indicating the number of columns, the area occupied by each item is a rectangular area in which the letter D is located in the area 508. It will be appreciated that the rectangular area is the same as the rectangular area in the rows described above.

Specifically, determining the second moving speed of the spreader according to the current second position includes: determining a difference value of the abscissa according to the current abscissa and the abscissa corresponding to the transverse target moving position of the lifting appliance; determining a time difference value according to the current time corresponding to the current abscissa and the target time corresponding to the transverse target moving position; and determining the transverse moving speed of the lifting appliance according to the difference value of the abscissa and the time difference value.

For example, from the foregoing, the lateral movement speed of the spreader is (the abscissa position S1-the abscissa position S0)/(T1-T0), where T1 and T0 correspond to the time of the position S1 and the position S0, respectively. The abscissa positions corresponding to different times can be determined in the manner described above.

In addition, the method 100 further comprises: determining a difference value of the vertical distances according to the currently determined vertical distance and the vertical distance corresponding to the longitudinal target moving position of the lifting appliance; determining a time difference value according to the current time corresponding to the currently determined vertical distance and the target time corresponding to the longitudinal target moving position; and determining the longitudinal movement speed of the lifting appliance according to the distance difference and the time difference.

For example, as can be seen from the foregoing, when the spreader moves vertically and loads, the vertical moving speed of the spreader is (height M1 of the spreader-height M0 of the spreader)/(T1-T0), where T1 and T0 correspond to the time of position M1 and position M0, respectively.

It will be appreciated that the height of the spreader at different times may be determined in the manner described hereinbefore.

Therefore, the server and/or the computer in the central system can calculate the moving speed between two transverse coordinates and corresponding two times and the moving speed between two heights and corresponding two times for the lifting appliance, and can use the speeds as control information, as shown in fig. 2, the central system 203 sends the control information, such as the moving speed, to the control system 204 of the gantry crane, or the control system 204 of the gantry crane collects the control information, and then controls the lifting appliance to move according to the collected control information.

Then, the method 100 further comprises: and controlling the gantry crane equipment to move according to the first moving speed and the second moving speed so as to finish the operation.

As already explained above, it is not described here in detail.

The utility model provides an install additional and reform transform equipment that this application embodiment relates to is few, only needs to install the wireless camera equipment of wiFi, need not to install the sensor and carry out PLC digital-to-analog conversion, and installation and debugging are simple and easy, with low costs.

In addition, the intelligent video image recognition technology is comprehensively applied, the problems of mark shielding and position deviation are solved by adopting a unique calculation algorithm, and the detection calculation of the gantry crane body and the hanger position with high accuracy and the detection calculation of the operation target container position are realized.

Therefore, the first camera at the position of the gantry crane body and the second camera at the position of the lifting appliance take pictures respectively, the images are returned to the central system, and the central system calculates to obtain the coordinate positions and the box areas of the current gantry crane body and the lifting appliance, such as the number of lines and the number of columns.

And the control system for the container operation calculates the distance and the speed of the gantry crane and the lifting appliance to move according to the coordinate positions of the gantry crane body and the lifting appliance, which are acquired from the central system, and the coordinate position of the box area of the target container.

As shown in fig. 6, in the control system for container handling, the gantry crane body 306 is remotely controlled to move along the gantry crane track 303 to the coordinate position of the row where the yard of the target container is located, the remote control spreader moves in parallel to the coordinate position of the column where the yard of the target container is located, and the remote control spreader is controlled to move vertically to the position of grabbing the container above the container 601.

In addition, the portal crane equipment still includes: the gantry crane track is arranged on the ground; when the gantry crane apparatus hoists the cargo, the method 100 further includes: determining the difference value of the vertical coordinates according to the current vertical coordinate and the relative vertical coordinate of the goods yard to the gantry crane track; and determining the number of rows of the goods to be hoisted according to the difference value of the vertical coordinates and the row width.

In the operation of hoisting goods by using the gantry crane equipment, such as containers, the position of an operation target container can be calculated by the position of a spreader, and the calculation algorithm is as follows:

the number of rows of the bin positions, that is, the number of rows of the goods to be hoisted ═ (the ordinate position of the gantry crane body-the relative ordinate position of the yard versus the gantry crane track)/the bin row spacing +1, where the bin row spacing refers to the row height or row width of one row, that is, the row width. In addition, the relative ordinate position of the yard to the gantry crane rail is a preset value, and is determined by the position basic parameter configuration module in the manner described above.

In addition, the portal crane equipment still includes: the gantry crane track is arranged on the ground; the gantry crane body comprises a gantry crane bridge parallel to the ground; when the gantry crane apparatus hoists the cargo, the method 100 further includes: determining the difference value of the abscissa according to the current abscissa and the relative abscissa of the goods yard to the gantry crane track; determining the number of columns of the goods to be hoisted according to the difference value of the horizontal coordinates and the column width; and determining the height difference value of the portal crane bridge and the determined vertical distance, and determining the number of layers of the goods to be hoisted according to the height difference value and the height of the body of the goods to be hoisted.

For example, as can be seen from the foregoing, the number of rows of boxes, i.e., the number of rows of goods to be hoisted (the abscissa position of the spreader-the relative abscissa position of the yard versus the gantry track)/the width of a box row +1, where the width of a box row, i.e., the distance between box rows, refers to the row width of one row. In addition, the relative abscissa position of the yard to the gantry crane rail is a preset value, and is determined by the position basic parameter configuration module in the manner described above.

The number of the container positions is (the height of the gantry crane bridge-the vertical height of the spreader)/the standard layer height of the container +1, wherein the number of the container positions is the number of the layers where the goods to be hoisted are located, and the standard layer height of the container is the standard height of one goods, namely the height of the goods body, which is a preset value, and is determined by the position basic parameter configuration module in the manner described above. The height of the gantry crane bridge is also a preset value, and is determined by the position basic parameter configuration module in the manner described above.

In addition, the method 100 further comprises: after the gantry crane equipment is controlled to move to the target position, the lifting appliance is controlled to move longitudinally to the lifting position of the goods to be lifted; and in the process of controlling the lifting appliance to longitudinally move to the lifting position of the goods to be lifted, determining a current first position and a current second position, and determining a corresponding first moving speed and a corresponding second moving speed.

In the process of controlling the movement by grabbing the container, the first camera of the gantry crane body and the second camera of the lifting appliance respectively take pictures continuously in real time, so that the central system calculates the accurate coordinate position and speed of the gantry crane body and the lifting appliance in the movement according to the mode, namely images, and a control system (namely the control system) for container operation acquires the operation process safety in real time. The spreader can then be connected to the container, i.e. the cargo to be hoisted.

In addition, the method 100 further comprises: after the lifting appliance is connected with the goods to be lifted, determining the longitudinal moving distance corresponding to the gantry crane body, the transverse moving distance corresponding to the lifting appliance, the corresponding first moving speed and second moving speed according to the target position of the goods to be lifted so as to move the goods to be lifted to the corresponding target position; determining a current first position and a current second position and determining a corresponding first moving speed and a corresponding second moving speed in the process of moving to a target position of a cargo to be hoisted and placing the cargo to be hoisted; and determining that the current first position and the current second position correspond to the target position of the goods to be hoisted, and separating the lifting appliance from the goods to be hoisted after the height of the goods to be hoisted is correct so as to finish the operation of placing the goods to be hoisted.

The target position of the goods to be hoisted can be the coordinate position (abscissa and ordinate) of the box area of the target position, and the coordinate position with the number of layers can also be the height of the goods to be hoisted.

And the control system of the container operation calculates the distance and the speed of the gantry crane body and the lifting appliance to move according to the coordinate position of the container area of the target position for placing the containers and the coordinate position of the layer number in the manner described above. And then, the control system for container operation remotely controls the lifting appliance to lift the container, remotely controls the gantry crane body to move to the coordinate position of the row where the box area of the target position is located, and remotely controls the lifting appliance to move in parallel to the coordinate position of the column where the box area of the target container is located and vertically move to the position of the box placing layer number. In the process of controlling movement by placing containers, the first camera at the position of the gantry crane body and the second camera at the position of the lifting appliance continuously and respectively take pictures in real time, and the central system performs image calculation according to the mode to obtain the coordinate positions and the speed of the gantry crane body and the lifting appliance in movement, so that the control system for container operation can control the safety of the operation process in real time after acquiring. And the control system for container operation acquires the container area, the number of layers and the corresponding position coordinates of the gantry crane body and the lifting appliance again, separates the lifting appliance from the container after the correctness is confirmed, and the central system can store the container area, the number of layers and the corresponding position coordinates of the current target container to finish the container placing operation.

The central system may further include an identification module for identifying a position of the gantry crane body and an identification module for identifying a position of the spreader, which are respectively used for identifying and determining the first position and the first moving speed, and identifying and determining the second position and the second moving speed.

The embodiment of the application can perform real-time camera shooting and efficient identification, provides the position and the moving speed of the gantry crane body and the lifting appliance in operation for the control system of container operation to perform real-time acquisition, provides guarantee for accurate, efficient, safe and reliable remote operation control, and can realize tracking record of the whole operation process.

The embodiment of the application also provides a controlling means of portal crane equipment, and portal crane equipment includes: the image acquisition device comprises a gantry crane body vertically arranged on the ground and a lifting appliance sliding on the gantry crane body in parallel, wherein a first image acquisition device is arranged at the gantry crane body, and a second image acquisition device is arranged at the lifting appliance; as shown in fig. 7, the apparatus 700 includes:

the first obtaining module 701 is configured to obtain a first image through a first image collecting device, where the first image records a current line number corresponding to a position of a goods location on the ground of the gantry crane body.

A second obtaining module 702, configured to obtain a second image through a second image collecting device, where the second image records a current column number corresponding to the position of the cargo area of the lifting appliance on the ground.

The first identifying module 703 is configured to identify a current line number in the first image, determine a current first position corresponding to the gantry crane body according to the current line number, and determine a first moving speed of the gantry crane body according to the current first position.

The second identifying module 704 is configured to identify a current column number in the second image, determine a current second position corresponding to the spreader according to the current column number, and determine a second moving speed of the spreader according to the current first position.

Specifically, the second image capturing device includes: a second camera; the gantry crane body comprises a gantry crane bridge parallel to the ground, the number of rows of cargo areas is marked below the gantry crane bridge, and the number of rows of cargo areas marked below the gantry crane bridge corresponds to the number of rows of cargo areas on the ground; and at least one second camera is arranged on two sides of the lifting hook of the lifting appliance, and faces the lower part of the gantry crane bridge, so that the at least one second camera acquires the second image, and the second image records the number of rows of the cargo areas corresponding to the marks below the gantry crane bridge.

Specifically, the first identifying module 703 includes: a first recognition unit for recognizing a first straight line in the first image for marking a row of the cargo area on the ground based on a hough transform algorithm and determining an angle of the first straight line with the first image; a first determination unit configured to determine a position of the first straight line in the first image based on the angle; a first adjusting unit configured to adjust the first image based on a coordinate conversion algorithm so that the first straight line becomes a vertical line; the first detection unit is used for detecting a first mark used for marking the number of rows of the goods area on the ground in the first image based on the convolutional neural network, and the first mark is arranged between every two adjacent first straight lines; a first identification unit configured to identify each of the detected first identifiers based on a similarity algorithm; and the first determining unit is used for drawing a first marking line parallel to the first straight line from the middle point of the top boundary of the first image, intersecting the region where the number of rows of the goods area on the ground is located at the first intersection point, and determining the vertical coordinate of the first intersection point in the first image and the corresponding first identification based on the position of the first straight line in the first image and the identification of each first identification.

Furthermore, the first identification unit is further configured to: based on the identified neighboring first markers, an occluded first marker is determined.

Specifically, the second identifying module 704 includes: a second recognition unit, configured to recognize, based on a hough transform algorithm, a second straight line in the second image, the second straight line being used to mark the row of the on-ground cargo area, and determine an angle between the second straight line and the second image; a second determination unit configured to determine a position of the second straight line in the second image based on the angle; a second adjusting unit for adjusting the second image based on a coordinate conversion algorithm so that the second straight line becomes a vertical line; the second detection unit is used for detecting a second mark used for marking the number of columns of the goods area on the ground in the second image based on the convolutional neural network, and the second mark is arranged between two adjacent second straight lines; a second detection unit for identifying each of the second identifiers from the detected second identifiers based on a similarity calculation method; and the second determining unit is used for drawing a second marking line parallel to the second straight line from the middle point of the top boundary of the second image, intersecting the area where the number of columns of the goods area on the ground is located at a second intersection point, and determining the abscissa of the second intersection point in the second image and the corresponding second identifier based on the position of the second straight line in the second image and the identification of each second identifier.

Furthermore, the second detection unit is further configured to: determining the shielded second identifier based on the identified adjacent second identifier, and determining the length of the area in which the number of columns of the cargo area on the ground in the second image is located; and the second determining unit is used for determining the vertical distance of the lifting appliance relative to the lower part of the gantry crane bridge according to the length, the abscissa and the acquisition angle of the second image acquisition device.

In addition, the second determining unit is used for determining the vertical distance of the lifting appliance corresponding to each second image acquisition device relative to the lower part of the gantry crane bridge under the condition that at least two second image acquisition devices are arranged; and determining the vertical distance of the final lifting appliance relative to the lower part of the gantry crane bridge according to the at least two vertical distances.

Specifically, the first determining unit is used for determining a difference value of vertical coordinates according to the current vertical coordinate and the vertical coordinate corresponding to the target moving position of the gantry crane body; determining a time difference value according to the current time corresponding to the current ordinate and the target time corresponding to the target moving position; and determining the longitudinal moving speed of the gantry crane body according to the difference value of the vertical coordinates and the time difference value.

Specifically, the second determining unit is configured to: determining a difference value of the abscissa according to the current abscissa and the abscissa corresponding to the transverse target moving position of the lifting appliance; determining a time difference value according to the current time corresponding to the current abscissa and the target time corresponding to the transverse target moving position; and determining the transverse moving speed of the lifting appliance according to the difference value of the abscissa and the time difference value.

Further, a second determination unit for: determining a difference value of the vertical distances according to the currently determined vertical distance and the vertical distance corresponding to the longitudinal target moving position of the lifting appliance; determining a time difference value according to the current time corresponding to the currently determined vertical distance and the target time corresponding to the longitudinal target moving position; and determining the longitudinal movement speed of the lifting appliance according to the distance difference and the time difference.

In addition, the portal crane equipment still includes: the gantry crane track is arranged on the ground; the number of rows of the goods areas marked on the ground and the number of columns of the goods areas marked on the ground are set on the basis of the gantry crane track.

In addition, the apparatus 700 further comprises: and the control module is used for controlling the gantry crane equipment to move according to the first moving speed and the second moving speed so as to finish the operation.

In addition, the portal crane equipment still includes: the gantry crane track is arranged on the ground; when the gantry crane apparatus hoists the goods, the apparatus 700 further includes: the determining module is used for determining the difference value of the vertical coordinates according to the current vertical coordinate and the relative vertical coordinate of the goods yard to the gantry crane track; and determining the number of rows of the goods to be hoisted according to the difference value of the vertical coordinates and the row width.

In addition, the portal crane equipment still includes: the gantry crane track is arranged on the ground; the gantry crane body comprises a gantry crane bridge parallel to the ground; the determining module is used for determining the difference value of the abscissa according to the current abscissa and the relative abscissa of the goods yard to the gantry crane track when the gantry crane equipment hoists the goods; determining the number of columns of the goods to be hoisted according to the difference value of the horizontal coordinates and the column width; and determining the height difference value of the portal crane bridge and the determined vertical distance, and determining the number of layers of the goods to be hoisted according to the height difference value and the height of the body of the goods to be hoisted.

In addition, the control module is further configured to: after the gantry crane equipment is controlled to move to the target position, the lifting appliance is controlled to move longitudinally to the lifting position of the goods to be lifted; and in the process of controlling the lifting appliance to longitudinally move to the lifting position of the goods to be lifted, determining a current first position and a current second position, and determining a corresponding first moving speed and a corresponding second moving speed.

Further, a determination module to: after the lifting appliance is connected with the goods to be lifted, determining the longitudinal moving distance corresponding to the gantry crane body, the transverse moving distance corresponding to the lifting appliance, the corresponding first moving speed and second moving speed according to the target position of the goods to be lifted so as to move the goods to be lifted to the corresponding target position; determining a current first position and a current second position and determining a corresponding first moving speed and a corresponding second moving speed in the process of moving to a target position of a cargo to be hoisted and placing the cargo to be hoisted; and the control module is also used for determining that the current first position and the current second position correspond to the target position of the goods to be hoisted, and separating the lifting appliance from the goods to be hoisted after the height of the goods to be hoisted is correct so as to finish the operation of placing the goods to be hoisted.

It should be understood that the specific order or hierarchy of steps in the processes disclosed is an example of exemplary approaches. Based upon design preferences, it is understood that the specific order or hierarchy of steps in the processes may be rearranged without departing from the scope of the present disclosure. The accompanying method claims present elements of the various steps in a sample order, and are not intended to be limited to the specific order or hierarchy presented.

In the foregoing detailed description, various features are grouped together in a single embodiment for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments of the subject matter require more features than are expressly recited in each claim. Rather, as the following claims reflect, invention lies in less than all features of a single disclosed embodiment. Thus, the following claims are hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate preferred embodiment of the invention.

The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. To those skilled in the art; various modifications to these embodiments will be readily apparent, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the disclosure. Thus, the present disclosure is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

What has been described above includes examples of one or more embodiments. It is, of course, not possible to describe every conceivable combination of components or methodologies for purposes of describing the aforementioned embodiments, but one of ordinary skill in the art may recognize that many further combinations and permutations of various embodiments are possible. Accordingly, the embodiments described herein are intended to embrace all such alterations, modifications and variations that fall within the scope of the appended claims. Furthermore, to the extent that the term "includes" is used in either the detailed description or the claims, such term is intended to be inclusive in a manner similar to the term "comprising" as "comprising" is interpreted when employed as a transitional word in a claim. Furthermore, any use of the term "or" in the specification of the claims is intended to mean a "non-exclusive or".

Those of skill in the art will further appreciate that the various illustrative logical blocks, units, and steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate the interchangeability of hardware and software, various illustrative components, elements, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design requirements of the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present embodiments.

The various illustrative logical blocks, or elements, described in connection with the embodiments disclosed herein may be implemented or performed with a general purpose processor, a digital signal processor, an Application Specific Integrated Circuit (ASIC), a field programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a digital signal processor and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a digital signal processor core, or any other similar configuration.

The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may be stored in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. For example, a storage medium may be coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC, which may be located in a user terminal. In the alternative, the processor and the storage medium may reside in different components in a user terminal.

In one or more exemplary designs, the functions described above in connection with the embodiments of the invention may be implemented in hardware, software, firmware, or any combination of the three. If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media that facilitate transfer of a computer program from one place to another. Storage media may be any available media that can be accessed by a general purpose or special purpose computer. For example, such computer-readable media can include, but is not limited to, RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to carry or store program code in the form of instructions or data structures and which can be read by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. Additionally, any connection is properly termed a computer-readable medium, and, thus, is included if the software is transmitted from a website, server, or other remote source via a coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL), or wirelessly, e.g., infrared, radio, and microwave. Such discs (disk) and disks (disc) include compact disks, laser disks, optical disks, DVDs, floppy disks and blu-ray disks where disks usually reproduce data magnetically, while disks usually reproduce data optically with lasers. Combinations of the above may also be included in the computer-readable medium.

The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are merely exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.

Claims

1. A method for determining state information of gantry crane equipment is characterized in that the gantry crane equipment comprises the following steps: the image acquisition device comprises a gantry crane body vertically arranged on the ground and a lifting appliance sliding on the gantry crane body in parallel, wherein a first image acquisition device is arranged at the gantry crane body, and a second image acquisition device is arranged at the lifting appliance; the method comprises the following steps:

acquiring a first image through the first image acquisition device, wherein the first image records the current line number corresponding to the position of the goods area on the ground of the gantry crane body;

acquiring a second image through the second image acquisition device, wherein the second image records the current column number corresponding to the position of the cargo area of the lifting appliance on the ground;

identifying the current line number in the first image, determining a current first position corresponding to the gantry crane body according to the current line number, and determining a first moving speed of the gantry crane body according to the current first position;

and identifying the current column number in the second image, determining a current second position corresponding to the lifting appliance according to the current column number, and determining a second moving speed of the lifting appliance according to the current second position.

2. The method of claim 1, wherein the first image acquisition device comprises: a first camera; the gantry crane comprises a gantry crane body and is characterized in that the gantry crane body comprises a gantry crane bridge parallel to the ground, and a first camera facing the ground is arranged below the gantry crane bridge, so that the first camera collects the number of lines of a cargo area marked on the ground and acquires a first image recorded with the current number of lines.

3. The method of claim 1, wherein the second image acquisition device comprises: a second camera; the gantry crane body comprises a gantry crane bridge parallel to the ground, the number of rows of cargo areas is marked below the gantry crane bridge, and the number of rows of cargo areas marked below the gantry crane bridge corresponds to the number of rows of cargo areas on the ground;

and at least one second camera is arranged on two sides of the lifting hook of the lifting appliance and faces the lower part of the gantry crane bridge, so that the at least one second camera acquires a second image, and the second image records the number of rows of the cargo areas corresponding to the marks below the gantry crane bridge.

4. The method of claim 1, wherein identifying a current number of rows in the first image and determining a current first position of the gantry crane body according to the current number of rows comprises:

identifying a first straight line in the first image for marking the row of the on-ground cargo area based on a Hough transform algorithm and determining an angle of the first straight line with the first image;

determining a position of the first straight line in the first image based on the angle;

adjusting the first image based on a coordinate conversion algorithm so that the first straight line becomes a vertical line;

detecting a first marker for marking the number of lines of the above-ground cargo area in the first image based on a convolutional neural network, the first marker being disposed between two adjacent first lines;

identifying each first identifier from the detected first identifiers based on a similarity algorithm;

drawing a first marking line parallel to the first straight line from the middle point of the top boundary of the first image, intersecting the region where the row number of the cargo area on the ground is located at a first intersection point, and determining the ordinate of the first intersection point in the first image and the corresponding first identifier based on the position of the first straight line in the first image and the identification of each first identifier.

5. The method of claim 4, further comprising:

based on the identified neighboring first markers, an occluded first marker is determined.

6. The method of claim 1, wherein the identifying a current number of columns in the second image and determining a current second location for the spreader based on the current number of columns comprises:

identifying a second line in the second image for marking the column of the on-ground cargo area based on a Hough transform algorithm and determining an angle of the second line with the second image;

determining a position of the second line in the second image based on the angle;

adjusting the second image based on a coordinate conversion algorithm so that the second straight line becomes a vertical line;

detecting a second mark used for marking the number of columns of the goods area on the ground in the second image based on a convolutional neural network, wherein the second mark is arranged between two adjacent second straight lines;

identifying each second identifier from the detected second identifiers based on a similarity algorithm;

drawing a second marking line parallel to the second straight line from the middle point of the top boundary of the second image, intersecting the area where the number of columns of the on-ground cargo area is located at a second intersection point, and determining the abscissa of the second intersection point in the second image and the corresponding second identifier based on the position of the second straight line in the second image and the identification of each second identifier.

7. The method of claim 6, further comprising:

determining the second identification which is shielded based on the identified adjacent second identification, and determining the length of the area where the number of columns of the goods area on the ground is located in the second image;

and determining the vertical distance of the lifting appliance relative to the lower part of the gantry crane bridge according to the length, the abscissa and the acquisition angle of the second image acquisition device.

8. The method of claim 7, further comprising:

when the number of the second image acquisition devices is at least two, determining the vertical distance of a lifting appliance corresponding to each second image acquisition device relative to the lower part of the gantry crane bridge;

and determining the final vertical distance of the lifting appliance relative to the lower part of the gantry crane bridge according to the at least two vertical distances.

9. The method of claim 4 or 5, wherein said determining a first moving speed of said gantry crane body according to said current first position comprises:

determining a difference value of vertical coordinates according to the current vertical coordinate and the vertical coordinate corresponding to the target moving position of the gantry crane body;

determining a time difference value according to the current time corresponding to the current ordinate and the target time corresponding to the target moving position;

and determining the longitudinal moving speed of the gantry crane body according to the difference value of the vertical coordinates and the time difference value.

10. The method according to any of claims 6-8, wherein said determining a second speed of movement of the spreader from the current second position comprises:

determining a difference value of the abscissa according to the current abscissa and the abscissa corresponding to the transverse target moving position of the lifting appliance;

determining a time difference value according to the current time corresponding to the current abscissa and the target time corresponding to the transverse target moving position;

and determining the transverse moving speed of the lifting appliance according to the difference value of the abscissa and the time difference value.