OPTICAL SENSING AND CONTROL SYSTEM .
This invention relates to an optical/visual sensing and control system for accurate positioning of a stacker crane, especially a hoisting rig or/and a crane for handling of containers within a container stack yard.
Prior art
Larger container terminals around the world are aiming for an increasing automatization of the container handling. One critical step in this automatization is the positioning of the hoisting rig over a container by the crane. Track bound cranes may be positioned relatively exactly on the tracks - within 25-50 mm - but many other factors like wind, stretching of the crane wire, oscillations etc. have as a result that an exact positioning of the hoisting rig hanging from the crane is quite intricate. Today this is done entirely manual by the crane operator and is com¬ pletely relying on his experience and judgement.
Many container terminals today, besides the so called Straddle Carriers, as well use huge cranes which are not driven on fixed tracks but are freely moving machines on big rubber tyres also called R.T.G. (Rubber Tyred Gantry). The existing systems today for automatic/semiautomatic control are formed most often either by means of induction loops or by some kind of sensed tags. One example of an induction loop is disclosed in e.g. the document EP-A-0098896 by Sumitomo Electric Industries. An induction loop as such is operating, but many docks or loading and unloading yards do not permit induction cables to be dug into the ground. For track bound technique are disclosed also different optical code readers in e.g. SU 1194-750-A and SU-907-564. Mechanical gyro systems are cumbersome and need a long period of time for stabilizing and have not been shown in this kind of application in mind here and are probably not fulfilling the demands set up for control, e.g., of equipment for container handling.
Further some other different systems have been tried for the determination of position of large freely moving machines for
handling of goods. These systems include techniques having decoders against the wheels of the vehicle which will not provide enough accuracy. Additionally have been tested contact rollers towards the ground surface which commonly give the proper accuracy but those demand that the ground surface is absolutely free from litter or small stones etc. A third system depends on discrete readable tags spread out in the yard area and which will be read by the machine when it passes such a tag whereby its position will be recognized. A large number of such systems are known from trackbound technique. One example of such a system is disclosed in the patent document GB 2112982 by Mitsui Engineer¬ ing and Shipbuilding Co. Ltd. (Japan) applied to a type RTG container crane. In the patent document DE 38 25 097 by Lanfer and Kugler, Germany is disclosed a device for positioning measurements applied to booms using color markings on the boom read by a simple optical sensor. Another system for identifying a container placed in a large stack served by a RTG is disclosed in the document GB 2 221 212 by Davy Morris Limited, Lough- borough, UK. This system uses a computer for the identification of and storing how the di ferent containers are located or were placed in the stack of containers.
For an effective handling of containers a system is needed which permits the vehicle to relatively freely move within a defined yard to handle the goods and that the system at a certain situation when loading more or less automatically may identify the position of the spreader relative to the container to be hoisted. Such a system for spreaders is published in the International Patent Document O-A1-90/09336 by Bromma Conquip AB. Another system disclosed in the American Document US 4 139 107 is using a detector plate on the container which is detected for positioning of the spreader. Up to now no complete system for the control of these machines handling containers is not known.
Closed TV-circuits are applied to cranes e.g. in the Patent Document DT 2642373 showing the use of a video camera attached to a lifting crane. The problem with a crane handling containers
is that the feeding of a direct video signal through a simple coaxial cable is difficult due to the mechanical stress such a cable would experience as such a connection has to be made through the crane wire to the hoisting rig where the camera must be positioned. Optical sensing systems having for example a video camera are as such also known from surveillance systems with one or more cameras fixedly mounted for surveillance of one or more surveillance sectors. Additionally there are such systems working with a certain set of standard pictures to be able to judge whether an intruder is present within a given area of surveil¬ lance. However this type of pure surveillance does not have to operate as fast as compared to a corresponding system in a moving application, which directly or indirectly controls the motion, and accordingly from this reason these known surveillance systems are not suitable for solving the aforementioned problems.
Description of the invention
The present invention provides a positioning sensing system for the hoisting rig of a loading crane, especially a machine handling containers. The system may normally be integrated with other computerized processing and automatics within the machine but it may also be installed as a separate unit in an already existing fixed or transportable hoisting crane to identify the position of the spreader relative to the container being hoisted.
The system for identification of the position of the spreader comprises one or more optical sensors, in a preferred embodiment constituting a standard resolving video camera using a CCD cell, a light source, and a microprocessor based interface which senses, converts and evaluates the signals from the optical sensor and compares obtained data with previously stored standard data for judgement of the position of the spreader relative to the standard corner box of a container.
The present invention further provides an arrangement for urging a goods handling machine to follow a system of colored lines painted on the surface of the container stack yard. The system ensures by means of one or more optical sensors attached to the machine that it maintains a path and direction without introduc¬ ing skew or position deviation relative to the colored lines painted on the surface of the yard.
The system in accordance to the present invention includes along with one or more optical sensors also lighting units attached to the goods handling machine for tracking of the colored lines on the yard surface, a data processing unit with an associated control and display device and a control unit arranged in the cabin of the machine to manually or semiautomatically move the machine in accordance with the colored lines on the surface of the yard. The colored line system on the surface yard also through incorporation of an encoding system provides that absolute positioning information will at every instant be obtained by the data processing unit within the machine.
The system for identification of the position of the hoisting rig further provides the following advantages:
A) the system only needs limited guidance arms eventually protruding at any side of a container,
B) the system even functions when the containers are stacked so that they are close together,
C) the system functions for all containers having standard corner boxes independent if those are lacking roof or sides,
D) the system functions for direct control of the crane as it gives exact information of distance and direction to the standard corner box, whereby offsets in the x- and y- direction are given in absolute measure, especially in mm,
E) the crane control and the operator do get during the lowering of the spreader continual information of the position of the spreader relative to the underlying con¬ tainer. Thereby corrections in x- and y-direction of the
spreader and the crane, respectively, are carried out during the lowering, which operation in principal does not have to be interrupted,
F) the system externally only demands small units for the optical sensing and lighting and a local processing unit, which easily may be positioned onto the hoisting rig,
G) the system functions in a height interval, which for the hoisting rig may vary between 0.1 and about 3 meters or even more,
H) the system provides the operator in a maneuvering cabin of the machine or, for a larger semiautomatic loading and unloading system, in a central control room also a kind of visual information of the position of the spreader relative to the container to be hoisted.
Brief description of the figures
The present invention will be described in several preferred embodiments to be read together with the enclosed drawings in which
Fig. 1 shows a spreader having an optical sensor comprising a video camera and an associated local processing unit including also a local electronic circuitry,
Fig. 2 shows a schematic block diagram for a control system of a spreader in a crane for handling of containers in accordance with the present invention,
Fig. 3 shows a preferred embodiment of an optical sensor comprising a CCD-type video camera in accordance with the invention,
Fig. 4 shows a preferred embodiment of a monitor for the control system with presentation of data regarding the position of the spreader and its motion illustrated on the screen,
Fig. 5 shows schematically in a preferred embodiment a portion of line markings on the surface of a container stack yard for control of a hoisting crane for con¬ tainers in accordance with the present invention,
Fig. 6 shows a schematic block diagram of a control system of a hoisting crane for containers in accordance with the invention, and
Fig. 7 shows the same monitor display as in Fig. 3 but having a presentation of data when maneuvering the crane machine.
Preferred embodiments
When a hoisting crane for container handling, e.g. a gantry has placed itself in a preliminary desired position to hoist a container, a spreader will start to be lowered down. In Fig. 1 is shown a principle plane view from above of a standard type spreader 1 for containers and having its four guidance arms 2 in a raised position. The spreader is further provided with an optical sensor 3 in the form of a CCD type videocamera 35 having a lighting arrangement 36 (Fig. 3) arranged at a corner 4. In a first embodiment the video camera is provided with fixed focus optics. Additionally is arranged onto the spreader a first local processing unit 7 comprising an electronic device PCU 71 (Fig. 2) for conversion of picture data from the CCD camera into digital pattern data, which are input to a number of first parallel processors 72 and then transmitted through a communica¬ tion path 74, e.g. an optical fiber, a cable system with a preferably two wire system or a directed radio link system, preferably in the Ghz region, transferring already partly processed and compressed information to a central processing unit 8. In Fig. 2 is shown a simplified block diagram of the optical sensing arrangement for positioning of the spreader. As may be seen from Fig. 2 there is also a height detection device 73 which continuously gives additional help information in form of a
height which partly in one embodiment of the present invention facilitates control of the focusing and the focal distance of the CCD camera 35 and partly for general information to the central processing unit 8 of the height of the spreader in relation to the highest surface of an underlaying subject, besides the information which the computer system is able derive from picture data using for example the size of one of the four locking holes present in the upper corner box surfaces of a container. Such a locking hole is for example according to ISO oval and has the standard measures 63 x 124 mm and the size of this locking hole in the picture generated by the camera is recorded by means of a processing unit. The height detector 73 gives the distance to the highest point of an object present within an area correspond¬ ing to slightly more than the area of the spreader by using well known measurement technique within the preferred embodiment constituting a small electronically connected ultrasonic measuring device, which however has a certain limit for contain¬ ers lacking a roof, while the picture processing besides the position sideways primarily also gives the distance to just the locking hole of a localized corner box, which is recognized by the local processing unit 7 and/or by the central processing unit 8 by comparison of the size of the standard holes with stored standard data e.g. as a function of the depicturing height. During the lowering of the spreader the optical sensor 3 identifies the corner having the normally oval locking hole at the corner box of the container closest to the corner 4 and the sensor 3 of the spreader, and the central processing unit 8 gives semiautomatically/automatically via a crane control interface 84 corrections to the crane for smaller adjustment so that the hoisting rig will land exactly against the standard corner box of the container. As the processing unit continually receives information about the position of the spreader eventual necessary adjustment sideways in position will be done (or forward or backward with the machine itself during continuous lowering of the hoisting rig, which operations in principal does not have to stop the lowering motion for a minor correction of the position. As the optical sensor 3 only searches for the corner or the
locking hole at the upper frame of the standard corner box it makes no difference if the container is lacking roof or sides. The operator receives simultaneously picture information on a monitor 85 with a screen as shown in Fig. 4. Thus the operator can also observe the container in a kind of side view and the position of the locking arrangement of the spreader relative to the locking hole in a simplified line sketch and calculated coordinate deviations, which give a very exact feeling of how the hoisting rig is positioned, which in turn will result in that the whole process may further be executed faster, besides that it will be more safe. If the spreader should be lowered below the normal height of a standard container the operator will be alarmed by the central processing unit that an abnormal situation is present.
In Fig. 3 is shown in a preferred embodiment an optical sensor consisting of a CCD video camera 35 together with a lighting unit 36. The lighting unit 36 at the CCD video camera primarily operates in the preferred embodiment with continuous lighting, but in a second embodiment it is working with intermittent lighting in the form of flashes synchronized with the electronic device for conversion of the picture information from the CCD camera into digital pattern information for further processing by the data processors 72 in Fig. 2. The camera system 3, 35 always regularly operates with continuous light when no extra light is used or when also the extra lighting 36 is continuous dependent of the type of lighting device.
In a first embodiment a processing unit compares the obtained picture data with previously stored parameters according to prior art. In the second embodiment the central processing unit 8 instead operates in two modes, in part a mode of learning and in part an operation mode. The mode of learning is used to create pattern sets in the memory device for comparation with actual obtained patterns in the normal operating mode i.e. during positioning of the hoisting rig. In the mode of learning the hoisting rig is manually operated to different heights and
positions over the standard corner box of a container and the simplified patterns thus created via the processing unit 7, and primarily obtained from the optical sensor 3, are stored in the memory device 76 and partly in the memory device 86 of the central processing unit, which comprises write and read memories (RAM), by means of established technique using a preprepared program for the processors. Thus the obtained reference patterns are transferred by means of an electronic device (not shown) to ROM circuits which then are inserted into the memory devices 76, 86 and will according to established technique form part of the read memory in the memory devices 76, 86, from which different reference patterns can be obtained during the operating mode.
Thus processing unit 7 in the second embodiment operates to partly create standard corner box reference picture patterns of a container and besides similar to the first embodiment obtaining actual picture patterns during the positioning of the spreader to the standard corner box. The ROM circuits obtained according to the procedure above which contains digital pattern data are preferably primarily placed into the local processing unit 7 onto the spreader itself so that an essential part of the picture processing will be done locally by a digital processor or several primary parallel processors 72, which preferably are so called transputers, to as far as possible locally reduce the amount of data, essentially in the form of coordinate information, which is then transmitted to and received by the central processing unit 8 in a compressed form according to known technique via a communication path 74 formed by for example an optical fiber or an electrical two wire system or a directed radio link system. A conventional physical connection with even more than two wires is of course possible for an eventually higher rate of transmis¬ sion, but a physical connection with as few wires as possible is the most proper when having in mind the stress which such a cable will experience. Thereby is also the possibility of incorrect transmission of information reduced to a large sense by the fact that an interruption of one of the two wires opposite to the use of many wires will result in that the transmission completely
will be interrupted as an indication that the cable has to be repaired or exchanged.
In Fig. 5 is illustrated a part of a larger line pattern in a preferred embodiment of the present invention painted with a color of preferably e. g. white, green, blue or red paint on the surface of the stack yard where a crane especially a machine handling containers is moving around. In Fig. 6 is demonstrated a corresponding schematic block diagram of the main parts in an arrangement for control of the crane itself. Fig. 6 demonstrates in the preferred embodiment two optical sensing units 3', 3", each with a CCD video camera 35', 35" and lighting units 36', 36" which in principle are identical to the optical sensing unit 3 shown in Fig. 3 and attached to the spreader. One camera unit 3' is here placed at the front of the crane while the other unit 3" is correspondingly placed at the rear of the crane. The arrange¬ ment shown in Fig. 6 further comprises an electronic device 71* for conversion of the picture signals into digital signals and a number of primary parallel processors 72' directly connected to the CCD video cameras via the electronic device 71' similar to the arrangement shown in Fig. 2. In a preferred embodiment for this application the processing unit 7' can of course be incorporated in the central processing unit 8, but from reasons of unitarity in this description the unit 7' is displayed as a separate unit. Additionally there is present similar to the previous picture the central processing unit 8 with the addition¬ al secondary parallel processors to which the primary parallel processors are connected over a data bus 74* . The central processing unit 8 processes the picture signals similarly to the previous description into the form of simplified picture patterns from the primary parallel processors 72' and partly shows the actual obtained measurement data on the monitor 85 (Fig. 7) and controls via a crane control interface 84 in Fig. 6 (not shown control means) the machine in its motion along the pattern of lines arranged within the container stack yard area.
In Fig. 5 a small portion of lines is visualized, in a preferred embodiment, along which the machine is maneuvered during transportation along a dock or in a loading and unloading area. Fig. 5 is showing a four way crossing of these tracking lines. If the crane is moving forward in the direction of the arrow 110 it is possible to choose between three possible track alterna¬ tives by means of a maneuvering unit (not shown) or by in advance inputting data to the central processing unit via a not shown keyboard. If the operator had chosen the command "left" the crane will follow the curved line 13, 31 and after that the line 30 and so on. If the operator instead had chosen the command "right", the crane follows the curved line 14, 41 and further the line 40 in the opposite direction of the arrow 140. If the operator finally would have chosen the command "straight forward" the crane will go via the line 12, 21 and further along the line 20 opposite the direction of the arrow 120. Correspondingly if the crane comes along the line 40 in the direction of the arrow 140 there are two possibilities of path alternatives by commands from the maneuvering unit or the not shown keyboard. Either is chosen a path to the left via the curved line 41, 14 and further along the line 10 opposite the direction of the arrow 110 or a path straight forward via the lines 43, 34 and further along the line 30. Accordingly there is here no possibility by following the system of lines to choose a path to the right and the crane cannot otherwise than fully manually be turned to the right as the optical sensors then will loose contact with the tracking line pattern and the central processing unit would not allow this during program controlled driving. From now on is therefore assumed computer controlled driving.
If the crane comes along the direction of the arrow 120 there is in this line crossing no alternative of choice but the crane has to proceed forward via the lines 21, 12 and further along the line 10 opposite the direction of the arrow 110. As is further shown by Fig. 5 there is always a short break in the line before a position where several alternative line paths are present to further ensure that the central processing unit 8 will catch the
change in the pattern along the path of motion. In a motion along the direction of the arrow 120 the interruption at 13, 14 indicates that alternative line paths join while in motion along the direction of the arrow 110 the interruption at 13, 14 indicates that alternative paths of motion are present. As the machine is provided with identical optical sensors in the front 3' and at the rear 3" the system will function in the same way independent if the machine is driven forward or backward. In reverse motion the rear optical sensor 3" is connected with its camera 35" as a primary sensor by the central processing unit 8 while during forward driving the forward optical sensor 3' with the camera 35' is the primary optical sensor.
Each line segment further comprises in a first embodiment binary encodings which is partly exemplified in Fig. 5. The central processing unit 8 reads the binary data of the simplified digital pattern received from any of the primary parallel processors 72' . In the first embodiment this is done by means of a synchroniza¬ tion mark which is longer than the other 8 marks in the embodi¬ ment in Fig. 5. These are placed on either side of the lines dependent whether they should indicate a logical "one" or a logical "zero". From the synchronization mark is the most significant bit MSB is given for the binary number facilitating that the patterns unambiguously can be read from both directions. In a second embodiment this binary word of data contains considerably more than 8 bits, preferably 16 bits and are found at numerous places along a segment of line why the central processing unit 8 by means of these will be able to decide a very exact position of the crane, partly by the position of the digital pattern information along the line, and partly by the time between each such read pattern information which also simultaneously gives information about the speed of the machine. In a third embodiment these binary pattern number are formed by a redundant bar code representing a number of binary bits by means of lines with different thicknesses. By in the preferred embodiment reading at both sides of the line also redundant information is achieved to further ensure correct reading. In
each position in the first and second embodiments the coding marks for logical "one" and logical "zero" can be found only on either side of the line excluding the longer synchronization marks why the binary number will be read twice, direct and reverse, whereby the actual processor controls that the two registrations correspond. If such a correspondence is not achieved the operator will be alerted, and he can check whether the painted colored line might have been damaged or a subject of the right color and right size and shape as a mark can be found or if some unqualified item is covering the line. In case of an eventual damage the line is simply adjusted by means of new color and by means of a templet according to well known technique. In the embodiment there is close to the CCD camera 35', 35" and its lighting arrangement 36', 36" an air pressure unit (not shown) blowing towards the area which is detected by the CCD camera. In this way small amounts of grit or snow is blown away from the line to obtain correct scanning of the line.
In a first embodiment a processor compares the picture data, achieved by a program, with previously stored parameters according to prior art. In the second embodiment the central processing unit 8 operates according to the previous description in two modes, one mode of learning and one operation mode. The learning mode is then used as previously described, to create pattern sets in the memory device 86 for comparison with actually obtained patterns during an action of motion. In the mode of learning the crane is manually driven to different types of crossings in the line network and simplified patterns derived by the optical sensors 3', 3" are stored in RAM similar to the previously mentioned example in connection to Fig. 2 by means of a preprepared program for the processors according to established technique. The reference patterns thus obtained are transferred by means of the electronic device (not shown) to ROM capsules which then are inserted into the memory device 76 or 86 alterna¬ tively, and will according to known procedure constitute a read memory from which the reference patterns are obtained during the operation mode which already has been described previously in
connection to the description of the positioning of the hoisting rig in the preferred embodiment.
In Fig. 7 is shown in a first and a second embodiment the monitor screen 85 connected to the central processing unit 8. By means of a picture according to Fig. 7 on the monitor screen 85 the operator receives actual information about the position of the crane relative to desired path of motion. The operator has a possibility to choose between manual maneuvering or semiautomatic maneuvering and additionally if measures should be given in metric unit i.e. m and cm or according to for example measures using foot and inches etc. The monitor screen also shows how the motion is planned to continue according to eventually already fed information to the central processing unit 8.
The way of operation of already wellknown parts in the processing units 7, 7' and 8 with the control functions and computer programs as well as a detailed way of function for the electronic devices 71, 71' have here been excluded as these parts do not constitute the invention but only constitutes parts to provide the solution of the total maneuvering problem which is the objective of the present invention. Characterizing marks of the invention is defined by the independent claims 1 and 2 and the associated dependent claims.