WO2009052854A1

WO2009052854A1 - Device, method and system for recording inspection data about a freight container

Info

Publication number: WO2009052854A1
Application number: PCT/EP2007/061292
Authority: WO
Inventors: Uno Bryfors; Claes Heidenback; Clara Holmgren
Original assignee: Abb Ab
Priority date: 2007-10-22
Filing date: 2007-10-22
Publication date: 2009-04-30

Abstract

A device, method and system for recording inspection data about a freight container comprising making an image of at least one side the container. In one aspect a sensor platform or sensor holder means (2) device is disclosed that is moveable with respect to the crane in a predetermined direction and controlled dependent on a movement of said container. The sensor on the platform is moved so as to synchronise with a planned path for the container such that the camera may be moved to a point where it may focus on the container and make an image of the container. A method for operating the device and a container tracking system comprising the device and one or more computer programs are also described.

Description

Device, method and system for recording inspection data about a freight container

TECHNICAL FIELD.

The invention relates to method of recording inspection data about a freight container and a system for carrying out the method. The method may involve automatic and/or manual identification and inspection of cargo containers in a loading or unloading situation, and especially for capturing an image of at least a part of a freight container.

BACKGROUND ART

Standard shipping containers are used to transport a great and growing volume of freight around the world. Transshipment is a critical function in freight handling. Transshipment may occur at each point of transfer from one transport means to another, from ship to shore in ports and harbours for example. There is usually a tremendous number of containers that must be unloaded, transferred to a temporary stack, and later loaded on to another ship, back onto the same ship or loaded instead onto another form of transport . Loading and unloading containers to and from a ship takes a great deal of time. The development of automated cranes has improved loading and unloading and made the productivity more predictable, and also eliminated many situations in which port workers have been exposed to danger and injury.

Traditionally, for containers being loaded or unloaded to or from a ship or a train the IDs have been read or handled manually by an operator or a checker and entered manually into a computer system for controlling operations in the cargo container terminal. There are several problems with this type of handling: -keeping staff on the ground is a safety risk due to heavy traffic; -there is a risk of entering wrong information into the system: -there is no "evidence" saved for later review;

-manual handling tends to take a long time and can be a bottle neck in handling.

For a time period in which the identity of a container being lifted is not known by the crane control system, the identification of a where the container should be landed either has to wait, or else the container being lifted has to be landed at a temporary holding place.

During the last few years a number of systems have been developed to automatically apply Optical Character Recognition technique to images of the Container Code. For example US 2003/0167214, entitled Scheduling method for loading and unloading containers at the terminal; describes a method in which Container ID numbers are tracked during loading on to a ship and unloading off the ship and also used for organising a container storage system. US6,768,931 entitled: Method and apparatus for quay container crane-based automated optical container code recognition with positional identification; assigned to Paceco Corp., describes an optical characteristic system providing container code recognition from a quay container crane of a container identified by a container code to a container inventory management system.

The tasks earlier performed manually by a person such as the checker involved gathering more information about the container than just the ID-number. Part of this information is only available on the container short sides. However, it is technically very demanding to photograph the short side of a freight container.

SUMMARY OF THE INVENTION The aim of the present invention is to provide an improved device, method and system for recording inspection and image data for a freight container. This and other aims are obtained by a method, and a system wherein the attached independent claims. Advantageous embodiments are described in sub-claims to the above independent claims.

to be completed when claims are settled....

The system comprises one or more cameras mounted on the crane structure, preferably on one or more crane legs. To always get the best possible images, at least one camera is moveable. Other sensors or cameras can be either fixed or be movable. The moveable camera is moved in a predetermined direction to a position from where it may focus on the container. Movement is carried out and controlled by using the container position and speed as reference to control the camera movement. The camera control system may obtain its control reference for camera position from a crane control system or from external sensors, or from a combination of both. The cameras used can be of any type of non-contact sensor suitable for the application, and preferably line scan cameras, High Definition cameras or traditional PAL system cameras.

The images may be used in any type of container identification, damage inspection or for other purposes. The images can be used for automatic analysis as well as manual inspection. A Container ID number for the container is obtained by processing or digitally processing image data recorded by a sensor mounted on a moveable sensor platform. The image data is analysed to resolve a plurality of information items including any of a container identification number, an IMO placard, door type or door direction, condition of a door seal, damage to the container. The Container ID number input is analysed to test if it is of a valid form in terms of number of characters, whether letters or numbers and so on. If the ID number is valid, the number can be used to plan the desired or optimum destination for landing the container and a record is made or updated in a database to list the current handling, which crane, date and time etc., and together with or linked to the recorded image.

In cases where the container number is found to be not readable, or not to be a valid number, the image is automatically processed and stored in the archive with an internal database identifier only, and the container designated for a manual inspection process .

For valid ID numbers a search is made by the archive system to find if the container is known to the system. Information about each container already handled in a container yard or port is comprised in some way as Property information for each container. The image data and other container information is stored in a predetermined way. One or more data storage systems are arranged capable of retrieving previous container image data and comparing a picture of the back, or front etc of the container to an earlier or a later image.

The processed image data for a known container is stored in a database or an archive. When retrieved, the image may be examined as 2D image . The image data per container may also be provided as a 3-D image or a 3-D rendered image of the container. The image may be rotated and manipulated on screen or via a graphic user interface by an operator.

The main advantage is that important information about a container may be quickly and automatically generated from an image of the container taken by means of a sensor or camera mounted on the movable sensor platform. Rapid identification and/or establishment of door position, rapid identification and/or establishment of a door seal or a missing door seal; establishment and reading of an IMO (International Maritime Organisation) placard in respect of hazardous loads according to International Maritime Dangerous Goods (IMDG) Code, all this information determines how the container should be handled, where in a stack of containers a given container shall be stacked, whether any inspection is necessary for the container, and thus where it is to be landed.

Container information including damage information may be automatically compared to previous container damage information so that for a given damage it is established before or after which dates and events the given damage to the container was sustained. Information from manual inspections may be input and records updated where necessary. A database or archive provides a reliable and comprehensive record of date and extent of damage to each or any container handled. Reports may be produced and graphic information in 2D or 3D examined and inspected on-line or off-line as necessary. Early identification means that the container may be lifted and moved to a desired or optimum location directly and automatically.

Another object of the present invention is to provide an improved computer program product and a computer readable medium having a program recorded thereon, for recording inspection data about a freight container handled by a crane.

BRIEF DESCRIPTION OF THE DRAWINGS A more complete understanding of the method and system of the present invention may be had by reference to the following detailed description when taken in conjunction with the accompanying drawings wherein:

Figure 1 shows in a simplified schematic diagram a container crane and a sensor platform or sensor holder according to an embodiment of the invention;

Figure 2 shows a flowchart diagram of a method for processing inspection data for a freight container according to another aspect of the invention; Figure 3 shows a simplified diagram for a part of a container crane arranged with two sensor platforms according to another embodiment ;

Figure 4 shows a flowchart for a method of recording inspection data about a container using at least two cameras according to another embodiment;

Figure 5 shows schematically an interface for displaying inspection data gathered about a freight container;

Figure 6 is a schematic diagram of a container short side with a container identity number.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Figure 1 shows a simplified diagram of a crane structure holding a container 4. The crane 6 may be of the ship-to-shore (STS) type, arranged on a dock or in another part of a container yard. The crane is arranged to transfer containers 44 to or from a transport vehicle such as a container ship 10. The crane has vertical support structures including the parts usually referred to as a leg 15. At the bottom of the leg suitable wheels may be mounted which often are arranged to run on a rail. The rail would run in the Y direction in and out of the page, a direction also known as the gantry direction. A spreader 7 is suspended from a trolley 8 which runs in the X direction from ship to shore (or from shore crane to ship) , which direction is also known as the trolley direction.

A camera 1 is arranged on one first leg 15 of the crane 6. It is arranged on a measurement platform or sensor holder 2 which is arranged moveable. The sensor platform may, for example, be moved in a vertical or near vertical direction V_app. When handled, the containers 44 are lifted out of the ship approximately along a path P. When lifting commences in the ship 10 the container may be out of line of sight of a fixed point on the crane leg for much of the duration of the lift.

As the container is lifted eg out of the ship, the platform or sensor holder 2 carrying a camera is controlled to move up or down so that the camera 1 may focus on the moving container at the earliest opportunity during the lift. One way to describe this is to say that a window for focussing is defined within which a sensor or camera fixed on the crane structure can rapidly make an image of a container that has begun to be handled, even if the container is not always in line of sight of the camera. By the time the container has been moved over the dock to be lowered between the crane legs, the camera has been moved to focus on the container, make an image, and the image data has been processed, registered, stored, matched and identified to the container of interest. Thus camera 1 may be moved up or down along the approximately vertical direction V_app in order to be in position to focus on a container while it is being moved. The sensor platform is driven using information from the crane control system. The crane control system will lift the container along a path, for example path P from ship to shore. A movement path is normally planned and information used to control the movement of the spreader holding the container is supplied to the sensor platform. The sensor platform is driven according to crane control system information so that the camera is moved to a position where it can focus on the moving container and make an image .

It is also possible that the camera may be moved to focus using a movement in a horizontal or approximately horizontal direction, indicated by the horizontal double arrow H_app as shown in Fig 1. The principle is similar. Information about the container position and/or a movement path is provided from the crane control system to the sensor platform and the sensor platform is moved so that a camera is moved to a position where it can quickly record an image of the container.

Spreader position may be used to provide position data for the container that is being lifted. Preferably, a Load Position System (LPS) from ABB is used to determine, from the trolley position and the spreader position, a position of a container in space. The lifting and movement of a container by a container crane is controlled by a crane control system. This system calculates a movement path for each container as it is handled. Typically in a ship-to-shore lift a container is lifted out of the ship along a path such as path P in Fig 1, and landed either on a vehicle under the crane or, the crane may move for example in the gantry (Y) direction and place the container on a designated container slot or on a stack of containers. In order to determine the correct place to land the container the identification of the container must be known. Given the identity of the container, it may be determined where it shall be best placed ready for subsequent handling and a movement path including the landing may then be calculated. Thus early identification of the container allows for movement directly to the correct desired landing place.

Correspondingly, when a container is lifted from the land side to be loaded into a ship, as soon as a container identity is determined from the recorded image then the correct stacking position in the ship may be found and a movement path to land the container on the correct place or container stack may be calculated.

Figure 2 shows a simplified flowchart for a method for recording image data for a freight container using the sensor platform or sensor holder 2. The figure shows the actions of: 22 a new or present container position is calculated from a spreader position or by a load positioning system LPS;

24 present camera position is found or retrieved from memory; 26 container position information is received 19 from crane control system 18 for a movement path planned for container 28 target camera position to be able to focus on container at time t+1 is calculated,

30 sensor platform or sensor holder is moved to the calculated focus position to arrive ready for time t+1

32 is container in focus?

N repeats steps such as 24-32 34 if Y make and record image of the container

The image is processed to identify information such as Container identity code IMO placard Door direction Damage Seal condition The images taken by the camera 1 may be sent to an image processing unit together with an internal database object identifier number. The images per container comprise at least one view of a short side of the container (B or F, Fig 5) . Depending on camera position and time, up to five views may be photographed, these 5 being images of 4 sides: 2 long (L, R, and two short sides F, B) and the top of each container.

Figure 6 shows a schematic diagram of a short side of a freight container 60. The figure shows schematically a container ID number 62 and an IMO or IMDG- type placard.

The Container ID is processed to determine that it includes a valid number and/or combination of letters, numbers, spaces, hyphens etc. If no record is found for the present container ID the Container ID is stored as a new Container ID and sent to a database or archive for storage. If the container ID is already known, the existing container information is updated block in a container information database or archive. The updated record is stored for a predetermined length of time, such as five years, so that the databases accumulate information over time about a series of handling operations for each container.

If the Container ID number is not of a valid form, indicating that numbers or letters are missing or obliterated or otherwise not distinguishable, then the container, identified by the internal database object number, is placed in a Work Order queue for a manual inspection. The container may be landed in a specified location to await inspection. The image data is stored in image database associated with the internal database number stored temporarily in the information database. After a manual inspection the Container ID is input manually to the system and checked to see if it is already known. If yes, existing information is updated and if not then a record for a new container is created.

The sensor platform or holder means may be arranged with one or more non-contact sensors being any from the group of: active or passive Radio frequency sensors, infra red sensors, video camera, CCD camera, 3-D laser. Magnetic tags or RFID (Radio Frequency Identification) tags are sometimes used for identification purposes on some containers. Such magnetic or RFID tags may be read by a reader arranged on the sensor platform when it is moved within range of the container in a similar way to moving a video camera within focussing range of the short side. Other types of transponder tags may also be read by a suitable reader mounted on the sensor platform.

The image data recorded by a video line scan camera, High Definition camera, traditional PAL system camera, CCD or other camera when processed may show damage to the container. This may be recorded only, or the detection of damage may trigger a report or warning that an inspection may be necessary. A damage inspection may be signaled or even carried out by an operator examining database information and comparing older views of the same container to present views. The operator may issue a report based on visual and/or numerical and date information provided by the system or he may request a manual inspection if, for example, the damage is extensive, goods may be falling out or leaking out of the container, of if a door seal appears to be broken or lost. A damage inspection would then provide an answer that the container is either damaged, and a damage report is generated, or not damaged. In both cases, the information in the Container information database or archive concerning damage to the container may be updated. The images from previous handlings may be combined or superimposed or animated or processed in another way to show the progress and extent of the damage during successive time intervals.

An IMO placard is a sign according to the International Maritime Organisation that has information that the container includes items belonging to one or more of the IMO classes of Dangerous Goods or Hazardous cargoes .

An information about a door direction may describe the type and number of doors, and opening direction. This information is useful to determine in what orientation the container shall be placed.

Information about a door seal is important. A missing door seal may mean that the container must be placed in a special area and scheduled for manual inspection and customs inspection before any further transport may take place. Similarly a damaged door seal may require that the container is inspected.

Visible damage is recorded and the image data kept. The damage information may be compared with previous information or image information for the container of interest to determine between which handling dates the damage has occurred. A damage report may also lead to a container being set aside for an inspection before further transport takes place.

A graphical user interface (GUI) may be used to display one or more of the information or values obtained using the system and methods described above. Figure 5 shows schematically a simplified representation of a GUI 50 which in the example depicted displays at least a front F and/or back side B (short sides) view of a container. A pointing device or other input device 52 may be used by an operator or manager etc to examine or manipulate 2D images or even 3D images comprising at least the short side image data. An old damage image may be compared to a later or present image using the interface. Such a comparison of image data for the same container from different dates allows a computer process or a human operator to determine, for example, if new damage has occurred or not. The graphic user interface for manipulation by the operator to call up, examine or register additional information, dates, values and so on may be provided by a thin client program based on internet protocols generating the GUI from live and/or stored data about container images and handling events.

As described above, a Load Position System (LPS) is preferably used to determine, from the trolley position and the spreader position, the instantaneous position of a container in space. However it is also possible to determine the position of the container under the spreader by means of external sensors . In addition, data from a LPS may also be supplemented by data from external sensors .

As mentioned the processing of the container image comprising the container inspection data may be carried out automatically by one or more or computerised processes without any need for supervision by or actions from an operator. For the cases where a Container ID is not valid, or when a Manual Inspection is required work orders for manual intervention may be issued. At any time an operator or other authorised person may access the system to display, view, inspect or analyse live data on-line or off line as required.

In another embodiment of the invention at least two cameras may be mounted such that at least one of them is moveable. Figure 3 shows a schematic diagram for a part of a container crane. The figure shows two crane legs 105, 106 either of which may correspond to the crane leg 15 of Figure 1. Figure 3 also shows two sensor installations 101 and 102, camera installations. At least one of them, 102 is capable of being moved in a vertical or near vertical direction up to a height indicated as H. The two cameras may focus on the short sides of a short container shown 109, 110 or on the short sides of a long container 111,

112.

Figure 4 shows a flowchart for a method for making an image of a short side of a container. The method comprises block:

200 new container picked up by spreader

202 first position of the new container found 205 from information held by an automatic crane control system about the spreader position, and/or

206 information from external sensors to determine position of the new container 206 calculation for camera position necessary to focus on the new container according to a movement path

208 camera 1 and or a second camera are moved towards a planned position for focusing dependent on the planned movement path in the crane control system for this container 210 camera 1 and/or the second camera makes an image of the short side of the container

212 image processed to extract and/or calculate or otherwise determine any of: a container ID number, a door direction, a door seal condition, an IMO placard, damage 214 generate information for a container ID number and other data for the current container

In another embodiment of the invention, the image data may be stored in one or a plurality of databases. It is advantageous keep the retrieval time low despite the amount of storage space required to store the entire database and maintain image data for a period of five years and more.

In other embodiments the image may be rotated and manipulated on screen or via a graphic user interface by an operator by means of a graphic user interface displayed on a touch screen. Users may also use movements of their hands instead of the pointing device or a stylus to signal some of the selections, rotation, or commands. Instead of, or as well as, touching the display screen a movement or an action for example, by a user on or close to the touch screen surface may be detected or determined, for example by means of an inertial sensor or laser-based position or movement measuring arrangement. The graphic user interface may be provided by a thin client program based on internet protocols generating the GUI from live and/or stored data about container images and handling events.

It should be noted that while the above describes exemplifying embodiments of the invention, there are several variations and modifications which may be made to the disclosed solution without departing from the scope of the present invention as defined in the appended claims.

Claims

1. A sensor platform device for recording inspection data about a freight container, said sensor platform device being arranged on a structure of a container crane (6) in a freight terminal and comprising at least one sensor (1) arranged to make an image of part of a container (4) being handled by the crane, characterised by a sensor platform or sensor holder means (2) is arranged moveable with respect to the crane in a predetermined direction and controlled dependent on a movement of said container.

2. A device according to claim 1, wherein the platform or holder means is arranged for movement in a vertical or near vertical direction (V_ap_p) .

3. A device according to claim 1, wherein the platform or holder means is arranged for movement in a horizontal or near horizontal direction (H_app) .

4. A device according to claim 1, wherein a control means is arranged for controlling movement of said sensor platform or holder means in a predetermined direction dependent on information (x) from a crane control system (z) .

5. A device according to claim 1, wherein a control means for controlling movement of said sensor platform or holder means in a predetermined direction dependent on information of a planned movement path (x) from the crane control system.

6. A device according to claim 1, wherein a control means for synchronising movement of said sensor platform or holder means in a predetermined direction dependent on information from the crane control system (z) of a planned movement path (x) .

7. A device according to claim 1, wherein a drive means for moving said sensor platform or holder means in a predetermined direction.

8. A device according to any previous claim, wherein the sensor platform or holder means also comprises one or more sensors being any from the group of: active or passive Radio frequency sensors, infra red sensors, video camera, CCD camera, 3-D laser.

9. A device according to claim 1 or 8 , further comprising camera means for making an image of said container.

10. A device according to claim 1, further comprising means for communicating an image of said container to said container crane tracking system.

11. A method for recording inspection data about a freight container in a freight terminal using a container crane comprising at least one sensor being arranged on a structure of the container crane (6) where said at least one sensor (1) is arranged for making an image of part of a container (4) being handled by the crane, characterised in that the method comprises -moving by means of a sensor platform device the at least one sensor (1) in a predetermined direction (V_app, H_app) to a position dependent on a position of a said container being handled by the container crane, and -making an image of at least one side of said freight container.

12. A method according to claim 11, further comprising controlling movement of said sensor platform or holder means in a predetermined direction dependent on information of a planned movement path of said freight container from a crane control system.

13. A method according to claim 11 or 12, further comprising synchronising movement of said sensor platform or holder means to movement of said freight container dependent on information from the crane control system.

14. A method according to claim 11, 12 or 13, further comprising controlling movement of said sensor platform or holder means in a predetermined direction by a control means associated with a sensor platform or sensor holder drive means.

15. A method according to claim 10, further comprising processing the image of one side of said freight container and generating a container identification code number calculated from data comprised in the image.

16. A method according to claim 10, comprising making an image of at least one short side (B, F) of said freight container.

17. A method according to claim 11, further comprising communicating the container identification code number to a container handling system.

18. A method according to claim 11, further comprising processing the image of one side of said freight container, saving the image for later retrieval, and generating any data from the group: a door direction, an IMO placard, presence of a door seal.

19. A method according to claim 11, further comprising processing the image of one side of said freight container, saving the image for later retrieval, and generating data in respect of damage to the container.

20. A method according to claim 11, comprising the further step of analysing a Container ID number, and determining it to be of a valid form.

21. A method according to claim 11, comprising the further steps of analysing a Container ID number, determining it to be of invalid form and creating an order for a manual inspection to determine a valid Container ID.

22. A method according to claim 16 or 17, comprising the further step of updating the stored property information data for a container with an updated Container ID number in the Container information text database (12) .

23. A method according to claim 11, further comprising sending the image data to the container crane tracking system communicated wirelessly in part.

24. A method according to claim 11, comprising moving two optical sensors or cameras a predetermined direction (V_app, H_app) dependent on a position of the said freight container being handled by the container crane.

25. A method according to any previous claim 11-24, further comprising processing the image of one side of said freight container and manipulating the image by means of a graphic user interface provided by a thin client program based on internet protocols generating the GUI from live and/or stored data about container images and handling events.

26. A computer program which when read into a computer or processor will cause the computer or processor to carry out a method according to the steps of any of claims 11-25.

27. A computer readable medium comprising a computer program which when read into a computer or processor will cause the computer or processor to carry out a method according to the steps of any of claims 11-25.

28. A system for recording inspection data about a freight container comprising at least one sensor arranged on a structure of a container crane (6) in a freight terminal wherein the at least one sensor (1) is arranged to make an image of part of a container (4) being handled by the crane, characterised in that the system comprises a sensor platform or sensor holder means (2) for holding or bearing the at least one sensor, the sensor platform being arranged with means for moving the sensor platform with respect to the crane in a predetermined direction and dependent on information about a movement of said freight container.

29. A system according to claim 28, further comprising means for by controlling movement of said sensor platform or holder means in a predetermined direction dependent on information of a planned movement path of said freight container from a crane control system.

30. A system according to claim 28, further comprising means for synchronising movement of said sensor platform or holder means to movement of said freight container dependent on information from the crane control system.

31. A system according to claim 28, further comprising means for transmitting information from a crane control system for controlling lifting and handling of the containers to the sensor platform or sensor holder, or control system thereof.

32. A system according to claim 28, further comprising means such as for of a Load Position System for calculating a position of a freight container held by a crane.

33. A system according to claim 28, further comprising means comprising one or more sensors mounted on the spreader and/or crane for determining and calculating a position of a freight container held by a crane.

34. A system according to claim 28, in which the system further comprises means (5) for receiving an image of said container, from at least one sensor (2) and means for processing image data .

35. A system according to claim 28, in which the system further comprises a graphical user interface, wherein the graphical user interface further comprises a display arranged with means to present and manipulate data about the container comprised in and extracted from said image data.

36. A system according to claim 35, in which the graphical user interface comprises means to display or manipulate changes or differences between a part of said image data relative a part of a previous image data for the container.

37. A system according to claim 35, in which the graphic user interface is provided at least in part by a thin client program based on internet protocols generating the GUI from live and/or stored data about container images and handling events.

38. Use of a system for recording inspection data about a freight container for determining an identity and a condition of the freight container according to any of claims 26-37.