KR102687873B1 - Method for estimating location of unmanned vehicle using container location - Google Patents

Abstract

According to one embodiment of the present invention, receiving a container number image taken from an unmanned vehicle, recognizing a container number from the container number image, and selecting a container number corresponding to the container number based on pre-stored location data for each container. A method for estimating the location of an unmanned vehicle is disclosed, including the steps of recognizing the location of a container and estimating the location of the unmanned vehicle by estimating the distance between the container location and the unmanned vehicle.

Description

Method for estimating the location of an unmanned vehicle using container location {METHOD FOR ESTIMATING LOCATION OF UNMANNED VEHICLE USING CONTAINER LOCATION}

The present invention relates to a method for estimating the location of an unmanned vehicle using the location of a container.

Currently, technology is being developed to minimize the berthing time of ships by processing the loading/unloading work in a short time for large quantities of large ships at container terminals.

In particular, due to the development of autonomous driving technology, the transportation of containers at container terminals is also being carried out by unmanned vehicles.

Accordingly, in the driving management of yard trucks (YT: YT) running in a container terminal, manned YT and unmanned YT are mixed.

In the conventional case, the stationary position of the unmanned YT in the port equipment is recognized using radio frequency identification (RFID), or the unmanned YT recognizes its current location using GPS and DGPS.

However, if GPS reception is not possible in an unmanned yard truck or the error range is greater than 5M, the accuracy of BAY and ROW locations may decrease, making it difficult to confirm whether the vehicle has properly arrived at the destination location.

The problem to be solved according to an embodiment of the present invention includes confirming the location of the unmanned yard truck through the location of the container corresponding to the container number recognized by the unmanned yard truck.

The technical problem to be achieved by the present invention is not limited to the technical problem mentioned above, and other technical problems not mentioned can be clearly understood by those skilled in the art from the description below. There will be.

In order to solve the above problem, a method for estimating the position of an unmanned vehicle according to an aspect of the present invention includes steps of receiving a container number image taken from an unmanned vehicle, recognizing a container number from the container number image, and pre-stored containers. Based on location data, the method may include recognizing a container location corresponding to the container number and estimating a distance between the container location and the unmanned vehicle to estimate the location of the unmanned vehicle.

Here, the container number image taken from the unmanned vehicle may be an image of one side of each container whose equipment has been completed at the container terminal.

Here, the step of recognizing the container number in the container number image includes extracting a region of interest (ROI) image from the container number image using a character recognition program for the image, and extracting the region of interest (ROI) image by image processing. It may include a step of extracting a container number composed of letters and numbers from the region of interest image and recognizing the letters.

Here, the stored location data for each container may be data that stores the device location of the installed container and the number of the container after at least one container is installed by an external truck.

Here, estimating the location of the unmanned vehicle by estimating the distance between the container location and the unmanned vehicle includes obtaining location coordinates of the container corresponding to the container number, using the location coordinates of the container, Calculating first position coordinates of the unmanned vehicle when the container number image is photographed as a rectangular image, calculating a coordinate transformation matrix based on the shape of the previously photographed container number image, and calculating the coordinate transformation matrix. It may include calculating second position coordinates by converting the first position coordinates.

A method for estimating the location of an unmanned vehicle according to another aspect of the present invention includes identifying current location information of an unmanned vehicle, assigning a task to the unmanned vehicle based on the current location information, and content of the assigned task. calculating the destination and route based on the route, receiving a container number image taken from an unmanned vehicle traveling on the route, recognizing the container number from the container number image, based on pre-stored location data for each container. , recognizing the container location corresponding to the container number, and estimating the location of the unmanned vehicle by estimating the distance between the container location and the unmanned vehicle.

Here, receiving a plurality of landmark images taken from an unmanned vehicle traveling on the route, recognizing a location corresponding to the landmark image based on pre-stored location data for each landmark, and the container location and It may further include estimating the location of the unmanned vehicle by estimating the distance between the unmanned vehicles.

As described above, according to the embodiment and various aspects of the present invention, the information gap can be resolved by creating a landmark using the container number and correcting the unmanned yard truck location information.

The effects of the present invention are not limited to the effects described above, and should be understood to include all effects that can be inferred from the configuration of the invention described in the description or claims of the present invention.

Figure 1 is a block diagram showing a control system for performing a method for estimating the location of an unmanned vehicle according to an embodiment of the present invention.
Figure 2 is a flowchart showing a method for estimating the location of an unmanned vehicle according to an embodiment of the present invention.
Figure 3 is a diagram for explaining a situation in which an unmanned vehicle photographs a container number image according to an embodiment of the present invention.
Figure 4 is a flowchart showing the step of recognizing a container number in the method for estimating the location of an unmanned vehicle according to an embodiment of the present invention.
Figure 5 is a diagram for explaining a container number image and container number according to an embodiment of the present invention.
Figure 6 is a diagram for explaining location data for each container according to an embodiment of the present invention.
7 and 8 are diagrams for explaining a method of estimating the distance between a container location and an unmanned vehicle according to an embodiment of the present invention.
Figure 9 is a diagram for explaining a work path assigned to an unmanned vehicle according to an embodiment of the present invention.
Figure 10 is a diagram for explaining a landmark image according to various embodiments of the present invention.

Hereinafter, the present invention will be described with reference to the attached drawings. However, the present invention may be implemented in various different forms and, therefore, is not limited to the embodiments described herein. In order to clearly explain the present invention in the drawings, parts unrelated to the description are omitted, and similar parts are given similar reference numerals throughout the specification.

Throughout the specification, when a part is said to be "connected (connected, contacted, combined)" with another part, this means not only "directly connected" but also "indirectly connected" with another member in between. "Includes cases where it is. Additionally, when a part is said to “include” a certain component, this does not mean that other components are excluded, but that other components can be added, unless specifically stated to the contrary.

The terms used herein are only used to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that it does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

One embodiment of the present invention relates to a method for estimating the location of an unmanned vehicle using the location of a container.

Figure 1 is a block diagram showing a control system for performing a method for estimating the location of an unmanned vehicle according to an embodiment of the present invention.

Referring to FIG. 1, the control system 10 for performing a method for estimating the location of an unmanned vehicle according to an embodiment of the present invention includes an unmanned vehicle control unit 20, a terminal operating system 30, and an integrated DB 40. can be connected

The yard truck control system 1 may include a server 11.

The server 11 may be implemented as a computer device or a plurality of computer devices that communicate with at least one unmanned vehicle control unit 20 over a network to provide commands, codes, files, content, services, etc.

Specifically, the computer device may be a user client PC.

In addition, the unmanned vehicle control unit 20 connects to the server 11 under the control of an operating system (OS) or at least one program (for example, a browser or the installed application) and provides information provided by the server 11. Services or content can be provided.

Additionally, the control system 10 can receive location information from the manned truck (T1) and the unmanned truck (T2) and recognize the manned truck (T1) and the unmanned truck (T2).

According to an embodiment of the present invention, the unmanned vehicle control unit 20 may communicate with other electronic devices and/or the server 11 through a network using a wireless or wired communication method.

The communication method is not limited, and may include not only a communication method utilizing a communication network that the network may include (for example, a mobile communication network, wired Internet, wireless Internet, and a broadcasting network), but also short-range wireless communication between devices.

For example, networks include personal area network (PAN), local area network (LAN), campus area network (CAN), metropolitan area network (MAN), wide area network (WAN), broadband network (BBN), Internet, etc. It may include one or more arbitrary networks among the networks.

Additionally, the network may include, but is not limited to, any one or more of network topologies including a bus network, star network, ring network, mesh network, star-bus network, tree or hierarchical network, etc. .

Specifically, the server 11 and the terminal 20 may include memory, a processor, a communication module, and an input/output interface.

Memory is a computer-readable recording medium and may include non-permanent mass storage devices such as random access memory (RAM), read only memory (ROM), and disk drives.

Additionally, an operating system or at least one program code (for example, code for an application installed and running on the electronic device 3) may be stored in the memory.

These software components may be loaded from a computer-readable recording medium separate from the memory.

Such separate computer-readable recording media may include computer-readable recording media such as floppy drives, disks, tapes, DVD/CD-ROM drives, and memory cards.

In another embodiment, software components may be loaded into memory through a communication module rather than a computer-readable recording medium.

For example, at least one program may be loaded into memory based on a program (e.g., the application described above) that is installed by files provided over a network by developers or a file distribution system that distributes installation files for applications. there is.

A processor may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input/output operations.

Instructions may be provided to the processor by a memory or communication module.

The communication module may provide a function for the unmanned vehicle control unit 20 and the server 11 to communicate with each other through a network, and may provide a function for communication with other electronic devices or other servers.

The input/output interface may be a means for interfacing with an input/output device.

For example, an input device may include a device such as a keyboard or mouse, and an output device may include a device such as a display for displaying a communication session of an application.

As another example, an input/output interface may be a means of interfacing with a device that integrates input and output functions into one, such as a touch screen.

Additionally, in other embodiments, the unmanned vehicle control unit 20 and the server 11 may include more components than those shown in FIG. 1 .

Meanwhile, the Terminal Operating System (TOS) 30 receives information about containers from ships scheduled to be brought into the terminal, and operates container unloading equipment and transport equipment on the main ship so that all containers can be moved to the correct location at the correct time. It is a system for managing the workflow of containers, including establishing work plans and delivering work instructions to individual unloading equipment.

The terminal operating system 30 includes a server 31 and is connected to the integrated DB 40 to store and manage terminal operation information.

The server 31 may be implemented as a computer device or a plurality of computer devices that communicate with at least one manned vehicle terminal 20 through a network to provide commands, codes, files, content, services, etc.

Specifically, the computer device may be a user computer PC of a terminal operation control system.

Hereinafter, a specific example of a method for estimating the location of an unmanned vehicle performed by the control system 10 according to an embodiment of the present invention will be described.

Figure 2 is a flowchart showing a method for estimating the location of an unmanned vehicle according to an embodiment of the present invention.

Referring to FIG. 2, the method for estimating the location of an unmanned vehicle according to an embodiment of the present invention is performed by the control system 10, and the current location information of the unmanned vehicle can be identified in step S110.

In step S120, a task may be assigned to the unmanned vehicle based on the current location information.

In step S130, the destination and route can be calculated based on the assigned task content.

At this time, work details can also be input from an external system.

Here, the external system may be the terminal operating system 30.

In step S200, a container number image taken from an unmanned vehicle traveling on a route can be input.

Here, the container number image taken from the unmanned vehicle may be an image of one side of each container whose equipment has been completed at the container terminal.

Accordingly, the unmanned vehicle is equipped with a photographing unit and can capture container number images while driving.

In addition, the container number image captured by CCTV located on the path of the unmanned vehicle can also be input and used.

In step S300, the container number can be recognized from the container number image.

In one embodiment of the present invention, the step (S300) of recognizing the container number of the method for estimating the location of an unmanned vehicle is described in detail in FIGS. 4 and 5 below.

In step S400, the container location corresponding to the container number may be recognized based on pre-stored location data for each container.

Here, the stored location data for each container may be data that stores the device location of the installed container and the container number after at least one container has been installed by an external truck.

This will be explained in detail in FIG. 6 below.

In step S500, the location of the unmanned vehicle can be estimated by estimating the distance between the container location and the unmanned vehicle.

Specifically, the coordinate value is obtained based on the shape of the previously photographed container number image.

This will be explained in detail in FIGS. 7 and 8 below.

Meanwhile, according to various embodiments of the present invention, a plurality of landmark images taken from an unmanned vehicle traveling on a route can be received in step S201.

Here, the plurality of landmark images may include the BAY number of the floor, the location number of a specific building, etc.

Thereafter, in step S301, the location corresponding to the landmark image may be recognized based on pre-stored location data for each landmark.

In step S501, the location of the unmanned vehicle can be estimated by estimating the distance between the container location and the unmanned vehicle.

Figure 3 is a diagram for explaining a situation in which an unmanned vehicle photographs a container number image according to an embodiment of the present invention.

According to an embodiment of the present invention, in step S200, a container number image taken from an unmanned vehicle traveling on a route can be input.

Here, the container number image taken from the unmanned vehicle may be an image of one side of each container whose equipment has been completed at the container terminal.

Accordingly, the unmanned vehicle is equipped with a photographing unit and can capture container number images while driving.

Specifically, the control system 10 according to an embodiment of the present invention can assign tasks to the unmanned vehicle based on current location information.

Afterwards, as shown in FIG. 3, the unmanned vehicle can drive a three-dimensional virtual path (P1).

Here, the 3D virtual path may display at least one of going straight, turning left, and turning right.

A plurality of containers with completed equipment are located on both sides of the path (P1) along which the unmanned vehicle travels.

Additionally, container numbers (C1, C2, C3, C4) are written on one side of each of the plurality of containers.

While driving along the path P1, the unmanned vehicle can move closer to or farther away from each container, and through this, the location of the unmanned vehicle can be estimated from the container.

According to various embodiments of the present invention, the image shown in FIG. 3 may be displayed on a separate display.

Additionally, the control system 10 may display a warning message (W) at the top of the screen.

The warning message (W) may be displayed when a vehicle departure from the route is detected, and may be composed of a combination of the text “Occurrence of route deviation” and a symbol.

Meanwhile, when the vehicle is driven according to the virtual route, the warning message (W) disappears from the screen, and the warning message (W) disappears from the screen and indicates that driving includes at least one of the following: straight ahead, left turn, right turn, pause, stop, stop after OO seconds, or OO km to the destination. Information about this may be displayed.

Additionally, alternative routes (A1, A2, A3, A4, and A5) and waiting line congestion message information (D) that are not displayed on the screen can be prepared in advance.

As alternative routes (A1, A2, A3, A4, and A5) are prepared and stored in advance for each vehicle location, if the vehicle deviates from the route, the route from the corresponding route to the destination is not searched but immediately saved. You can select and display alternative routes.

Additionally, if necessary, the terminal can select and display the entire route to the destination according to the newly appeared alternative route.

Accordingly, the worker can immediately check the route status to the destination.

Meanwhile, even when a queue congestion section occurs, one of the alternative routes (A1, A2, A3, A4, and A5) may be selected and displayed.

At this time, the message “Congestion in waiting line has occurred” may be displayed on the screen.

In addition, the calculated route can be divided into time sections, and information on the maximum speed, driving direction, and driving lane of the manned vehicle can be generated for each divided route.

For example, the message “Driving on the 3rd section of the route” (131) can be displayed, and the messages “Maximum speed (132)” and “Turn right in OO seconds” (133) can be displayed.

Figure 4 is a flowchart showing the step of recognizing a container number in the method for estimating the location of an unmanned vehicle according to an embodiment of the present invention.

Referring to FIG. 4, in an embodiment of the present invention, the step of recognizing the container number of the method for estimating the position of an unmanned vehicle (S300) involves recognizing a region of interest in the container number image by using a character recognition program for the image prepared in step S310. (ROI, Region Of Interest) images can be extracted.

In step S320, a container number consisting of letters and numbers can be extracted from the extracted region of interest image through image processing to recognize the letter.

Figure 5 is a diagram for explaining a container number image and container number according to an embodiment of the present invention.

As shown in Figures 5 (a) and (b), the container number image can be prepared in various forms.

The image character recognition program according to an embodiment of the present invention can apply a conventional character recognition program.

In addition, in the case of the method for estimating the location of an unmanned vehicle according to an embodiment of the present invention, even if character recognition is performed roughly rather than accurately, it can be compared with previously stored location data for each container, so specific digits of the container number It can also be set to recognize only

For example, in (a) of Figure 5, only the 7 middle digits can be recognized, or in (b) of Figure 5, only numbers excluding the alphabet can be recognized.

Accordingly, the character recognition program according to an embodiment of the present invention may be composed of a simpler program than a conventional character recognition program.

Figure 6 is a diagram for explaining location data for each container according to an embodiment of the present invention.

Referring to FIG. 6, in one embodiment of the present invention, the stored location data for each container includes the device location of the installed container and the number of the container after at least one container C5 is installed by an external truck. It may be data stored together.

Specifically, container operation information is collected in advance, the location of the container is confirmed before moving, and the location is stored.

Afterwards, after the external truck transfers and loads the container and reports completion of the container transfer operation, information about the location (container number, location information) can be updated.

7 and 8 are diagrams for explaining a method of estimating the distance between a container location and an unmanned vehicle according to an embodiment of the present invention.

Referring to FIG. 7, the step (S500) of estimating the location of the unmanned vehicle by estimating the distance between the container location and the unmanned vehicle according to an embodiment of the present invention includes the location of the container corresponding to the container number in step S510. Coordinates can be obtained.

In step S520, the first position coordinates of the unmanned vehicle when the container number image is captured as a rectangular image can be calculated using the position coordinates of the container.

In step S530, a coordinate transformation matrix can be obtained based on the shape of the previously captured container number image.

In step S540, the first position coordinates can be converted using a coordinate transformation matrix to calculate the second position coordinates.

Specifically, referring to FIG. 8 , ideally, when the container number image is captured as a rectangular image 521, it can be determined that the position of the unmanned vehicle is located perpendicular to the container.

Accordingly, the degree to which the unmanned vehicle's position has fallen is calculated through the rate at which the rectangular image 521 becomes smaller.

Afterwards, the location of the container and the distance and angle information from the camera can be estimated according to the shape 531 of the actual container number image.

The method of calculating the second position coordinates by transforming the first position coordinates using a coordinate transformation matrix according to an embodiment of the present invention may use a conventional coordinate transformation technology.

Figure 9 is a diagram for explaining a work path assigned to an unmanned vehicle according to an embodiment of the present invention.

As shown in FIG. 9, the work path assigned to the unmanned vehicle according to an embodiment of the present invention is set as the optimal path to arrive at the destination in the shortest time among a plurality of candidate paths generated from GATE to the destination (D1). It may be a virtual path (P1).

Since the vehicle can move through multiple intersection points from the gate to the destination (D1), a movement line between intersections that can be prepared for when an unexpected situation occurs along the intersection of the vehicle's movement line is created, creating multiple alternative routes. (A1, A2, A3, A4, and A5).

In addition, after exploring various routes from the vehicle's location to the destination, the generated routes can be ranked according to the smoothest traffic situation and shortest time.

According to one embodiment of the invention, the first vehicle moves to the first location (L1), the second location (L2), the third location (L3), the fourth location (L4) and the fifth location in chronological order with respect to the path P1. It can be assigned to move location (L5).

Additionally, the second vehicle may be assigned to travel on the route P1 and then move to the sixth location L6 and the seventh location L7 in chronological order with respect to the alternative route A1.

Accordingly, a route can be provided to each unmanned vehicle, and it can be determined whether it is driving according to the route using the container image taken from the unmanned vehicle.

Figure 10 is a diagram for explaining a landmark image according to various embodiments of the present invention.

According to various embodiments of the present invention, a plurality of landmark images taken from an unmanned vehicle traveling on a route can be received in step S201.

Here, the plurality of landmark images may include the BAY number of the floor, the location number of a specific building, etc.

Referring to FIG. 10, the unmanned vehicle T2 can confirm its location by photographing the BAY number 541 on the floor.

Additionally, the unmanned vehicle (T2) can confirm the location by photographing the location number 551 of a specific building.

Landmarks are not limited to this and may include objects that exist within a port and whose location can be confirmed.

In addition, according to one embodiment of the present invention, there is provided a computer-readable recording medium recording a program for performing on a computer, a computer-readable recording medium recording a program for performing on a computer a method for estimating the location of an unmanned vehicle. .

Such computer-readable media may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on the recording medium may be those specifically designed and configured for the present invention, or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -Includes magneto-optical media and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

The description of the present invention described above is for illustrative purposes, and those skilled in the art will understand that the present invention can be easily modified into other specific forms without changing the technical idea or essential features of the present invention. will be. Therefore, the embodiments described above should be understood in all respects as illustrative and not restrictive. For example, each component described as single may be implemented in a distributed manner, and similarly, components described as distributed may also be implemented in a combined form.

The scope of the present invention is indicated by the claims described below, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be construed as being included in the scope of the present invention.

Claims (7)

A first container number image taken from an unmanned vehicle and a second container number image taken by a CCTV located on the path of the unmanned vehicle - the first container number image taken from the unmanned vehicle is a plurality of devices for which the device has been completed at the container terminal. A step of receiving input that one side of each container is an image including a photographed number and alphabet;
Recognizing a container number from the first container number image and the second container number image;
After at least one container is installed by an external truck, the location of the container corresponding to the container number is recognized based on location data for each container, which is data stored together with the device location of the installed container and the number of the container. step; and
It includes estimating the location of the unmanned vehicle by estimating the distance between the container location and the unmanned vehicle,
The step of recognizing the container location corresponding to the container number is,
In the recognized first container number image and the second container number image, only specific digits of the container number are recognized, or only numbers excluding alphabets are recognized, and the recognized partial data is compared with the stored location data for each container. A method for estimating the location of an unmanned vehicle that recognizes the location of a container. delete According to paragraph 1,
The step of recognizing the container number in the container number image is,
Extracting a region of interest (ROI) image from the container number image using a previously prepared image character recognition program; and
A method for estimating the location of an unmanned vehicle, comprising the step of extracting a container number consisting of letters and numbers from the extracted region of interest image through image processing and recognizing the letters. delete In paragraph 3
The step of estimating the location of the unmanned vehicle by estimating the distance between the container location and the unmanned vehicle,
Obtaining location coordinates of a container corresponding to the container number;
Using the position coordinates of the container, calculating first position coordinates of the unmanned vehicle when the container number image is captured as a rectangular image;
Obtaining a coordinate transformation matrix based on the shape of the previously photographed container number image; and
A method for estimating the position of an unmanned vehicle, comprising converting the first position coordinates using the coordinate transformation matrix to calculate second position coordinates. Identifying current location information of the unmanned vehicle;
assigning a task to the unmanned vehicle based on the current location information;
calculating a destination and route based on the assigned task content;
A first container number image taken from an unmanned vehicle traveling on the route and a second container number image taken by a CCTV located on the route of the unmanned vehicle - the first container number image taken from the unmanned vehicle is a container terminal A step in which the device receives an input that each side of a plurality of completed containers is an image including a photographed number and alphabet;
Recognizing a container number from the first container number image and the second container number image;
After at least one container is installed by an external truck, the location of the container corresponding to the container number is recognized based on location data for each container, which is data stored together with the device location of the installed container and the number of the container. step; and
It includes estimating the location of the unmanned vehicle by estimating the distance between the container location and the unmanned vehicle,
The step of recognizing the container location corresponding to the container number is,
In the recognized first container number image and the second container number image, only specific digits of the container number are recognized, or only numbers excluding alphabets are recognized, and the recognized partial data is compared with the stored location data for each container. A method for estimating the location of an unmanned vehicle that recognizes the location of a container. According to clause 6,
Receiving a plurality of landmark images taken from an unmanned vehicle traveling on the route;
Recognizing a location corresponding to the landmark image based on pre-stored location data for each landmark; and
A method for estimating the location of an unmanned vehicle further comprising estimating the location of the unmanned vehicle by estimating the distance between the container location and the unmanned vehicle.

Images