JP2023143180A - Package sorting system, package sorting method, information processing device, and method for controlling the same - Google Patents

Abstract

To sort packages more efficiently and accurately in a package sorting system.SOLUTION: There is provided a package sorting system configured to include a plurality of sorting clients that respectively monitor a plurality of sorting areas constituting a conveyance path, the package sorting system comprising: an acquisition unit that acquires respective images of the plurality of sorting areas photographed by a photographing unit provided in each of the plurality of sorting clients; a generation unit that generates an overall image of the conveyance path using respective images of the plurality of sorting areas; and a control unit that uses the overall image to control the conveyance of packages being conveyed on the conveyance path.SELECTED DRAWING: Figure 8

Description

The present disclosure relates to a baggage sorting system, a baggage sorting method, an information processing device, and a control method thereof.

With the increase in economic activity in recent years, the amount of parcels in circulation is increasing. In the baggage distribution process, the sorting process of sorting packages by destination is a time-consuming process that has traditionally relied on manual labor, but technologies have also been proposed to automate at least part of the sorting process.

Patent Document 1 tracks a package moving on a transport route, determines an image to be displayed based on information about the package read from the package and information on the location of the package, and projects the image onto the package from a projector to display the image. The company is disclosing a system that displays this on luggage.

US Patent No. 7,090,134

However, in recent years, the amount of parcels in circulation has been increasing more and more, and the types of parcels have become more diverse, and there is a need for technology for sorting parcels more efficiently and accurately.

The present disclosure was devised in view of the above-mentioned conventional circumstances, and relates to a technique for sorting packages more efficiently and accurately. In particular, the present invention aims to provide a baggage sorting system, a baggage sorting method, an information processing device, and a control method thereof.

The present disclosure is a baggage sorting system that includes a plurality of sorting clients that monitor each of a plurality of sorting areas constituting a transport route, and in which images are taken by a photographing unit provided in each of the plurality of sorting clients. an acquisition unit that acquires an image of each of the plurality of sorting areas; a generation unit that generates an entire image of the transport route using the images of each of the plurality of sorting areas; Provided is a cargo sorting system that includes a control unit that controls the transportation of cargo that is being transported in the cargo sorting system.

The present disclosure also provides a baggage sorting method in a baggage sorting system including a plurality of sorting clients that monitor each of a plurality of sorting areas constituting a transport route, the method comprising: an acquisition step of acquiring an image of each of the plurality of sorting areas photographed by a photographing unit; a generating step of generating an entire image of the transport route using the images of each of the plurality of sorting areas; and a control step of controlling the conveyance of the articles being conveyed on the conveyance path using the present invention.

Further, the present disclosure includes a processor and a memory, and the processor cooperates with the memory to provide an imaging camera provided in each of a plurality of sorting clients that monitors each of a plurality of sorting areas constituting a conveyance route. acquire images of each of the plurality of sorting areas photographed by the department, generate an overall image of the transport route using the images of each of the plurality of sorting areas, and generate a total image of the transport route using the overall image. Provided is an information processing device that controls the transportation of cargo being transported.

In addition, the present disclosure provides that the processor cooperates with the memory to capture images of each of the plurality of sorting areas photographed by a photographing unit provided in each of the plurality of sorting clients that monitor each of the plurality of sorting areas constituting the transport route. acquiring an image, using the images of each of the plurality of sorting areas to generate an entire image of the transport route, and using the entire image to control transport of the cargo being transported on the transport route; A method for controlling an information processing device is provided.

Note that arbitrary combinations of the above components and expressions of the present disclosure converted between methods, devices, systems, storage media, computer programs, etc. are also effective as aspects of the present disclosure.

According to the present disclosure, it is possible to sort packages more efficiently and accurately.

A block diagram showing a configuration example of a baggage sorting system according to Embodiment 1 Block diagram illustrating a configuration example of an information processing device according to Embodiment 1 Schematic diagram for explaining transportation of luggage according to Embodiment 1 Schematic diagram for explaining detection and projection of luggage according to Embodiment 1 Processing sequence of baggage sorting system according to Embodiment 1 Processing sequence of baggage sorting system according to Embodiment 1 Schematic diagram for explaining recognition of cargo during transportation according to Embodiment 1 Processing sequence for generating an entire transport route image according to Embodiment 1 Schematic diagram for explaining stitching processing according to Embodiment 1

DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments specifically disclosing a baggage sorting system, a baggage sorting method, an information processing device, and a control method thereof according to the present disclosure will be described in detail with reference to the accompanying drawings as appropriate. However, more detailed explanation than necessary may be omitted. For example, detailed explanations of well-known matters or redundant explanations of substantially the same configurations may be omitted. This is to avoid unnecessary redundancy in the following description and to facilitate understanding by those skilled in the art. The accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter recited in the claims.

<Embodiment 1>
[System configuration]
FIG. 1 is a block diagram showing a configuration example of a luggage sorting system 100 according to an embodiment. The baggage sorting system 100 is a system that supports the work of a worker who sorts a plurality of parcels conveyed by a conveyor (not shown in FIG. 1). The package sorting system 100 is installed, for example, in a distribution center owned by a retailer, wholesaler, Internet distributor, or the like. The baggage to be sorted generally has a substantially rectangular parallelepiped shape, but there are no particular limitations on its external shape or size, and there are no particular limitations on the type of baggage. The conveyor that conveys the luggage may be, for example, a belt conveyor or a roller conveyor. Furthermore, the conveyance path for the cargo may be configured such that the conveyance direction changes or a part of the conveyance path branches by combining a plurality of conveyors.

The baggage sorting system 100 includes a sorting management server 101, a cooperation server 102, a label reader 103, a sorting client 104, a projector 105, a distance sensor 106, and an image sensor 107. A plurality of client systems 110 are provided in which the sorting client 104, the projector 105, the distance sensor 106, and the image sensor 107 are set as one set, and are associated with each of the plurality of sorting areas provided on the transport conveyor that transports packages. Therefore, a plurality of sorting clients 104, projectors 105, distance sensors 106, and image sensors 107 are each installed according to the number of sorting areas. In the following description, when an individual device needs to be explained, subscripts (a, b, . . . ) are added, and when the devices are explained collectively, the subscripts are omitted.

The sorting management server 101 is an information processing device that controls the entire package sorting system 100 according to this embodiment. The sorting management server 101 acquires the detection results of the label reader 103 and acquires various baggage information from the cooperation server 102. Further, the sorting management server 101 issues various instructions to each of one or more sorting clients 104 and collects various information. Further, the sorting management server 101 may control the operation of a conveyor, which will be described later, and collect and manage operation information.

The cooperation server 102 stores and manages label information and package information in association with each other. The cooperation server 102 then provides package information corresponding to the label information read by the label reader 103 in response to a request from the sorting management server 101. In the example of FIG. 1, one sorting management server 101 and one cooperation server 102 are shown, but they may be configured with a plurality of devices from the viewpoint of functional redundancy and load distribution. Further, label information and sorting patterns may be managed in association with each other on the sorting management server 101 side, and based on this, processing related to sorting of each baggage, which will be described later, may be performed. In such cases, the configuration of the cooperation server 102 is omitted, and the sorting management server 101 is configured to perform processing related to sorting of each package, which will be described later, based on the pattern of acquired label information. good.

The label reader 103 is a device for reading labels attached to packages being transported by a transport conveyor, and includes optical components such as a lens and an image sensor. By reading the label attached to the package, various information shown on the label (hereinafter referred to as "label information") is acquired. The format or method of the label is not particularly limited, and for example, a barcode or two-dimensional code (QR code (registered trademark), etc.) can be used. In this embodiment, a barcode will be used as an example of a label. Further, the label information may include, for example, identification information for uniquely identifying the package. In this case, the type of characters and the number of digits constituting the identification information are not particularly limited.

The label reader 103 uses, for example, a general imaging camera to extract a label area from a captured image through image recognition and read various information, or performs OCR (Optical Character Recognition) on characters written on the label. /Reader). The label reader 103 may be a dedicated device for reading labels, or may be integrated with other devices. Further, the label reader 103 may be configured to detect the three-dimensional coordinates of the label. Furthermore, a plurality of label readers 103 may be provided to accommodate various sizes of packages, or a configuration may be used to photograph packages from multiple directions in response to various label attachment positions. good.

The package information according to this embodiment includes, for example, a package identification number that is individually assigned to a package (e.g., identification information for uniquely identifying a package), the name, address, and telephone number of the package sender, Examples include the name, address, telephone number, and type of package of the destination (for example, recipient). Note that the information is not limited to these, and the items included in the baggage information may be increased or decreased as necessary. In this embodiment, the sorting management server 101 acquires label information from the label reader 103, and uses this label information to make an inquiry to the cooperation server 102, thereby acquiring corresponding package information. As described above, in the case of a configuration in which cooperation with the cooperation server 102 is omitted, label information and pattern information related to sorting may be stored in association with each other on the sorting management server 101 side.

The sorting client 104 is an information processing device that controls projection processing and the like for the cargo being transported based on instructions from the sorting management server 101. Further, the sorting client 104 acquires the photographic results of the distance sensor 106 and the image sensor 107, and detects and manages packages based on the photographic results. Further, the sorting client 104 provides the sorting management server 101 with position information, size information, detected time information, etc. of the detected baggage. Note that the configuration may be such that the various information acquired on the sorting client 104 side is provided to the sorting management server 101 in a timely manner, and the sorting management server 101 side processes the various information.

The projector 105 is a projection device that irradiates a projection image including a predetermined projection pattern onto the cargo on the transport route based on instructions from the sorting client 104 . Note that in this embodiment, the "projection image" refers to the entire image projected by the projector 105 onto the entire sorting area in a superimposed manner. Further, the "projection pattern" refers to an image that is displayed in a superimposed manner corresponding to each baggage detected in the sorting area. Therefore, when a package is detected in the sorting area, the projected image will include one or more projection patterns depending on the package. Examples of projection patterns will be described later.

The distance sensor 106 is a sensor for detecting distance information between the distance sensor 106 and an object to be imaged (for example, the surface of a package), and acquires the detection result as a distance image. The distance image may include something that cannot be recognized as an image by the human eye, such as a table listing numerical values indicating distance. That is, the distance image may be information indicating the relationship between coordinates and distance within the imaged area, and its data structure is not particularly limited. As the distance sensor 106, for example, an infrared sensor (IR sensor), a stereo camera composed of a plurality of cameras, or the like may be used.

The image sensor 107 is an imaging device that includes optical components such as a lens and an image sensor. The image sensor 107 is composed of, for example, an imaging camera. As the imaging camera, a three-dimensional camera or a plurality of two-dimensional cameras may be used. The image sensor 107 captures an image of the conveyor including the baggage being conveyed by the conveyor, and generates a color image. The color image here refers to an image in which the color of the surface of the object to be photographed is expressed in a predetermined gradation. In addition to the 256 gradations of RGB (Red, Green, and Blue), the gradation may include a gray scale.

The number of client systems 110 varies depending on the length of the conveyor, the detection range of each sensor, and the like. Note that the sorting client 104 is configured as one unit, and this one sorting client 104 controls a plurality of projectors 105, a plurality of distance sensors 106, and a plurality of image sensors 107 corresponding to a plurality of sorting areas. It may be a configuration. Further, in this embodiment, an example is shown in which the distance sensor 106 and the image sensor 107 are used as sensors, but a sensor in which these are integrated may be used, or a configuration in which only one of them is used is also possible. It may be. Also, other types of sensors may be used. The conveyance format of the conveyor is not particularly limited, and for example, the conveyor may be configured to convey the loads in one line, or may be configured to convey the loads in multiple lines.

Each device is communicably connected by wire/wireless. The communication method and connection method here are not particularly limited, and the connection may be made using a predetermined network cable, or may be made by combining a plurality of methods.

(Information processing device)
FIG. 2 is a diagram illustrating an example of the hardware configuration of an information processing apparatus that can be applied as the sorting management server 101, the cooperation server 102, and the sorting client 104 according to this embodiment. Although each device will be described here as having the same configuration, each device may have a different configuration.

The information processing device 200 includes a CPU (Central Processing Unit) 201, a memory 202, a storage device 203, an input/output section 204, a communication section 205, and an external I/F 206. The CPU 201 implements various functions by reading various programs and data stored in the memory 202 and the storage device 203 and executing processes. Note that the CPU 201 may be another arithmetic circuit such as a GPU (Graphics Processing Unit), or may be used in combination with another arithmetic circuit. The memory 202 is a storage area for storing and retaining various information, such as ROM (Read Only Memory) which is a non-volatile storage area, RAM (Random Access Memory) which is a volatile storage area, etc. configured. The storage device 203 is a storage area for storing and retaining various information, and is composed of an HDD (Hard Disk Drive) or the like.

The input/output unit 204 receives instructions from a user from a mouse or keyboard (not shown), and outputs various information from a display (not shown), for example. The communication unit 205 communicates with an external device via a wired/wireless network, and sends and receives various data and signals. The communication method by the communication unit 205 is not particularly limited, and may be compatible with a plurality of communication methods. For example, WAN (Wide Area Network), LAN (Local Area Network), power line communication, short-range wireless communication (eg, Bluetooth (registered trademark)), etc. may be used. External I/F 206 is an interface for transmitting and receiving data with an external device. Each part of the information processing device 200 is communicably connected via an internal bus (not shown) or the like.

[Example of transportation]
FIG. 3 is a schematic diagram for explaining the transportation of luggage according to this embodiment. A label 301 is attached to each baggage 300 to be transported. The label 301 is read during transportation by a label reader 103 installed at an arbitrary position on the transportation path. In the example of FIG. 3, the label 301 is attached to the top surface of the luggage 300 and is read from the top by the label reader 103, but the label 301 is not limited to this. For example, a label 301 may be attached to the side of the luggage 300, and the label reader 103 may be configured to read the label from the side.

The cargo 300 is transported by the transport conveyor 302 at an arbitrary speed. A rotary encoder 303 is installed on the conveyor 302 to detect the conveyance distance in a timely manner and provide it to the sorting management server 101. The rotary encoder 303 is attached to a rotating mechanism (not shown) that rotates as the conveyor 302 transports, and outputs the number of rotations of the rotating mechanism. When the conveyor 302 is a belt conveyor, this rotation mechanism is a roller in contact with the belt or a motor that moves the belt. Moreover, when the transport conveyor 302 is a roller conveyor, it is a roller which comprises the transport conveyor 302. Here, it is assumed that the cargo is being transported along the direction indicated by the arrow. The conveyor 302 changes its conveyance speed or stops conveyance according to instructions from the user. Conveyance in the opposite direction may be performed as necessary. Further, the conveyance speed of the conveyor 302 may change due to factors other than the user's designation, such as the physical characteristics of each mechanism of the conveyor 302. However, since the rotary encoder 303 accurately detects the rotation speed of the rotation mechanism, it is less susceptible to changes in the conveyance speed. Conveyance by the conveyor 302 may be controlled, for example, by the sorting management server 101 or by a separately provided control device (not shown).

On the downstream side of the label reader 103 in the transport direction, a plurality of various sensors (in this example, a distance sensor 106 and an image sensor 107) and a plurality of projectors 105 are installed corresponding to a plurality of sorting areas. FIG. 3 shows the most upstream sorting area A among the plurality of sorting areas. Various sensors corresponding to the sorting area A acquire various images by photographing the sorting area A, and using these images, the sorting client 104a or the sorting management server 101 detects the cargo being transported. Specifically, when the package 300 reaches the starting position 305a of the sorting area A, the package 300 enters the photographing range of various sensors. The photographing range 304 of the distance sensor 106a and the image sensor 107a includes at least the range of the sorting area A. Further, within the range of the sorting area A, the projector 105a projects a predetermined projection image based on instructions from the sorting management server 101 and the sorting client 104. A projection range 306 by the projector 105a includes at least the sorting area A. Note that the distance from the reading position of the label reader 103 to the starting position 305a may be defined according to the processing speed of the processing sequence described later, the reading speed of various sensors, and the like.

A worker M who performs work on the cargo 300 is located around the transport conveyor 302, and performs various tasks as necessary. At this time, the worker M can perform work based on the projection image (projection pattern, etc.) projected by the projector 105.

FIG. 4 is a diagram for explaining a plurality of sorting areas corresponding to each of a plurality of client systems 110. Here, two sorting areas A and B will be taken as an example and explained as being monitored by client systems 110a and 110b, respectively. The number of sorting areas and their range (for example, length and width in the transport direction) may be defined according to the functions of the client system 110. Therefore, further downstream of the sorting area B in the conveyance direction, one or more sorting areas may be provided.

The sorting area A is photographed by the distance sensor 106a and the image sensor 107a, and the image is provided to the sorting client 104a. Further, in the sorting area A, a projection image including an arbitrary projection pattern is projected by the projector 105a. Similarly, sorting area B adjacent to sorting area A on the downstream side in the transport direction is photographed by distance sensor 106b and image sensor 107b, and the image is provided to sorting client 104b. Furthermore, in the sorting area B, a projection image including an arbitrary projection pattern is projected by the projector 105b. In the sorting area B, monitoring is carried out in a manner that takes over the monitoring of the packages conveyed from the sorting area A. At this time, as shown in FIG. 4, the sorting areas of the respective client systems 110 partially overlap, and hereinafter, this area will also be referred to as a "handover area." The start position 305b indicates the start position of the monitoring of the sorting area B, and is different from the end position (not shown) of the monitoring of the sorting area A adjacent to the upstream side of the sorting area B. Details of the handover area will be described later. Note that the images acquired from each sensor may be further provided from the sorting client 104 to the sorting management server 101.

[Processing sequence]
The processing sequence of the baggage sorting system 100 according to this embodiment will be explained using FIGS. 5 and 6. The processing of each device may be realized by the processing entity (CPU, etc.) of each processing executing programs corresponding to various functions. It is assumed that the operation of the transport conveyor 302 is started when the processing shown in FIGS. 5 and 6 is started, and thereby the packages are sequentially transported. Note that the subject of each process in the processing sequence shown below is an example, and a part thereof may be executed by another device. For example, the configuration may be such that the sorting client 104 executes part of the processing of the sorting management server 101 described below. Alternatively, the reverse may be possible, and a configuration may be adopted in which the sorting management server 101 executes a part of the processing of the sorting client 104 described below. An example of such a different configuration will be described later as a modification. Alternatively, each server and sorting client shown in FIG. 1 may be configured as one device, and the following processing sequence may be executed by this one device.

The label reader 103 detects a label attached to a package being transported on the transport conveyor 302, and reads the label information (step S501).

The label reader 103 transmits the label information read in S501 to the sorting management server 101 (step S502). The information transmitted at this time may include, in addition to the information shown on the label, the time of reading and positional information (for example, three-dimensional information) detected when reading the label.

The sorting management server 101 acquires the label information transmitted from the label reader 103 (step S503). At this time, the sorting management server 101 acquires the detected values from the rotary encoder 303 provided in the transport conveyor 302, and stores them in association with each other. The value detected by the rotary encoder 303 corresponds to the moving distance of the conveyor 302 based on its rotation speed. Further, the sorting management server 101 uses the acquired label information to inquire of the cooperation server 102 about package information.

The cooperation server 102 searches a predetermined DB (database) for package information corresponding to the label information indicated in the inquiry from the sorting management server 101 and identifies it (step S504). The DB may be provided internally in the cooperation server 102, or may be configured in an external device (not shown) connected to the cooperation server 102 via a network.

The cooperation server 102 transmits the package information specified in step S504 to the sorting management server 101 as a response to the inquiry (step S505).

The sorting management server 101 determines a projection pattern based on the package information acquired from the cooperation server 102 (step S506). The projection pattern according to this embodiment is defined and held in advance as a plurality of patterns. The projection pattern may be determined, for example, depending on the destination (for example, the destination address) indicated in the package information, or may be determined depending on the sorting area.

The sorting management server 101 transmits a projection instruction including the projection pattern determined in step S506 to the sorting client 104 (step S507). Here, as shown in FIG. 3, the explanation will be made assuming that the information is transmitted to the sorting client 104a of the client system 110a corresponding to the sorting area A located at the most upstream side of the conveyance route. That is, each package is first transported toward the sorting area A. In addition to the projection pattern, the projection instruction may include the package information acquired in step S506, the label information read by the label reader 103 in step S501, and information on the time at which it was read. The projection instruction here may be transmitted from the sorting management server 101 to the sorting client 104a based on the sorting client 104a detecting the package using sensors, inquiring information regarding the package. Alternatively, the label information may be transmitted to the sorting client 104a due to the sorting management server 101 receiving label information from the label reader 103.

Step S520 (steps S508 to S515) is a process repeatedly performed by the client system 110 (here, the client system 110a) to monitor each package being transported within the sorting area (here, sorting area A). It is. Here, for convenience, the distance sensor 106 and the image sensor 107 (here, the distance sensor 106a and the image sensor 107a) are collectively referred to as "sensors."

The sensors photograph the inside of the sorting area A and obtain the image (step S508). Note that in this embodiment, the sorting client 104a will be described as one that detects packages using images captured by sensors, but a configuration in which part of the processing is performed on the sensors side is also possible. It may be. For example, sensors may be configured to identify the area corresponding to the baggage (presence or absence of baggage), or detect the three-dimensional coordinates (position) and size (length, width, height) of each baggage. good.

The sensors transmit the obtained results to the sorting client 104a (step S509). At this time, the sensors may perform predetermined processing on the acquired images and then transmit them to the sorting client 104a. For example, in the case of the image sensor 107a, arbitrary image processing (for example, image compression or correction processing) may be performed on the acquired image information.

The sorting client 104a performs a package detection process based on the acquisition results sent from the sensors in step S509 (step S510). Specifically, the sorting client 104a identifies one or more packages being conveyed by the conveyor 302 in the sorting area A based on the results obtained from sensors. Processing for detecting baggage may be performed using any known method, and is not particularly limited. For example, the area corresponding to the baggage may be detected by area segmentation using machine learning. Alternatively, packages may be detected by pattern matching using a predefined pattern.

The sorting client 104a correlates each piece of information based on the projection instruction transmitted from the sorting management server 101 in step S507 and the detection result in step S510 (step S511). As a result, each of the detected one or more packages is associated with various information (projection pattern, etc.) indicated in the projection instruction transmitted from the sorting management server 101.

The projection pattern is, for example, an image of numbers surrounded by a circular frame and has a color indicating the sorting location corresponding to the delivery address of the package. Here, the numbers correspond to, for example, the number of the truck that transports the sorted cargo (the number of the truck itself, the parking lot number, etc.), the number of the shelf or box to be carried into the truck, etc. In addition, the image of the number may not directly correspond to the number of a shelf or box, but may correspond to the number of a shooter (not shown) that moves the picked up cargo to another location or truck, etc. good. Since the parking positions of trucks and the like frequently change depending on traffic conditions, it may be difficult to change the sorting destination as seen from around the transport conveyor 302 at any time. Therefore, by sandwiching a chute between the transport conveyor 302 and a transport truck, etc., and projecting the number of the chute around the transport conveyor 302, it is possible to change the arrangement of the exit of the chute without changing the configuration around the transport conveyor 302 at any time. can respond to changes in sorting destinations. In this case, the contents of the projection pattern may be switched according to changes in the situation.

Other examples of displaying numbers as the projection pattern include the postal code corresponding to the delivery address, the number of the worker who should pick up the package P, etc. Further, as an example of displaying information other than numbers, an arrow indicating the sorting direction (such as right or left with respect to the transport direction of the transport conveyor 302) or characters (such as "left" or "right") may be used. . Further, the display form is not limited to numbers surrounded by a circular frame, but may be various configurations such as numbers surrounded by a square frame ("3", "359", "24735"). Further, the projection pattern is not limited to one in which numbers or letters are surrounded by a frame, but may be white numbers or letters with the background filled in. Further, depending on the information to be displayed, the shapes of the numbers or characters to be displayed, such as circles, triangles, squares, etc., may be changed. Alternatively, a picture that can be individually associated with each piece of information to be displayed may be displayed.

Further, the projection pattern is not limited to a still image, but may be an animation. As examples of animation, each of the above examples may be made to blink, be enlarged/reduced, or change color. Furthermore, an animation reflecting the sorting direction may be projected. Examples of animations that reflect the sorting direction include moving light rays or light spots toward the sorting direction, forming all or part of a projected pattern toward the sorting direction, changing color, Examples include displaying the image by moving an arrow. If you want to animate only a part of the projected pattern, do not change the parts that have a large influence on the operator's judgment of the sorting destination, such as numbers and arrows, and change the parts that have less influence on the sorting destination, such as the frame line. It may be changed. However, in situations where it is more efficient to intuitively convey the sorting direction rather than the meaning of numbers etc. projected within the frame, such as when there are few options for sorting destinations, numbers or arrows may be displayed within a fixed frame. may be moved in the sorting direction. Further, the animation may be projected repeatedly or may be projected only once.

Further, the projection pattern is not limited to one indicating a sorting destination, but may also be a notification of an error occurring in the baggage sorting system 100 or a notification when a worker has overlooked a baggage.

Further, instead of directly projecting the projection pattern onto the luggage, the operator may be made to perceive the projection pattern as if it were being projected onto the luggage in a pseudo manner through glasses that can display an image. In other words, if the worker wears special glasses that can display images, the projected pattern may be superimposed on the image of the package that is viewed through the glasses.

Furthermore, the projection pattern may be projected onto a location other than the baggage. For example, general notification contents to the worker may be projected onto an area of the conveyor 302 where no cargo exists. Examples of such notifications include advance notice of increases and decreases in the amount of arriving packages, remaining work time, and greetings to workers.

The sorting client 104a processes the one or more packages detected in step S510 in the three-dimensional space (step S512). In the processing in the three-dimensional space here, processing is performed so that each package is expressed in a three-dimensional shape in a three-dimensional coordinate system on the transport route. Note that depending on the number and arrangement of sensors, information about surfaces located in blind spots of the sensors may not be obtained. In this case, the three-dimensional shape of each baggage may be complemented to match the general shape of baggage (for example, a rectangular parallelepiped shape, etc.). In addition, if the three-dimensional shape of the surface on which the projection image is to be projected among the surfaces constituting the baggage can be recognized, instructions can be projected. A set of surfaces including the surface may be expressed as the three-dimensional shape of the luggage. In this embodiment, the surface to be projected is the top surface of the luggage, but it may be the side surface or the like. The sensors of this embodiment are arranged above so as to obtain at least information on the top surface of the baggage, but if the surface to be projected is the side surface etc., the baggage is photographed from a position that matches that surface. do.

Regarding the three-dimensional coordinate system on the transport route, it is assumed that the origin and each coordinate axis are defined in advance within the client system 110a. It is also assumed that the three-dimensional coordinate systems of the label reader 103 and the plurality of client systems 110 are associated in advance and adjusted in a timely manner. The plurality of three-dimensional coordinate systems here may share an absolute coordinate system throughout the baggage sorting system 100, and may be associated with each other using that absolute coordinate system. Furthermore, the plurality of three-dimensional coordinate systems may be associated using a relative coordinate system. When using a relative coordinate system, for example, the client system 110 located upstream of the baggage sorting system 100 is used as a reference, and the coordinate systems used by other client systems 110 are set relative to the client system 110 that is the reference. It may also be defined using a coordinate system. Note that any coordinate system within the baggage sorting system 100 may be used as the reference coordinate system for the relative coordinate system. For example, it may be another coordinate system, such as a coordinate system used by another client system or a group of sensors. Note that this process may be performed in a two-dimensional space, that is, a planar space, if adjustment in the height direction is not necessary due to the shape of the cargo to be transported or the configuration of the projector 105. good.

The sorting client 104a draws a projection image including a projection pattern associated with each of the one or more three-dimensional packages processed in step S512 (step S513). Specifically, the sorting client 104a calculates the projection position of the projection pattern for each individual baggage, and determines the projection image for the entire sorting area A. This projected image is, for example, an image in which a region corresponding to each baggage includes a projection pattern for each baggage, and a region where no baggage exists includes a black image. By using such a projection image, it is possible to simultaneously perform projection on a large number of packages while suppressing the number of projectors. In addition, with a general projector, the closer the distance between the surface to be projected and the projector, the smaller the image, and the farther away, the larger the image is projected. Therefore, the size of the projection pattern for each package in the projection image drawn in this step may be adjusted so that it becomes larger as the distance between each package and the projector becomes shorter.

The sorting client 104a instructs the projector 105a to project according to the projection image determined in step S513 (step S514).

The projector 105a projects a projection image onto the sorting area A based on the projection instruction from the sorting client 104a in step S514 (step S515). Since each baggage is continuously transported by the transport conveyor 302, the process shown in step S520 may be continuously executed while the transport continues.

(Luggage handover processing)
In FIG. 5, the processing has been described focusing on the first (most upstream) sorting area A on the downstream side in the transport direction from the reading position of the label reader 103. As described above, a plurality of sorting areas are provided on the conveyance route by the conveyor 302, and the monitoring of each package is handed over (hereinafter also referred to as "handover") between the client systems 110 between the sorting areas. conduct.

A processing sequence related to handover processing will be explained using FIG. 6. Here, handover processing between a client system 110a corresponding to sorting area A and a client system 110b corresponding to sorting area B adjacent to sorting area A on the downstream side in the conveying direction on the conveying route will be taken as an example. I will explain. Since the number of sorting areas may vary depending on the length of the transport route, etc., handover processing is performed at appropriate times between client systems 110 corresponding to adjacent sorting areas. It is assumed that the client system 110a is performing the processing sequence described in FIG. 5 simultaneously and in parallel.

Step S620 (steps S601 to S603) is a process repeatedly performed by the client system 110a in order to monitor each package being transported within the sorting area A. Here, for convenience, the distance sensor 106a and the image sensor 107a are collectively referred to as sensors A.

The sensors A photograph the inside of the sorting area A and obtain the image (step S601). Here, as shown in FIG. 5, the description will be made assuming that the sorting client 104a side detects packages using various images taken by sensors A. If a plurality of packages are being transported to the sorting area A, each package is detected. In the case of the client system 110a, this process corresponds to step S508 in FIG.

Sensors A transmit the obtained results to the sorting client 104a (step S602). At this time, the sensors A may perform predetermined processing on the acquired images and then transmit them to the sorting client 104a. In the case of the client system 110a, this process corresponds to step S509 in FIG.

The sorting client 104a detects packages based on the acquisition results sent from the sensors A in step S602 (step S603). Furthermore, the sorting client 104a transmits the transportation information of the detected package to the sorting management server 101 (step S603). Specifically, depending on the results of the processing in the three-dimensional space that has already been performed in the processing sequence shown in FIG. ) may be identified, and information regarding the cargo may be transmitted as transport information. Processing for detecting baggage may be performed using any known method, and is not particularly limited. For example, the area corresponding to the baggage may be detected by area segmentation using machine learning. Alternatively, packages may be detected by pattern matching using a predefined pattern.

Step S630 (steps S604 to S606, steps S610 to S614) is a process repeatedly performed by the client system 110b in order to monitor each package being transported within the sorting area B. Here, for convenience, the distance sensor 106b and the image sensor 107b are collectively referred to as sensors B.

Sensors B photograph the inside of sorting area B and obtain the image (step S604). Here, similarly to the sorting area A, the description will be made assuming that the sorting client 104b side detects packages using various images taken by the sensors B. If a plurality of packages are being transported to the sorting area B, each package is detected.

Sensors B transmit the obtained results to the sorting client 104b (step S605). At this time, the sensors B may perform predetermined processing on the acquired information and then transmit it to the sorting client 104b.

The sorting client 104b detects packages based on the acquisition results sent from the sensors B in step S605 (step S606). Furthermore, the sorting client 104b transmits transportation information of the detected baggage to the sorting management server 101. For example, among the detected packages, a package located at the boundary with the upstream sorting area (here, sorting area A) (i.e., a newly transported package) is identified, and information about that package (location, etc.) is identified. size, etc.) may be transmitted as transport information.

The sorting management server 101 determines between the sorting areas (here, the sorting area The baggage to be handed over between the sorting area A and sorting area B is identified (step S607). In other words, as a parcel is transported, the person monitoring the parcel is handed over from the client system 110a to the client system 110b. , and are associated as being the same package. For example, the sorting management server 101 may make the correspondence by specifying which piece of managed baggage information corresponds to a piece of baggage that straddles the sorting area.

The sorting management server 101 selects the downstream sorting area (here, sorting area B) based on the association in step S607 and the package information used in the upstream sorting area (here, sorting area A). A projection pattern is determined (step S608). The projection pattern according to this embodiment is defined and held in advance as a plurality of patterns. Note that the projection pattern may be the same as the upstream sorting area (here, sorting area A), or may be configured to be switched depending on the sorting area during transportation even if the same baggage is being transported. .

The sorting management server 101 transmits a projection instruction including the projection pattern determined in step S608 to the sorting client 104 (step S609). Here, a projection instruction is sent to the sorting client 104b of the client system 110b to which monitoring of the package is handed over. Note that the projection instruction may include, in addition to the projection pattern, baggage information of the corresponding baggage.

The sorting client 104b correlates each piece of information based on the projection instruction transmitted from the sorting management server 101 in step S609 and the detection result in step S606 (step S610). Specifically, the sorting client 104b identifies one or more packages being conveyed by the conveyor 302 in the sorting area B based on the detection results of the sensors B. Further, the sorting client 104b associates various information included in the projection instruction transmitted from the sorting management server 101 with each of the identified one or more packages. As a result, each of the one or more packages detected by the sorting client 104b is associated with various information (projection pattern, etc.) indicated in the projection instruction transmitted from the sorting management server 101.

The sorting client 104b performs processing in the three-dimensional space for the one or more packages identified in step S610 (step S611). In the processing in the three-dimensional space here, processing is performed so that each package is expressed in a three-dimensional shape in a three-dimensional coordinate system on the transport route. The processing here may be the same as the processing in step S512 in FIG.

The sorting client 104b draws a projection image including a projection pattern associated with each of the one or more three-dimensional packages processed in step S611 (step S612). Specifically, the sorting client 104b calculates the projection position of the projection pattern for each individual baggage, and determines the projected image of the entire sorting area B. The projected image here may have the same configuration as step S513 described in FIG. 5.

Note that a plurality of projectors may be installed in association with one sorting area. For example, one sorting area may be further subdivided to set a range to be handled by each of a plurality of projectors. In this case, the sorting client 104b generates a projection image corresponding to the area in charge of each of the plurality of projectors. An example of a projected image in this case is that the positions of packages that exist within the assigned area include a projection pattern for each package, and the positions where no packages exist within the assigned area include an image that includes a black image. Can be mentioned. By doing this, even if it is difficult to project onto the entire sorting area with one projector due to the performance of the projector or the limit on the height at which the projector can be installed, it is possible to project onto the entire sorting area. I can do it.

Furthermore, each projector may be in charge of one baggage and project only to that baggage. In this case, each of the multiple projectors can change the irradiation angle according to the movement of the baggage it is responsible for, or project a black image to a position other than the position of the baggage it is responsible for, so that only the baggage it is responsible for Projection towards. By doing so, each of the plurality of projectors can perform projection using the optimal parameters for the baggage that it is in charge of. For example, since the focal length of each of the plurality of projectors can be automatically adjusted, even if there are variations in the height of the packages, it is possible to project images that are in focus on each package.

The sorting client 104b instructs the projector 105b to project according to the projection image determined in step S612 (step S613).

The projector 105b projects a projection image onto the sorting area B based on the projection instruction from the sorting client 104b in step S613 (step S614). Since each baggage is continuously transported by the transport conveyor 302, each of the client systems 110 responsible for monitoring the sorting area B and beyond may continue to execute the process shown in step S630 while the transport continues. .

[Baggage recognition]
FIG. 7 is a diagram for explaining recognition of packages in the sorting area according to the present embodiment. Here, a certain sorting area 701 on the conveyance path of the conveyor 302, and sorting areas 702 and 703 adjacent to it on the upstream and downstream sides in the conveying direction will be described as an example. It is also assumed that three packages 706, 707, and 708 are being transported by the transport conveyor 302.

A handover area 704 is provided in the sorting area 701 and the sorting area 702. The handover area 704 is an area where detection by sensors is performed in both the client system corresponding to the sorting area 701 and the client system corresponding to the sorting area 702. Similarly, a handover area 705 is provided in the sorting area 701 and the sorting area 703. The handover area 705 is an area where detection by sensors is performed in both the client system 110 corresponding to the sorting area 701 and the client system 110 corresponding to the sorting area 703.

In each sorting area according to the present embodiment, the range of the cargo being transported is specified in a timely manner. By specifying the range of the baggage, the size and shape of the baggage can be recognized. The range here does not need to completely match the shape of the baggage, and a predetermined margin may be provided. Then, the size, shape, and position of the projection pattern to be projected onto each piece of baggage are adjusted according to the range of each piece of baggage. Note that the adjustment method is not particularly limited, and may be performed in consideration of, for example, visibility of the projection pattern, work efficiency, etc.

Note that in this embodiment, the label reader 103 and the image sensor 107 are described as being different, but the label reader 103 may be used instead of the image sensor 107. Specifically, the position of the label read by the label reader 103 is treated as the position of the package, and a projection image is projected onto the position of the label. Since the identification information of the package can be obtained from the label, by adopting such a configuration, the process of linking the same package between client systems 110 becomes unnecessary. However, since the area that the label reader 103 can recognize is limited to the area of the label, the degree of freedom in the size of the projected image and the position of the projected image is reduced compared to tracking by the image sensor 107 that can track the entire package. do. Similarly, image sensor 107 may be used in place of label reader 103. Specifically, label information may be acquired from an image of a label included in an image acquired by the image sensor 107. In the following, unless otherwise specified, an example of tracking a package using an image captured by the image sensor 107 will be shown, but the label reader 103 may be used instead of the image sensor 107.

[Management of entire transport route]
As shown in FIGS. 3 and 4, the cargo conveyance path is composed of a plurality of sorting areas. In order to appropriately monitor and manage the transportation status of packages, it is necessary to appropriately understand the entire transportation route and manage the transportation status of packages in each sorting area.

Image information for each sorting area can be obtained from the corresponding sensors in a timely manner. In this embodiment, this image information is linked to generate and manage an overall image representing the entire transport route. Then, the baggage sorting system 100 manages and controls the conveyance state of each baggage based on the overall image.

FIG. 8 is a flowchart of the stitching process for the entire conveyance route consisting of a plurality of sorting areas. Note that the constituent elements of each of the plurality of client systems 110 perform similar processing, and will be described together here. This processing flow may be performed in parallel when transporting cargo, or may be performed as an initial calibration when the system is installed.

The processes from step S801 to step S803 are processes executed by each of the plurality of client systems 110. Here, for convenience, the distance sensor 106 and the image sensor 107 are collectively referred to as sensors.

The sensors photograph the inside of the target sorting area and acquire the image (step S801). The images acquired here may include not only images taken by the image sensor 107 but also distance images taken by the distance sensor 106 .

The sensors transmit the obtained results to the sorting client 104 (step S802). At this time, the sensors may perform predetermined processing on the acquired images and then transmit the images to the sorting client 104. For example, in the case of the image sensor 107, arbitrary image processing (for example, image compression or correction processing) may be performed on the captured image.

The sorting client 104 further transmits the acquisition results sent from the sensors in step S802 to the sorting management server 101. At this time, arbitrary image processing (for example, image compression or correction processing) may be performed on the acquisition results sent from the sensors.

The sorting management server 101 acquires images of each sorting area from all client systems 110 (step S804).

The sorting management server 101 performs a stitching process between the sorting areas acquired in step S805. FIG. 9 is a diagram for explaining an overview of the stitching process according to this embodiment. In FIG. 9, for convenience, the x-axis is shown as the transport direction of the cargo, and the y-axis is shown as the width direction of the transport conveyor 302. Here, images 901 and 902 corresponding to two sorting areas will be described as an example. As described using FIG. 7 and the like, adjacent sorting areas partially overlap as handover areas. Therefore, the explanation will be given here assuming that images 901 and 902 are partially photographed overlappingly. Further, in this embodiment, it is assumed that a marker for stitching processing is attached to a part of the transport conveyor 302. Here, markers 903 and 904 attached near the edges in the width direction of the conveyor 302 will be described as an example. Note that the handover area shown in FIG. 7 and the overlapping area used in the stitching process may be the same, or may be partially different.

First, the sorting management server 101 aligns each image in the y-axis direction. The sorting management server 101 detects the ends of the conveyor 302 in the width direction from each image. In the case of FIG. 9, the upper end and the lower end are detected. Then, the sorting management server 101 performs adjustment by enlarging or reducing the image so that the edges match. Since the size of the transport conveyor 302 is predefined, adjustments may be made based on this information. Next, the sorting management server 101 aligns each image in the x-axis direction. The sorting management server 101 detects markers 903 and 904 included in each image, and adjusts the arrangement of the images so that their positions match. By performing the above-described stitching process between all adjacent images, a conveyance path image corresponding to the space of the entire conveyance path is generated. Further, the sorting management server 101 defines the absolute coordinate system of the entire transport route by setting the coordinate system in the generated transport route image. Note that the coordinate system may be set as a relative coordinate system between images.

In the above example, adjustment in the x-axis direction in the stitching process was performed using markers attached in advance. However, the configuration is not limited to this, and objects commonly included in adjacent images may be used. For example, if this processing flow is performed while a package is being transported, a package that has been photographed overlappingly may be detected from among the packages included in the image, and the region of the package may be used as a marker. For example, in the example shown in FIG. 9, three packages 905, 906, and 907 are being transported. In this case, baggage 906 is included in images 901 and 902 overlappingly. Therefore, the adjustment may be performed by detecting the edges of the area of the baggage 906 from the images 901 and 902, respectively, and performing alignment so that the edges match. Further, the marker is not limited to a configuration that is used for adjustment only in one direction, and a marker that allows positioning in both the x-axis direction and the y-axis direction may be used.

The sorting management server 101 feeds back information on the coordinate system set in step S805 to the plurality of sorting clients 104 (step S806). Since the plurality of sorting clients 104 manage different sorting areas, the sorting management server 101 feeds back coordinate system information according to the sorting area of each of the plurality of sorting clients 104.

The sorting client 104 uses the coordinate system information fed back from the sorting management server 101 in step S806 to adjust the settings of the coordinate system that it manages. In this adjustment, for example, the coordinate values specified by the sorting management server 101 are adjusted so as to match the values of the coordinate system managed by the sorting client 104. Furthermore, the sorting client 104 may be configured to also adjust the coordinate system of the projector 105. Note that the adjustment process here is not limited to changing the setting values, but also changes the coordinate values sent from the sorting management server 101 and the values of the coordinate system managed by the sorting client 104. The configuration may be such that errors are stored in association with each other as calibration information. In this case, when actually transporting packages, each package is accurately identified and tracked by correcting the coordinate values from the sorting management server 101 using the above calibration information. good. Then, this processing flow ends.

As described above, the process flow shown in FIG. 8 may be performed concurrently and timely during the cargo transport operation, or may be performed as an initial setting when the system is installed. The entire transport route of the transport conveyor 302 obtained through the above processing is managed by the sorting management server 101, and the information is fed back to each of the plurality of sorting clients 104. Then, each process shown in FIGS. 5 and 6 is performed using the feedback information.

This enables more precise cooperation between devices in transporting cargo. In particular, in processes such as steps S511 to S513 in FIG. 5, the accuracy of correspondence is improved in processes that require correspondence between parcels detected on the sorting client 104 side and coordinate information sent from the sorting management server 101. It becomes possible to do so. As a result, it becomes possible to sort luggage more efficiently and accurately.

Note that in this embodiment, an image of the entire transport route is generated, but at this time, the transport route may be reproduced in a three-dimensional shape and managed. This makes it possible to appropriately recognize cases where the transport path is flat, slopes depending on the sorting area, and the shape of the cargo being transported.

In addition, the sorting management server 101 manages and controls the conveyance state of packages being conveyed along the entire conveyance route, and the contents of the management and control are not particularly limited. As described above, the sorting management server 101 may control the projection pattern for each package, or may mechanically control the operation of the conveyor, depending on the conveyance state along the entire conveyance path. In addition, control corresponding to each of the plurality of client systems 110 may be performed, or control common to the plurality of client systems 110 as a whole may be performed.

As described above, the baggage sorting system 100 according to the first embodiment uses the sorting management server 101 that acquires a plurality of images of the conveyor 302 photographed by sensors and the plurality of images to create an overall image of the conveyor 302. The sorting management server 101 and the sorting client 104 control the conveyance of packages being conveyed on the conveyor 302 using the entire image.

This makes it possible to sort luggage more efficiently and accurately. In particular, the accuracy of coordinate system adjustment between the plurality of sorting clients 104 and the sorting management server 101 is improved, and it becomes possible to manage packages being transported more appropriately.

Furthermore, the sorting management server 101 performs alignment between two adjacent images among the plurality of images based on objects that are included in duplicate in the two images.

This makes it possible to stitch images using any object that is duplicated in a plurality of images.

Further, the object is a marker provided on the transport conveyor 302.

This makes it possible to stitch images using markers provided in advance on the transport conveyor 302 that are redundantly included in a plurality of images.

Further, the object is a load being transported on a transport route.

This makes it possible to stitch images using the position of the baggage being transported on the transport conveyor 302, which is redundantly included in a plurality of images. Furthermore, it is not necessary to attach objects to the conveyor 302 in advance.

Further, the sorting management server 101 and the sorting client 104 set the coordinate system of the conveyor 302 using the entire image.

This improves the accuracy of coordination of coordinate systems between the plurality of sorting clients 104 and the sorting management server 101, and enables more appropriate management of packages being transported.

The baggage sorting system 100 also includes a sorting client 104 that monitors each of the plurality of sorting areas that constitute the conveyor 302,
The plurality of images are images of each of the plurality of sorting areas.

As a result, even if the transport conveyor 302 is huge, it can be controlled by dividing it into multiple sorting areas corresponding to multiple sorting clients 104, and it is possible to improve the accuracy of cooperation between the devices at that time. Become.

[Modified example]
As described above, in the processing sequences shown in FIGS. 5 and 6, it is also possible to switch the processing subject. For example, the configuration may be such that the processing load on the client system 110 side is reduced and more processing is performed centrally on the sorting management server 101 side.

For example, among the processing in step S520 in FIG. 5, which is the processing on the client system 110 side, the processing in steps S510 to S513 may be performed on the sorting management server 101 side. That is, the sorting client 104 performs only the processing of transferring information detected by sensors to the sorting management server 101 and the processing of instructing the projector 105 to project based on the information acquired from the sorting management server 101. It may be a configuration. In this case, the sorting management server 101 may generate and transmit to each client system 110 an image to be projected by the projector 105 of each client system. In this configuration, the sorting management server 101 performs detection of packages being transported, identification of package information, etc. on behalf of the client system 110 based on information acquired on the client system 110 side. That is, control related to transportation is timely performed on the sorting management server 101 side based on the transportation path generated in the stitching process shown in FIG. 8 and the state of the cargo being transported.

Similarly, among the processing of step S630 in FIG. 6, which is the processing of the downstream client system, the processing of steps S606 to S612 can be configured to be performed on the sorting management server 101 side.

In the case of the above-mentioned configuration, the sorting management server 101 side can manage the conveyance state of packages along the entire conveyance route. As a result, it is also possible to omit baggage handover processing between client systems 110. In other words, by managing the entire transportation status of packages on the sorting management server 101 side, the client system 110 needs to detect newly transferred packages in the corresponding sorting area and monitor packages being transferred. It disappears. By omitting handover processing and having the sorting management server 101 perform comprehensive transport management, for example, tracking errors and handover errors between client systems 110 can be suppressed, and the tracking accuracy of packages being transported can be improved. It becomes possible to configure the system so as to improve the performance.

Note that the division of processing between the sorting management server 101 and the client system 110 is not limited to the above. For example, among the plurality of client systems 110, the client system 110 corresponding to a specific sorting area may be assigned a different responsibility than other client systems. As a specific sorting area, for example, the client system 110 corresponding to the most upstream sorting area, the most downstream sorting area, or a sorting area including a branch in the transport route shares processing with the sorting management server 101. may be different from other client systems 110.

Further, the information to be fed back may be changed depending on the configuration of processing allocation between the sorting management server 101 and the client system 110. For example, when the coordinate system for the entire transport route is managed on the sorting management server 101 side, there is no need for calibration on the client system 110 side each time, so information regarding this may be omitted. In addition to the respective processing loads of the sorting management server 101 and the client system 110, the processing assignment may be determined by taking into account the communication load between the sorting management server 101 and the client system 110.

<Other embodiments>
In the embodiment described above, each of the plurality of sorting clients 104 may feed back to the sorting management server 101 the transportation status of packages in the sorting area that it is in charge of. As a result, the sorting management server 101 can flexibly switch the projection images projected onto packages in each of the plurality of sorting clients 104, and can give flexible sorting instructions to workers who sort packages in each sorting area. (sorting instructions).

The transportation status of packages fed back by each of the plurality of sorting clients 104 may be, for example, the number of packages currently existing in each sorting area. Having received this feedback, the sorting management server 101 performs sorting at the upstream sorting client so as to reduce the number of new packages to be conveyed to the sorting area, for example, when there are many packages in the downstream sorting area. Instructions may be changed. Specifically, a projection image showing a transport route connected to another sorting client may be projected onto the package, and the package may be moved between the transport routes by a worker in an upstream sorting area.

As a result, when managing the conveyance status on the entire conveyance route, it is possible to switch the sorting instructions on the upstream side depending on the number of packages on the downstream side, that is, the congestion situation, and further suppress congestion on the downstream side. becomes possible.

Further, the transportation status of the cargo fed back by each of the plurality of sorting clients 104 may be, for example, the time required for sorting in each sorting area. This time can be obtained by measuring the time from when the baggage enters the corresponding sorting area to when the baggage is picked up by a worker and moved out of the sorting area. In this case as well, the sorting management server 101 may change the sorting instructions in the upstream sorting client 104, for example, so as to reduce the number of packages newly transported to the sorting area where sorting takes time.

As a result, when managing the conveyance status of the entire conveyance route, the upstream sorting instructions can be switched depending on the work time of downstream loads, that is, the work efficiency in real time, and further downstream sorting instructions can be changed. It becomes possible to suppress congestion.

Note that since the sorting client 104 has a function of projecting images onto packages, the number of packages may be more accurate as information fed back to the sorting management server 101 than the time required for sorting. Highly sexual. However, information on the number of packages does not easily reflect worker performance, so for example, if there is a worker in the sorting area who can quickly sort a large number of packages, it is difficult to determine whether or not to reduce the number of packages to be transported. There is a risk of making a mistake in judgment. Therefore, whether the number of packages or the time required for sorting is to be fed back may be changed depending on the user's selection of the package sorting system 100.

Furthermore, the sorting management server 101 may change the contents of sorting instructions between an upstream sorting client and a downstream sorting client even for the same package. For example, if there are multiple conveyance routes that can be sorted to the same sorting destination downstream, the upstream can be instructed to sort to the conveyance route with a margin, and the downstream can be instructed to sort to the same destination. Good too. Furthermore, it is possible that the originally planned destination for sorting may change while the cargo is being transported from upstream to downstream. For example, if the destination for downstream sorting is a truck, and the cargo is loaded in excess of the capacity of the truck before it is transported from upstream to downstream, if the cargo is the same, It is also necessary to change the upstream sorting destination (original truck) and the downstream sorting destination (changed truck). In this case, the sorting management server 101 determines whether or not to change the sorting instructions at the downstream sorting client by receiving feedback from the sorting client 104, such as the number of packages sorted to each sorting destination. You can.

This makes it possible to switch and notify the operator of sorting instructions even if the conveyance conditions for the same package change from moment to moment.

Further, programs and applications for realizing the functions of one or more embodiments described above may be supplied to a system or device using a network or a storage medium, and one or more processors in a computer of the system or device may This can also be achieved by reading and executing a program.

Further, it may be realized by a circuit (for example, an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array)) that realizes one or more functions.

Although various embodiments have been described above with reference to the drawings, it goes without saying that the present disclosure is not limited to such examples. It is clear to those skilled in the art that various changes, modifications, substitutions, additions, deletions, and equivalents may be made within the scope of the claims; It is understood that it naturally falls within the technical scope of the present disclosure. Moreover, each component in the various embodiments described above may be combined arbitrarily within a range that does not depart from the spirit of the invention.

The present disclosure is useful as a baggage sorting system, a baggage sorting method, an information processing device, and a control method thereof that can improve the convenience of actions according to rights for multiple users.

100... Baggage sorting system 101... Sorting management server 102... Cooperation server 103... Label reader 104... Sorting client 105... Projector 106... Distance sensor 107... Image sensor 110... Client system 300... Baggage 301... Label 302... Transport conveyor 303... Rotary encoder

Claims (13)

A baggage sorting system including a plurality of sorting clients that monitor each of a plurality of sorting areas that constitute a transport route,
an acquisition unit that acquires images of each of the plurality of sorting areas photographed by a photographing unit provided in each of the plurality of sorting clients;
a generation unit that generates an entire image of the conveyance route using images of each of the plurality of sorting areas;
a control unit that uses the entire image to control transport of the cargo being transported on the transport route;
has,
Baggage sorting system. Each of the plurality of sorting clients includes an instruction device for instructing the sorting of packages within the sorting area,
The control unit changes the content of the sorting instruction in at least one sorting client based on the conveyance status of the cargo throughout the conveyance route specified based on the overall image.
The baggage sorting system according to claim 1. The conveyance status of the baggage is the number of bags existing in each of the plurality of sorting areas,
The control unit changes the content of a sorting instruction for packages existing in a sorting area located upstream of the transport route based on the number of packages existing in a sorting area located downstream of the transport route.
The baggage sorting system according to claim 2. The conveyance status of the baggage is the time required for sorting the baggage in each of the plurality of sorting areas,
The control unit changes the content of a sorting instruction for packages existing in a sorting area located upstream of the transport route based on the time required for sorting the packages in a sorting area located downstream of the transport route. do,
The baggage sorting system according to claim 2. The control unit may issue a sorting instruction for the same package based on the transportation status of the package on the entire specified transportation route, when the package is located upstream of the transportation route, and when the package is located upstream of the transportation route. It differs depending on whether it is located downstream of
The baggage sorting system according to claim 2. The instruction device is a projection device that projects an image indicating the sorting instructions onto the baggage in the sorting area.
The baggage sorting system according to any one of claims 2 to 5. The generation unit performs alignment between the images of the two sorting areas based on an object that is redundantly included in the images of two adjacent sorting areas among the images of the plurality of sorting areas.
The baggage sorting system according to any one of claims 1 to 6. The object is a marker provided on the transport route,
The baggage sorting system according to claim 7. The object is a load being transported on a transport route,
The baggage sorting system according to claim 7. The control unit uses the entire image to set a coordinate system of the conveyance path.
The baggage sorting system according to any one of claims 1 to 9. A baggage sorting method in a baggage sorting system including a plurality of sorting clients that monitor each of a plurality of sorting areas constituting a transport route, the method comprising:
an acquisition step of acquiring images of each of the plurality of sorting areas photographed by a photographing unit provided in each of the plurality of sorting clients;
a generation step of generating an entire image of the conveyance route using images of each of the plurality of sorting areas;
a control step of controlling the conveyance of the cargo being conveyed on the conveyance route using the entire image;
has,
How to sort luggage. a processor;
Equipped with memory and
The processor cooperates with the memory to:
Obtaining images of each of the plurality of sorting areas taken by a photographing unit provided in each of a plurality of sorting clients that monitor each of the plurality of sorting areas constituting the transport route,
generating an entire image of the transport route using images of each of the plurality of sorting areas;
controlling the conveyance of the cargo being conveyed on the conveyance route using the entire image;
Information processing device. Processor works with memory to
Obtaining an image of each of the plurality of sorting areas taken by a photographing unit provided in each of the sorting clients that monitors each of the plurality of sorting areas constituting the conveyance route,
generating an entire image of the transport route using images of each of the plurality of sorting areas;
controlling the conveyance of the cargo being conveyed on the conveyance route using the entire image;
A method for controlling an information processing device.

Images