JP2023143182A - Baggage search system, baggage search method, information processing device, and control method thereof - Google Patents

Abstract

To perform sort of baggage further efficiently and accurately, in a system for sorting the baggage.SOLUTION: A baggage search system comprises: a photographing unit which photographs a plurality of continuous images of baggage being transported; and a detection unit which detects the baggage by using each of the plurality of images. The detection unit, after detecting the baggage by using a first image among the plurality of images, when detecting the baggage by using a second image following the first image, sets a search range of the baggage in the second image on the basis of a position of the baggage detected from the first image, and detects the baggage, from the search range, among the second images.SELECTED DRAWING: Figure 8

Description

The present disclosure relates to a baggage search system, a baggage search method, an information processing device, and a control method thereof.

With the increase in economic activity in recent years, the amount of parcels in circulation is increasing. In the baggage distribution process, the sorting process of sorting packages by destination is a time-consuming process that has traditionally relied on manual labor, but technologies have also been proposed to automate at least part of the sorting process.

Patent Document 1 tracks a package moving on a transport route, determines an image to be displayed based on information about the package read from the package and information on the location of the package, and projects the image onto the package from a projector to display the image. The company is disclosing a system that displays this on luggage.

US Patent No. 7,090,134

However, in recent years, the amount of parcels in circulation has been increasing more and more, and the types of parcels have become more diverse, and there is a need for technology for sorting parcels more efficiently and accurately.

The present disclosure was devised in view of the above-mentioned conventional circumstances, and relates to a technique for sorting packages more efficiently and accurately. In particular, the present invention aims to provide a baggage search system, a baggage search method, an information processing device, and a control method thereof.

The present disclosure includes a photographing unit that captures a plurality of consecutive images of a package being transported, and a detection unit that detects the package using each of the plurality of images, and the detection unit includes a plurality of images of the package being transported. After detecting a package using a first image of the images, when detecting a package using a second image subsequent to the first image, the position of the package detected from the first image. The present invention provides a baggage search system that sets a search range for the baggage in the second image based on the above, and detects the baggage from the search range in the second image.

Further, the present disclosure includes a photographing step of photographing a plurality of consecutive images of the baggage being transported by a photographing unit, and a detection step of detecting the baggage using each of the plurality of images, and the detection step After detecting a package using a first image of the plurality of images, when detecting a package using a second image subsequent to the first image, A baggage search method is provided, in which a search range for the baggage in the second image is set based on the detected position of the baggage, and the baggage is detected from the search range in the second image. .

Further, the present disclosure includes a processor and a memory, and the processor cooperates with the memory to acquire a plurality of consecutive images of the cargo being transported, and uses each of the plurality of images to Detecting a package, and in the detection, detecting the package using a first image of the plurality of images, and then detecting the package using a second image subsequent to the first image. , a search range for the package in the second image is set based on the position of the package detected from the first image, and the package is detected from the search range in the second image. , provides an information processing device.

Further, the present disclosure provides a method in which a processor cooperates with a memory to acquire a plurality of consecutive images of the transported baggage, detects the baggage using each of the plurality of images, and in the detection, the plurality of After detecting a package using a first image of the images, when detecting a package using a second image subsequent to the first image, the position of the package detected from the first image. A search range for the baggage in the second image is set based on the search range for the baggage in the second image, and the baggage is detected from the search range in the second image.

Note that arbitrary combinations of the above components and expressions of the present disclosure converted between methods, devices, systems, storage media, computer programs, etc. are also effective as aspects of the present disclosure.

According to the present disclosure, it is possible to sort packages more efficiently and accurately.

A block diagram showing a configuration example of a baggage sorting system according to Embodiment 1 Block diagram illustrating a configuration example of an information processing device according to Embodiment 1 Schematic diagram for explaining transportation of luggage according to Embodiment 1 Schematic diagram for explaining detection and projection of luggage according to Embodiment 1 Processing sequence of baggage sorting system according to Embodiment 1 Processing sequence of baggage sorting system according to Embodiment 1 Schematic diagram for explaining recognition of cargo during transportation according to Embodiment 1 Schematic diagram for explaining the detection range of cargo being transported according to Embodiment 1 Flowchart of processing for associating baggage information according to Embodiment 1

DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments specifically disclosing a baggage search system, a baggage search method, an information processing device, and a control method thereof according to the present disclosure will be described in detail with reference to the accompanying drawings as appropriate. However, more detailed explanation than necessary may be omitted. For example, detailed explanations of well-known matters or redundant explanations of substantially the same configurations may be omitted. This is to avoid unnecessary redundancy in the following description and to facilitate understanding by those skilled in the art. The accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter recited in the claims.

<Embodiment 1>
[System configuration]
FIG. 1 is a block diagram showing a configuration example of a luggage sorting system 100 according to an embodiment. The baggage sorting system 100 is a system that supports the work of a worker who sorts a plurality of parcels conveyed by a conveyor (not shown in FIG. 1), and also operates as a baggage search system. The package sorting system 100 is installed, for example, in a distribution center owned by a retailer, wholesaler, Internet distributor, or the like. The baggage to be sorted generally has a substantially rectangular parallelepiped shape, but there are no particular limitations on its external shape or size, and there are no particular limitations on the type of baggage. The conveyor that conveys the luggage may be, for example, a belt conveyor or a roller conveyor. Furthermore, the conveyance path for the cargo may be configured such that the conveyance direction changes or a part of the conveyance path branches by combining a plurality of conveyors.

The baggage sorting system 100 includes a sorting management server 101, a cooperation server 102, a label reader 103, a sorting client 104, a projector 105, a distance sensor 106, and an image sensor 107. A plurality of client systems 110 are provided in which the sorting client 104, the projector 105, the distance sensor 106, and the image sensor 107 are set as one set, and are associated with each of the plurality of sorting areas provided on the transport conveyor that transports packages. Therefore, a plurality of sorting clients 104, projectors 105, distance sensors 106, and image sensors 107 are each installed according to the number of sorting areas. In the following description, when an individual device needs to be explained, subscripts (a, b, . . . ) are added, and when the devices are explained collectively, the subscripts are omitted.

The sorting management server 101 is an information processing device that controls the entire package sorting system 100 according to this embodiment. The sorting management server 101 acquires the detection results of the label reader 103 and acquires various baggage information from the cooperation server 102. Further, the sorting management server 101 issues various instructions to each of one or more sorting clients 104 and collects various information. Further, the sorting management server 101 may control the operation of a conveyor, which will be described later, and collect and manage operation information.

The cooperation server 102 stores and manages label information and package information in association with each other. The cooperation server 102 then provides package information corresponding to the label information read by the label reader 103 in response to a request from the sorting management server 101. In the example of FIG. 1, one sorting management server 101 and one cooperation server 102 are shown, but they may be configured with a plurality of devices from the viewpoint of functional redundancy and load distribution. Further, label information and sorting patterns may be managed in association with each other on the sorting management server 101 side, and based on this, processing related to sorting of each baggage, which will be described later, may be performed. In such cases, the configuration of the cooperation server 102 is omitted, and the sorting management server 101 is configured to perform processing related to sorting of each package, which will be described later, based on the pattern of acquired label information. good.

The label reader 103 is a device for reading labels attached to packages being transported by a transport conveyor, and includes optical components such as a lens and an image sensor. By reading the label attached to the package, various information shown on the label (hereinafter referred to as "label information") is acquired. The format or method of the label is not particularly limited, and for example, a barcode or two-dimensional code (QR code (registered trademark), etc.) can be used. In this embodiment, a barcode will be used as an example of a label. Further, the label information may include, for example, identification information for uniquely identifying the package. In this case, the type of characters and the number of digits constituting the identification information are not particularly limited.

The label reader 103 uses, for example, a general imaging camera to extract a label area from a captured image through image recognition and read various information, or performs OCR (Optical Character Recognition) on characters written on the label. /Reader). The label reader 103 may be a dedicated device for reading labels, or may be integrated with other devices. Further, the label reader 103 may be configured to detect the three-dimensional coordinates of the label. Furthermore, a plurality of label readers 103 may be provided to accommodate various sizes of packages, or a configuration may be used to photograph packages from multiple directions in response to various label attachment positions. good.

The package information according to this embodiment includes, for example, a package identification number that is individually assigned to a package (e.g., identification information for uniquely identifying a package), the name, address, and telephone number of the package sender, Examples include the name, address, telephone number, and type of package of the destination (for example, recipient). Note that the information is not limited to these, and the items included in the baggage information may be increased or decreased as necessary. In this embodiment, the sorting management server 101 acquires label information from the label reader 103, and uses this label information to make an inquiry to the cooperation server 102, thereby acquiring corresponding package information. As described above, in the case of a configuration in which cooperation with the cooperation server 102 is omitted, label information and pattern information related to sorting may be stored in association with each other on the sorting management server 101 side.

The sorting client 104 is an information processing device that controls projection processing and the like for the cargo being transported based on instructions from the sorting management server 101. Further, the sorting client 104 acquires the photographic results of the distance sensor 106 and the image sensor 107, and detects and manages packages based on the photographic results. Further, the sorting client 104 provides the sorting management server 101 with position information, size information, detected time information, etc. of the detected baggage.

The projector 105 is a projection device that irradiates a projection image including a predetermined projection pattern onto the cargo on the transport route based on instructions from the sorting client 104 . Note that in this embodiment, the "projection image" refers to the entire image projected by the projector 105 onto the entire sorting area in a superimposed manner. Further, the "projection pattern" refers to an image that is displayed in a superimposed manner corresponding to each baggage detected in the sorting area. Therefore, when a package is detected in the sorting area, the projected image will include one or more projection patterns depending on the package. Examples of projection patterns will be described later.

The distance sensor 106 is a sensor for detecting distance information between the distance sensor 106 and an object to be imaged (for example, the surface of a package), and acquires the detection result as a distance image. The distance image may include something that cannot be recognized as an image by the human eye, such as a table listing numerical values indicating distance. That is, the distance image may be information indicating the relationship between coordinates and distance within the imaged area, and its data structure is not particularly limited. As the distance sensor 106, for example, an infrared sensor (IR sensor), a stereo camera composed of a plurality of cameras, or the like may be used.

The image sensor 107 is an imaging device that includes optical components such as a lens and an image sensor. The image sensor 107 is composed of, for example, an imaging camera. As the imaging camera, a three-dimensional camera or a plurality of two-dimensional cameras may be used. The image sensor 107 captures an image of the conveyor including the baggage being conveyed by the conveyor, and generates a color image. The color image here refers to an image in which the color of the surface of the object to be photographed is expressed in a predetermined gradation. In addition to the 256 gradations of RGB (Red, Green, and Blue), the gradation may include a gray scale.

The number of client systems 110 varies depending on the length of the conveyor, the detection range of each sensor, and the like. Note that the sorting client 104 is configured as one unit, and this one sorting client 104 controls a plurality of projectors 105, a plurality of distance sensors 106, and a plurality of image sensors 107 corresponding to a plurality of sorting areas. It may be a configuration. Further, in this embodiment, an example is shown in which the distance sensor 106 and the image sensor 107 are used as sensors, but a sensor in which these are integrated may be used, or a configuration in which only one of them is used is also possible. It may be. Also, other types of sensors may be used. The conveyance format of the conveyor is not particularly limited, and for example, the conveyor may be configured to convey the loads in one line, or may be configured to convey the loads in multiple lines.

Each device is communicably connected by wire/wireless. The communication method and connection method here are not particularly limited, and the connection may be made using a predetermined network cable, or may be made by combining a plurality of methods.

(Information processing device)
FIG. 2 is a diagram illustrating an example of the hardware configuration of an information processing device that can be used as the sorting management server 101, the cooperation server 102, and the sorting client 104 according to this embodiment. Although each device will be described here as having the same configuration, each device may have a different configuration.

The information processing device 200 includes a CPU (Central Processing Unit) 201, a memory 202, a storage device 203, an input/output section 204, a communication section 205, and an external I/F 206. The CPU 201 implements various functions by reading various programs and data stored in the memory 202 and the storage device 203 and executing processes. Note that the CPU 201 may be another arithmetic circuit such as a GPU (Graphics Processing Unit), or may be used in combination with another arithmetic circuit. The memory 202 is a storage area for storing and retaining various information, such as ROM (Read Only Memory) which is a non-volatile storage area, RAM (Random Access Memory) which is a volatile storage area, etc. configured. The storage device 203 is a storage area for storing and retaining various information, and is composed of an HDD (Hard Disk Drive) or the like.

The input/output unit 204 receives instructions from a user from a mouse or keyboard (not shown), and outputs various information from a display (not shown), for example. The communication unit 205 communicates with an external device via a wired/wireless network, and sends and receives various data and signals. The communication method by the communication unit 205 is not particularly limited, and may be compatible with a plurality of communication methods. For example, WAN (Wide Area Network), LAN (Local Area Network), power line communication, short-range wireless communication (eg, Bluetooth (registered trademark)), etc. may be used. External I/F 206 is an interface for transmitting and receiving data with an external device. Each part of the information processing device 200 is communicably connected via an internal bus (not shown) or the like.

[Example of transportation]
FIG. 3 is a schematic diagram for explaining the transportation of luggage according to this embodiment. A label 301 is attached to each baggage 300 to be transported. The label 301 is read during transportation by a label reader 103 installed at an arbitrary position on the transportation path. In the example of FIG. 3, the label 301 is attached to the top surface of the luggage 300 and is read from the top by the label reader 103, but the label 301 is not limited to this. For example, a label 301 may be attached to the side of the luggage 300, and the label reader 103 may be configured to read the label from the side.

The cargo 300 is transported by the transport conveyor 302 at an arbitrary speed. A rotary encoder 303 is installed on the conveyor 302 to detect the conveyance distance in a timely manner and provide it to the sorting management server 101. The rotary encoder 303 is attached to a rotating mechanism (not shown) that rotates as the conveyor 302 transports, and outputs the number of rotations of the rotating mechanism. When the conveyor 302 is a belt conveyor, this rotation mechanism is a roller in contact with the belt or a motor that moves the belt. Moreover, when the transport conveyor 302 is a roller conveyor, it is a roller which comprises the transport conveyor 302. Here, it is assumed that the cargo is being transported along the direction indicated by the arrow. The conveyor 302 changes its conveyance speed or stops conveyance according to a user's designation. Conveyance in the opposite direction may be performed as necessary. Further, the conveyance speed of the conveyor 302 may change due to factors other than the user's designation, such as the physical characteristics of each mechanism of the conveyor 302. However, since the rotary encoder 303 accurately detects the rotation speed of the rotation mechanism, it is less susceptible to changes in the conveyance speed. Conveyance by the conveyor 302 may be controlled, for example, by the sorting management server 101 or by a separately provided control device (not shown).

On the downstream side of the label reader 103 in the transport direction, a plurality of various sensors (in this example, a distance sensor 106 and an image sensor 107) and a plurality of projectors 105 are installed corresponding to a plurality of sorting areas. FIG. 3 shows the most upstream sorting area A among the plurality of sorting areas. Various sensors corresponding to the sorting area A acquire various images by photographing the sorting area A, and using these images, the sorting client 104a detects the cargo being transported. Specifically, when the package 300 reaches the starting position 305a of the sorting area A, the package 300 enters the photographing range of various sensors. The photographing range 304 of the distance sensor 106a and the image sensor 107a includes at least the range of the sorting area A. Further, within the range of the sorting area A, the projector 105a projects a predetermined projection image based on instructions from the sorting management server 101 and the sorting client 104. A projection range 306 by the projector 105a includes at least the sorting area A. Note that the distance from the reading position of the label reader 103 to the starting position 305a may be defined according to the processing speed of the processing sequence described later, the reading speed of various sensors, and the like.

A worker M who performs work on the cargo 300 is located around the transport conveyor 302, and performs various tasks as necessary. At this time, the worker M can perform work based on the projection image (projection pattern, etc.) projected by the projector 105.

FIG. 4 is a diagram for explaining a plurality of sorting areas corresponding to each of a plurality of client systems 110. Here, two sorting areas A and B will be taken as an example and explained as being monitored by client systems 110a and 110b, respectively. The number of sorting areas and their range (for example, length and width in the transport direction) may be defined according to the functions of the client system 110. Therefore, further downstream of the sorting area B in the conveyance direction, one or more sorting areas may be provided.

The sorting area A is photographed by the distance sensor 106a and the image sensor 107a, and the image is provided to the sorting client 104a. Further, in the sorting area A, a projection image including an arbitrary projection pattern is projected by the projector 105a. Similarly, sorting area B adjacent to sorting area A on the downstream side in the transport direction is photographed by distance sensor 106b and image sensor 107b, and the image is provided to sorting client 104b. Furthermore, in the sorting area B, a projection image including an arbitrary projection pattern is projected by the projector 105b. In the sorting area B, monitoring is carried out in a manner that takes over the monitoring of the packages conveyed from the sorting area A. At this time, as shown in FIG. 4, the sorting areas of the respective client systems 110 partially overlap, and hereinafter, this area will also be referred to as a "handover area." The start position 305b indicates the start position of the monitoring of the sorting area B, and is different from the end position (not shown) of the monitoring of the sorting area A adjacent to the upstream side of the sorting area B. Details of the handover area will be described later.

[Processing sequence]
The processing sequence of the baggage sorting system 100 according to this embodiment will be explained using FIGS. 5 and 6. The processing of each device may be realized by the processing entity (CPU, etc.) of each processing executing programs corresponding to various functions. It is assumed that the operation of the transport conveyor 302 is started when the processing shown in FIGS. 5 and 6 is started, and thereby the packages are sequentially transported. Note that the subject of each process in the processing sequence shown below is an example, and a part thereof may be executed by another device. For example, the configuration may be such that the sorting client 104 executes part of the processing of the sorting management server 101 described below. Alternatively, each server and sorting client shown in FIG. 1 may be configured as one device, and the following processing sequence may be executed by this one device.

The label reader 103 detects a label attached to a package being transported on the transport conveyor 302, and reads the label information (step S501).

The label reader 103 transmits the label information read in S501 to the sorting management server 101 (step S502). The information transmitted at this time may include, in addition to the information shown on the label, the time of reading and positional information (for example, three-dimensional information) detected when reading the label.

The sorting management server 101 acquires the label information transmitted from the label reader 103 (step S503). At this time, the sorting management server 101 acquires the detected values from the rotary encoder 303 provided in the transport conveyor 302, and stores them in association with each other. The value detected by the rotary encoder 303 corresponds to the moving distance of the conveyor 302 based on its rotation speed. Furthermore, the sorting management server 101 uses the acquired label information to inquire of the cooperation server 102 about package information.

The cooperation server 102 searches a predetermined DB (database) for package information corresponding to the label information indicated in the inquiry from the sorting management server 101 and identifies it (step S504). The DB may be provided internally in the cooperation server 102, or may be configured in an external device (not shown) connected to the cooperation server 102 via a network.

The cooperation server 102 transmits the package information specified in step S504 to the sorting management server 101 as a response to the inquiry (step S505).

The sorting management server 101 determines a projection pattern based on the package information acquired from the cooperation server 102 (step S506). The projection pattern according to this embodiment is defined and held in advance as a plurality of patterns. The projection pattern may be determined, for example, depending on the destination (for example, the destination address) indicated in the package information, or may be determined depending on the sorting area.

The sorting management server 101 transmits a projection instruction including the projection pattern determined in step S506 to the sorting client 104 (step S507). Here, as shown in FIG. 3, the explanation will be made assuming that the information is transmitted to the sorting client 104a of the client system 110a corresponding to the sorting area A located at the most upstream side of the conveyance route. That is, each package is first transported toward the sorting area A. In addition to the projection pattern, the projection instruction may include the package information acquired in step S506, the label information read by the label reader 103 in step S501, and information on the time at which it was read. The projection instruction here may be transmitted from the sorting management server 101 to the sorting client 104a based on the sorting client 104a detecting the package using sensors, inquiring information regarding the package. Alternatively, the label information may be transmitted to the sorting client 104a due to the sorting management server 101 receiving label information from the label reader 103.

Step S520 (steps S508 to S515) is a process repeatedly performed by the client system 110 (here, the client system 110a) to monitor each package being transported within the sorting area (here, sorting area A). It is. Here, for convenience, the distance sensor 106 and the image sensor 107 (here, the distance sensor 106a and the image sensor 107a) are collectively referred to as "sensors."

The sensors photograph the inside of the sorting area A and obtain the image (step S508). Note that in this embodiment, the sorting client 104a will be described as one that detects packages using images captured by sensors, but a configuration in which part of the processing is performed on the sensors side is also possible. It may be. For example, sensors may be configured to identify the area corresponding to the baggage (presence or absence of baggage), or detect the three-dimensional coordinates (position) and size (length, width, height) of each baggage. good.

The sensors transmit the obtained results to the sorting client 104a (step S509). At this time, the sensors may perform predetermined processing on the acquired images and then transmit them to the sorting client 104a. For example, in the case of the image sensor 107a, arbitrary image processing (for example, image compression or correction processing) may be performed on the acquired image information.

The sorting client 104a performs a package detection process based on the acquisition results sent from the sensors in step S509 (step S510). Specifically, the sorting client 104a identifies one or more packages being conveyed by the conveyor 302 in the sorting area A based on the results obtained from sensors. Processing for detecting baggage may be performed using any known method, and is not particularly limited. For example, the area corresponding to the baggage may be detected by area segmentation using machine learning. Alternatively, packages may be detected by pattern matching using a predefined pattern.

The sorting client 104a correlates each piece of information based on the projection instruction transmitted from the sorting management server 101 in step S507 and the detection result in step S510 (step S511). As a result, each of the detected one or more packages is associated with various information (projection pattern, etc.) indicated in the projection instruction transmitted from the sorting management server 101.

The projection pattern is, for example, an image of numbers surrounded by a circular frame and has a color indicating the sorting location corresponding to the delivery address of the package. Here, the numbers correspond to, for example, the number of the truck that transports the sorted cargo (the number of the truck itself, the parking lot number, etc.), the number of the shelf or box to be carried into the truck, etc. In addition, the image of the number may not directly correspond to the number of a shelf or box, but may correspond to the number of a shooter (not shown) that moves the picked up cargo to another location or truck, etc. good. Since the parking positions of trucks and the like frequently change depending on traffic conditions, it may be difficult to change the sorting destination as seen from around the transport conveyor 302 at any time. Therefore, by sandwiching a chute between the transport conveyor 302 and a transport truck, etc., and projecting the number of the chute around the transport conveyor 302, it is possible to change the arrangement of the exit of the chute without changing the configuration around the transport conveyor 302 at any time. can respond to changes in sorting destinations. In this case, the contents of the projection pattern may be switched according to changes in the situation.

Other examples of displaying numbers as the projection pattern include the postal code corresponding to the delivery address, the number of the worker who should pick up the package P, etc. Further, as an example of displaying information other than numbers, an arrow indicating the sorting direction (such as right or left with respect to the transport direction of the transport conveyor 302) or characters (such as "left" or "right") may be used. . Further, the display form is not limited to numbers surrounded by a circular frame, but may be various configurations such as numbers surrounded by a square frame ("3", "359", "24735"). Further, the projection pattern is not limited to one in which numbers or letters are surrounded by a frame, but may be white numbers or letters with the background filled in. Further, depending on the information to be displayed, the shapes of the numbers or characters to be displayed, such as circles, triangles, squares, etc., may be changed. Alternatively, a picture that can be individually associated with each piece of information to be displayed may be displayed.

Further, the projection pattern is not limited to a still image, but may be an animation. As examples of animation, each of the above examples may be made to blink, be enlarged/reduced, or change color. Furthermore, an animation reflecting the sorting direction may be projected. Examples of animations that reflect the sorting direction include moving light rays or light spots toward the sorting direction, forming all or part of a projected pattern toward the sorting direction, changing color, Examples include displaying the image by moving an arrow. If you want to animate only a part of the projected pattern, do not change the parts that have a large influence on the operator's judgment of the sorting destination, such as numbers and arrows, and change the parts that have less influence on the sorting destination, such as the frame line. It may be changed. However, in situations where it is more efficient to intuitively convey the sorting direction rather than the meaning of numbers etc. projected within the frame, such as when there are few options for sorting destinations, numbers or arrows may be displayed within a fixed frame. may be moved in the sorting direction. Further, the animation may be projected repeatedly or may be projected only once.

Further, the projection pattern is not limited to one indicating a sorting destination, but may also be a notification of an error occurring in the baggage sorting system 100 or a notification when a worker has overlooked a baggage.

Further, instead of directly projecting the projection pattern onto the luggage, the operator may be made to perceive the projection pattern as if it were being projected onto the luggage in a pseudo manner through glasses that can display an image. In other words, if the worker wears special glasses that can display images, the projected pattern may be superimposed on the image of the package that is viewed through the glasses.

Furthermore, the projection pattern may be projected onto a location other than the baggage. For example, general notification contents to the worker may be projected onto an area of the conveyor 302 where no cargo exists. Examples of such notifications include advance notice of increases and decreases in the amount of arriving packages, remaining work time, and greetings to workers.

The sorting client 104a performs processing in a three-dimensional space on the one or more packages detected in step S510 (step S512). In the processing in the three-dimensional space here, processing is performed so that each package is expressed in a three-dimensional shape in a three-dimensional coordinate system on the transport route. Note that depending on the number and arrangement of sensors, information about surfaces located in blind spots of the sensors may not be obtained. In this case, the three-dimensional shape of each baggage may be complemented to match the general shape of baggage (for example, a rectangular parallelepiped shape, etc.). In addition, if the three-dimensional shape of the surface on which the projection image is to be projected among the surfaces constituting the baggage can be recognized, instructions can be projected. A set of surfaces including the surface may be expressed as the three-dimensional shape of the luggage. In this embodiment, the surface to be projected is the top surface of the luggage, but it may be the side surface or the like. The sensors of this embodiment are arranged above so as to obtain at least information on the top surface of the baggage, but if the surface to be projected is the side surface etc., the baggage is photographed from a position that matches that surface. do.

Regarding the three-dimensional coordinate system on the transport route, it is assumed that the origin and each coordinate axis are defined in advance within the client system 110a. It is also assumed that the three-dimensional coordinate systems of the label reader 103 and the plurality of client systems 110 are associated in advance and adjusted in a timely manner. The plurality of three-dimensional coordinate systems here may share an absolute coordinate system throughout the baggage sorting system 100, and may be associated with each other using that absolute coordinate system. Furthermore, the plurality of three-dimensional coordinate systems may be associated using a relative coordinate system. When using a relative coordinate system, for example, the client system 110 located upstream of the baggage sorting system 100 is used as a reference, and the coordinate systems used by other client systems 110 are set relative to the client system 110 that is the reference. It may also be defined using a coordinate system. Note that any coordinate system within the baggage sorting system 100 may be used as the reference coordinate system for the relative coordinate system. For example, it may be another coordinate system, such as a coordinate system used by another client system or a group of sensors. Note that this process may be performed in a two-dimensional space, that is, a planar space, if adjustment in the height direction is not necessary due to the shape of the cargo to be transported or the configuration of the projector 105. good.

The sorting client 104a draws a projection image including a projection pattern associated with each of the one or more three-dimensional packages processed in step S512 (step S513). Specifically, the sorting client 104a calculates the projection position of the projection pattern for each individual baggage, and determines the projection image for the entire sorting area A. This projected image is, for example, an image in which a region corresponding to each baggage includes a projection pattern for each baggage, and a region where no baggage exists includes a black image. By using such a projection image, it is possible to simultaneously perform projection on a large number of packages while suppressing the number of projectors. In addition, with a general projector, the closer the distance between the surface to be projected and the projector, the smaller the image, and the farther away, the larger the image is projected. Therefore, the size of the projection pattern for each package in the projection image rendered in step S512 may be adjusted so that it becomes larger as the distance between each package and the projector becomes shorter.

The sorting client 104a instructs the projector 105a to project according to the projection image determined in step S513 (step S514).

The projector 105a projects a projection image onto the sorting area A based on the projection instruction from the sorting client 104a in step S514 (step S515). Since each baggage is continuously transported by the transport conveyor 302, the process shown in step S520 may be continuously executed while the transport continues.

(Luggage handover processing)
In FIG. 5, the processing has been described focusing on the first (most upstream) sorting area A on the downstream side in the transport direction from the reading position of the label reader 103. As described above, a plurality of sorting areas are provided on the conveyance route by the conveyor 302, and the monitoring of each package is handed over (hereinafter also referred to as "handover") between the client systems 110 between the sorting areas. conduct.

A processing sequence related to handover processing will be explained using FIG. 6. Here, handover processing between a client system 110a corresponding to sorting area A and a client system 110b corresponding to sorting area B adjacent to sorting area A on the downstream side in the conveying direction on the conveying route will be taken as an example. I will explain. Since the number of sorting areas may vary depending on the length of the transport route, etc., handover processing is performed at appropriate times between client systems 110 corresponding to adjacent sorting areas. It is assumed that the client system 110a is performing the processing sequence described in FIG. 5 simultaneously and in parallel.

Step S620 (steps S601 to S603) is a process repeatedly performed by the client system 110a in order to monitor each package being transported within the sorting area A. Here, for convenience, the distance sensor 106a and the image sensor 107a are collectively referred to as sensors A.

The sensors A photograph the inside of the sorting area A and obtain the image (step S601). Here, as shown in FIG. 5, the description will be made assuming that the sorting client 104a side detects packages using various images taken by sensors A. If a plurality of packages are being transported to the sorting area A, each package is detected. In the case of the client system 110a, this process corresponds to step S508 in FIG.

Sensors A transmit the obtained results to the sorting client 104a (step S602). At this time, the sensors A may perform predetermined processing on the acquired images and then transmit them to the sorting client 104a. In the case of the client system 110a, this process corresponds to step S509 in FIG.

The sorting client 104a detects packages based on the acquisition results sent from the sensors A in step S602 (step S603). Furthermore, the sorting client 104a transmits the transportation information of the detected package to the sorting management server 101 (step S603). Specifically, depending on the results of the processing in the three-dimensional space that has already been performed in the processing sequence shown in FIG. ) may be identified, and information regarding the cargo may be transmitted as transport information. Processing for detecting baggage may be performed using any known method, and is not particularly limited. For example, the area corresponding to the baggage may be detected by area segmentation using machine learning. Alternatively, packages may be detected by pattern matching using a predefined pattern.

Step S630 (steps S604 to S606, steps S610 to S614) is a process repeatedly performed by the client system 110b in order to monitor each package being transported within the sorting area B. Here, for convenience, the distance sensor 106b and the image sensor 107b are collectively referred to as sensors B.

Sensors B photograph the inside of sorting area B and obtain the image (step S604). Here, similarly to the sorting area A, the description will be made assuming that the sorting client 104b side detects packages using various images taken by the sensors B. If a plurality of packages are being transported to the sorting area B, each package is detected.

Sensors B transmit the obtained results to the sorting client 104b (step S605). At this time, the sensors B may perform predetermined processing on the acquired information and then transmit it to the sorting client 104b.

The sorting client 104b detects packages based on the acquisition results sent from the sensors B in step S605 (step S606). Furthermore, the sorting client 104b transmits transportation information of the detected baggage to the sorting management server 101. For example, among the detected packages, a package located at the boundary with the upstream sorting area (here, sorting area A) (i.e., a newly transported package) is identified, and information about that package (location, etc.) is identified. size, etc.) may be transmitted as transport information.

The sorting management server 101 determines between the sorting areas (here, the sorting area The baggage to be handed over between the sorting area A and sorting area B is identified (step S607). In other words, as a parcel is transported, the person monitoring the parcel is handed over from the client system 110a to the client system 110b. , and are associated as being the same package. For example, the sorting management server 101 may make the correspondence by specifying which piece of managed baggage information corresponds to a piece of baggage that straddles the sorting area.

The sorting management server 101 selects the downstream sorting area (here, sorting area B) based on the association in step S607 and the package information used in the upstream sorting area (here, sorting area A). A projection pattern is determined (step S608). The projection pattern according to this embodiment is defined and held in advance as a plurality of patterns. Note that the projection pattern may be the same as the upstream sorting area (here, sorting area A), or may be configured to be switched depending on the sorting area during transportation even if the same baggage is being transported. .

The sorting management server 101 transmits a projection instruction including the projection pattern determined in step S608 to the sorting client 104 (step S609). Here, a projection instruction is sent to the sorting client 104b of the client system 110b to which monitoring of the package is handed over. Note that the projection instruction may include, in addition to the projection pattern, baggage information of the corresponding baggage.

The sorting client 104b correlates each piece of information based on the projection instruction transmitted from the sorting management server 101 in step S609 and the detection result in step S606 (step S610). Specifically, the sorting client 104b identifies one or more packages being conveyed by the conveyor 302 in the sorting area B based on the detection results of the sensors B. Further, the sorting client 104b associates various information included in the projection instruction transmitted from the sorting management server 101 with each of the identified one or more packages. As a result, each of the one or more packages detected by the sorting client 104b is associated with various information (projection pattern, etc.) indicated in the projection instruction transmitted from the sorting management server 101.

The sorting client 104b performs processing in the three-dimensional space for the one or more packages identified in step S610 (step S611). In the processing in the three-dimensional space here, processing is performed so that each package is expressed in a three-dimensional shape in a three-dimensional coordinate system on the transport route. The processing here may be the same as the processing in step S512 in FIG.

The sorting client 104b draws a projection image including a projection pattern associated with each of the one or more three-dimensional packages processed in step S611 (step S612). Specifically, the sorting client 104b calculates the projection position of the projection pattern for each individual baggage, and determines the projected image of the entire sorting area B. The projected image here may have the same configuration as step S513 described in FIG. 5.

Note that a plurality of projectors may be installed in association with one sorting area. For example, one sorting area may be further subdivided to set a range to be handled by each of a plurality of projectors. In this case, the sorting client 104b generates a projection image corresponding to the area in charge of each of the plurality of projectors. An example of a projected image in this case is that the positions of packages that exist within the assigned area include a projection pattern for each package, and the positions where no packages exist within the assigned area include an image that includes a black image. Can be mentioned. By doing this, even if it is difficult to project onto the entire sorting area with one projector due to the performance of the projector or the limit on the height at which the projector can be installed, it is possible to project onto the entire sorting area. I can do it.

Furthermore, each projector may be in charge of one baggage and project only to that baggage. In this case, each of the multiple projectors can change the irradiation angle according to the movement of the baggage it is responsible for, or project a black image to a position other than the position of the baggage it is responsible for, so that only the baggage it is responsible for Projection towards. By doing so, each of the plurality of projectors can perform projection using the optimal parameters for the baggage that it is in charge of. For example, since the focal length of each of the plurality of projectors can be automatically adjusted, even if there are variations in the height of the packages, it is possible to project images that are in focus on each package.

The sorting client 104b instructs the projector 105b to project according to the projection image determined in step S612 (step S613).

The projector 105b projects a projection image onto the sorting area B based on the projection instruction from the sorting client 104b in step S613 (step S614). Since each baggage is continuously transported by the transport conveyor 302, each of the client systems 110 responsible for monitoring the sorting area B and beyond may continue to execute the process shown in step S630 while the transport continues. .

[Baggage recognition]
FIG. 7 is a diagram for explaining recognition of packages in the sorting area according to the present embodiment. Here, a certain sorting area 701 on the conveyance path of the conveyor 302, and sorting areas 702 and 703 adjacent to it on the upstream and downstream sides in the conveying direction will be described as an example. It is also assumed that three packages 706, 707, and 708 are being transported by the transport conveyor 302.

A handover area 704 is provided in the sorting area 701 and the sorting area 702. The handover area 704 is an area where detection by sensors is performed in both the client system corresponding to the sorting area 701 and the client system corresponding to the sorting area 702. Similarly, a handover area 705 is provided in the sorting area 701 and the sorting area 703. The handover area 705 is an area where detection by sensors is performed in both the client system 110 corresponding to the sorting area 701 and the client system 110 corresponding to the sorting area 703.

In each sorting area according to the present embodiment, the range of the cargo being transported is specified in a timely manner. By specifying the range of the baggage, the size and shape of the baggage can be recognized. The range here does not need to completely match the shape of the baggage, and a predetermined margin may be provided. Then, the size, shape, and position of the projection pattern to be projected onto each piece of baggage are adjusted according to the range of each piece of baggage. Note that the adjustment method is not particularly limited, and may be performed in consideration of, for example, visibility of the projection pattern, work efficiency, etc.

Note that in this embodiment, the label reader 103 and the image sensor 107 are described as being different, but the label reader 103 may be used instead of the image sensor 107. Specifically, the position of the label read by the label reader 103 is treated as the position of the package, and a projection image is projected onto the position of the label. Since the identification information of the package can be obtained from the label, by adopting such a configuration, the process of linking the same package between client systems 110 becomes unnecessary. However, since the area that the label reader 103 can recognize is limited to the area of the label, the degree of freedom in the size of the projected image and the position of the projected image is reduced compared to tracking by the image sensor 107 that can track the entire package. do. Similarly, image sensor 107 may be used in place of label reader 103. Specifically, label information may be acquired from an image of a label included in an image acquired by the image sensor 107. In the following, unless otherwise specified, an example of tracking a package using an image captured by the image sensor 107 will be shown, but the label reader 103 may be used instead of the image sensor 107.

[Package tracking]
Consider a case where the same package is detected (tracked) from a plurality of consecutive images when tracking the package being transported on the transport conveyor 302. The image sensor 107 according to this embodiment is a high-speed camera capable of capturing images at high speed (ie, high frame rate). When using such a high-speed camera, the movement (change in position) of the same baggage in continuously captured images is relatively small, considering the transport speed of the baggage by the conveyor 302. . In other words, the position of a package detected in a certain image (frame) is unlikely to change significantly immediately after or within a certain range of images. Note that the term "high speed" here does not particularly limit the speed, and may be defined by the relative relationship with the transport speed of the transport conveyor 302. For example, as the high-speed camera according to the present embodiment, a camera having performance of several hundred frames per second (fps) or more may be used.

Considering this situation, when tracking a package, rather than searching the entire image information, it is possible to track a package based on the position of the package detected in the image immediately before the image of interest or a certain range of images before the image of interest. It is possible to track packages more efficiently by narrowing down the search range.

FIG. 8 is a diagram for explaining reduction of the search range for luggage according to the present embodiment. FIG. 8A shows an example of an image of the conveyance surface of the conveyor 302 taken by the image sensor 107 at a certain point in time. In this example, two items 801 and 802 are being conveyed by the conveyor 302. FIG. 8(b) shows the next image after the image shown in FIG. 8(a) among the consecutive images taken by the image sensor 107. Here, the direction indicated by the arrow will be described as the conveyance direction of the conveyor 302. In FIG. 8(b), packages 801 and 802 have each moved a certain distance along the conveyance direction. A broken line 811 indicates the position of the baggage 801 in the immediately previous image (that is, the image shown in FIG. 8(a)). Furthermore, a broken line 812 indicates the position of the baggage 802 in the immediately previous image (that is, the image shown in FIG. 8(a)).

As shown in FIG. 8(b), the position of a certain baggage can be predicted to be within a certain range based on the position detected in the immediately previous image. FIG. 8(c) shows the search range of each baggage in the image shown in FIG. 8(b). The search range 821 indicates the range in which the package 801 is searched. Similarly, search range 822 indicates the range in which baggage 802 is searched. The size of each search range is set for each package depending on, for example, the size of the package detected in FIG. It's fine.

Specifically, the distance that the package is expected to travel is estimated by multiplying the elapsed time between consecutive images, which is determined as the reciprocal of the frame rate, by the conveyance speed of the conveyor. Then, the range in which the baggage is moved by this distance in the transport direction from the range where the baggage was present in the previous image is set as the reference of the search range. The standard of this search range corresponds to the range in which the cargo can move if it is transported faithfully in the transport direction of the transport conveyor 302. On the other hand, if the position or orientation of the baggage changes due to someone touching the baggage or colliding with another baggage, there is a risk that the actual baggage will end up outside the search range as a result of being transported. Therefore, in the present embodiment, a predetermined margin is added in all directions parallel to the transport surface of the transport conveyor 302 with respect to the search range reference so that the baggage is included in the search range even in such a case. The width of the margin may be determined, for example, in consideration of the range in which the load can move when it receives a force acting from a source other than the conveyance surface of the conveyor 302. For example, the standard may be a value obtained by multiplying the physical limit value of the moving speed of a human hand by the elapsed time between successive images. By providing such a margin, the search range can include the entire range in which the baggage can move due to human touch. Furthermore, if the cargo may be moved due to interference from an object such as a robot arm that is faster than a human hand, the margin may be set based on the limit value of the moving speed of such an object. Further, a margin may be set in consideration of movement caused by collisions between pieces of baggage.

Further, the arrangement of each search range may be set based on, for example, the position of the package detected in FIG. 8(a). In addition, although the example of FIG. 8 shows the case where the conveyance direction is one direction, when conveyance occurs in a direction different from the conveyance direction of the conveyor 302, the search range is set according to that direction. You may do so. For example, when a worker touches a package, the package is transported in a direction that is a combination of the transport direction of the transport conveyor 302 and the direction in which the worker touched it. In this case, the transport speed in a direction different from the transport direction may be, for example, the moving speed of the package between images. As described above, by providing a margin in the search range, packages that have moved in a direction different from the transport direction of the transport conveyor 302 can be included in the search range. Therefore, the transportation speed can be estimated by dividing the amount of movement of the package between images by the elapsed time between images.

FIG. 9 is a flowchart of the package tracking process according to the present embodiment. In this embodiment, this processing is realized by the CPU of the sorting client 104 reading out programs corresponding to each function from the storage unit and executing them.

The sorting client 104 acquires image information from the image sensor 107 (step S901). In this embodiment, the interval of image information acquired from the image sensor 107 may be defined according to the frame rate at which images can be captured by the image sensor 107 and the conveyance speed by the conveyor 302. For example, when the conveyance speed by the conveyor 302 is slow, the time interval for acquiring image information may be set longer than when it is fast.

The sorting client 104 acquires information on the conveyance speed of the conveyor 302 (step S902). In this embodiment, the moving distance (rotation speed) detected by the rotary encoder 303 is provided from the sorting management server 101 to each of the sorting clients 104 in a timely manner. The conveyance speed of the cargo by the conveyor 302 can be calculated using this information. In this embodiment, if the search range is set wide, the accuracy of the speed of the conveyor 302 is not strictly required. Therefore, the rotary encoder 303 may not be installed, and the speed or the like on the catalog of the transport conveyor 302 may be used as information on the transport speed. Further, at this time, the information on the conveyance speed may also include information on the conveyance direction.

The sorting client 104 acquires information on one or more packages detected in the previous image information (step S903). As mentioned above, the transport conveyor 302 is transporting one or more packages that have been detected from previous image information and stored in a timely manner. Note that the immediately previous image information here refers to the immediately previous image information in the order acquired in step S901, and refers to the immediately previous image information in the order of the images (frames) photographed by the image sensor 107. They do not necessarily match.

The sorting client 104 focuses on unprocessed packages among the information on the one or more packages acquired in step S903 (step S904).

The sorting client 104 sets a search range for the corresponding baggage in the main image information based on the position information detected in the image information immediately before the baggage of interest and the conveyance speed acquired in step S902 (step S905). Furthermore, the frame rate of continuous image information, time information, etc. may be used. In this embodiment, the search range is set so as to limit the range within the entire image indicated by the image information where the corresponding baggage is expected to be detected. In this embodiment, the search range is set as a rectangle, as shown in FIG. 8(c).

The sorting client 104 searches for the corresponding package from the search range set in step S906 (step S907). The search process here may be performed using a known method, and is not particularly limited. For example, the area corresponding to the baggage may be detected by area segmentation using machine learning. Alternatively, packages may be detected by pattern matching using a predefined pattern.

The sorting client 104 stores the package detected in step S907 in association with the package detected in the previous image information (step S908). As a result, the baggage in the image information and the baggage information related to the baggage are associated with each other.

The sorting client 104 determines whether there is unprocessed information among the information on the one or more packages acquired in step S903 (step S908). If there is information on unprocessed packages (step S908; YES), the processing of the sorting client 104 returns to step S904 and repeats the process on the information on unprocessed packages. On the other hand, if there is no information on unprocessed packages (step S908; NO), the processing of the sorting client 104 proceeds to step S909.

The sorting client 104 determines whether to end the tracking process (step S909). For example, the tracking process may be determined to end when the operation of the transport conveyor 302 stops, when the entire system stops, when various information cannot be acquired, or the like. If it is determined that the tracking process is to be ended (step S909; YES), this processing flow is ended. On the other hand, if it is determined to continue the tracking process (step S909; NO), the sorting client 104 returns to step S901, acquires the next image information, and continues the process.

Note that in the example of FIG. 9, detection is performed as repeated processing for one or more packages included in the image, but the configuration is not limited to this, and detection processing for multiple packages can be performed simultaneously. It may also be a process that is performed automatically. In any case, in the entire image, a search range is set that is narrowed by the number of packages detected in the immediately previous image information.

Further, the baggage sorting system 100 may also detect a baggage from outside the search range if no baggage is detected within the search range. This increases the possibility that the baggage will be detected even if the baggage moves suddenly and moves out of the search range. Alternatively, the package may be detected only from within the search range, and if the package is not found within the search range, the search for the package may be stopped. By doing so, the processing load required for detecting packages can be reduced.

As described above, the baggage sorting system 100 according to the first embodiment includes: an image sensor 107 that captures a plurality of consecutive images of a baggage being transported; a sorting client 104 that detects a baggage using each of the plurality of images; After the sorting client 104 detects a package using the first image of the plurality of images, when detecting the package using a second image subsequent to the first image, A search range for the package in the second image is set based on the position of the package detected from the first image, and the package is detected from the search range in the second image.

This makes it possible to sort luggage more efficiently and accurately. In particular, it becomes possible to efficiently manage packages being transported while reducing the processing load when tracking each package.

Further, in the second image, the search range is given priority over the outside of the search range to detect the baggage.

This makes it possible to sort items more efficiently.

In addition, packages are detected only from within the search range in the second image.

This makes it possible to sort items more efficiently.

Further, the package is conveyed by the conveyor 302, and the sorting client 104 sets a search range for the package in the second image based on the conveyance speed of the conveyor 302.

This makes it possible to set a search range according to the conveyance speed of each package, making it possible to efficiently track packages.

Further, the baggage is transported by the transport conveyor 302, and the sorting client 104 sets the search range for the baggage in the second image to be the longest in the transport direction of the transport conveyor 302.

This makes it possible to set a wider search range in consideration of the transport direction and the range where the cargo is most likely to be transported.

Further, the plurality of images used for detecting the baggage are acquired from among the images photographed by the image sensor 107 based on a frame rate defined according to the conveyance speed of the conveyor 302.

Thereby, an appropriate search image can be set according to the conveyance speed of the conveyor 302, and the load of search processing can be reduced.

Further, the sorting client 104 sets the entire range of a predetermined length along the direction in which the package is transported from upstream in the second image as the search range for the package.

This makes it possible to increase the possibility that the entire package will be recognized before the package leaves a certain range.

Further, the shape of the search range for the package is set based on the shape of the package detected from the immediately previous image.

As a result, the search range for the package in the next image is set based on the shape of the package that has already been detected, making it possible to perform an efficient search.

Further, the shape of the search range for the package is a rectangular shape that is set based on the shape of the package detected from the previous image.

As a result, the search range for the package in the next image is set in a rectangular shape based on the shape of the detected package, making it possible to perform an efficient search.

Image sensor 107 is a high speed camera.

This makes it possible to use a plurality of images for detecting packages at a frame rate corresponding to the conveyance speed of the conveyor 302.

<Other embodiments>
In the embodiment described above, the process in FIG. 8 may be performed in any of the plurality of client systems 110. Further, in the embodiment described above, the image sensor 107 has been described as an example of a high-speed camera, but the present invention is not limited to this. For example, if the distance sensor 106 is capable of capturing distance images at high speed, similar processing may be implemented using the distance sensor.

Further, as shown in FIG. 7, one sorting area partially overlaps an adjacent sorting area (handover area). Therefore, the process shown in FIG. 9 may be configured to be performed in a region other than the handover region on the upstream side of the sorting region. In this case, since it is assumed that a new package will be transported in the upstream handover area, the entire area may be set as the search range. Alternatively, if the position and size of the newly transported baggage can be specified based on the information included in the projection instruction, the search range in the handover area may be set based on that information.

Further, in the most upstream sorting area A, a configuration may be adopted in which a certain range from the starting position 305a is searched and the above-described processing is performed thereafter. This is because within this range, the entire package may not be recognized, and the search range cannot be accurately narrowed down. Note that, if the maximum size of the cargo to be transported is specified, the certain range may be set to be larger than or equal to the maximum size. This is because by doing so, there is a high possibility that the entire package will be recognized before the package leaves a certain range. Furthermore, when handover between sorting areas is not possible, a configuration may be adopted in which a certain range is searched from the starting position of the downstream sorting area, and thereafter the above-described processing is performed. Specifically, a case in which handover between sorting areas cannot be performed may be a case where sorting areas cannot be set to overlap each other due to the structure of the facility where the transport conveyor 302 is placed or the relationship with the arrangement of other equipment. It will be done.

Further, in the example shown in FIG. 8, the search range is shown as a rectangle, but the search range is not limited to this. For example, the search range may be set in the range indicated by a circle, or the search range may be set in the range indicated by a fan shape that spreads in the transport direction based on the position of the immediately preceding package. good. Furthermore, since the shapes and sizes of luggage vary, the shape and size of the search range may be adjusted depending on the shape and size. Note that when the transport conveyor 302 is used, the length of the search range may be adjusted to be the longest in the transport direction of the transport conveyor 302. This is because this direction is the direction in which the cargo is most likely to move compared to other directions.

Further, if the baggage cannot be found within the search range, it may be determined that the baggage has been lost, or the baggage may be additionally searched for, including outside the search range. That is, the search range may be at least a range in which the search for luggage is performed preferentially.

Furthermore, although the above embodiment has been described using a luggage sorting system as an example, the configuration of the above embodiment may be applied to a system that does not involve sorting luggage. This embodiment can be applied to a system that has at least a function of searching for luggage.

Further, programs and applications for realizing the functions of one or more embodiments described above may be supplied to a system or device using a network or a storage medium, and one or more processors in a computer of the system or device may This can also be achieved by reading and executing a program.

Further, it may be realized by a circuit (for example, an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array)) that realizes one or more functions.

Although various embodiments have been described above with reference to the drawings, it goes without saying that the present disclosure is not limited to such examples. It is clear to those skilled in the art that various changes, modifications, substitutions, additions, deletions, and equivalents may be made within the scope of the claims; It is understood that it naturally falls within the technical scope of the present disclosure. Moreover, each component in the various embodiments described above may be combined arbitrarily within a range that does not depart from the spirit of the invention.

The present disclosure is useful as a baggage sorting system, a baggage sorting method, an information processing device, and a control method thereof that can improve the convenience of actions according to rights for multiple users.

100... Baggage sorting system 101... Sorting management server 102... Cooperation server 103... Label reader 104... Sorting client 105... Projector 106... Distance sensor 107... Image sensor 110... Client system 300... Baggage 301... Label 302... Transport conveyor 303... Rotary encoder

Claims (13)

a photography department that takes multiple consecutive images of the cargo being transported;
a detection unit that detects luggage using each of the plurality of images;
has
The detection unit detects a package using a first image of the plurality of images, and then detects the package using a second image subsequent to the first image. setting a search range for the baggage in the second image based on the position of the baggage detected from the image;
Detecting the baggage from the search range in the second image;
Baggage search system. The baggage search system according to claim 1, wherein the detection unit detects the baggage by giving priority to the search range in the second image over the outside of the search range. The baggage search system according to claim 1, wherein the detection unit detects the baggage only from within the search range in the second image. The baggage is transported by a transport conveyor,
The baggage search system according to claim 1, wherein the detection unit sets a search range for the baggage in the second image based on a conveyance speed of the conveyor. The baggage is transported by a transport conveyor,
The baggage search system according to claim 1, wherein the detection unit sets the baggage search range in the second image to be the longest in the transport direction of the transport conveyor. 5. The baggage search system according to claim 4, wherein the plurality of images are acquired from among the images photographed by the photographing section based on a frame rate defined according to the conveyance speed of the conveyor. The detection unit sets the entire range as a search range for the package within a predetermined length range from upstream in the direction in which the package is transported in the second image. The baggage search system described in 1. 8. The luggage search system according to claim 1, wherein the shape of the luggage search range in the second image is set based on the shape of the luggage detected from the first image. 9. The baggage search system according to claim 8, wherein the shape of the baggage search range in the second image is a rectangular shape set based on the shape of the baggage detected from the first image. The baggage search system according to any one of claims 1 to 9, wherein the photographing unit is a high-speed camera. a photographing step in which a photographing unit photographs a plurality of consecutive images of the cargo being transported;
a detection step of detecting baggage using each of the plurality of images;
has
In the detection step, after detecting a package using a first image of the plurality of images, when detecting a package using a second image subsequent to the first image, the first setting a search range for the baggage in the second image based on the position of the baggage detected from the image;
The baggage is detected from the search range in the second image,
How to search for luggage. a processor;
Equipped with memory and
The processor cooperates with the memory to:
Obtain multiple consecutive images of the cargo being transported,
Detecting the package using each of the plurality of images,
In the detection, after detecting a package using a first image of the plurality of images, when detecting a package using a second image subsequent to the first image, the first A search range for the baggage in the second image is set based on the position of the baggage detected from the image,
The baggage is detected from the search range in the second image,
Information processing device. Processor works with memory to
Obtain multiple consecutive images of the cargo being transported,
Detecting the package using each of the plurality of images,
In the detection, after detecting a package using a first image of the plurality of images, when detecting a package using a second image subsequent to the first image, the first A search range for the baggage in the second image is set based on the position of the baggage detected from the image,
The baggage is detected from the search range in the second image,
A method of controlling an information processing device.

Images