WO2019003688A1 - 投影指示装置、荷物仕分けシステムおよび投影指示方法 - Google Patents
投影指示装置、荷物仕分けシステムおよび投影指示方法 Download PDFInfo
- Publication number
- WO2019003688A1 WO2019003688A1 PCT/JP2018/018945 JP2018018945W WO2019003688A1 WO 2019003688 A1 WO2019003688 A1 WO 2019003688A1 JP 2018018945 W JP2018018945 W JP 2018018945W WO 2019003688 A1 WO2019003688 A1 WO 2019003688A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- package
- image
- projection
- color image
- processor
- Prior art date
Links
- 238000000034 method Methods 0.000 claims description 30
- 230000008569 process Effects 0.000 description 9
- 238000012545 processing Methods 0.000 description 8
- 238000003384 imaging method Methods 0.000 description 7
- 230000006866 deterioration Effects 0.000 description 4
- 239000011521 glass Substances 0.000 description 4
- 238000012546 transfer Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000011179 visual inspection Methods 0.000 description 2
- 238000007792 addition Methods 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/292—Multi-camera tracking
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
- B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
- B07C7/00—Sorting by hand only e.g. of mail
- B07C7/005—Computer assisted manual sorting, e.g. for mail
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
- B65G—TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS
- B65G1/00—Storing articles, individually or in orderly arrangement, in warehouses or magazines
- B65G1/02—Storage devices
- B65G1/04—Storage devices mechanical
- B65G1/137—Storage devices mechanical with arrangements or automatic control means for selecting which articles are to be removed
-
- G—PHYSICS
- G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
- G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
- G03B21/00—Projectors or projection-type viewers; Accessories therefor
- G03B21/14—Details
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/90—Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/74—Projection arrangements for image reproduction, e.g. using eidophor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/08—Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Definitions
- the present disclosure relates to a projection instruction device, a baggage sorting system, and a projection instruction method useful for sorting luggage.
- the sorting operation for sorting the packages according to the destination is a time-consuming process, and has relied on manual operations from before, but a technique for automating at least a part of the sorting operation has also been proposed.
- Patent Document 1 tracks moving packages, determines an image to be displayed based on information on a package read from the package and information on the location of the package, projects an image on the package from the projector, and places the image on the package Disclosed is a system for displaying.
- the present disclosure relates to techniques for more efficiently and accurately sorting packages.
- the present disclosure is a projection instruction device that generates a projection image to be projected onto a package based on sensing information of a package, comprising: a processor and a memory, wherein the processor cooperates with the memory to perform sensing
- the package is identified based on the distance image of the package included in the information, the package is traced based on the color image of the package included in the sensing information, and the color image of the package on which the projection image is projected is the original floor.
- the package is tracked based on a distance image of the package.
- the present disclosure projects the projection instruction device described above, a label reader that reads package identification information from a label attached to a package, an image sensor that acquires the distance image and the color image, and the projection image on the package And an image projection device.
- the present disclosure is a projection instruction method for generating a projection image to be projected onto a package based on sensing information on the package, and the processor cooperates with the memory to determine the package based on a distance image of the package included in the sensing information. And the package is tracked based on the color image of the package included in the sensing information, and the color image of the package on which the projection image is projected includes a white area represented by white that is not the original gradation. In the case, the package is tracked based on the distance image of the package.
- sorting of packages can be performed more efficiently and accurately, and further increase in the circulation volume of packages can be coped with.
- even when overexposure occurs due to the projection of the projection image it is possible to prevent deterioration in the tracking accuracy of the package.
- FIG. 1 is a block diagram showing the configuration of a package sorting system according to the embodiment.
- FIG. 2 is a block diagram showing the configuration of the projection instruction apparatus according to the embodiment.
- FIG. 3A is a conceptual view showing a state in which the package sorting system is installed in the distribution center and in operation.
- FIG. 3B is a view showing a projected image including numbers on a top surface of the luggage.
- FIG. 4 is a view showing an example of a projection image generated by the projection instruction apparatus according to the embodiment.
- FIG. 5 is a flowchart showing an outline procedure of an operation mainly performed by the projection instruction apparatus.
- FIG. 6 is a flowchart showing a procedure of switching images used for tracking a package.
- the present embodiment which specifically discloses a projection instruction device, a package sorting system, and a projection instruction method according to the present disclosure will be described in detail with reference to the drawings as appropriate.
- the detailed description may be omitted if necessary.
- detailed description of already well-known matters and redundant description of substantially the same configuration may be omitted. This is to avoid unnecessary redundancy in the following description and to facilitate understanding by those skilled in the art.
- the attached drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and they are not intended to limit the claimed subject matter.
- FIG. 1 is a block diagram showing the configuration of a package sorting system according to the embodiment.
- the package sorting system 100 shown in FIG. 1 is installed, for example, at a distribution center.
- the package sorting system 100 includes a label reader 10, an image sensor 20, a projection instruction device 30, and a projector 40.
- the package sorting system 100 is a system that supports the work of a worker who sorts the packages transported by the transport conveyor.
- the package sorting system 100 is installed at a distribution center owned by, for example, a retailer, a wholesaler, an internet distributor, and the like.
- the luggage to be sorted generally has a substantially rectangular parallelepiped shape, its outer shape is not particularly limited, and the type of luggage is also not particularly limited.
- the configuration of the package sorting system is not limited to that described in FIG.
- the quantity of each component can be appropriately changed according to the purpose, such as connecting a plurality of image sensors 20, the projection instruction device 30, and the projector 40 to one label reader 10.
- the label reader 10 as a reading device is a device including various elements such as a lens, an image sensor and the like (not shown). By using the label reader 10, it is possible to read label recording information in which various information related to the package is recorded from a label attached to the package transported by the transport conveyor. By using the read label recording information, it is possible to specify the package. Package specific information is defined by the read information.
- the image sensor 20 is an imaging device including various elements such as a lens, an image sensor, and the like (not shown).
- the image sensor 20 is generally configured as an imaging camera.
- the imaging camera is a three-dimensional camera, a plurality of two-dimensional cameras, or the like.
- the image sensor 20 includes a distance image sensor 22 and a color image sensor 24.
- the distance image sensor 22 picks up an item transported by the transfer conveyor and generates a distance image.
- the generated distance image is used as information indicating the position of the package, the distance to the package, the size of the package, and the like.
- the “distance image” refers to an image containing distance information indicating the distance from the imaging position to the position (including the surface of the luggage) indicated by each pixel (that is, in the present disclosure, the word “image” is a distance image). including). Further, the term “distance image” includes those that can not be recognized as an image by human eyes, such as a table listing numerical values indicating distances.
- the “distance image” may be any information as long as it indicates the relationship between the coordinates in the imaged area and the distance, and the data structure does not matter.
- the distance image sensor 22 is used to identify the position of the package. Therefore, the distance image sensor 22 may be replaced by another sensing device (ultrasonic sensor, infrared sensor, stereo camera, monocular video camera).
- the color image sensor 24 captures a package for which a distance image has been generated, and generates a color image.
- Color image refers to an image in which the color of the surface of the package is expressed in a predetermined gradation, and "gradation” includes not only 256 gradations of RGB but also all kinds such as gray scale.
- the color image sensor 24 is used to track each package for the package specified by the distance image sensor 22.
- the color image sensor 24 can also be replaced with another sensing device (ultrasonic sensor, infrared sensor, stereo camera, single-eye video camera).
- the term "image" includes both a distance image and a color image.
- information output from an image sensor as a sensing device including a distance image sensor and a color image sensor is referred to as sensing information.
- the present embodiment will be described using an image sensor 20 (including a distance image sensor 22 and a color image sensor 24) as an example of a sensing device. Further, in the present embodiment, as an example of sensing information, the description will be made using a distance image which is an output of the distance image sensor 22 and a color image which is an output of the color image sensor 24.
- the projection instruction device 30 plays a role as an arithmetic device in the luggage sorting system 100.
- the projection instruction device 30 includes an input unit 32, a processor 34, a memory 36, and an output unit 38 connected via a bus.
- the input unit 32 receives package specification information that can specify the package acquired from the label recording information read by the label reader 10, the distance image generated by the distance image sensor 22, and the color image generated by the color image sensor 24.
- the processor 34 is configured by a general computing device, and generates a projection image to be projected onto a package based on the package specific information, the distance image and the color image.
- the memory 36 as a storage device performs operations such as reading of control programs necessary for various processes by the processor 34 and saving of data.
- processor 34 and the memory 36 cooperate to control various processes by the projection instruction device 30.
- the output unit 38 outputs the projection image generated by the processor 34 to the projector 40.
- processor does not mean only a single processor.
- processor is a term that means an operation subject in the case where a plurality of processors with the same purpose and processors with different purposes (for example, a general-purpose CPU (Central Processing Unit) and a GPU (Graphic Processing Unit)) perform processing jointly. Also used as.
- the projector 40 is constituted by a general projection device, projects the projection light including the projection image received from the projection instructing device 30 onto a package, and displays the projection light on the package.
- the package sorting system 100 can be constructed by connecting the label reader 10, the image sensor 20 (the distance image sensor 22 and the color image sensor 24), the projection instruction device 30, and the projector 40 by wired communication or wireless communication. Further, two or more devices among the label reader 10, the image sensor 20, the projection instruction device 30, and the projector 40 can be constructed as an integrated device. For example, the image sensor 20 and the projector 40 can be combined to construct an integrated imaging and projection apparatus (see FIG. 3A).
- FIG. 3A is a conceptual view showing a state in which the package sorting system 100 is installed in the distribution center and in operation.
- each worker M visually checks the label affixed to each package P transported by the transport conveyor 50 in the direction of arrow X, and the package for which the worker himself is in charge of delivery arrives Then, it was necessary to pick up the package (pickup) and place it in the vicinity of the worker's own foot, a car, a truck bed and the like.
- sorting by visual inspection of the workers since there is a limit to the work efficiency of the workers, it was necessary to suppress the transport speed to a predetermined value or less.
- positioned above the conveyance conveyor 50 reads the label stuck on each package P conveyed by the conveyance conveyor 50.
- the label describes label recording information including various information on the package.
- the label record information includes information similar to the package identification number individually assigned to the package, the sender's name, address, telephone number, recipient's name, address, telephone number, package type, and the like. The reading of the label may be performed by the worker in charge corresponding to the barcode in the label with the barcode reader as the label reader 10 by hand.
- the image sensor 20 picks up an image (distance image and color image) of the package P transported by the transport conveyor 50, and the position of the package P, the distance to the package P, the size of the package P (the package P is a rectangular parallelepiped) In the case of three side lengths), the color of the package P, the pattern of the package P, etc. are acquired.
- positioning position of the label reader 10 and the image sensor 20, the kind of sensing device, and the order of a process in particular are not limited to the form shown in figure.
- the image sensor 20 and the projector 40 are constructed as an integrated imaging and projecting device 60 and disposed above the transport conveyor 50.
- the projection instruction device 30 which is not shown in FIGS. 3A and 3B, is constituted by, for example, a computer disposed in the vicinity of the transport conveyor 50 or in another room, and information for identifying the package acquired by the label reader 10 and the image Based on the distance image and the color image generated by the sensor 20, a projection image to be displayed on the package P (for example, the upper surface when the package P is a rectangular parallelepiped) is generated.
- the projection instructing device 30 sends to the projector 40 a projection instruction to project the projection image onto the package P.
- the projector 40 as an image projection apparatus projects the projection light including the projection image generated by the projection instructing device 30 onto the package P, and displays the projection image on the package P.
- the projection image displayed on the package P is, for example, a circled number image having a color indicating the sorting place corresponding to the delivery address of the package P (see FIG. 3B).
- the circled numbers correspond to, for example, the number of the truck carrying the sorted luggage P (the number of the truck itself, the parking lot number, etc.), the number of the shelf or box to be carried into the truck, etc. ing.
- a zip code corresponding to the delivery address, or a number of a worker who should pick up the package P may be used.
- arrows indicating the sorting direction such as right or left with respect to the transport direction of the transport conveyor 50
- characters such as "left” or "right”
- the form of the display is not limited to the circled numbers, and various numbers may be considered such as the numbers enclosed by squares ("3", “359", "24735") as shown in FIG.
- the projected image is not limited to one enclosing a number or letter by a frame, and may be an open number or letter with a background.
- the shapes of numbers or characters to be displayed may be switched.
- the projection image is not limited to a still image, and may be animation.
- the animation it is conceivable to blink, scale, or change the color of each example described above.
- an animation reflecting the sorting direction may be projected.
- Examples of animation reflecting the sorting direction include, for example, moving a light beam or a light spot in the sorting direction, forming all or part of a projected image in the sorting direction, or changing the color, Various things can be considered, such as moving and displaying an arrow in the sorting direction.
- moving and displaying an arrow in the sorting direction When only part of the projected image is targeted for animation, the part that has a large influence on the judgment of the sorting destination by the operator, such as numbers and arrows, is not changed, and the part that has little influence on the sorting destination such as a frame It is conceivable to change it. However, in situations where it is more efficient to intuitively convey the sorting direction than the meaning of numbers etc.
- the processor 34 moves the package between the label reader 10 and the distance image sensor 22 from the distance (known) between the label reader 10 and the distance image sensor 22 and the speed of the transfer conveyor 50. Calculate the time required. Then, by subtracting the time from the time when the distance image is acquired, it is possible to estimate the time when the package present in the distance image is given an ID by the label reader 10 (and the processor 34). Then, it can be estimated that the ID given close to the estimated time is the ID corresponding to the package present in the distance image.
- the processor 34 compares the time information recorded in advance in the memory 36 with the estimated time, and if these values are close (for example, when the time difference is less than or equal to a predetermined threshold time), It can be determined that the corresponding ID exists in the memory 36. If it is determined that the ID corresponding to the package exists in the memory 36 (step S30; Yes), the process proceeds to the process of step S60 and subsequent steps.
- the color image sensor 24 In parallel with the above-described steps, the color image sensor 24 generates a color image of each package for which the distance image has been acquired. Then, based on the color image from the color image sensor 24 acquired by the input unit 32, the processor 34 tracks the luggage with an ID which is transported and moved by the transport conveyor 50 (step S60). Then, the processor 34 determines, based on the color image, whether or not the package being tracked is picked up by the operator (step S70). If it is determined that the package is not picked up by the worker (Step S70; No), the processor 34 determines whether the package exists in a specific area (a predetermined sorting area where the package is to be picked up) described later. judge.
- a specific area a predetermined sorting area where the package is to be picked up
- step S80 when it is determined that the package is present (reached) in the specific area (step S80; Yes), the processor 34 generates a projection image and transmits it to the projector 40 (step S90). If not determined (step S80; No), the process returns to step S60 and the processor 34 continues tracking the package.
- step S70 is the determination in step S70.
- contact determination with a worker's hand and a package, and a color image, a distance image, etc. for implementation of determination of step S70.
- the processor 34 basically identifies the package and gives an ID based on the distance image of the package included in the sensing information, and then proceeds to step S60.
- the package is tracked based on the color image of the package also included in the sensing information.
- the processor 34 ends tracking by the color image and switches to tracking by the distance image. This is to prevent deterioration of the tracking accuracy of the package.
- the processor 34 may perform tracking by extracting the feature amount of the package to be tracked from the color image generated by the color image sensor 24.
- the feature amount various things can be considered such as the distribution of color and luminance, and the pattern of a specific part in a color image.
- various methods of extraction and evaluation thereof including those exemplified above, various methods are known in the field of image recognition, and thus are omitted in the present disclosure.
- the feature amount when used, even if a part of the color image, such as a portion corresponding to the end of the package, is white-out, it may be possible to identify the package. Therefore, when the occurrence of whiteout is detected, the ratio of the area where whiteout is generated exceeds a predetermined value or the area is a region where whiteout is generated instead of switching to tracking by the distance image.
- the tracking by the color image may be continued until it can not be recognized. Since tracking by color image tends to be more accurate than tracking by distance image, tracking by color image is more reliable than tracking by distance image even if it is partially overexposed There is a possibility that the result can be obtained.
- the tracking by the color image is stopped and the package is tracked based on only the distance image, but the present invention is not limited to this.
- tracking by the color image may also be able to be continued. Therefore, when the occurrence of overexposure is detected, the package tracking may be continued based on both the color image tracking and the distance image tracking.
- the degree of reliability of each tracking result may be different. As the degree of overexposure increases, it is conceivable to lower the confidence result of the result by the color image and to increase the reliability of the result by the distance image.
- the weight on the white region may be increased as the position closer to the center of the package having the higher determination reliability is obtained.
- the ratio of the white region described above includes the ratio evaluated after such weighting.
- the processor 34 has a function of obtaining color image information generated by the color image sensor 24 and a function of generating a projection image to be projected onto a package. Therefore, by artificially combining the color image information and the projection image in the processor 34, it is possible to predict whether overexposure will occur when the image is actually projected onto a package. In this case, the tracking method is switched more smoothly than the above-described embodiment in which processing is performed after the occurrence of the whiteout by predicting the occurrence of the whiteout and switching to the distance image used for tracking the package in advance. be able to.
- the processor 34 since the occurrence of overexposure is also affected by factors that the processor 34 can not control, such as a change in ambient light, it is used for tracking after detecting the occurrence of overexposure as in the above embodiment.
- the configuration for switching the image to be transmitted is also effective from the viewpoint of accuracy. Therefore, an image used for tracking the package in consideration of both the result of the prediction of the occurrence of the overexposure due to the pseudo composition and the detection of the occurrence of the overexposure based on the color image actually generated by the color image sensor 24 May be switched. For example, when it is predicted that whiteout will occur from the pseudo synthesis result, it may be considered to switch to tracking using a distance image even if whiteout has not yet occurred in an actual color image.
- the color of the package ID in which the whiteout is predicted to occur and the area of the distance image corresponding to the color image of the package are associated in advance and managed. It is possible to suppress tracking errors when switching from tracking using an image to tracking using a distance image.
- the present disclosure is useful for providing a projection instruction device, a package sorting system, and a projection instruction method that can prevent deterioration in package tracking accuracy even when overexposure occurs.
Abstract
Description
図1は、実施の形態にかかる荷物仕分けシステムの構成を示すブロック図である。図1に示す荷物仕分けシステム100は、例えば物流センターに設置される。荷物仕分けシステム100は、ラベルリーダ10と、画像センサ20と、投影指示装置30と、プロジェクタ40とを備える。荷物仕分けシステム100は、搬送コンベヤにより搬送されてくる荷物を仕分ける作業員の作業を支援するシステムである。荷物仕分けシステム100は、例えば小売業者、卸売業者、インターネット流通業者などが所有する物流センターに設置される。仕分け対象の荷物は、一般的には略直方体の形状を有するものであるが、その外形は特に限定はされず、荷物の種類も特に限定されない。なお、荷物仕分けシステムの構成は図1に記載したものに限定されない。例えば、一台のラベルリーダ10に複数の画像センサ20と、投影指示装置30と、プロジェクタ40を接続させるなど、各構成要素の数量は目的に応じて適宜変更させることができる。
図3Aは、荷物仕分けシステム100が物流センターに設置され、稼働している状況を示す概念図である。従来の方法によれば、各作業者Mが、搬送コンベヤ50により矢印X方向に搬送されてくる各荷物Pに貼付されたラベルを目視によりチェックし、作業者自身が配送を担当する荷物が到着したら当該荷物を拾い上げて(ピックアップ)、作業者自身の足元、かご、トラックの荷台等の近傍に一旦置く必要があった。しかしながら、作業者の目視による仕分けの場合、作業者の作業効率には限界があるから、搬送速度を所定の値以下に抑える必要があった。結果として、作業者が単位時間内に仕分けることが可能な荷物の量の限界値は低かった。また、作業者がラベルの目視に際して誤認を行うことにより、仕分けに際して誤りを引き起こす可能性があった。近年、荷物の流通量は増大しており、これらの問題はより注目されつつある。
図5は、本実施形態の投影指示装置30、特に投影指示装置30のプロセッサ34が主として行う動作手順の概要を示すフローチャートである。まず、ラベルリーダ10による荷物のラベルのラベル記録情報の読み取り後、投影指示装置30の入力部32がラベルリーダ10から、ラベル記録情報に対応した荷物特定情報を取得する(ステップS1)。荷物特定情報とは、当該荷物に個別に付与される荷物特定番号、発送人の氏名、住所、電話番号、受取人の氏名、住所、電話番号、荷物の種類、等に類似する情報の少なくとも一つ含む情報である。プロセッサ34は、当該荷物特定情報に対して、当該荷物を特定する荷物特定番号としてのIDを付与し、IDを付与した時刻に対応する時刻情報とともにメモリ36に記録する(ステップS2)。メモリ36に記録するIDは、荷物特定情報に元から記録されていた荷物特定番号でもよいし、投影指示装置30が新しいIDを生成して付与してもよい。
従来より、デジタルカメラの分野では、いわゆる白飛びの問題が知られている。撮影対象内における明るい撮影領域では、デジタルカメラの階調が再現できる範囲を超えると、生成画像の階調が失われ、生成画像が白色になってしまうことがある。この問題が、投影指示装置30、荷物仕分けシステム100にても生じ得る。例えば図5の例では、ステップS90で述べたように、投影指示装置30から投影画像を取得したプロジェクタ40が、当該投影画像を対応する荷物に投影する。この際、投影光により荷物が明るく照らされると、荷物の投影された領域の明るさが色画像センサ24により捉えられる明るさを超えてしまうことが生じ得る。そして、色画像に上述した白色領域が発生し(白飛びの発生)、本来の荷物の色が表現できなくなり、荷物の特定が困難となり、さらには荷物の追従ができなくなる可能性が生じ得る。
20 画像センサ
22 距離画像センサ
24 色画像センサ
30 投影指示装置
32 入力部
34 プロセッサ
36 メモリ
38 出力部
40 プロジェクタ
50 搬送コンベヤ
60 撮像投影装置
100 荷物仕分けシステム
P 荷物
Claims (9)
- 荷物のセンシング情報に基づき、当該荷物に投影する投影画像を生成する投影指示装置であって、
プロセッサと、
メモリと、を備え、
前記プロセッサは、前記メモリと協働して、
センシング情報に含まれる荷物の距離画像に基づき当該荷物を特定し、
前記センシング情報に含まれる前記荷物の色画像に基づき当該荷物を追跡し、
投影画像が投影された荷物の色画像が、本来の階調ではない白色で表現された白色領域を含む場合、当該荷物の距離画像に基づき、前記荷物を追跡する、
投影指示装置。 - 前記プロセッサは、前記荷物の色画像に前記白色領域が含まれていても、前記荷物を識別することが可能であれば、前記色画像に基づく荷物の追跡を継続する、
請求項1に記載の投影指示装置。 - 前記プロセッサは、前記荷物の色画像に前記白色領域が含まれていても前記色画像に基づく荷物の追跡を継続する場合、前記色画像に基づく当該荷物の追跡結果の信頼度を、前記白色領域が含まれていない場合の当該荷物の追跡結果の信頼度より下げて、前記荷物の追跡を行う、
請求項2に記載の投影指示装置。 - 前記プロセッサは、前記荷物の色画像に前記白色領域が含まれていても前記色画像に基づく荷物の追跡を継続する場合、前記荷物の色画像に含まれる前記白色領域の比率に応じて、前記色画像に基づく追跡結果と前記距離画像に基づく追跡結果のどちらにしたがって前記荷物を追跡するかを決定する、
請求項2に記載の投影指示装置。 - 前記プロセッサは、前記荷物の色画像と前記投影画像とを合成することにより、前記投影画像が投影された荷物の色画像が白色領域を含むか否かを、前記投影画像を当該荷物に投影する前に判断する、
請求項1に記載の投影指示装置。 - 前記プロセッサは、前記投影画像を前記荷物に投影する前に前記荷物の色画像が白色領域を含むと判断した場合、前記荷物の色画像に対応する前記距離画像の領域と前記荷物を識別する情報とを対応付けて管理する、
請求項5に記載の投影指示装置。 - 前記プロセッサは、前記投影画像が投影された荷物の前記色画像が、前記白色領域を含む場合、当該荷物の距離画像のみに基づき、前記荷物を追跡する、
請求項1に記載の投影指示装置。 - 請求項1から7のいずれか1項に記載の投影指示装置と、荷物に貼付されたラベルから荷物特定情報を読み取るラベルリーダと、前記距離画像および前記色画像を取得する画像センサと、前記投影画像を前記荷物に投影する画像投影装置と、を備える荷物仕分けシステム。
- 荷物のセンシング情報に基づき、当該荷物に投影する投影画像を生成する投影指示方法であって、
プロセッサがメモリと協働して、
センシング情報に含まれる荷物の距離画像に基づき当該荷物を特定し、
前記センシング情報に含まれる前記荷物の色画像に基づき当該荷物を追跡し、
投影画像が投影された荷物の色画像が、本来の階調ではない白色で表現された白色領域を含む場合、当該荷物の距離画像に基づき、前記荷物を追跡する、
投影指示方法。
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2019526671A JP7129617B2 (ja) | 2017-06-30 | 2018-05-16 | 投影指示装置、荷物仕分けシステムおよび投影指示方法 |
EP18823103.9A EP3647234B1 (en) | 2017-06-30 | 2018-05-16 | Projection indication device, parcel sorting system, and projection indication method |
US16/624,678 US11250578B2 (en) | 2017-06-30 | 2018-05-16 | Projection indication device, parcel sorting system, and projection indication method |
CN201880042438.2A CN110785362B (zh) | 2017-06-30 | 2018-05-16 | 投影指示装置、货物分拣系统以及投影指示方法 |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2017129576 | 2017-06-30 | ||
JP2017-129576 | 2017-06-30 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2019003688A1 true WO2019003688A1 (ja) | 2019-01-03 |
Family
ID=64742995
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2018/018945 WO2019003688A1 (ja) | 2017-06-30 | 2018-05-16 | 投影指示装置、荷物仕分けシステムおよび投影指示方法 |
Country Status (5)
Country | Link |
---|---|
US (1) | US11250578B2 (ja) |
EP (1) | EP3647234B1 (ja) |
JP (1) | JP7129617B2 (ja) |
CN (1) | CN110785362B (ja) |
WO (1) | WO2019003688A1 (ja) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2020175085A1 (ja) * | 2019-02-26 | 2020-09-03 | コニカミノルタ株式会社 | 画像処理装置、及び画像処理方法 |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110781824B (zh) * | 2019-10-25 | 2023-03-14 | 阿波罗智联(北京)科技有限公司 | 目标检测及跟踪方法、装置、电子设备及存储介质 |
CN115112508B (zh) * | 2022-08-29 | 2023-01-24 | 民航成都物流技术有限公司 | 民航机场托运行李软硬包识别装置和方法 |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7090134B2 (en) | 2003-03-04 | 2006-08-15 | United Parcel Service Of America, Inc. | System for projecting a handling instruction onto a moving item or parcel |
JP2006349924A (ja) * | 2005-06-15 | 2006-12-28 | Seiko Epson Corp | 画像表示装置及び画像表示方法 |
WO2015145982A1 (ja) * | 2014-03-28 | 2015-10-01 | 日本電気株式会社 | 情報処理装置、情報処理システム、物流システム、情報処理方法およびプログラム記憶媒体 |
Family Cites Families (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5923017A (en) * | 1997-01-23 | 1999-07-13 | United Parcel Service Of America | Moving-light indicia reader system |
US5920056A (en) * | 1997-01-23 | 1999-07-06 | United Parcel Service Of America, Inc. | Optically-guided indicia reader system for assisting in positioning a parcel on a conveyor |
US7063256B2 (en) * | 2003-03-04 | 2006-06-20 | United Parcel Service Of America | Item tracking and processing systems and methods |
US8825200B2 (en) * | 2007-11-07 | 2014-09-02 | Siemens Industry, Inc. | Method and system for tracking of items |
US9192965B2 (en) * | 2013-02-07 | 2015-11-24 | Logical Turn Services Inc. | Package vision sort system and method |
JP6411204B2 (ja) * | 2014-12-22 | 2018-10-24 | トーヨーカネツソリューションズ株式会社 | プロジェクタ利用ピッキング装置 |
US20180044112A1 (en) * | 2015-03-06 | 2018-02-15 | Hitachi Transport System, Ltd. | Task aid system, terminal device, and server |
US10148918B1 (en) * | 2015-04-06 | 2018-12-04 | Position Imaging, Inc. | Modular shelving systems for package tracking |
CN106358027A (zh) * | 2016-08-25 | 2017-01-25 | 深圳吉祥星科技股份有限公司 | 一种投影装置、投影方法及投影系统 |
US11481567B2 (en) * | 2018-12-21 | 2022-10-25 | Hand Held Products, Inc. | Methods and systems of package processing in a material handling environment |
-
2018
- 2018-05-16 WO PCT/JP2018/018945 patent/WO2019003688A1/ja active Application Filing
- 2018-05-16 US US16/624,678 patent/US11250578B2/en active Active
- 2018-05-16 JP JP2019526671A patent/JP7129617B2/ja active Active
- 2018-05-16 EP EP18823103.9A patent/EP3647234B1/en active Active
- 2018-05-16 CN CN201880042438.2A patent/CN110785362B/zh active Active
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7090134B2 (en) | 2003-03-04 | 2006-08-15 | United Parcel Service Of America, Inc. | System for projecting a handling instruction onto a moving item or parcel |
JP2006349924A (ja) * | 2005-06-15 | 2006-12-28 | Seiko Epson Corp | 画像表示装置及び画像表示方法 |
WO2015145982A1 (ja) * | 2014-03-28 | 2015-10-01 | 日本電気株式会社 | 情報処理装置、情報処理システム、物流システム、情報処理方法およびプログラム記憶媒体 |
Non-Patent Citations (1)
Title |
---|
See also references of EP3647234A4 |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2020175085A1 (ja) * | 2019-02-26 | 2020-09-03 | コニカミノルタ株式会社 | 画像処理装置、及び画像処理方法 |
JP7375806B2 (ja) | 2019-02-26 | 2023-11-08 | コニカミノルタ株式会社 | 画像処理装置、及び画像処理方法 |
Also Published As
Publication number | Publication date |
---|---|
US11250578B2 (en) | 2022-02-15 |
EP3647234B1 (en) | 2023-02-22 |
US20200111221A1 (en) | 2020-04-09 |
CN110785362B (zh) | 2021-08-17 |
JPWO2019003688A1 (ja) | 2020-04-23 |
EP3647234A1 (en) | 2020-05-06 |
EP3647234A4 (en) | 2020-10-14 |
JP7129617B2 (ja) | 2022-09-02 |
CN110785362A (zh) | 2020-02-11 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20200282429A1 (en) | Package sorting system, projected instruction device, and package sorting method | |
US20190084009A1 (en) | Projection instruction device, parcel sorting system, and projection instruction method | |
US20190084008A1 (en) | Instruction projecting device, package sorting system and instruction projecting method | |
WO2019003688A1 (ja) | 投影指示装置、荷物仕分けシステムおよび投影指示方法 | |
US11120286B2 (en) | Projection indication device, parcel sorting system, and projection indication method | |
US20200290093A1 (en) | Projection instruction device, package sorting system, and projection instruction method | |
JP7129616B2 (ja) | 投影指示装置、荷物仕分けシステムおよび投影指示方法 | |
JP7129614B2 (ja) | 荷物認識装置、荷物仕分けシステムおよび荷物認識方法 | |
JP7194884B2 (ja) | 荷物仕分けシステム、投影指示装置、投影指示方法 | |
JP7045621B2 (ja) | 荷物認識装置、荷物仕分けシステムおよび荷物認識方法 | |
US11145066B2 (en) | Parcel recognition device, parcel sorting system, and parcel recognition method | |
JP2017171446A (ja) | 投影指示装置、荷物仕分けシステムおよび投影指示方法 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 18823103 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2019526671 Country of ref document: JP Kind code of ref document: A |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2018823103 Country of ref document: EP |
|
ENP | Entry into the national phase |
Ref document number: 2018823103 Country of ref document: EP Effective date: 20200130 |