JP2017048052A - Support system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a support system that can be attached to an existing picking system.SOLUTION: A support system 10 is attached to a picking system 1 that sequentially indicates a marker to a region 21 of an article to be taken out next of a plurality of regions 21 assigned to each article. The support system 10 includes a monitoring device 12, and the monitoring device 12 specifies a frontage of the region where the marker is indicated in image data by detecting the marker in the image data on the plurality of regions 21, tracks a position of hands of a picking operator, and determines whether or not the specified frontage corresponds to the tracked position of the hands.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a support system that supports picking work.

  Picking products, products, etc. is performed in various fields such as distribution and production. Picking means picking up ordered items based on information about ordered items in the warehouse, distribution center, and distribution center in the distribution field, and collecting and distributing necessary items from materials and parts in the factory in the production field. It is to be. Patent Documents 1 to 3 are known as systems that support picking work.

  In Patent Document 1, picking mistakes are prevented as much as possible by alternating the following two types of confirmation using a tag. One is to check the items taken out from the pallets installed in the incoming pallet storage, and the other is to check the items stored in the pallets installed in the shipping pallet storage.

  In Patent Document 2, in a picking cabinet in which a plurality of trays are arranged in the vertical direction on the main body frame, an indicator having an indicator lamp is provided on the front surface of each tray, so that even when the tray is pulled out, the display is performed. The information displayed on the means can be confirmed from any position.

  In Patent Document 3, a data input device that inputs part information in a pallet that is newly supplied to a parts stock area, and an efficiency management computer that manages and stores stock data in the parts stock area in an integrated manner, A load instructing device that calculates the optimum storage position of a pallet that enables a typical picking operation and instructs the worker, and the purpose of calculating the storage position of the target pallet that can perform the picking operation most efficiently and instructing the operator By connecting the pallet indicating devices to each other, the operator can perform quick and efficient picking.

JP 2007-169048 A JP 2001-340151 A JP-A-5-278816

  However, in the techniques relating to picking disclosed in Patent Documents 1 to 3, it is necessary to replace an already installed picking cabinet or to modify the picking system.

  Accordingly, an object of the present invention is to provide a support system that can be retrofitted to an existing picking system.

The concept of the present invention is as follows.
[1] Attached to a picking system that sequentially displays a sign on an area in which an article to be taken out next is stored among a plurality of areas assigned to each article.
By detecting the sign in the image data related to the plurality of areas, the frontage of the area where the sign is displayed in the image data is specified, the position of the hand of the operator who picks is tracked, and the specified frontage And a monitoring device that determines whether or not the position of the tracked hand corresponds.
[2] The support according to [1], wherein the monitoring device outputs an abnormal signal to the outside when detecting that the position of the hand is displaced in the depth direction other than the frontage where the sign is displayed. system.
[3] The support system according to [1] or [2], wherein the monitoring device detects the marker based on a color of the marker in image data.
[4] The support system according to any one of [1] to [3], wherein the monitoring device tracks a position of a worker's hand based on a color of a glove worn by the worker.
[5] The monitoring device comprises:
A video camera,
A position measuring camera;
An area in which the sign is displayed based on the output data from the position measurement camera, while detecting the sign and tracking the position of the operator's hand picking based on the output data from the video camera A processing unit for identifying a frontage and measuring a position of a hand of a worker who picks and tracking a hand obtained from output data from the video camera;
The support system according to any one of [1] to [4].

  According to the present invention, the monitoring device detects signs in the image data relating to a plurality of areas, thereby identifying the frontage of the area where the signs are displayed in the image data and tracking the position of the operator's hand picking. Then, it can be determined whether or not the specified frontage corresponds to the position of the tracked hand. Therefore, it can be retrofitted to an existing picking system.

1 is an overall configuration diagram of a picking system incorporating a support system according to an embodiment of the present invention. It is a top view which shows the positional relationship of a camera sensor and a cabinet. It is a side view which shows the positional relationship of a camera sensor and a cabinet. It is a block block diagram of a support system. It is a flowchart which shows the process sequence of detection and tracking of the position of a picking worker's hand.

  Hereinafter, the present invention will be described in detail based on embodiments. However, the support system according to the embodiments of the present invention is not limited to the one described with reference to the drawings, and technical progress related to ICT (Information Communication Technology). And those that are changed as appropriate according to the applicable society and system.

(Configuration of the entire system incorporating the support system)
FIG. 1 is an overall configuration diagram of a picking system 1 incorporating a support system 10 according to an embodiment of the present invention. The picking system 1 includes a support system 10 according to an embodiment of the present invention, a cabinet 20, and a picking management system 30.

  The cabinet 20 is configured such that a plurality of areas 21 assigned to each article are divided, and a sign is displayed in each of the plurality of areas 21. In the form shown in FIG. 1, the display unit 22 is attached to each area 21, and the cabinet 20 is connected to the picking management system 30. In the picking management system 30, at least the following two are management targets, and the picking work is managed. One of the management targets is a frontage in each area 21 assigned in article units. Another management target is the order of taking out articles in the region 21.

  The picking work is managed by the picking management system 30 displaying the signs in order for the area where the article to be taken out next is present. Specifically, at least one display unit 22 is sequentially turned on by a signal from the picking management system 30. The display unit 22 includes a switch for turning off the light, and when the picking operator takes out the article, the display unit 22 may be turned off by pressing the switch.

  Here, “a sign is displayed” refers to urging the picking operator to pick up an article from the area 21 where the sign is displayed. The display form of the sign is arbitrarily set depending on the site of picking work, the type of article, and the number of articles. For example, the display unit 22 may be turned on sequentially. You may make it display the number of articles | goods which should be taken out by displaying a character, a figure, a symbol, etc. on the display part 22. FIG. In the picking operation for always taking out one article, the picking order may be indicated by a number.

  The picking management system 30 confirms that one picking is completed by receiving a signal from the cabinet 20, for example, every time when the operator takes out the article from the area 21 according to the sign. If confirmed, a command to display a sign is transmitted from the picking management system 30 to the cabinet 20 for the region 21 of the article to be taken out next.

(Support system)
The support system 10 according to the embodiment of the present invention includes a camera sensor 11 for imaging a plurality of areas in the cabinet 20, a monitoring device 12 that processes a signal input from the camera sensor 11 and monitors a picking operation, Is provided.

  The camera sensor 11 is arranged so as to capture a plurality of areas 21 of the cabinet 20. FIG. 2 is a plan view showing the positional relationship between the camera sensor 11 and the cabinet 20, and FIG. 3 is a side view showing the positional relationship between the camera sensor 11 and the cabinet 20. The camera sensor 11 is installed so as to face the plurality of regions 21. The number of camera sensors 11 may be one, or two as shown in FIGS. 1 to 3 may be provided on the left and right sides of the cabinet 20 diagonally, or three may be provided on the left and right and the front. The number of camera sensors 11 is set according to the overall dimensions of the plurality of regions 21.

  FIG. 4 is a block configuration diagram of the support system 10. The camera sensor 11 includes a color sensor 13 and a position distance sensor 14. The color sensor 13 is, for example, an RGB color camera, corresponds to a normal video camera, and outputs a moving image signal as image data.

  The position distance sensor 14 corresponds to a position measurement sensor such as a 3D depth sensor or a depth camera, and outputs position information from the position measurement sensor with respect to a point designated by image data. The 3D depth sensor is connected to the irradiation unit 15 that irradiates light to the target, the light receiving unit 16 that receives reflected light of the light from the target, the irradiation unit 15 and the light receiving unit 16, and irradiation, light reception, and external data And a control unit 17 that controls the output. The irradiation unit 15 converts the oscillated light from the infrared laser into a single irradiation pattern and irradiates the target object. Examples of the irradiation pattern include a quadrangular pattern in which random dots are scattered. The irradiation pattern is distorted when it hits the target object, and the distortion spreads greatly as the distance from the irradiation unit 15 increases. The light receiving unit 16 detects a plurality of irradiation patterns at different distances from the irradiation unit 15 with compound eyes. The detected signal is input to the control unit 17 and subjected to statistical processing, whereby the distance to the target object, that is, the depth can be detected.

The position distance sensor 14 divides the video image area in the light receiving unit 16 into, for example, 640 dots × 480 dots, and outputs the distance from the depth camera to the image corresponding to the dot for each dot. By associating the positional relationship between the dots as pixels in the video camera and the dots of the position measurement sensor, the coordinates (x _i, y _i ) and the distance from the depth camera with respect to the lattice points and the hands constituting the frontage Is obtained.

  In the embodiment of the present invention, the position measurement method is not limited to this, and the following method may be used. That is, a method of obtaining the distance between the position measurement sensor and the designated position and the coordinates of the designated position starting from the position measurement sensor from the delay time until the infrared rays are emitted and returned.

  The monitoring device 12 includes a processing unit 18. The processing unit 18 receives input from the color sensor 13 of the camera sensor 11 and data from the position distance sensor 14. The processing unit 18 determines whether or not the picking work is properly performed from the movement of the picking worker based on the input data. Specifically, the processing unit 18 detects the sign displayed in the image data regarding the plurality of areas 21 from the camera sensor 11 to identify the frontage of the area where the sign is displayed in the image data, Further, the position of the hand of the worker who picks is tracked, and it is determined whether or not the specified frontage corresponds to the position of the tracked hand.

Here, a processing procedure in the processing unit 18 will be specifically described.
Initial settings are made as follows. First, the camera sensor 11 is installed on the cabinet 20. Next, each pixel of the image output from the color sensor 13 is associated with each pixel of the image output from the positional distance sensor 14, and each grid point of the plurality of regions 21 in the cabinet 20 which is a real space. Define coordinate data for. As shown in FIGS. 1 to 3, when the lattice point is defined by xy plane coordinates, the i-th lattice point is defined as coordinates (x _i, y _i ).

As a specific initial setting, in the image data from the color sensor 13, after selecting lattice points constituting each frontage, in the image data from the position / distance camera 14, the coordinates of the lattice points on the coordinate axis of the position / distance camera 14 are selected. Measure (x _i, y _i ) and distance. Thus, it is possible to identify the frontage in front of the picking operator's hand from the coordinates (x _i, y _i ) of the four corners of each frontage measured in advance, and to determine the distance and hand to the four corners. It is possible to identify that the hand has entered the frontage from the distance.

  Monitoring of the picking work by the worker is performed as follows. First, the picking management system 30 displays a sign on the area 21 of the article to be taken out next among the plurality of areas allocated for each article.

  Image data from the color sensor 13 is sequentially input to the processing unit 18. The processing unit 18 identifies the frontage of the area where the sign is shown based on the input image data. At the same time, data on the position and distance is sequentially input from the position distance sensor 14 to the processing unit 18. Here, the “data relating to position and distance” includes each dot in the image area of the image data input from the position distance sensor 14 and the distance for each dot from a reference position such as the installation position of the position distance sensor 14. It is data.

  In this way, the processing unit 18 planarly tracks the position of the picking operator's hand from the image data from the color sensor 13 and spatially determines the position of the picking operator's hand from the data from the position distance sensor 14. Chase. The processing unit 18 checks whether or not these tracked positions correspond to the grid points of the area where the sign is indicated, that is, whether or not it corresponds to the frontage of the area to be extracted next. To do. On the condition that the collation is correct, it is determined whether or not the depth data is farther from the camera sensor 11 than the plane that defines the frontage of the area where the sign is indicated. Thereby, it is possible to determine whether or not a hand is inserted in the depth direction with respect to the plane in which the sign is drawn in the area where the sign is shown. It can also be determined that a hand has been inserted into the frontage other than the area to be taken out.

  The monitoring of the picking work by the worker is not limited to the above, and may be performed as follows. First, the picking management system 30 displays a sign on the area 21 of the article to be taken out next among the plurality of areas allocated for each article.

  Image data from the color sensor 13 is sequentially input to the processing unit 18, and the marker is detected in the image data. Then, the processing unit 18 transmits data related to the detected position to the position / distance sensor 14, and the position / distance sensor 14 specifies the grid point corresponding to the region 21 to be extracted next, thereby managing the frontage to be managed. Recognize

  Next, the processing unit 18 receives the sequential input of the image data from the color sensor 13 and tracks the position of the hand of the picking operator. The tracking position is expressed for each time as coordinates (Xt, Yt), for example. Here, “t” in the coordinates (Xt, Yt) indicates a time parameter.

  The processing unit 18 outputs the tracked position data to the position distance sensor 14. Then, the position distance sensor 14 measures the depth indicating the approach degree of the hand at the position together with the actually measured position data, and outputs the position data and the depth data to the processing unit 18. When the processing unit 18 obtains the position data and the depth data, it checks whether or not it corresponds to the grid point of the area where the sign is indicated, that is, whether or not it corresponds to the frontage of the area to be extracted next. To do. On the condition that the collation is correct, it is determined whether or not the depth data is farther from the camera sensor 11 than the plane that defines the frontage of the area where the sign is indicated. As a result, it is possible to determine whether or not a hand is inserted in the depth direction in the area where the sign is indicated, with respect to the surface defining the frontage of the area where the sign is indicated. It can also be determined that a hand has been inserted into the frontage other than the area to be taken out.

  When the processing unit 18 determines that the frontage determined that the hand has entered is the same as the frontage on which the sign is displayed, the processing unit 18 outputs a display or sound that means “pass”. On the other hand, if it is determined that the frontage where the hand is put is different from the frontage where the sign is displayed, a display or sound indicating “fail” is output.

  According to the embodiment of the present invention, the processing unit 18 detects the sign in the image data related to the plurality of areas 21, so that the operator can identify the frontage of the area indicated by the sign in the image data and pick it. Is determined, and it is determined whether or not the specified frontage corresponds to the position of the tracked hand. Therefore, it is possible to provide a support system that can be retrofitted to an existing picking system.

  Furthermore, in the embodiment of the present invention, when the processing unit 18 detects that the position of the hand is displaced in the depth direction other than the specified frontage, an abnormal signal is output to the outside, thereby “fail”. Information such as characters indicating the meaning of "" can be displayed or a sound can be played.

  In the embodiment of the present invention, after identifying the frontage where the sign is shown in the processing unit 18, the picking operator's hand is tracked, and no hand is inserted into the frontage within a predetermined time. When the determination is made, it is understood that the picking operator does not take out the article from the frontage to be taken out. Even in such a case, a display or sound indicating “fail” is output.

  The processing unit 18 may output an abnormal signal to the picking management system 30 when determining “fail”. By doing so, it is possible to grasp the abnormality of the work performed by the picking operator in the picking system 1, so that it is possible to take systematic measures and contribute to the maintenance of the quality of the picking work.

  Next, characteristic items in the embodiment of the present invention will be described in detail.

(Label detection)
A sign is shown on the display unit 22 of the cabinet 20. When a lamp of a specific color such as “red”, “green”, or “blue” is lit as the sign, the lit sign can be detected as follows.

  As an initial setting, the color sensor 13 captures the lighting of the sign and outputs it to the processing unit 18. The processing unit 18 converts the input image data into, for example, HSV data. And it divides | segments into the HSV channel of brightness, saturation, and hue, and the histogram regarding a label | marker is created from a hue channel. This created histogram will be referred to as a “reference histogram”.

  In the monitoring of the picking work by the worker, the processing unit 18 converts the image data sequentially input from the color sensor 13 into, for example, HSV data. Then, the color is divided into HSV channels of lightness, saturation, and hue, and a color distribution image is created by comparing the hue channel with the reference histogram. Filters created from the brightness / saturation values of HSV on the created color distribution image are used to eliminate out-of-range colors. Further, noise scattered in the image data output from the color sensor 13 is removed using a median filter. Thereby, it is possible to leave only the color of the sign and a color close to it, and to easily detect the sign.

  As described above, the monitoring device detects the label based on the color of the label in the image data, so that the label can be detected efficiently and accurately.

(Detection and tracking of picker's hand position)
In the embodiment of the present invention, the color information may be tracked by detecting color information of a work gloves worn by the picking operator from the image data output from the color sensor 13. At the position where the color information is tracked, position data and depth are measured from the position distance sensor 14. This facilitates detection and tracking of the hand position.

  A specific processing procedure will be described. FIG. 5 is a flowchart showing a processing procedure for detecting / tracking the position of the hand of the picking worker.

  The image data output from the color sensor 13 is converted into bitmap data (STEP 1). The bitmap data is used for the monitor GUI (STEP 2).

  Bitmap data is converted into HSV (STEP 3), and further divided into HSV channels, thereby being divided into three numerical values of hue, saturation, and value (STEP 4). Color information can be extracted from the hue channel.

  Prior to the picking operation, a hue histogram is created from the hue channel data for the work gloves (STEP 5). Further, the minimum value and maximum value of brightness and the minimum value of saturation in the work environment are manually set (STEP 6), and a mask image for extracting only colors close to the color of work gloves from the HSV data is created (STEP 7).

  The image data from the position / distance sensor 14 is subjected to STEP 1 to STEP 4 to detect colors and obtain a color distribution image (STEP 8). Then, the detection area is narrowed down using the mask image (STEP 9), the noise of the median filter is removed (STEP 10), and the coordinates of the coincidence area are obtained by color tracking using the CamShift method (STEP 11).

  In this way, the hue histogram acquired in advance is compared with the color information obtained during the picking operation (STEP 12), and only the matched ones are output and checked against the histogram of the work gloves. It is possible to track the work gloves by color by color (STEP 13).

(Preventing misjudgment)
By using the camera sensor 11 with high accuracy, it is possible to prevent misjudgment caused by data processing. However, since one monitoring device is used in one cabinet 20, use of a highly accurate camera sensor 11 increases the cost due to the introduction of the monitoring device, and consequently the cost of products and services. Invite high. Thus, misuse can be prevented by data processing even when a camera sensor 11 having a low quality is used. The data processing is based on the following concept. In the picking operation, the area where the mark is shown at the time of the current picking is different from the area where the mark is shown at the time of picking one time before or several times before. That is, a sign is not shown for the same area unless a certain time has passed.

  As the data processing, for example, when the area found by tracking the picking operator's hand is found to be the same area without elapse of a fixed time, the determination result is abnormal.

  In the embodiment of the present invention, the order of picking up articles such as merchandise, work-in-progress, material, and product is managed, and the frontage of the picking area where each article exists, and the frontage of the area where the next article to be taken out exists. The system that notifies the picking object to the picking operator by turning on the lamp or the like attached to the head has the following characteristics.

  A camera sensor 11 in which a video camera as the color sensor 13 and a depth camera as the position distance sensor 14 are integrated, and the camera sensor 11 that detects the position of a designated location is used. By detecting the lighted lamp, the front door of the article is specified, and the front position of the article is detected by tracking the position of the hand by the camera sensor 11. Then, it is determined whether or not an article has been taken out from the frontage corresponding to the lit lamp.

  At that time, the camera sensor 11 may be used to detect the color of the lamp when lit. The camera sensor 11 is used to track the color of the worker's gloves as the position of the hand, and by measuring the coordinates at that position, the frontage where the article has been taken out is detected.

  It is determined whether or not an article has been taken out from the frontage corresponding to the lit lamp, and the operator is notified by displaying it with Patlite (registered trademark) or sounding an alarm sound.

  As described above, according to the support system according to the embodiment of the present invention, the monitoring device 12 detects the sign in the image data related to the plurality of areas 21, thereby the frontage of the area 21 where the sign is displayed in the image data, That is, the exit in the region 21 is specified, the position of the hand of the worker who picks is tracked, and it is determined whether or not the specified frontage corresponds to the position of the tracked hand. Therefore, it can be retrofitted to an existing picking system.

1: Picking system 10: Support system 11: Camera sensor 12: Monitoring device 13: Color sensor 14: Position distance sensor 15: Irradiation unit 16: Light receiving unit 17: Control unit 18: Processing unit 20: Cabinet 21: Area 22: Display Part 30: Picking management system

Claims (5)

Attached to a picking system that sequentially displays signs to an area in which an article to be taken out next is stored among a plurality of areas assigned to each article,
By detecting the sign in the image data related to the plurality of areas, the frontage of the area where the sign is displayed in the image data is specified, the position of the hand of the operator who picks is tracked, and the specified frontage And a monitoring device that determines whether or not the position of the tracked hand corresponds.   The support system according to claim 1, wherein the monitoring device outputs an abnormal signal to the outside when detecting that the position of the hand is displaced in the depth direction other than the frontage where the sign is displayed.   The support system according to claim 1, wherein the monitoring device detects the sign based on a color of the sign in image data.   The support system according to any one of claims 1 to 3, wherein the monitoring device tracks the position of a worker's hand based on a color of a glove worn by the worker. The monitoring device includes:
A video camera,
A position measuring camera;
An area in which the sign is displayed based on the output data from the position measurement camera, while detecting the sign and tracking the position of the operator's hand picking based on the output data from the video camera A processing unit for identifying a frontage and measuring a position of a hand of a worker who picks and tracking a hand obtained from output data from the video camera;
The support system according to claim 1, comprising:

Images