JP2020040811A - Determination device and determination method - Google Patents

Abstract

To determine whether or not a fork lift faces a pallet.SOLUTION: A determination unit (1) comprises a facing determination unit (104) that determines whether or not a fork lift (5) faces a target pallet, by using a learnt model obtained by mechanically learning a correlation between whether or not the fork lift faces the pallet and an image of the pallet with a cargo object placed thereon, the image being taken by a photographing device (3) installed on the fork lift (5).SELECTED DRAWING: Figure 1

Description

  The present invention relates to a determination device and the like for determining whether a forklift is directly facing a pallet loaded with a load.

  In a cargo handling operation using a forklift, a driver of the forklift performs a series of operations of inserting a fork into a fork pocket of a pallet loaded with a load, lifting the load along with the pallet, and transporting the load. Of these series of operations, the difficulty of inserting the fork into the fork pocket is particularly high, and depending on the driver, it may take time to insert the fork or cause the fork to collide with the pallet. is there.

  As a technique for facilitating the fork insertion operation, for example, Patent Document 1 is cited. Patent Document 1 discloses that a pallet having a light reflecting surface for reflecting a laser beam is irradiated with a laser beam having the same height as a fork, and the height of the fork is adjusted with respect to the pallet using the reflected light as a mark for a driver. Is disclosed.

JP 2005-298000 A

  According to the technique described in Patent Literature 1, height alignment of the fork with respect to the pallet becomes easy. However, even if the height position of the fork and the pallet is matched, if the forklift is not directly facing the pallet, the fork insertion operation may not proceed smoothly or the fork may contact the pallet.

  The present invention has been made in view of the above-described problems, and an object of the present invention is to provide a determination device and the like that can determine whether a forklift is directly facing a pallet.

  In order to solve the above-described problems, a determination device according to one embodiment of the present invention includes an image capturing device that is mounted on a forklift, an image of a pallet on which a material to be unloaded is placed, and the forklift Using a learned model that machine-learned the correlation with whether or not the pallet is directly facing, from the image of the target pallet on which the cargo handling object of the forklift is placed by the imaging device, A facing determination unit that determines whether or not the facing pallet is facing is provided.

  Further, in order to solve the above problem, a determination method according to an aspect of the present invention is a determination method using a determination device, in which a cargo handling target object of the forklift is mounted on an imaging device installed on the forklift. Using a learned model in which the correlation between the image of the pallet taken and the forklift facing the pallet or not is machine-learned, the object pallet on which the forklift cargo handling object is placed is photographed. A facing determination step of determining whether or not the forklift is facing the target pallet from an image captured by the apparatus.

  According to one embodiment of the present invention, it is possible to determine whether or not a forklift is facing the target pallet based on an image of the target pallet.

It is a block diagram showing an example of the important section composition of the judging device concerning Embodiment 1 of the present invention. FIG. 1 is a diagram for describing an outline of a first embodiment of the present invention. It is a flowchart which shows an example of the process which the said determination apparatus performs. It is a figure for explaining an outline of Embodiment 2 of the present invention. It is a block diagram showing an example of the important section composition of the control part in the judgment device concerning Embodiment 2 of the present invention. It is a block diagram showing an example of important section composition of a control part in a judgment device concerning Embodiment 3 of the present invention.

[Embodiment 1]
(Overview)
First, with reference to FIG. 2, an outline of a process performed by the determination device 1 according to the first embodiment will be described. FIG. 2 is a diagram for describing an outline of a process performed by the determination device 1. FIG. 2A illustrates a state in which the forklift 5 performs a cargo handling operation on a pallet (target pallet) 800 loaded with a cargo (loading target) 900 placed on a gantry. FIG. 2B shows the state of FIG. 2A viewed from above.

  The forklift 5 performs a cargo handling operation in which the load 900 loaded on the pallet 800 is moved by inserting the fork 50 of the forklift 5 into the fork pocket 801 of the pallet 800. The determination device 1 is a device that determines whether the forklift 5 faces the pallet 800 in the cargo handling operation. The fork pocket 801 includes a pair of left and right openings into which a pair of left and right forks 50 are inserted, as shown in FIG. 2B.

  Here, in order to properly insert the fork 50 of the forklift 5 into the fork pocket 801 of the pallet 800, the forklift 5 needs to face the pallet 800. This is because if the forklift 5 does not face the pallet 800, the fork 50 of the forklift 5 does not enter the fork pocket 801 or gets stuck in the fork pocket 801.

  Even if the forklift 5 is not completely opposed to the pallet 800 (the direction in which the fork 50 projects and the direction in which the fork pocket 801 extends is parallel), the forklift 5 is in a state close to the pallet 800. Cargo handling work is performed without any problem. For this reason, the term “facing” in the present specification includes not only a perfect facing state but also a state deviating from the perfect facing state within a range that does not hinder cargo handling work.

  In the present embodiment, as shown in FIG. 2A, the photographing device 3 for photographing the pallet 800 from the position of the fork 50 of the forklift 5 is provided. Then, the determination device 1 determines whether or not the forklift 5 faces the pallet 800 from the image captured by the imaging device 3. Here, when it is determined that the pallet 800 is not directly facing, the determining device 1 notifies that the forklift 5 is not directly facing the pallet 800 as shown in FIG. In addition, in the example shown in FIG. 2B, it is informed that “It is not directly facing the pallet. Please adjust the direction of the vehicle body”.

  As described above, the image capturing device 3 captures an image for determining whether the forklift 5 faces the pallet 800 or not. The image photographed by the photographing device 3 may be a still image or a moving image. In the present embodiment, it is assumed that a still image is captured.

  In the example of FIG. 2, the photographing device 3 is mounted at an intermediate position between the bases of the pair of forks 50, and the photographing direction is the front direction (horizontal direction) of the forklift 5. Of course, the photographing device 3 may be mounted at a position where the side surface of the pallet 800 where the fork pocket 801 is open can be photographed, and is not limited to the illustrated example. However, the pallet 800 and the fork pocket 801 can be photographed from the front or a direction close to the front by installing the photographing device 3 at a position where the pallet 800 and the fork 50 are raised and lowered in accordance with the lifting and lowering of the fork 50, such as the base of the fork 50. It is preferable to install the photographing device 3 at such a position.

(Main Configuration of Judgment Device 1)
Next, a configuration of a main part of the determination device 1 will be described with reference to FIG. FIG. 1 is a block diagram illustrating an example of a main configuration of the determination device 1. As shown in FIG. 1, the determination device 1 includes a control unit 10 that integrally controls each unit of the determination device 1, a storage unit 20 that stores various data used by the determination device 1, and a voice that outputs a voice. An output unit 40 is provided. In addition, the control unit 10 includes an operation detection unit 101, an imaging control unit 102, an input data generation unit 103, a facing determination unit 104, and a notification control unit 105.

  The operation detection unit 101 detects that a predetermined operation has been performed on the forklift 5. Specifically, the operation detection unit 101 detects that an operation of lifting and lowering the fork 50 and an operation of adjusting the direction of the forklift 5 have been performed. The detection method is not particularly limited. For example, the detection may be performed by communicating with the forklift 5, the detection may be performed by analyzing an image of the forklift 5 or its driver, or the sensor may be detected. May be used for detection. As described below, detection of the above operation by the operation detection unit 101 triggers imaging by the imaging control unit 102.

  The photographing control unit 102 controls the operation of the photographing device 3 to photograph the above-described photographed image. In the present embodiment, since the photographing device 3 moves up and down in conjunction with the fork 50, the photographing control unit 102 causes the photographing device 3 to photograph when the pallet 800 enters the photographing range of the photographing device 3. Specifically, the photographing control unit 102 causes the photographing device 3 to photograph at the timing when the lifting and lowering of the fork 50 is completed. The timing at which the raising and lowering of the fork 50 is completed can be specified from the timing at which the operation is no longer detected after the operation detecting unit 101 detects an operation to raise and lower the fork 50.

  In this way, by causing the photographing to be performed at the end of the vertical movement, the photographing can be automatically performed at the timing when the pallet 800 enters the photographing range of the photographing device 3. In addition, since the number of times of photographing can be minimized, the number of times of processing and the processing time of the determination device 1 using captured images are also minimized, and efficient processing with a small load on the determination device 1 is realized. Is done.

  It should be noted that the timing of shooting is not limited to the above example. For example, when inserting the fork 50 into the fork pocket 801, a tilt operation for adjusting the inclination of the fork 50 is often performed. Therefore, the operation detection unit 101 may detect a tilt operation, and the imaging control unit 102 may perform imaging when the operation is detected. In addition to this, for example, the operation detecting unit 101 may use the forklift 5 in a specific manner at the start of the cargo handling operation, such as performing fine adjustment of the traveling direction while the forklift 5 moves forward and backward at a low speed, as an opportunity for shooting. May be detected as being operated.

(About the input data generation unit 103 and the input data)
The input data generation unit 103 generates input data to be input to the learned model used by the facing determination unit 104. More specifically, the input data generation unit 103 acquires a captured image from the imaging device 3 and generates input data from the acquired captured image. In the present embodiment, an example will be described in which a captured image acquired from the imaging device 3 is used as input data as it is.

  The captured image used for generating the input data is preferably an image captured under the same capturing conditions as the image of the teacher data used for generating the learned model. The photographing conditions include, for example, the positional relationship between the photographing device 3 and the subject (palette 800), the background of the subject, and the surrounding brightness. In order to make the shooting conditions uniform, for example, if the image used for the teacher data is an image shot by a shooting device fixed to the base of the fork, the shot image is also shot by the shooting device 3 fixed to the base of the fork 50. Preferably,

  It should be noted that the input data only needs to be data indicating characteristics of the positional relationship between the forklift 5 and the pallet 800 (whether the forklift 5 and the pallet 800 face each other), and it is not always necessary to use the captured image as it is. For example, the input data generation unit 103 may use an image obtained by performing a predetermined process on a captured image as input data. The predetermined process may be any process that can reduce information not related to the positional relationship between the forklift 5 and the pallet 800 without losing the feature. For example, when the captured image is a color image, the input data generating unit 103 may supply grayscale data of the captured image to the facing determination unit 104 as input data. Alternatively, when the captured image is a moving image, the input data generation unit 103 extracts a part of the frame constituting the captured image, and supplies the extracted part of the frame to the facing determination unit 104 as input data. Is also good.

  Further, the input data generation unit 103 may detect a predetermined subject (for example, a pallet 800) from the subjects appearing in the captured image, and may generate the input data only from an area where the detected subject appears. Accordingly, the influence of the background of the subject can be eliminated, so that the accuracy of the facing determination unit 104 can be improved. Note that a learned model such as a CNN (Convolutional Neural Network) can be used to detect a predetermined subject.

  However, the input data generation unit 103 needs to generate, as input data, data in the same format as the teacher data. For example, if the teacher data is a grayscale still image, the input data also needs to be a grayscale still image.

(About the facing determination unit 104 and the learned model)
The facing determination unit 104 determines whether the forklift 5 faces the pallet 800 using a learned model generated by machine learning. For generation of the learned model, for example, an image of the pallet 800 taken in a state where the forklift 5 and the pallet 800 face each other and an image of the pallet 800 taken in a state where the forklift 5 and the pallet 800 do not face each other are used as teacher data. be able to.

  When the teacher data is a still image, highly reliable output data can be obtained by using the trained model of the CNN. In addition, it is preferable that a large number of images that are boundaries between a face-to-face state and a face-to-face state are not used as teacher data. This makes it possible to appropriately distinguish between a state in which the fork 50 is directly facing (there is no problem in inserting the fork 50) and a state in which the face is not directly facing (there is no problem in inserting the fork 50). As described above, in the facing determination of the present embodiment using the trained model, even if the vehicle is not facing strictly, it is facing if it does not hinder insertion of the fork 50. Therefore, it can be said that the determination device 1 has high practicality.

  The facing determination unit 104 inputs the input data generated from the captured image by the input data generation unit 103 to the learned model generated as described above. As a result, the learned model outputs output data indicating the probability that the forklift 5 is not directly facing the pallet 800, and outputs output data indicating the probability that the forklift 5 is facing the pallet 800. The facing determination unit 104 determines whether the forklift 5 faces the pallet 800 based on the output data. For example, the facing determination unit 104 determines that the user is facing when the probability of facing is higher than the probability of not facing, and conversely, if the probability of not facing is higher than the probability of facing directly. It may be determined that they are not facing each other.

  Note that the facing determination unit 104 uses the image of the pallet 800 taken in the face-to-face state as the teacher data, while the image of the pallet 800 taken in the face-to-face state is generated without being included in the teacher data. A completed model may be used. Similarly, the facing determination unit 104 generates the image of the pallet 800 taken in the face-to-face state as the teacher data while generating the image of the pallet 800 taken in the face-to-face state without including it in the teacher data. A trained model may be used. In these cases, the facing determination unit 104 may determine whether or not the facing is based on whether or not the probability indicated by the output data output by the trained model is equal to or greater than a threshold. For example, when the former learned model is used, the facing determination unit 104 determines that the user is not facing the user if the probability indicated by the output data output by the learned model is equal to or greater than a threshold, and determines that the user is not facing the user if the probability is less than the threshold. It is determined that it is.

(About the notification control unit 105 and an example of notification)
When the facing determination unit 104 determines that the forklift 5 is not facing the pallet 800, the notification control unit 105 causes the audio output unit 40 to notify that fact. The mode of the notification is not limited to this example. For example, the notification control unit 105 may perform the notification by displaying information indicating that the forklift 5 is not directly facing the pallet 800 on the display device. This display device may be provided in the determination device 1 or may be provided outside the determination device 1. For example, when the forklift 5 includes a display device, the notification control unit 105 may display the information on the display device of the forklift 5 to notify the display device. Further, for example, the above information may be displayed on a device (for example, a wearable device such as an eyeglass type, a mobile phone, a tablet terminal, or the like) carried by the driver of the forklift 5. The same applies to the case of notifying by voice output. The notification control unit 105 outputs a voice to a device installed outside the determination device 1 to notify the driver that the forklift 5 is not directly facing the pallet 800. You may be notified.

(Processing flow)
Next, the flow of processing executed by the determination device 1 will be described with reference to FIG. FIG. 3 is a flowchart illustrating an example of a process (determination method) performed by the determination device 1.

  In S1, the imaging control unit 102 determines whether the pallet 800 has entered the imaging range of the imaging device 3. Specifically, the photographing control unit 102 determines whether or not the pallet 800 has entered the photographing range of the photographing device 3 based on the detection result of the operation of moving the fork 50 up and down by the operation detecting unit 101. Here, if it is determined that the image has entered the shooting range (YES in S1), the process proceeds to S2. On the other hand, if it is determined that it is not within the shooting range (NO in S1), the process returns to S1, and the same processing is repeated.

  In S2, the photographing control unit 102 causes the photographing device 3 to photograph a photographed image including the pallet 800, and proceeds to S3. Then, in S3, the input data generation unit 103 acquires a captured image from the imaging device 3, and generates input data. In the first embodiment, as described above, the input data generation unit 103 uses a captured image as it is as input data.

  In S4 (facing determination step), the facing determining unit 104 inputs the input data generated in S3 to the above-described learned model, and the forklift 5 moves the pallet 800 based on the output data from the learned model. It is determined whether or not it is directly facing. When it is determined that the user is facing directly (YES in S4), the processing in FIG. 3 ends. In this case, the notification control unit 105 may cause the audio output unit 40 to output a message indicating that the user is facing the user.

  On the other hand, if it is determined in S4 that they are not facing each other (NO in S4), the process proceeds to S5. In S5, the notification control unit 105 causes the sound output unit 40 to notify that the forklift 5 is not directly facing the pallet 800. Then, the process proceeds to S6.

  In S6, the operation detection unit 101 determines whether an operation for adjusting the direction of the forklift 5 has been performed. Here, when it is determined that the operation of adjusting the direction has been performed (YES in S6), the process returns to S2. In this case, the facing determination unit 104 determines whether the forklift 5 faces the pallet 800 from the image of the pallet 800 taken by the photographing device 3 after the adjustment. Thus, it is possible to feed back to the driver who has performed the operation of adjusting the direction whether or not the forklift 5 has faced the pallet 800 by the operation. Therefore, the driver can smoothly face the forklift 5 to the pallet 800.

(Summary of Embodiment 1)
As described above, the facing determination unit 104 is configured such that the image capturing apparatus 3 installed on the forklift 5 captures an image of the pallet 800 on which the cargo handling object of the forklift 5 is mounted, and the forklift 5 faces the pallet 800 directly. The forklift 5 faces the pallet 800 from an image obtained by photographing the pallet 800 on which the cargo handling object of the forklift 5 is placed with the photographing device 3 using a learned model in which the correlation with whether or not the pallet 800 has been machine-learned. Is determined.

  This makes it possible to determine whether or not the forklift 5 faces the pallet 800 based on the image of the pallet 800 taken. In addition, when the forklift 5 is not directly facing the pallet 800, the notification control unit 105 notifies the driver of the forklift 5 so that failure of the cargo handling work can be prevented beforehand.

[Embodiment 2]
Another embodiment of the present invention will be described below. For convenience of description, members having the same functions as the members described in the first embodiment are denoted by the same reference numerals, and description thereof will not be repeated. This is the same in the third and subsequent embodiments.

(Overview)
The determination device 1 of the first embodiment determines whether or not the forklift 5 is directly facing the pallet 800, and reports that the forklift 5 is not facing the pallet 800. On the other hand, the determination device 1 of the present embodiment determines not only whether the forklift 5 faces the pallet 800 but also how much the forklift 5 deviates from the facing state, and reports the determination result. Thereby, the driver can smoothly finely adjust the direction of the forklift 5 and face the forklift 5 to the pallet 800.

  An outline of the present embodiment will be described with reference to FIG. FIG. 4 is a diagram for explaining the outline of the present embodiment. FIGS. 4A to 4C show the forklift 5 for carrying out the cargo handling operation as viewed from above. As shown in FIG. 4A, when the forklift 5 does not face the pallet 800, the sound output unit 40 of the determination device 1 notifies that the forklift 5 does not face the pallet 800. This notification is the same as in the first embodiment.

  Thereafter, the direction of the forklift 5 is adjusted, and in FIG. 4B, the forklift 5 approaches the pallet 800 more closely than in FIG. 4A, but it can be said that it is again facing. There is no state. In this state, the audio output unit 40 reports that the player is approaching the facing state but has not yet faced it. In the present embodiment, the degree of deviation from the face-to-face state is determined, so that the notification as shown in FIG.

  In FIG. 4C in which the direction of the forklift 5 is further adjusted, the forklift 5 faces the pallet 800. In this state, the audio output unit 40 notifies that the forklift 5 is directly facing the pallet 800. In the state of FIG. 4C, the notification may be omitted.

(Main Configuration of Judgment Device 1)
Next, a configuration of a main part of the control unit 10 of the determination device 1 according to the present embodiment will be described with reference to FIG. FIG. 5 is a block diagram illustrating an example of a main configuration of the control unit 10. The control unit 10 of the present embodiment includes a deviation degree determination unit 111 in addition to the components (see FIG. 1) included in the control unit 10 of the first embodiment.

  The deviation degree determination unit 111 determines the degree of deviation of the forklift 5 from the state of directly facing the pallet 800 based on output data (output value) from the learned model used by the facing determination unit 104. More specifically, the deviation degree determination unit 111 corresponds to the probability value output from the learned model used by the facing determination unit 104, that is, the probability corresponding to the non-facing state and the probability corresponding to the facing state. The above determination is made based on at least one of the probabilities. In the following, an example will be described in which a determination is made based on a probability corresponding to a state in which the user does not face directly.

  Here, in the captured image of the pallet 800, when the pallet 800 is clearly tilted, that is, when the characteristic of the state in which the pallet 800 does not face directly appears in the captured image, the output data of the learned model is 100%. Shows a probability close to. On the other hand, in the captured image of the pallet 800, when the degree of inclination of the pallet 800 is smaller, that is, when the feature of the pallet 800 is not clearly shown in the captured image, the probability that the output data of the trained model indicates Is a lower value. The deviation degree determination unit 111 makes the above determination using this property.

  Specifically, the deviation degree determination unit 111 determines the degree of deviation of the forklift 5 from the state in which the forklift 5 is directly facing the pallet 800 according to which numerical range the probability value indicated by the output data of the learned model belongs to. Is determined. For example, three numerical ranges of less than 50%, 50% or more and less than 80%, and 80% or more are set for the probability value corresponding to the state of not facing directly for the determination by the deviation degree determination unit 111. Is also good. Then, the deviation degree determination unit 111 determines that the deviation degree is large when the probability value is 80% or more, and determines that the deviation degree is small when the probability value is 50% or more and less than 80%. May be. If the probability value is less than 50%, the facing determination unit 104 determines that the vehicle is facing directly, and the determination by the deviation degree determination unit 111 is omitted.

  Further, the notification control unit 105 of the present embodiment performs notification according to the determination result of the deviation degree determination unit 111. For example, when the deviation degree determination unit 111 determines that the degree of deviation is large, the notification control unit 105 indicates that the state is greatly deviated from the state of facing directly, or that the direction of the forklift 5 needs to be largely adjusted. May be notified to the driver. Similarly, when the deviation degree determination unit 111 determines that the degree of deviation is small, the notification control unit 105 needs to slightly deviate from the facing state or slightly adjust the direction of the forklift 5. This may be notified to the driver.

  Further, the shift degree determining unit 111 may determine the shift degree when an operation of adjusting the direction of the forklift 5 is performed. Then, the notification control unit 105 may perform notification according to the deviation degree before the direction adjustment of the forklift 5 and the deviation degree after the direction adjustment. For example, when the degree of deviation is determined to be large before the direction adjustment as in the example of FIG. 4A, and the degree of deviation is determined to be small after the direction adjustment as in the example of FIG. The notification control unit 105 may notify that the direction of the adjustment is correct. Conversely, when it is determined that the degree of deviation is small before the direction adjustment, and when it is determined that the degree of deviation is large after the direction adjustment, the notification control unit 105 determines that the direction of the adjustment is incorrect or that the adjustment is performed in the opposite direction. You may be notified of what to do.

(Summary of Embodiment 2)
As described above, the determination device 1 of the second embodiment determines the degree of deviation of the forklift 5 from the state of directly facing the pallet 800 based on the output data from the learned model used by the facing determination unit 104. The shift degree determining unit 111 is provided. As a result, it is possible to feed back to the driver how much the forklift 5 deviates from the state where the forklift 5 faces the pallet 800 directly. Therefore, the driver can smoothly finely adjust the direction of the forklift 5 and face the forklift 5 to the pallet 800.

  The degree of deviation can be determined also by using a learned model generated by machine learning using teacher data having different degrees of deviation between the forklift 5 and the pallet 800. However, in the case of using the output data of the trained model to determine whether or not the robot is facing directly, as in the present embodiment, the cost of generating the trained model and the calculation using the trained model This is preferable because the degree of deviation can be determined without increasing the cost.

[Embodiment 3]
As described above, in the cargo handling operation, the load is moved together with the pallet 800 by inserting the fork 50 of the forklift 5 into the fork pocket 801 of the pallet 800. Here, the fork 50 of the forklift 5 can be tilted. If the fork 50 is not horizontal when inserted into the fork pocket 801, the fork 50 may come into contact with the fork pocket 801. is there. In other words, even if the forklift 5 is directly facing the pallet 800, if the fork 50 is in a tilt state (a state in which the tip of the fork is located above or below the base of the fork 50), the cargo handling operation fails. There is a risk.

  Thus, in the present embodiment, an example will be described in which it is determined whether the forklift 5 faces the pallet 800 and whether the fork 50 is in the tilt state. With reference to FIG. 6, the main configuration of the control unit 10 of the determination device 1 in the present embodiment will be described. FIG. 6 is a block diagram illustrating an example of a main configuration of the control unit 10. The control unit 10 of the present embodiment includes a tilt determination unit 121 in addition to the components included in the control unit 10 (FIG. 5) of the second embodiment. When it is not necessary to determine the degree of deviation, the degree of deviation determination unit 111 may be omitted.

  The tilt determination unit 121 determines whether the fork 50 is in a tilt state (whether the fork 50 is horizontal or inclined). The tilt determination is performed by using a teacher data generated from an image of the fork 50 in a tilted state from a predetermined position and a teacher data generated from an image of the fork 50 in a horizontal state from the predetermined position. A learned model generated by learning can be used.

  As described in the first embodiment, when the photographing device 3 is arranged in the middle of each base of the pair of forks 50 and photographed forward in the horizontal direction, the photographed image shows the fork 50. Therefore, in the present embodiment, the input data generated by the input data generation unit 103 from the captured image is also used for the determination by the tilt determination unit 121. Thereby, both the facing determination and the tilt determination can be performed from one captured image. Needless to say, input data generated by using an image photographed separately from that for the facing determination may be used for the tilt determination. Further, even when the same photographed image is used, input data for tilt determination, which is different from that for facing determination, may be generated.

  When the tilt determination unit 121 inputs the input data to the learning model, the learned model outputs output data indicating the probability that the fork 50 is in the tilt state, and the fork 50 is not in the tilt state. Outputs output data indicating the probability. Then, the tilt determination unit 121 determines whether the fork 50 is in the tilt state based on the output data. This determination is the same as the determination by the facing determination unit 104, and a description thereof will be omitted.

  When the tilt determination unit 121 determines that the vehicle is in the tilt state, the notification control unit 105 of the present embodiment notifies the driver of the forklift 5 of the fact. The manner of the notification is not particularly limited, as in the notification of the facing determination result and the like in the above embodiments.

(Summary of Embodiment 3)
As described above, the determination device 1 of the present embodiment uses the learned model obtained by machine learning using the image of the fork in the tilted state as the teacher data, from the image of the fork 50 in the tilted state. A tilt determining unit 121 that determines whether or not there is a counter is provided. Therefore, the tilt state can be automatically detected. In addition, when it is determined that the vehicle is in the tilt state, the notification control unit 105 notifies the driver of the forklift 5, so that it is possible to prevent the failure of the cargo handling work before it occurs.

(Modification)
The configuration of each of the embodiments described above can also be applied to a forklift capable of performing a cargo handling operation automatically without a driver's operation. In this case, the determination device 1 may include a forklift control unit for adjusting the direction of the forklift (the direction facing the forklift) instead of the notification control unit 105. This allows the forklift 5 and the pallet 800 to automatically face each other.

[About distributed processing]
Part of the processing performed by the determination device 1 described in each of the above embodiments may be performed by one or more devices that are communicatively connected to the determination device 1. For example, part of the processing performed by the facing determination unit 104 may be performed by an AI server that is communicably connected to the determination device 1. In this case, for example, the determination device 1 transmits the input data to the AI server, receives the output data from the AI server, and determines whether the forklift 5 faces the pallet 800 or not.

[Example of software implementation]
The control block of the determination device 1 (particularly, each unit included in the control unit 10) may be realized by a logic circuit (hardware) formed on an integrated circuit (IC chip) or the like, or may be realized by software. .

  In the latter case, the determination device 1 includes a computer that executes instructions of a program that is software for realizing each function. This computer includes, for example, one or more processors and a computer-readable recording medium storing the program. Then, in the computer, the object of the present invention is achieved by the processor reading the program from the recording medium and executing the program. As the processor, for example, a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit) can be used. Examples of the recording medium include “temporary tangible medium” such as a ROM (Read Only Memory), a tape, a disk, a card, a semiconductor memory, and a programmable logic circuit. Further, a RAM (Random Access Memory) for expanding the program may be further provided. Further, the program may be supplied to the computer via an arbitrary transmission medium (a communication network, a broadcast wave, or the like) capable of transmitting the program. Note that one embodiment of the present invention can also be realized in the form of a data signal embedded in a carrier wave, in which the above-described program is embodied by electronic transmission.

  The present invention is not limited to the embodiments described above, and various modifications are possible within the scope shown in the claims, and embodiments obtained by appropriately combining technical means disclosed in different embodiments. Is also included in the technical scope of the present invention.

DESCRIPTION OF SYMBOLS 1 Judgment apparatus 3 Photographing apparatus 5 Forklift 50 Fork 104 Face-to-face judging part 111 Deflection degree judging part 121 Tilt judging part 800 Pallet (target pallet)
801 Fork pocket 900 Cargo (loading target)

Claims (5)

  An image obtained by photographing a pallet on which a cargo handling object of the forklift is mounted by a photographing device installed on the forklift, and a learned model obtained by machine learning a correlation between whether the forklift is directly facing the pallet or not. Using an image of the target pallet on which the cargo handling object of the forklift is placed by the photographing device, provided with a facing determination unit that determines whether the forklift is facing the target pallet or not. A determining device.   The facing determination unit, when an operation of adjusting the direction facing the forklift is performed, from the image of the target pallet photographed by the imaging device after the adjustment, the forklift facing the target pallet The determination device according to claim 1, wherein the determination device determines whether or not there is an error.   3. The apparatus according to claim 1, further comprising: a deviation degree determination unit configured to determine a degree of deviation of the forklift from a state of directly facing the target pallet based on an output value from the learned model. 4. The determination device according to the above.   A tilt determining unit that determines whether or not the fork is in a tilt state from an image of a fork of the forklift using a learned model obtained by machine learning using an image of a fork in a tilt state as teacher data; The determination device according to any one of claims 1 to 3, wherein: A determination method by a determination device,
An image obtained by photographing a pallet on which a cargo handling object of the forklift is mounted by a photographing device installed on a forklift, and a learned model obtained by machine learning a correlation between whether the forklift is directly facing the pallet or not. A facing determination step of determining whether or not the forklift is facing the target pallet from an image obtained by capturing the target pallet on which the cargo handling object of the forklift is placed by the photographing device. A determination method characterized by the following.

Images