JP5904608B2 - Vehicle motion estimation device and cargo handling vehicle - Google Patents

Description

  The present invention relates to a vehicle motion estimation device that estimates a motion mode of a vehicle body using a camera, and a cargo handling vehicle including the device.

  2. Description of the Related Art In recent years, in general passenger cars, a front or rear image of a vehicle is taken with a camera, and the video is used for driving assistance or the like. Moreover, an apparatus for estimating an operation mode including a traveling speed and a traveling direction of a vehicle based on an image photographed by a camera has been developed (see, for example, Patent Document 1).

  In Patent Document 1, a road surface is photographed by a camera provided on a moving body, and a displacement vector ΔS representing a moving amount of the moving body at a minute time interval Δt is estimated based on a camera image photographed by the camera. A quantity estimation device is described. This movement amount estimation device includes a geometric conversion means for converting a camera image into a video pattern (road surface image) on a plane Λ having a normal direction to a direction perpendicular to a unit vector e in the moving direction of the moving object; On-plane movement amount calculation means for calculating the displacement vector Δs1 representing the movement amount of the entire video pattern on the plane Λ, and movement amount conversion means for converting the displacement vector Δs1 into a displacement vector ΔS representing the movement amount of the moving body. Have. Further, in [0038] of Patent Document 1, in order to lower the calculation cost, the road surface image is projected onto the Y axis to make the road surface image a one-dimensional pattern, and the time dimension of the one-dimensional pattern in the axial direction is calculated. It describes that the amount of movement is estimated based on movement.

JP 2003-178309 A

  However, since the movement amount estimation apparatus of Patent Document 1 performs geometric transformation on the entire camera image, when the two-dimensional image that is the camera image becomes large, the movement mode of the vehicle body that is the moving body is estimated. The load increases. Moreover, according to the structure which estimates the moving amount | distance of a vehicle main body based on the one-dimensional pattern converted from the two-dimensional image, although the load for estimating an operation | movement aspect can be reduced, driving | running | working and turning of a vehicle main body And the operation mode of the vehicle main body cannot be estimated with high accuracy. That is, there is a problem that it is impossible to achieve both reduction in load for estimating the operation mode of the vehicle body and improvement in estimation accuracy.

  The present invention has been made in view of the above circumstances, and in the estimation of the operation mode of the vehicle main body using a camera, to achieve both reduction of the load for estimating the operation mode and improvement of the estimation accuracy. It is an object of the present invention to provide a vehicle motion estimation device and a cargo handling vehicle that can perform a vehicle.

In order to solve the above-mentioned problem, the vehicle motion estimation device according to the first aspect of the present invention is a vehicle motion estimation device that estimates a motion mode of a vehicle body that can travel on a road surface and can turn, and is a subject. By capturing the road surface, a two-dimensional image composed of a plurality of pixels arranged along a first coordinate axis corresponding to the front-rear direction of the vehicle main body and a second coordinate axis corresponding to the left-right direction of the vehicle main body is acquired. Acquired by the camera, a first area extraction unit that extracts a first area corresponding to a location aligned with the vehicle body in at least one of the left and right directions of the vehicle body from the two-dimensional image acquired by the camera, and the camera A second region extraction unit that extracts a second region corresponding to a location on the vehicle body in at least one of the vehicle body in the front-rear direction of the vehicle body; A first displacement amount calculation unit that calculates a displacement amount of a subject included in a region and in a direction parallel to the first coordinate axis; and a displacement amount of a subject included in the second region and the second displacement amount. Based on a second displacement amount calculator that calculates a displacement amount in a direction parallel to the coordinate axis, a displacement amount calculated by the first displacement amount calculator, and a displacement amount calculated by the second displacement amount calculator. And an operation mode estimation unit that estimates an operation mode of the vehicle body , wherein the camera is configured by an omnidirectional camera capable of capturing a 360 ° range when viewed from the vertical direction, and the first region extraction unit and The second region extraction unit extracts the first region and the second region from the two-dimensional image acquired by the same omnidirectional camera .

  Further, the vehicle motion estimation device according to claim 2 is the vehicle motion estimation device according to claim 1, wherein the first region extraction unit and the second region extraction unit are configured to use the first coordinate axis in the two-dimensional image. A rectangular region surrounded by two sides parallel to each other and two sides parallel to the second coordinate axis is extracted.

  The vehicle motion estimation apparatus according to claim 3 is the vehicle motion estimation apparatus according to claim 1 or 2, wherein the motion mode estimation unit has a predetermined displacement amount calculated by the first displacement amount calculation unit. It is estimated that the vehicle main body is stopped without traveling in the front-rear direction.

  According to a fourth aspect of the present invention, there is provided the vehicle motion estimation apparatus according to the third aspect, wherein the motion mode estimation unit is configured such that the displacement amount calculated by the first displacement amount calculation unit is a predetermined stop. When the vehicle body is not a determination amount and the ratio of the displacement amount calculated by the second displacement amount calculation unit to the displacement amount calculated by the first displacement amount calculation unit is less than a predetermined determination boundary value Is estimated to be traveling forward or backward.

  The vehicle motion estimation apparatus according to claim 5 is the vehicle motion estimation apparatus according to claim 3 or 4, wherein the motion mode estimation unit has a predetermined displacement amount calculated by the first displacement amount calculation unit. When the ratio of the displacement amount calculated by the second displacement amount calculation unit to the displacement amount calculated by the first displacement amount calculation unit is not less than a predetermined determination boundary value. It is estimated that the vehicle body is turning right or left.

A cargo handling vehicle according to claim 6 is a cargo handling vehicle comprising the vehicle main body provided with a head guard, and the vehicle operation estimation device according to any one of claims 1 to 5, wherein The camera is provided on the head guard.

  According to the present invention, there is provided a vehicle motion estimation device and a cargo handling vehicle capable of achieving both reduction of a load for estimating a motion mode and improvement of estimation accuracy in estimation of a motion mode of a vehicle body using a camera. Can be provided.

BRIEF DESCRIPTION OF THE DRAWINGS It is a figure which simplifies and shows the external appearance of the cargo handling vehicle which concerns on one Embodiment of this invention, Comprising: (A) is a side view, (B) is a top view. It is a block diagram which shows the structure of the cargo handling vehicle which concerns on the same embodiment. It is a schematic diagram which shows an example of the image image | photographed with the camera which concerns on the embodiment. (A) And (B) is a schematic diagram which shows a mode that the vehicle main body which concerns on the same embodiment advances, (C) is the object contained in the 1st area | region and 2nd area | region accompanying advance of a vehicle main body. It is a schematic diagram which shows the displacement of. (A) And (B) is a schematic diagram which shows a mode that the vehicle main body which concerns on the embodiment turns right, Comprising: (C) is the 1st area | region and 2nd area | region accompanying turning of a vehicle main body. It is a schematic diagram which shows the displacement of the to-be-photographed object. It is a figure of the orthogonal coordinate system in which the combination of the displacement amount of the subject included in the first region and the displacement amount of the subject included in the second region is expressed by coordinates. It is a flowchart which shows the flow of the vehicle motion estimation process which concerns on the same embodiment. It is a flowchart which shows the flow of the vehicle motion estimation process which concerns on the same embodiment.

  An embodiment of the present invention will be described with reference to the drawings. In the drawings, a front-rear direction X indicated by an arrow X, a left-right direction Y indicated by an arrow Y, and a vertical direction Z indicated by an arrow Z are directions orthogonal to each other.

As shown in FIG. 1, the reach-type forklift 1 is a cargo handling vehicle including a vehicle body 10, a traveling device 20, a cargo handling device 30, an operation device 40, a camera 50, and the like.
The vehicle main body 10 houses a battery (not shown) that is a power source of the traveling device 20 and the cargo handling device 30. The vehicle main body 10 is provided with a riding section 11 on which an operator (not shown) driving the forklift 1 rides, and a head guard 12 disposed above the riding section 11. The head guard 12 is constituted by a lattice-like structure, and is supported by a stay 12 </ b> A provided to stand on the vehicle main body 10. In FIG. 1B, the head guard 12 is not shown. The vehicle body 10 is configured to be able to travel on the road surface S and to turn when the traveling device 20 operates.

  The traveling device 20 includes a front wheel 21, a rear wheel 22, a traveling motor (not shown) that drives the rear wheel 22, a steering device (not shown) that changes the direction of the rear wheel 22, and the like. The cargo handling device 30 is provided in front of the vehicle main body 10 and includes a mast 31 that can be expanded and contracted in the vertical direction Z and a fork 32 that can be moved up and down along the mast 31. The traveling device 20 and the cargo handling device 30 operate based on an instruction from an operator input using the operation device 40.

  The operating device 40 is provided in the vehicle main body 10 and includes a steering handle 41, a traveling lever 42, and the like. When the operator rotates the steering handle 41, an instruction to change the direction of the rear wheel 22 is input to the forklift 1. Further, when the operator tilts the traveling lever 42 forward or backward, an instruction to drive the rear wheel 22 is input to the forklift 1.

  The camera 50 is provided on the head guard 12 and is composed of an omnidirectional camera capable of photographing a range of 360 ° as viewed from the vertical direction. The camera 50 photographs the road surface S, which is a subject existing below the camera 50, from above the riding section 11. The camera 50 captures the road surface S in color, and acquires a two-dimensional image composed of a plurality of pixels arranged along a first coordinate axis and a second coordinate axis that are orthogonal to each other. The camera 50 repeatedly acquires the above-described image showing the road surface S at a minute time interval (for example, an interval of about 33 milliseconds).

With reference to FIG. 2, the structure for estimating the operation | movement aspect of the vehicle main body 10 from the two-dimensional image (henceforth a "camera image") acquired with the camera 50 is demonstrated.
As shown in FIG. 2, the forklift 1 includes a vehicle motion estimation device 2, a control unit 71, a storage unit 72, and a display unit 73.

  The vehicle motion estimation device 2 includes a camera 50, a preprocessing unit 61, a first region extraction unit 62A, a second region extraction unit 62B, a first displacement amount calculation unit 63A, a second displacement amount calculation unit 63B, and a motion mode estimation unit. 64. The vehicle motion estimation device 2 estimates the motion mode of the vehicle main body 10 using the camera 50.

  The preprocessing unit 61 is configured by an integrated circuit that performs predetermined preprocessing on the camera image. The preprocessing unit 61 performs a process of converting a color image into a grayscale image on the image data related to the camera image. Each of the pixels constituting the gray scale image has a pixel value included in a range of 0 to 255, for example. The pixel value represents the gradation level.

  The first region extraction unit 62A and the second region extraction unit 62B are configured by an integrated circuit that partially extracts image data relating to a camera image. The first region extraction unit 62A extracts a first region W1 (see FIG. 3) that is a part of the camera image from the camera image, and the second region extraction unit 62B is a second region W2 (see FIG. 3) that is a part of the camera image. 3) is extracted from the camera image. That is, the first area extracting unit 62A extracts image data related to the subject included in the first area W1, and the second area extracting unit 62B is image data related to the subject included in the second area W2. To extract. The first region W1 and the second region W2 will be described later with reference to FIG.

  The first displacement amount calculation unit 63A and the second displacement amount calculation unit 63B are configured by an integrated circuit that calculates the displacement amount of the subject based on image data relating to two camera images acquired with a minute time interval. . The first displacement amount calculation unit 63A calculates the displacement amount of the subject included in the first region W1 and the displacement amount in the direction parallel to the first coordinate axis of the camera image (hereinafter referred to as “shift amount Δx”). The second displacement amount calculation unit 63B calculates the displacement amount of the subject included in the second region W2 and the displacement amount in the direction parallel to the second coordinate axis of the camera image (hereinafter referred to as “shift amount Δy”). A specific method for calculating the shift amounts Δx and Δy and the shift amounts Δx and Δy will be described later with reference to FIGS. 4 and 5.

The motion mode estimation unit 64 (hereinafter referred to as “estimation unit 64”) is based on the shift amount Δx calculated by the first displacement amount calculation unit 63A and the shift amount Δy calculated by the second displacement amount calculation unit 63B. Thus, the vehicle main body 10 is configured by an integrated circuit that estimates the operation mode. Specifically, the estimation unit 64 determines that the operation mode of the vehicle body 10 is any one of the following modes 1 to 5 based on the shift amounts Δx and Δy.
[Aspect 1] The vehicle main body 10 is stopped. [Aspect 2] The vehicle main body 10 is moving forward. [Aspect 3] The vehicle main body 10 is moving backward. [Aspect 4] The vehicle main body 10 turns to the right. [Mode 5] A specific estimation method of the operation mode of the vehicle body 10 based on the shift amounts Δx and Δy that the vehicle body 10 is turning to the left will be described later with reference to FIG. Further, when estimating that the vehicle body 10 is moving forward or backward, the estimation unit 64 estimates the traveling speed of the vehicle body 10 based on the shift amount Δx. The traveling speed of the vehicle body 10 is estimated by calculating the product of the shift amount Δx and a predetermined speed conversion rate.

  The control unit 71 includes an integrated circuit that controls the storage unit 72 and the display unit 73 based on the operation mode of the vehicle operation 10 estimated by the vehicle operation estimation device 2. Specifically, the control unit 71 stores the operation mode and the traveling speed of the vehicle main body 10 in the storage unit 72 and the display unit 73. Further, the control unit 71 determines the average traveling speed of the vehicle body 10 based on the traveling speed of the vehicle body 10 stored in the storage unit 72, that is, the past traveling speed of the vehicle body 10 estimated by the vehicle motion estimation device 2. And the average traveling speed is displayed on the display unit 73.

  The storage unit 72 includes a memory that stores the operation mode and the traveling speed of the vehicle body 10 estimated by the vehicle operation estimation device 2. The display unit 73 includes a display that displays the traveling speed of the vehicle body 10 estimated by the vehicle motion estimation device 2.

With reference to FIG. 3, the camera image, the first region W1, and the second region W2 will be described.
The camera image is composed of a plurality of pixels arranged along the x-axis as the first coordinate axis and the y-axis as the second coordinate axis. The x-axis corresponds to the front-rear direction X of the vehicle main body 10, and the y-axis corresponds to the left-right direction Y of the vehicle main body 10.

  In the center of the camera image, an operator who is on boarding section 11 located below head guard 12 is shown. The camera image shows a road surface S on which the vehicle body 10 travels, and a portion of the forklift 1 such as the head guard 12, the stay 12A, and the mast 31.

  Each of the first region W1 and the second region W2 is a region where the road surface S is reflected in the camera image, and is a rectangular region surrounded by two sides parallel to the x axis and two sides parallel to the y axis in the camera image. It is. The first area W1 and the second area W2 are included in the camera image and are separated from each other. The first region W1 corresponds to a place aligned with the vehicle main body 10 on the left side of the vehicle main body 10, and the second region W2 corresponds to a place aligned with the vehicle main body 10 behind the vehicle main body 10.

  With reference to FIG. 4 and FIG. 5, a specific calculation method of the shift amounts Δx and Δy and the shift amounts Δx and Δy will be described. 4 and 5, the locations corresponding to the first region W1 and the second region W2 on the road surface S are simplified to a rectangular shape. As the vehicle body 10 moves, the locations corresponding to the first area W1 and the second area W2 on the road surface S change, but the positions of the first area W1 and the second area W2 in the camera image do not change.

  The shift amount Δx represents the degree to which the subject included in the first region W1 has moved in the direction parallel to the x axis, and represents, for example, the relative movement amount of the virtual point Px in FIGS. 4 and 5 with respect to the vehicle body 10. . The shift amount Δy represents the degree to which the subject included in the second region W2 has moved in the direction parallel to the y-axis. For example, the relative movement amount of the virtual point Py in FIGS. 4 and 5 with respect to the vehicle main body 10. Represents.

  With reference to FIG. 4, the shift amount Δx generated when the vehicle body 10 travels forward will be described. FIG. 4A shows a state immediately before the vehicle main body 10 moves forward, and FIG. 4B shows a state immediately after the vehicle main body 10 moves forward. As shown in FIGS. 4A and 4B, when the vehicle body 10 moves forward, the position of the virtual point Px that is the subject changes in a direction parallel to the x axis in the first region W1 included in the camera image. To do. When the virtual point Px in FIG. 4A is “virtual point Px1” and the virtual point Px in FIG. 4B is “virtual point Px2”, the shift amount Δx is as shown in FIG. , The distance between the virtual point Px1 and the virtual point Px2 in the first region W1 in the direction parallel to the x-axis. When the virtual point Py in FIG. 4A is “virtual point Py1” and the virtual point Py in FIG. 4B is “virtual point Py2”, as shown in FIG. Since there is no interval between the virtual point Py1 and the virtual point Py2 in the direction parallel to the y-axis in the region W2, the shift amount Δy is zero. The shift amount Δx when the vehicle body 10 travels forward can be represented by a positive value, and the shift amount Δx when the vehicle body 10 travels backward can be represented by a negative value.

  Next, the shift amounts Δx and Δy that occur when the vehicle body 10 turns to the right will be described with reference to FIG. FIG. 5A shows a state immediately before the vehicle body 10 turns rightward, and FIG. 5B shows a state immediately after the vehicle body 10 turns rightward. When the vehicle body 10 turns to the right as shown in FIGS. 5A and 5B, the position of the virtual point Px that is the subject changes in the first region W1 included in the camera image, and the camera In the second region W2 included in the image, the position of the virtual point Py that is the subject changes. When the virtual point Px in FIG. 5A is “virtual point Px3” and the virtual point Px in FIG. 5B is “virtual point Px4”, the shift amount Δx is as shown in FIG. 5C. , The distance between the virtual point Px3 and the virtual point Px4 in the first region W1 in the direction parallel to the x-axis. When the virtual point Py in FIG. 5A is “virtual point Py3” and the virtual point Py in FIG. 5B is “virtual point Py4”, the shift amount Δy is shown in FIG. 5C. Thus, the interval in the direction parallel to the y-axis between the virtual point Py3 and the virtual point Py4 in the second region W2 is represented. The shift amounts Δx and Δy when the vehicle body 10 turns to the right can be expressed as positive values, and the shift amounts Δx and Δy when the vehicle body 10 turns to the left are respectively negative values. Can be expressed as

  As described above, the shift amount Δx can be represented by the displacement amount of the x coordinate of the virtual point Px, the shift amount Δy can be represented by the displacement amount of the y coordinate of the virtual point Py, and the shift amounts Δx and Δy are It can be represented by the number of pixels. In order to calculate the shift amount Δx, the first displacement amount calculation unit 63A attempts to match the subject appearing in the two first regions W1 extracted at a minute time interval. Specifically, the first displacement amount calculation unit 63A calculates one of the two first regions W1 in a direction parallel to the x-axis to calculate the degree of coincidence of the subject captured in the two first regions W1. . Then, the first displacement amount calculation unit 63A sets the movement amount of the first region W1 where the degree of coincidence of the subject is high as the shift amount Δx. In addition, in order to calculate the shift amount Δy, the second displacement amount calculation unit 63B attempts to match the subject appearing in the two second regions W2 extracted at a minute time interval. Specifically, the second displacement amount calculation unit 63B moves one of the two second regions W2 in a direction parallel to the y-axis to calculate the degree of coincidence of the subject captured in the two second regions W2. . Then, the second displacement amount calculation unit 63B sets the movement amount of the second region W2 where the degree of coincidence of the subject is high as the shift amount Δy. The degree of coincidence of the subject appearing in the two first regions W1 or the two second regions W2 can be calculated based on the difference between the pixel values. Further, the first displacement amount calculation unit 63A and the second displacement amount calculation unit 63B, when the subject cannot be matched, that is, when the matching degree of the subject is low, the shift amounts Δx and Δy are invalid values (for example, 99). Is calculated.

With reference to FIG. 6, the specific estimation method of the operation | movement aspect of the vehicle main body 10 based on shift amount (DELTA) x and (DELTA) y is demonstrated.
When the coordinates expressed by the combination of the shift amounts Δx and Δy (hereinafter referred to as “shift amount coordinates”) are located in the stop determination region shown in FIG. 6, it is estimated that the vehicle body 10 is stopped. Further, when the shift amount coordinates are located in the forward determination area shown in FIG. 6, it is estimated that the vehicle main body 10 is moving forward. When the shift amount coordinates are located in the reverse determination area shown in FIG. 6, the vehicle main body 10 moves backward. It is estimated that Further, when the shift amount coordinates are located in the right turn determination area shown in FIG. 6, it is estimated that the vehicle main body 10 is turning right, and when the shift amount coordinates are located in the left turn determination area shown in FIG. It is estimated that the main body 10 is turning leftward.

Specifically, when the shift amount Δx is 0, which is a predetermined stop determination amount, the estimating unit 64 determines that the vehicle body 10 is stopped without traveling in the front-rear direction X regardless of the shift amount Δy. presume. In addition, the estimation unit 64 determines which of the following conditions 1 to 5 is satisfied on the assumption that the shift amounts Δx and Δy are valid values and that the shift amount Δx is not a stop determination amount. The operation mode of the vehicle body 10 is estimated. The effective value of the shift amount Δx is a value included in the range of xmn to xmx (see FIG. 6), and the effective value of the shift amount Δy is a value included in the range of ymn to ymx (see FIG. 6). It is. Further, under the following conditions 1 to 5, the predetermined straight-ahead determination amount is a value included in the range of −ya to + ya (see FIG. 6), and the turn rate is the ratio of the shift amount Δy to the shift amount Δx (that is, Δy / Δx), the predetermined determination lower limit value is −0.5, the predetermined determination boundary value is 0.5, and the predetermined determination upper limit value is 1.5.
[Condition 1] The shift amount Δx is a positive value, and the shift amount Δy is a straight traveling determination amount. [Condition 2] The shift amount Δx is a positive value, and the turn rate is greater than or equal to the determination lower limit value. And the turn rate is less than the determination boundary value [Condition 3] The shift amount Δx is a positive value, the shift amount Δy is not a straight-ahead determination value, and the turn rate is greater than or equal to the determination boundary value. [Condition 4] The turn rate is equal to or less than the determination upper limit [Condition 4] The shift amount Δx is a negative value, and the shift amount Δy is a straight travel determination amount. [Condition 5] The shift amount Δx is a negative value. The turn rate is equal to or greater than the determination lower limit value, and the turn rate is less than the determination boundary value. [Condition 6] The shift amount Δx is a negative value, and the shift amount Δy is not a straight traveling determination value. , The turn rate is equal to or higher than the judgment boundary value, and Turn rate is below the determination limit

  The estimation unit 64 estimates that the vehicle body 10 is moving forward when the condition 1 or the condition 2 is satisfied, and estimates that the vehicle body 10 is turning right when the condition 3 is satisfied. The estimation unit 64 estimates that the vehicle main body 10 is moving backward when the condition 4 or the condition 5 is satisfied, and estimates that the vehicle main body 10 is turning to the left when the condition 6 is satisfied. . The range of xmn to xmx and the range of ymn to ymx are, for example, a range of -20 to +20 [unit: pixel]. Further, the range of −ya to + ya, which is the straight traveling determination amount, is, for example, a range of −3 to +3 [unit: pixel]. Since the fact that the vehicle body 10 is moving forward or backward is estimated based not only on the shift amount Δy but also on the turn rate, as shown in FIG. 6, the shift amount Δx is in the range of xmn to −xa and + xa to In the range of xmx, it is estimated that the vehicle body 10 is moving forward or backward also when the shift amount Δy is not the straight travel determination amount (that is, when the shift amount Δy is not included in the range of −ya to + ya). There is also.

  With reference to FIG. 7 and FIG. 8, the flow of the vehicle motion estimation process performed by the vehicle motion estimation device 2 will be described. Note that the vehicle motion estimation process shown in FIG. 7 starts on the assumption that the first region W1 and the second region W2 are extracted from two two-dimensional images acquired at a minute time interval.

  As shown in FIG. 7, the first displacement amount calculation unit 63A and the second displacement amount calculation unit 63B calculate shift amounts Δx and Δy (step S10), and then the estimation unit 64 determines that the shift amount Δx is an effective value. It is determined whether or not (step S1). When it is determined in step S1 that the shift amount Δx is an invalid value, the process of step S31 described later is performed.

  When it is determined in step S1 that the shift amount Δx is an effective value, the estimation unit 64 determines whether or not the shift amount Δx is a stop determination amount (step S2). When it is determined in step S2 that the shift amount Δx is the stop determination amount, the estimation unit 64 estimates that the vehicle body 10 is stopped (step S11).

  When it is determined in step S2 that the shift amount Δx is not the stop determination amount, the estimation unit 64 determines whether or not the shift amount Δy is an effective value (step S3). If it is determined in step S3 that the shift amount Δy is an invalid value, the process of step S31 described later is performed.

  When it is determined in step S3 that the shift amount Δy is an effective value, the estimation unit 64 determines whether or not the shift amount Δy is a straight travel determination amount (step S4). When it is determined in step S4 that the shift amount Δy is a straight traveling determination amount, a process in step S8 described later is performed.

  When it is determined in step S4 that the shift amount Δy is not the straight traveling determination amount, the estimation unit 64 determines whether or not the turn rate is a turning determination amount (step S5). The turning determination amount is a value included in the range from the above-described determination boundary value to the determination upper limit value, that is, in the range of 0.5 to 1.5. When it is determined in step S5 that the turn rate is not the turning determination amount, that is, when it is determined that the turn rate is not included in 0.5 to 1.5, the process of step S7 described later is performed.

  When it is determined in step S5 that the turn rate is the turning determination amount, that is, when it is determined that the turn rate is included in 0.5 to 1.5, the estimation unit 64 has a positive shift amount Δx. It is determined whether or not there is (step S6). When it is determined in step S6 that the shift amount Δx is a positive value, the estimation unit 64 estimates that the vehicle body 10 is turning right (step S12). On the other hand, when it is determined in step S6 that the shift amount Δx is a negative value, the estimation unit 64 estimates that the vehicle body 10 is turning left (step S13).

  As shown in FIG. 8, when it is determined in step S5 that the turn rate is not the turn determination amount, the estimation unit 64 determines whether or not the turn rate is a fine turn determination amount (step S7). The fine turning determination amount is a value included in the range from the above-described determination lower limit value to the determination boundary value, that is, in the range of −0.5 to 0.5. When it is determined in step S7 that the turn rate is not the slight turning determination amount, that is, when it is determined that the turn rate is not included in -0.5 to 0.5, the process of step S31 described later is performed.

  If it is determined in step S4 that the shift amount Δy is a straight travel determination amount, and if it is determined in step S7 that the turn rate is a slight turn determination amount, that is, the turn rate is -0.5 to 0.5. When it is determined that the shift amount Δx is included, the estimation unit 64 determines whether or not the shift amount Δx is a positive value (step S8). When it is determined in step S8 that the shift amount Δx is a positive value, the estimation unit 64 estimates that the vehicle body 10 is moving forward (step S14). On the other hand, when it is determined in step S8 that the shift amount Δx is a negative value, the estimation unit 64 estimates that the vehicle body 10 is moving backward (step S15).

  If it is estimated in steps S14 and S15 that the vehicle body 10 is moving forward or backward, the estimation unit 64 estimates the traveling speed of the vehicle body 10 based on the shift amount Δx (step S21).

  When it is determined in steps S1 and S3 that the shift amount Δx or the shift amount Δy is an invalid value, and when it is determined in step S7 that the turn rate is a slight turning determination amount, the estimation unit 64 performs step S10. It is prohibited to estimate the operation mode of the vehicle main body 10 based on the shift amounts Δx and Δy calculated in step. In this case, the estimation unit 64 estimates that the operation mode of the vehicle main body 10 has not changed. That is, based on the shift amounts Δx and Δy whose shift amount coordinates are not located in any of the stop determination region, forward determination region, reverse determination region, right turn determination region, and left turn determination region shown in FIG. The operation mode of the vehicle body 10 is not estimated.

In the above embodiment, the following effects can be obtained.
(1) The vehicle motion estimation device 2 includes a camera 50, a first region extraction unit 62A, a second region extraction unit 62B, a first displacement amount calculation unit 63A, a second displacement amount calculation unit 63B, and an operation mode. And an estimation unit 64. According to this configuration, the operation mode of the vehicle main body 10 is based on the displacement amount (shift amount Δx, Δy) of the subject included in the first region W1 and the second region W2, which are part of the two-dimensional image acquired by the camera 50. Therefore, it is possible to reduce the load for estimating the operation mode of the vehicle main body 10 as compared with the configuration in which the operation mode of the vehicle main body 10 is estimated based on the total displacement amount of the two-dimensional image. It is. The operation of the vehicle main body 10 is also based on the displacement amount (shift amount Δx) in the direction parallel to the first coordinate axis (x axis) and the displacement amount (shift amount Δy) in the direction parallel to the second coordinate axis (y axis). Since the mode is estimated, the mode of operation of the vehicle main body 10 can be estimated with higher accuracy than the configuration in which the mode of operation of the vehicle main body 10 is estimated based on the one-dimensional pattern converted from the two-dimensional image. Therefore, it is possible to achieve both reduction of the load for estimating the operation mode of the vehicle body 10 and improvement of the estimation accuracy. This makes it possible to obtain information on the position and traveling speed of the vehicle main body 10 even in a warehouse without using GPS or radio equipment.

  (2) The first region extraction unit 62A and the second region extraction unit 62B are surrounded by two sides parallel to the first coordinate axis (x axis) and two sides parallel to the second coordinate axis (y axis) in the two-dimensional image. Rectangular regions (first region W1 and second region W2) to be extracted are extracted. According to this configuration, the displacement amount of the subject included in the first area W1 and the displacement amount (shift amount Δx) in the direction parallel to the first coordinate axis, and the displacement amount of the subject included in the second area W2. Thus, the displacement amount (shift amount Δy) in the direction parallel to the second coordinate axis can be calculated by a simple calculation.

  (3) When the displacement amount (shift amount Δx) calculated by the first displacement amount calculation unit 63A is the predetermined stop determination amount (0), the motion mode estimation unit 64 travels in the front-rear direction X. Estimated to stop without. According to this configuration, even when the displacement amount (shift amount Δy) of the subject included in the second region W2 cannot be calculated with high accuracy, it is determined whether or not the vehicle main body 10 is stopped. It can be estimated based on the amount of displacement of the subject included in W1.

  (4) The motion mode estimation unit 64 uses the first displacement amount calculation unit 63A when the displacement amount (shift amount Δx) calculated by the first displacement amount calculation unit 63A is not the predetermined stop determination amount (0). When the ratio (turn rate) of the displacement amount (shift amount Δy) calculated by the second displacement amount calculation unit 63B to the calculated displacement amount is less than a predetermined determination boundary value, the vehicle body 10 travels forward or backward. Estimate that According to this configuration, the amount of displacement of the subject in the direction parallel to the first coordinate axis (x axis) is sufficiently larger than the amount of displacement of the subject in the direction parallel to the second coordinate axis (y axis). It can be estimated that the main body 10 is traveling forward or backward.

  (5) The motion mode estimation unit 64 uses the first displacement amount calculation unit 63A when the displacement amount (shift amount Δx) calculated by the first displacement amount calculation unit 63A is not the predetermined stop determination amount (0). When the ratio (turn rate) of the displacement amount (shift amount Δy) calculated by the second displacement amount calculation unit 63B to the calculated displacement amount is equal to or greater than a predetermined determination boundary value, the vehicle body 10 is on the right side or the left side. It is estimated that the car is turning. According to this configuration, it is possible to estimate that the vehicle body 10 is turning rightward or leftward by increasing the amount of displacement of the subject in the direction parallel to the second coordinate axis (y-axis).

  (6) The forklift 1 includes a vehicle main body 10 and a vehicle motion estimation device 2, and the camera 50 is provided on the head guard 12 and is configured by an omnidirectional camera that can capture a 360 ° range as viewed from the vertical direction. Has been. According to this configuration, a wide range of road surfaces including locations corresponding to the first area W1 and the second area W2 can be captured without using a plurality of cameras by photographing the lower part of the head guard 12 with the camera 50. 2D images can be acquired.

  The present invention is not limited to the above-described embodiment, and the configuration of the above-described embodiment can be changed as appropriate. For example, the above configuration can be changed as follows, and the following changes can be combined.

  If the reduction of the load for estimating the operation mode of the vehicle main body 10 and the improvement of the estimation accuracy can be achieved, the flow of the vehicle operation estimation process and the above modes 1 to 5 are estimated. These conditions may be changed as appropriate. For example, the stop determination amount may be a value included in a predetermined range including 0, the determination lower limit value, the determination boundary value, and the determination upper limit value may be appropriately changed. The range of the turning determination amount may be changed as appropriate.

  If the amount of displacement of the subject in the direction parallel to the first coordinate axis (x axis) can be calculated, the arrangement, shape, and size of the first region W1 may be changed as appropriate. In addition, the arrangement, shape, and size of the second region W2 may be appropriately changed as long as the amount of displacement of the subject in the direction parallel to the second coordinate axis (y-axis) can be calculated. For example, the first region W1 can be made to correspond to a place aligned with the vehicle body 10 on the right side of the vehicle body 10, and the second region W2 can be made to correspond to a place aligned with the vehicle body 10 in front of the vehicle body 10. You can also. In other words, the first region W1 extracted by the first region extraction unit 62A only needs to correspond to a place aligned with the vehicle body 10 in at least one of the vehicle body 10 in the left-right direction Y. Further, the second region W2 extracted by the second region extraction unit 62B only needs to correspond to a place in the vehicle body 10 in at least one of the vehicle body 10 in the front-rear direction X.

  If the road surface S that is the subject can be photographed, the configuration related to the camera 50 may be changed as appropriate. For example, the arrangement and the number of cameras 50 can be changed. Moreover, the camera 50 can also be comprised by cameras other than an omnidirectional camera.

  -The structure which concerns on the vehicle main body 10, the traveling apparatus 20, the cargo handling apparatus 30, and the operating device 40 of the forklift 1 can also be changed suitably. For example, the present invention can be applied to a counterbalance type forklift, and the present invention can also be applied to a cargo handling vehicle that does not include the fork 32.

1 Forklift (handling vehicle)
2 Vehicle Motion Estimation Device 10 Vehicle Main Body 11 Riding Section 12 Head Guard 20 Traveling Device 21 Front Wheel 22 Rear Wheel 30 Cargo Handling Device 31 Mast 32 Fork 40 Operating Device 41 Steering Handle 42 Traveling Lever 50 Camera 61 Preprocessing Unit 62A First Area Extraction Unit 62B second region extraction unit 63A first displacement amount calculation unit 63B second displacement amount calculation unit 64 operation mode estimation unit 71 control unit 72 storage unit 73 display unit S road surface (subject)
W1 First region W2 Second region X Front-rear direction Y Left-right direction Z Vertical direction Δx Shift amount (displacement amount of the subject included in the first region and in a direction parallel to the first coordinate axis)
Δy shift amount (displacement amount of the subject included in the second region and displacement amount in a direction parallel to the second coordinate axis)

Claims (6)

A vehicle motion estimation device for estimating an operation mode of a vehicle main body capable of traveling and turning on a road surface,
A two-dimensional image composed of a plurality of pixels lined up along a first coordinate axis corresponding to the front-rear direction of the vehicle body and a second coordinate axis corresponding to the left-right direction of the vehicle body by photographing the road surface that is the subject With a camera to get
A first region extraction unit that extracts a first region corresponding to a location on the vehicle main body in at least one of the left and right directions of the vehicle main body from the two-dimensional image acquired by the camera;
A second region extraction unit that extracts a second region corresponding to a location on the vehicle main body in at least one of the front and rear directions of the vehicle main body from the two-dimensional image acquired by the camera;
A first displacement amount calculation unit for calculating a displacement amount of a subject included in the first region and in a direction parallel to the first coordinate axis;
A second displacement amount calculation unit for calculating a displacement amount of a subject included in the second region and in a direction parallel to the second coordinate axis;
An operation mode estimation unit that estimates an operation mode of the vehicle body based on the displacement amount calculated by the first displacement amount calculation unit and the displacement amount calculated by the second displacement amount calculation unit ;
The camera is composed of an omnidirectional camera capable of photographing a 360 ° range as viewed from the vertical direction,
The vehicle motion estimation device, wherein the first region extraction unit and the second region extraction unit extract the first region and the second region from the two-dimensional image acquired by the same omnidirectional camera. . The first region extraction unit and the second region extraction unit extract a rectangular region surrounded by two sides parallel to the first coordinate axis and two sides parallel to the second coordinate axis in the two-dimensional image. The vehicle motion estimation apparatus according to claim 1. The operation mode estimation unit estimates that the vehicle main body is stopped without traveling in the front-rear direction when the displacement amount calculated by the first displacement amount calculation unit is a predetermined stop determination amount. The vehicle motion estimation apparatus according to claim 1, wherein: The operation mode estimation unit is configured such that the displacement amount calculated by the first displacement amount calculation unit is not a predetermined stop determination amount and the second displacement amount with respect to the displacement amount calculated by the first displacement amount calculation unit. The vehicle motion estimation according to claim 3, wherein when the ratio of the displacement amount calculated by the calculation unit is less than a predetermined determination boundary value, it is estimated that the vehicle body is traveling forward or backward. apparatus. The operation mode estimation unit is configured such that the displacement amount calculated by the first displacement amount calculation unit is not a predetermined stop determination amount and the second displacement amount with respect to the displacement amount calculated by the first displacement amount calculation unit. The vehicle body is estimated to be turning right or left when the ratio of the amount of displacement calculated by the calculation unit is equal to or greater than a predetermined determination boundary value. Vehicle motion estimation device. The vehicle body provided with a head guard;
A cargo handling vehicle comprising the vehicle motion estimation device according to any one of claims 1 to 5,
The cargo handling vehicle, wherein the camera is provided on the head guard.

Images