JP7533402B2 - Attitude estimation device, industrial vehicle, attitude estimation program, and storage medium - Google Patents

Description

The present disclosure relates to a posture estimation device, an industrial vehicle, a posture estimation program, and a storage medium.

A posture estimation method is known in which a marker is photographed by a camera and the posture between the marker and the camera is estimated based on the apparent distortion of the marker. In conventional posture estimation methods, for example, when a marker is photographed near a position where the marker and the camera are directly facing each other, the apparent distortion of the marker becomes small, and the accuracy of the posture estimation result may decrease. A posture estimation method described in Patent Document 1 is a conventional method for obtaining a highly accurate posture estimation result. A standard variable moiré pattern and a high-sensitivity variable moiré pattern that has a higher gaze angle resolution than the standard variable moiré pattern are arranged around the marker described in Patent Document 1. In this posture estimation method, the gaze angle is estimated based on a gaze angle tentatively determined using the standard variable moiré pattern and a candidate value for the gaze angle determined using the high-sensitivity variable moiré pattern.

International Publication No. 2018/135063

The marker described in Patent Document 1 uses a lenticular lens or lens array as two types of variable moiré patterns, which increases manufacturing costs and makes it difficult to increase the size of the marker. On the other hand, there is still an issue with posture estimation using general markers, which may reduce the accuracy of the posture estimation results.

The present disclosure has been made to solve the above problems, and aims to provide a posture estimation device, an industrial vehicle equipped with a posture estimation device, a posture estimation program, and a storage medium storing a posture estimation program, that can ensure that posture estimation results have a certain level of accuracy while suppressing increases in manufacturing costs.

A posture estimation device according to one aspect of the present disclosure includes a recognition unit that recognizes a marker from a captured image of the marker and acquires original coordinates of the marker in the captured image, a generation unit that generates shifted coordinates by shifting the coordinates of at least one point of the marker from the original coordinates, an estimation unit that generates a first posture estimation result using the original coordinates and a second posture estimation result using the shifted coordinates, and a determination unit that determines whether the first posture estimation result is reliable based on a comparison between the second posture estimation result and a preset threshold.

In this posture estimation device, at least a portion of the original coordinates is shifted to generate shifted coordinates. The markers indicated by the shifted coordinates appear to be more distorted than the markers indicated by the original coordinates. In this posture estimation device, the reliability of the first posture estimation result can be accurately determined by comparing the second posture estimation result using the shifted coordinates with a threshold value. Furthermore, this posture estimation device only needs to consider the computational costs required to calculate the shifted coordinates and compare them with the threshold value, and does not require processing of the markers themselves, thereby suppressing increases in manufacturing costs.

The generation unit may generate multiple shifted coordinates by shifting the coordinates of the four corners of the marker from the original coordinates. In this case, it becomes easier to identify the shape of the marker (rectangle), and the processing time required for the second orientation estimation result is reduced.

The generation unit may generate multiple shifted coordinates by moving each of the coordinates of the four corners of the marker from the original coordinates in the positive and negative directions of the X-axis and the positive and negative directions of the Y-axis. In this case, multiple shifted coordinates are generated by moving each of the coordinates of the four corners of the marker in four directions. By comparing multiple second posture estimation results with a threshold, it is possible to more accurately determine whether the first posture estimation result is reliable.

The generation unit may move each of the coordinates of the four corners of the marker by one pixel. In this case, the change in distortion of one pixel is reflected in the second posture estimation result. This makes it possible to further improve the accuracy of determining whether the first posture estimation result is reliable while minimizing the movement of the coordinates.

The determination unit may calculate the difference between the maximum and minimum values in the second posture estimation result and compare the difference with a threshold. In this case, the accuracy of determining whether the first posture estimation result is reliable can be ensured with a simple calculation.

The determination unit may calculate the variance of the second posture estimation result and compare the variance with a threshold. In this case, the accuracy of determining the reliability of the first posture estimation result can be ensured with a simple calculation.

When the determination unit determines that the first posture estimation result is unreliable, the determination unit may generate an assumed estimation result by excluding values outside a threshold from the second posture estimation result and calculating the average value of the excluded values. In this case, it is possible to obtain a posture estimation result from an assumed estimation result that has a certain degree of reliability while omitting the need to reacquire the original coordinates of the markers.

When the determination unit determines that the first posture estimation result is unreliable, the determination unit may discard the first posture estimation result. In this case, it is possible to prevent an unreliable first posture estimation result from being obtained as a posture estimation result.

The determination unit may determine whether the first posture estimation result is reliable based on a comparison of the first posture estimation result and the second posture estimation result with a threshold. By further using the first posture estimation result itself in determining whether the result is reliable, it is possible to further improve the accuracy of determining whether the first posture estimation result is reliable.

The estimation unit may generate at least one of the roll angle, pitch angle, yaw angle, and position information as the first attitude estimation result and the second attitude estimation result. In this case, the information necessary for attitude estimation can be appropriately obtained.

An industrial vehicle according to one aspect of the present disclosure is equipped with a camera that photographs a marker and the above-mentioned posture estimation device. In this industrial vehicle, at least a portion of the original coordinates is moved to generate shifted coordinates. The marker represented by the shifted coordinates appears more distorted than the marker represented by the original coordinates. In this industrial vehicle, the reliability of the first posture estimation result can be accurately determined by comparing the second posture estimation result using the shifted coordinates with a threshold value. Furthermore, with this industrial vehicle, it is only necessary to consider the calculation cost required to calculate the shifted coordinates and compare them with the threshold value, and since processing of the marker itself is not required, an increase in manufacturing costs can be suppressed.

A posture estimation program according to one aspect of the present disclosure causes a computer to execute the steps of recognizing a marker from a captured image of the marker and acquiring original coordinates of the marker in the captured image, generating shifted coordinates by shifting the coordinates of at least one point of the marker from the original coordinates, generating a first posture estimation result using the original coordinates and a second posture estimation result using the shifted coordinates, and determining whether the first posture estimation result is reliable based on a comparison between the second posture estimation result and a preset threshold. Also, a storage medium according to one aspect of the present disclosure is a computer-readable storage medium storing the posture estimation program.

In this posture estimation program and storage medium, at least a portion of the original coordinates is shifted to generate shifted coordinates. The marker indicated by the shifted coordinates appears more distorted than the marker indicated by the original coordinates. In this posture estimation program and storage medium, the reliability of the first posture estimation result can be accurately determined by comparing the second posture estimation result using the shifted coordinates with a threshold value. Furthermore, with this posture estimation program and storage medium, it is only necessary to consider the calculation cost required to calculate the shifted coordinates and compare them with the threshold value, and processing of the markers themselves is not required, so increases in manufacturing costs can be suppressed.

This disclosure makes it possible to ensure that posture estimation results have a certain level of accuracy while suppressing increases in manufacturing costs.

FIG. 1 is a block diagram showing a configuration of a posture estimation device. FIG. 13 is a diagram showing an example of a captured image of a marker. FIG. 13 is a diagram illustrating an example of a method for generating shift coordinates. 11A and 11B are diagrams illustrating an example of a first posture estimation result and a second posture estimation result. 13A and 13B are diagrams illustrating another example of the first and second posture estimation results. 10 is a flowchart illustrating an example of an operation of the posture estimation device. 10 is a flowchart showing another example of the operation of the posture estimation device. FIG. 4 is a diagram illustrating a configuration of a posture estimation program and a storage medium.

Below, preferred embodiments of the posture estimation device, industrial vehicle, posture estimation program, and storage medium relating to one aspect of the present disclosure will be described in detail with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of a posture estimation device according to an embodiment of the present disclosure. The posture estimation device 10 is a device that estimates the posture of an industrial vehicle 1 on which the posture estimation device 10 is mounted, based on a captured image of a marker M. The marker M is, for example, a rectangular planar marker such as an AR marker (see FIG. 2). The marker M is, for example, attached in advance to an object present around the industrial vehicle 1 by printing or pasting. An example of an industrial vehicle 1 to which the posture estimation device 10 is applied is a forklift. The industrial vehicle 1 is equipped with a camera 20 for capturing an image of the marker M. The camera 20 captures an image of the marker M and outputs data of the captured image to the posture estimation device 10.

The posture estimation device 10 is an electronic control unit having a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), etc. In the posture estimation device 10, for example, a program stored in the ROM is loaded into the RAM, and the program loaded into the RAM is executed by the CPU, thereby realizing various functions. The posture estimation device 10 has, as functional elements, a recognition unit 11, a generation unit 12, an estimation unit 13, a determination unit 14, and an output unit 15.

The recognition unit 11 recognizes the marker M from the captured image of the marker M. For example, the recognition unit 11 detects the presence or absence of the marker M by pattern matching or edge detection on the captured image. The method of recognizing the marker M is not limited to the above, and may be, for example, a method of recognizing the marker M by extracting contours and binarizing the input image and extracting connected components (a set of pixels) from the binarized image.

The recognition unit 11 acquires the original coordinates of the marker M in the captured image. The image coordinate system in the captured image is represented, for example, by XY coordinates with the upper left corner of the captured image as the origin. The image coordinate system can also be represented by pixel values. For example, the recognition unit 11 acquires the coordinates of the four corners of the marker M in the captured image as the original coordinates. The coordinates of the four corners refer to the coordinates corresponding to the vertices of each of the four corners. The original coordinates are not limited to the coordinates of the four corners as long as they are coordinates that can identify the shape of the marker M. For example, the marker M may be a polygon (such as an L-shape). The original coordinates may be coordinates corresponding to the vertices in eight directions (upper, upper right, right, lower right, lower, lower left, left, upper left) of the pixels representing the polygonal marker M.

The generation unit 12 generates shifted coordinates by shifting the coordinates of at least one point of the marker M from the original coordinates. "Shifted coordinates" are new coordinates generated by varying the values of the original coordinates in the coordinate system of the image. For example, the generation unit 12 generates multiple shifted coordinates by moving each of the coordinates of the four corners of the marker M from the original coordinates in the positive and negative directions of the X axis and the positive and negative directions of the Y axis. At this time, the generation unit 12 may move each of the coordinates of the four corners of the marker M by one pixel. A shifted marker is formed by connecting each of the shifted coordinates. The shifted marker is the marker M in the captured image to which an apparent distortion has been added.

The estimation unit 13 generates a first attitude estimation result using the original coordinates and a second attitude estimation result using the shifted coordinates. For example, the estimation unit 13 generates at least one of the roll angle, pitch angle, yaw angle, and position information as the first attitude estimation result and the second attitude estimation result.

The determination unit 14 determines whether the first posture estimation result is reliable based on a comparison between the second posture estimation result and a preset threshold. The determination unit 14 calculates a threshold in advance according to a method for generating shifted coordinates by, for example, moving by one pixel, and stores the threshold. The determination unit 14 may calculate a difference between a maximum value and a minimum value in the second posture estimation result, and compare the difference with the threshold. The determination unit 14 may calculate a variance in the second posture estimation result, and compare the variance with the threshold. The determination unit 14 may determine whether the first posture estimation result is reliable based on a comparison between the first posture estimation result and the second posture estimation result, and the threshold.

If the first posture estimation result is reliable, it indicates that the first posture estimation result has a certain level of accuracy. On the other hand, if the first posture estimation result is unreliable, it indicates that the first posture estimation result does not have a certain level of accuracy. If the determination unit 14 determines that the first posture estimation result is unreliable, it may exclude values outside a threshold from the second posture estimation result and calculate the average value of the excluded values to generate a deemed estimation result. If the determination unit 14 determines that the first posture estimation result is unreliable, it may discard the first posture estimation result.

The output unit 15 outputs the first attitude estimation result or the deemed estimation result to an ECU (Electronic Control Unit) or other devices such as an automatic driving control device. The output unit 15 may also transmit the result to other systems, etc. via a communication network.

The captured image P shown in FIG. 2 is an example of a captured image of the marker M. As shown in FIG. 2, the marker M is rectangular and has a planar pattern made up of white and black areas. The apparent distortion of the marker M in the captured image P changes depending on the positional relationship between the camera 20 and the marker M. The captured image P in FIG. 2 is an example in which the camera 20 captures the marker M from almost the front.

The coordinate system of the image in the captured image P has the upper left corner as the origin, the right direction is the positive direction of the X axis, and the downward direction is the positive direction of the Y axis. The recognition unit 11 acquires the coordinates of the four corners of the marker M in the captured image P as the original coordinates. For example, the recognition unit 11 acquires points C1 ( _x1 , _y1 ), C2 ( _x2 , _y2 ), C3 ( _x3 , _y3 ), and C4 ( _x4 , _y4 ) as the original coordinates.

Next, an example of a method for generating shifted coordinates will be described with reference to FIG. 3. FIG. 3 shows an example in which the coordinates of one of the four corners are moved in four directions. The four directions refer to the positive and negative directions of the X-axis and the positive and negative directions of the Y-axis in the image coordinate system. The dashed frames G1, G2, G3, and G4 show patterns in which only point C1, only point C2, only point C3, or only point C4 of the original coordinates are shifted in the four directions, respectively.

In the dashed frame G1, the shaded circle indicates point C1, the arrow indicates the direction in which point C1 is shifted, and the rectangle indicates a shift marker formed by shifting point C1. Four shift coordinates are generated by shifting point C1 in each of the four directions. Four shift markers are formed within the dashed frame G1.

The method of generating shifted coordinates described for dashed frame G1 is also applied to dashed frame G2, G3, and G4. For example, in dashed frame G2, four shifted coordinates are generated by shifting only point C2 in each of the four directions. A total of 16 shifted coordinates are generated for dashed frame G3 and dashed frame G4 by the same process.

Figure 4 is a diagram showing an example of the first posture estimation result R1 and the second posture estimation result R2. The estimation unit 13 generates a first posture estimation result R1a using the original coordinates and a second posture estimation result R2a using multiple shifted coordinates. The first posture estimation result R1a and the second posture estimation result R2a are, for example, yaw angle values. In the example of Figure 4, the value of the first posture estimation result R1a is [2]. The second posture estimation result R2a includes 16 values [1, 1, 5, 2, 3, 2, 1, 1, 3, 0, 4, 2, 1, 2, 3, 1]. The maximum value of the second posture estimation result R2a is [5] and the minimum value is [0].

Figure 5 shows another example of the first posture estimation result R1 and the second posture estimation result R2. The estimation unit 13 generates a first posture estimation result R1b using the original coordinates and a second posture estimation result R2b using multiple shifted coordinates. The first posture estimation result R1b and the second posture estimation result R2b are, for example, yaw angle values. In the example of Figure 5, the value of the first posture estimation result R1b is [2]. The second posture estimation result R2b includes 16 values [2, 4, 8, 3, 4, 8, 2, 3, 3, 2, 2, 5, 2, 1, 3, 14]. The maximum value of the second posture estimation result R2b is [14] and the minimum value is [1].

An example of the operation of the posture estimation device 10 will be described with reference to FIG. 6. FIG. 6 is a flowchart showing the operation of the posture estimation device 10 as a processing flow S1. First, the camera 20 of the industrial vehicle 1 acquires a captured image P of the marker M shown in FIG. 2 (step S11).

The recognition unit 11 recognizes the marker M from the captured image P of the marker M (step S12). If the marker M is present in the captured image P (YES in step S13), the process proceeds to step S14. On the other hand, if the marker M is not present in the captured image P (NO in step S13), the subsequent processes end. The process flow S1 is executed again, for example, after a predetermined time has elapsed. In one example, the process flow S1 is executed periodically by executing the process of step S11 at a predetermined time interval.

The recognition unit 11 acquires the original coordinates of the marker M in the captured image P (step S14). For example, the recognition unit 11 acquires points C1 ( _x1 , _y1 ), C2 ( _x2 , _y2 ), C3 ( _x3 , _y3 ), and C4 ( _x4 , _y4 ) as the original coordinates.

The generating unit 12 generates shifted coordinates by shifting the coordinates of at least one point of the marker M from the original coordinates (step S15). For example, the generating unit 12 generates a plurality of shifted coordinates by moving each of the coordinates of the four corners of the marker M from the original coordinates in the positive and negative directions of the X-axis and the positive and negative directions of the Y-axis. The generating unit 12 may move each of the coordinates of the four corners of the marker M by one pixel. In one example, the generating unit 12 moves the point C1 (x ₁ , y ₁ ) to the coordinate (x ₁ -1, y ₁ ) at a position shifted by one pixel in the negative direction of the X-axis. The generating unit 12 generates 16 shifted coordinates by shifting only the point C1, only the point C2, only the point C3, or only the point C4 of the original coordinates in the four directions.

The estimation unit 13 generates a first attitude estimation result R1 using the original coordinates and a second attitude estimation result R2 using the shifted coordinates (step S16). The estimation unit 13 generates at least one of the roll angle, pitch angle, yaw angle, and position information as the first attitude estimation result R1 and the second attitude estimation result R2.

The determination unit 14 determines whether the first posture estimation result R1 is reliable based on a comparison between the second posture estimation result R2 and a preset threshold (step S17). The determination unit 14 holds a preset threshold. Here, the threshold is [5]. The determination unit 14 compares the second posture estimation result R2 with the threshold, and determines that the first posture estimation result R1 is reliable if all values included in the second posture estimation result R2 are equal to or less than the threshold, and determines that the first posture estimation result R1 is unreliable if any one of the values included in the second posture estimation result R2 exceeds the threshold.

In step S17, the determination unit 14 may calculate the difference between the maximum and minimum values in the second posture estimation result R2, and compare the difference with a threshold value. In the example of FIG. 4, the difference between the maximum value [5] and the minimum value [0] in the second posture estimation result R2a is calculated to be [5]. In this example, since the difference [5] is equal to or smaller than the threshold value [5], the determination unit 14 determines that the first posture estimation result R1a is reliable. In the example of FIG. 5, since the difference [13] exceeds the threshold value [5], the determination unit 14 determines that the first posture estimation result R1b is unreliable.

In step S17, the determination unit 14 may calculate the variance of the second posture estimation result R2 and compare the variance with a threshold value. In the example of FIG. 4, the variance of the values [1, 1, 5, 2, 3, 2, 1, 1, 3, 0, 4, 2, 1, 2, 3, 1] of the second posture estimation result R2a is calculated to be [1.73]. In this example, since the variance [1.73] is equal to or smaller than the threshold value [5], the determination unit 14 determines that the first posture estimation result R1a is reliable. In the example of FIG. 5, the variance of the values [2, 4, 8, 3, 4, 8, 2, 3, 3, 2, 2, 5, 2, 1, 3, 14] of the second posture estimation result R2b is calculated to be [11.05]. In this example, since the variance [11.05] exceeds the threshold value [5], the determination unit 14 determines that the first posture estimation result R1b is unreliable.

In step S17, the determination unit 14 may determine whether the first posture estimation result R1 is reliable based on a comparison of the first posture estimation result R1 and the second posture estimation result R2 with a threshold value. In other words, the determination unit 14 may further use the first posture estimation result R1 when comparing with the threshold value.

If the first posture estimation result R1 is reliable (YES in step S17), the process proceeds to step S18. On the other hand, if the first posture estimation result R1 is not reliable (NO in step S17), the process proceeds to step S19.

The output unit 15 outputs the first posture estimation result R1 to another device (step S18). For example, the output unit 15 outputs the first posture estimation result R1a to the other device.

The determination unit 14 discards the first posture estimation result R1 (step S19). For example, the determination unit 14 discards the first posture estimation result R1b.

Processing flow S1 is executed again, for example, after a predetermined time has elapsed. In one example, processing flow S1 is executed periodically by executing the processing of step S11 at a predetermined time interval.

With reference to FIG. 7, another example of the operation of the posture estimation device 10 will be described. FIG. 7 is a flowchart showing the operation of the posture estimation device 10 as processing flow S2. Processing flow S2 differs from processing flow S1 in that it includes processing of step S20 instead of step S19. Below, explanations of steps that are the same as those in processing flow S1 will be omitted.

The determination unit 14 determines whether the first posture estimation result R1 is reliable based on a comparison between the second posture estimation result R2 and a preset threshold (step S17). The determination unit 14 holds a preset threshold. Here, the threshold is [5]. The determination unit 14 compares the second posture estimation result R2 with the threshold, and determines that the first posture estimation result R1 is reliable if all values included in the second posture estimation result R2 are equal to or less than the threshold, and determines that the first posture estimation result R1 is unreliable if any one of the values included in the second posture estimation result R2 exceeds the threshold. In the example of FIG. 4, the maximum value [5] is equal to or less than the threshold [5], so the determination unit 14 determines that the first posture estimation result R1a is reliable. In the example of FIG. 5, the maximum value [14] exceeds the threshold [5], so the determination unit 14 determines that the first posture estimation result R1b is unreliable.

If the first posture estimation result R1 is reliable (YES in step S17), the process proceeds to step S18. The output unit 15 outputs the first posture estimation result R1 to another device (step S18). For example, the output unit 15 outputs the first posture estimation result R1a to the other device.

If the first posture estimation result R1 is not reliable (NO in step S17), the process proceeds to step S20. The determination unit 14 generates an assumed estimation result based on the second posture estimation result R2 (step S20). For example, the determination unit 14 generates an assumed estimation result by excluding values outside the threshold from the second posture estimation result R2 and calculating the average value of the excluded values. In the example of FIG. 5, the value [8, 8, 14] exceeding the threshold value [5] is excluded from the values [2, 4, 8, 3, 4, 8, 2, 3, 3, 2, 2, 5, 2, 1, 3, 14] of the second posture estimation result R2b. Then, the average value of the excluded values [2, 4, 3, 4, 2, 3, 3, 2, 2, 5, 2, 1, 3] is calculated to be [2.77]. In this example, the average value [2.77] becomes the assumed estimation result. Then, the output unit 15 outputs the assumed estimation result to another device.

Next, a posture estimation program for executing the above-mentioned series of processes by the posture estimation device 10 will be described. As shown in FIG. 8, the posture estimation program 100 is stored in a program storage area 111 formed in a computer-readable storage medium 110 that is inserted into a computer and accessed, or that is provided in the computer. The storage medium 110 may be a non-transitory storage medium.

The posture estimation program 100 is configured to include a recognition module 101, a generation module 102, an estimation module 103, a judgment module 104, and an output module 105. The functions realized by executing the recognition module 101, the generation module 102, the estimation module 103, the judgment module 104, and the output module 105 are similar to the functions of the recognition unit 11, the generation unit 12, the estimation unit 13, the judgment unit 14, and the output unit 15 of the posture estimation device 10 described above, respectively.

The posture estimation program 100 may be configured so that a part or all of it is transmitted via a transmission medium such as a communication line, and is received and recorded (including installed) by another device. Furthermore, each module of the posture estimation program 100 may be installed on one of multiple computers, rather than on one computer. In that case, the above-mentioned series of processes are performed by a computer system consisting of the multiple computers.

As described above, in the posture estimation device 10, at least a portion of the original coordinates is shifted to generate shifted coordinates. The marker M indicated by the shifted coordinates appears to be more distorted than the marker M indicated by the original coordinates. In this posture estimation device 10, the reliability of the first posture estimation result R1 can be accurately determined by comparing the second posture estimation result R2 using the shifted coordinates with a threshold value. Furthermore, in this posture estimation device 10, it is only necessary to consider the calculation costs required for calculating the shifted coordinates and comparing them with the threshold value, and since processing of the markers themselves is not required, an increase in manufacturing costs can be suppressed.

In this embodiment, the generation unit 12 generates multiple shifted coordinates by shifting the coordinates of the four corners of the marker M from the original coordinates. In this case, it becomes easier to identify the shape (rectangle) of the marker M, and the processing time required for the second posture estimation result R2 is shortened.

In this embodiment, the generation unit 12 generates multiple shifted coordinates by moving each of the coordinates of the four corners of the marker M from the original coordinates in the positive and negative directions of the X-axis and the positive and negative directions of the Y-axis. In this case, multiple shifted coordinates are generated by moving each of the coordinates of the four corners of the marker M in four directions. By comparing with a threshold based on multiple second posture estimation results R2, the reliability of the first posture estimation result R1 can be determined with greater accuracy.

In this embodiment, the generation unit 12 moves each of the coordinates of the four corners of the marker M by one pixel. In this case, the change in distortion of one pixel is reflected in the second posture estimation result R2. This makes it possible to further improve the accuracy of determining whether the first posture estimation result R1 is reliable while minimizing the movement of the coordinates.

In this embodiment, the determination unit 14 calculates the difference between the maximum and minimum values in the second posture estimation result R2, and compares the difference with a threshold. In this case, the accuracy of determining whether the first posture estimation result R1 is reliable can be ensured with a simple calculation.

In this embodiment, the determination unit 14 calculates the variance of the second posture estimation result R2 and compares the variance with a threshold value. In this case, the accuracy of determining the reliability of the first posture estimation result R1 can be ensured with a simple calculation.

In this embodiment, when the determination unit 14 determines that the first posture estimation result R1 is unreliable, it generates an assumed estimation result by excluding values outside the threshold from the second posture estimation result R2 and calculating the average value of the excluded values. In this case, it is possible to obtain a posture estimation result based on an assumed estimation result that ensures a certain level of reliability while omitting the reacquisition of the original coordinates of the marker M.

In this embodiment, when the determination unit 14 determines that the first posture estimation result R1 is unreliable, the determination unit 14 discards the first posture estimation result R1. In this case, it is possible to prevent an unreliable first posture estimation result R1 from being obtained as a posture estimation result.

In this embodiment, the determination unit 14 determines whether the first posture estimation result R1 is reliable based on a comparison between the first posture estimation result R1 and the second posture estimation result R2 and a threshold value. By further using the first posture estimation result R1 itself to determine whether it is reliable, it is possible to further improve the accuracy of determining whether the first posture estimation result R1 is reliable.

In this embodiment, the estimation unit 13 generates at least one of the roll angle, pitch angle, yaw angle, and position information as the first attitude estimation result R1 and the second attitude estimation result R2. In this case, the information necessary for attitude estimation can be appropriately obtained.

The industrial vehicle 1 is equipped with a camera 20 that captures an image of the marker M and the above-mentioned posture estimation device 10. In this industrial vehicle 1, at least a portion of the original coordinates is moved to generate shifted coordinates. The marker M indicated by the shifted coordinates appears to be more distorted than the marker M indicated by the original coordinates. In this industrial vehicle 1, the reliability of the first posture estimation result R1 can be accurately determined by comparing the second posture estimation result R2 using the shifted coordinates with a threshold value. Furthermore, with this industrial vehicle 1, it is only necessary to consider the calculation costs required for calculating the shifted coordinates and comparing them with the threshold value, and since there is no need to process the marker itself, an increase in manufacturing costs can be suppressed.

The posture estimation program 100 causes a computer to execute the steps of recognizing marker M from a captured image of marker M and acquiring original coordinates of marker M in the captured image, generating shifted coordinates by shifting the coordinates of at least one point of marker M from the original coordinates, generating a first posture estimation result R1 using the original coordinates and a second posture estimation result R2 using the shifted coordinates, and determining whether or not the first posture estimation result R1 is reliable based on a comparison between the second posture estimation result R2 and a preset threshold. The storage medium 110 is a computer-readable storage medium 110 that stores the posture estimation program 100.

In this posture estimation program 100 and storage medium 110, at least a portion of the original coordinates is moved to generate shifted coordinates. The marker M indicated by the shifted coordinates appears to be more distorted than the marker M indicated by the original coordinates. In this posture estimation program 100 and storage medium 110, the reliability of the first posture estimation result R1 can be accurately determined by comparing the second posture estimation result R2 using the shifted coordinates with a threshold value. Furthermore, in this posture estimation program 100 and storage medium 110, it is only necessary to consider the calculation cost required to calculate the shifted coordinates and compare them with the threshold value, and since processing of the marker itself is not required, an increase in manufacturing costs can be suppressed.

In the above embodiment, the generation unit 12 shifts each of the coordinates of the four corners of the marker M by one pixel, but may shift by more than one pixel (e.g., two pixels or three pixels). Also, in the above embodiment, the generation unit 12 generates 16 shift coordinates as multiple shift coordinates, but the number of shift coordinates is not limited to this and may be any number.

1...industrial vehicle, 10...posture estimation device, 11...recognition unit, 12...generation unit, 13...estimation unit, 14...determination unit, 15...output unit, 20...camera, 100...posture estimation program, 110...storage medium, 101...recognition module, 102...generation module, 103...estimation module, 104...determination module, 105...output module, M...marker, P...captured image.

Claims (13)

a recognition unit that recognizes a marker from a captured image of the marker and acquires original coordinates of the marker in the captured image;
a generation unit that generates shifted coordinates by shifting coordinates of at least one point of the marker from the original coordinates;
an estimation unit that generates a first posture estimation result using the original coordinates and a second posture estimation result using the shifted coordinates;
a determination unit that determines whether or not the first posture estimation result is reliable based on a comparison between the second posture estimation result and a preset threshold;
A posture estimation device comprising: The posture estimation device according to claim 1, wherein the generation unit generates a plurality of shifted coordinates by shifting the coordinates of the four corners of the marker from the original coordinates. The posture estimation device according to claim 2, wherein the generation unit generates the multiple shifted coordinates by moving each of the coordinates of the four corners of the marker from the original coordinates in the positive and negative directions of the X axis and the positive and negative directions of the Y axis. The posture estimation device according to claim 3, wherein the generation unit moves each of the coordinates of the four corners of the marker by one pixel. The posture estimation device according to any one of claims 2 to 4, wherein the determination unit calculates a difference between a maximum value and a minimum value in the second posture estimation result and compares the difference with the threshold value. The posture estimation device according to any one of claims 2 to 4, wherein the determination unit calculates a variance in the second posture estimation result and compares the variance with the threshold value. The posture estimation device according to any one of claims 2 to 6, wherein the determination unit, when determining that the first posture estimation result is unreliable, generates a deemed estimation result by excluding values outside the threshold from the second posture estimation result and calculating an average value of the excluded values. The posture estimation device according to any one of claims 1 to 6, wherein the determination unit discards the first posture estimation result when it determines that the first posture estimation result is unreliable. The posture estimation device according to any one of claims 1 to 8, wherein the determination unit determines the reliability of the first posture estimation result based on a comparison of the first posture estimation result and the second posture estimation result with the threshold value. The attitude estimation device according to any one of claims 1 to 9, wherein the estimation unit generates at least one of a roll angle, a pitch angle, a yaw angle, and position information as the first attitude estimation result and the second attitude estimation result. An industrial vehicle equipped with a camera that captures the marker and a posture estimation device according to any one of claims 1 to 10. A step of recognizing the marker from a captured image of the marker and acquiring original coordinates of the marker in the captured image;
generating shifted coordinates by shifting coordinates of at least one point of the marker from the original coordinates;
generating a first pose estimation result using the original coordinates and a second pose estimation result using the shifted coordinates;
determining whether the first posture estimation result is reliable based on a comparison between the second posture estimation result and a preset threshold;
A posture estimation program that causes a computer to execute the above. A computer-readable storage medium storing the posture estimation program according to claim 12.

Images