JP2020112952A - Movement support device - Google Patents

Abstract

To provide a movement support device capable of causing an index image to appropriately appear.SOLUTION: A movement support device 100 includes an image acquisition unit 108 for acquiring an external image 20 obtained by imaging the outside, an index generation unit 112 for generating an index image to be used for drive control, and an image superposition unit 114 for superposing an index on the external image to generate a superposed image 28, and outputting the superposed image as data to a conveyance vehicle. Also, the movement support device may include a drive control unit 16 for extracting the index image from the superposed image to perform drive control of the vehicle.SELECTED DRAWING: Figure 4

Description

The present disclosure relates to a movement support device.

There is a device that automatically recognizes the movement route of the own vehicle in order to move the vehicle unmanned. For example, in Patent Document 1, the position of the automatic guided vehicle is derived from the direction and distance of the reflector and the position data of the reflector.

As an automated guided vehicle, for example, there is an optical automated guided vehicle (AGV). Such an optical automated guided vehicle optically reads a line marker attached to the movement route and moves while tracing the line marker.

JP, 2000-099145, A

However, it is not easy to properly place the line marker on the movement route while considering the resistance under various environments. For example, if the travel route extends to the outdoors, the line marker may accumulate deposits such as sand, or dirt may not be recognized, which may hinder the movement of the automated guided vehicle. is there.

Then, this indication aims at providing the movement support device which can make an index image appear appropriately.

In order to solve the above problems, a movement support device according to an aspect of the present disclosure includes an image acquisition unit that acquires an external image of the outside, an index generation unit that generates an index image used for drive control, and an external device. An image superimposing unit that superimposes the index image on the image to generate a superimposed image and outputs the superimposed image as data to the transport vehicle.

The movement support device may include a drive control unit that extracts the index image from the superimposed image and controls the drive of the vehicle.

The movement support device may include a position information generation unit that generates position information of the guided vehicle based on the external image acquired by the image acquisition unit, and the index generation unit may generate the index image based on the position information.

The image acquisition unit acquires the external image of the first imaging range and the external image of the second imaging range, respectively, and the position information generation unit acquires the position information of the transport vehicle based on the external image of the first imaging range. The generated image superimposing unit may superimpose the index image on the external image in the second imaging range.

The index generation unit may extract the index image required by the drive control unit within a predetermined time.

The index generation unit may generate the index image in which at least one of the shape, size, width, color, and pattern is specified regardless of the imaging state of the external image.

According to the present disclosure, it is possible to appropriately cause the index image to appear.

FIG. 1 is an explanatory diagram for explaining an outline of the optical automatic guided vehicle according to the present embodiment. FIG. 2 is an explanatory diagram for explaining the outline of the optical automated guided vehicle of the present embodiment. FIG. 3 is an explanatory diagram for explaining the outline of the optical automated guided vehicle of the present embodiment. FIG. 4 is a diagram illustrating a schematic configuration of the movement support device. FIG. 5 is a diagram for explaining the preprocessing. FIG. 6 is a diagram for explaining the preprocessing. FIG. 7 is a diagram for explaining the preprocessing. FIG. 8 is a flowchart showing a process relating to movement support of an optical automatic guided vehicle. FIG. 9 is a diagram for explaining the operation of the position information generation unit. FIG. 10 is a diagram for explaining the operation of the position information generation unit. FIG. 11 is a diagram for explaining the operation of the index generation unit. FIG. 12 is a diagram for explaining the operation of the image superimposing unit. FIG. 13 is an explanatory diagram for explaining the operation of the index generation unit. FIG. 14 is an explanatory diagram for explaining the operation of the index generation unit. FIG. 15 is an explanatory diagram for explaining another configuration of the movement support device. FIG. 16 is an explanatory diagram for explaining another configuration of the movement support device.

An embodiment of the present disclosure will be described below with reference to the accompanying drawings. The dimensions, materials, and other specific numerical values shown in the embodiments are merely examples for facilitating the understanding of the present disclosure, and are not limited unless otherwise specified. In this specification and the drawings, elements having substantially the same function and configuration are denoted by the same reference numerals to omit redundant description, and elements not directly related to the present disclosure are omitted. To do.

<Optical automated guided vehicle 10>
1 to 3 are explanatory views for explaining the outline of the optical automated guided vehicle 10 of the present embodiment. Here, first, the basic operation of the optical automatic guided vehicle 10 will be described. As shown in FIG. 1, the optical automated guided vehicle 10 includes an imaging unit 12, a drive mechanism 14, and a drive control unit 16. The imaging unit 12 is configured to include an imaging device, and images the external environment in front of the optical guided vehicle 10 to generate an external image. The drive mechanism 14 includes drive wheels that movably support the optical automated guided vehicle 10 in FIG. 1. The drive mechanism 14 moves the optical automatic guided vehicle 10 in the forward direction indicated by the white arrow, and steers the vehicle in the left-right direction of the traveling direction.

The drive control unit 16 of the optical automated guided vehicle 10 first performs an index image (representing an index from the superimposed image 28 in FIG. 2 in which the external image generated by the imaging unit 12 has been subjected to predetermined processing by the movement support device 100). The line marker 22 as an image is specified. Then, the drive control unit 16 derives the operation amount of the drive mechanism 14 indicated by the white arrow so that the line marker 22 overlaps with the reference line 24 located at the horizontal center of the superimposed image 28. Next, the drive control unit 16 drives and controls the drive mechanism 14 with the derived operation amount. In this way, the optical guided vehicle 10 can move while tracing on the line marker 22.

The optical automated guided vehicle 10 can be configured by modifying an existing optical automated guided vehicle. For example, in an existing optical guided vehicle, the optical axis of the imaging unit is set downward with respect to the vertical direction, and the horizontal and vertical angle of view is narrow. In place of or in addition to the image pickup unit of the existing optical automatic guided vehicle, by providing the image pickup unit 12 in which the optical axis is set in the horizontal direction and the left, right, up, and down angles of view are wide, the present embodiment is provided. The optical automated guided vehicle 10 can be configured. Note that the image pickup unit 12 may be arranged in such a posture that the angle of view spreads on both sides of a horizontal plane passing through the optical axis origin of the image pickup unit 12. Further, although the optical automated guided vehicle 10 of the present disclosure is a modification of the existing optical automated guided vehicle, the effect of the present disclosure can be obtained even if the existing optical automated guided vehicle is used as it is. Is.

Further, as shown in FIG. 3, the superimposed image 28 may include not only the line marker 22 but also the marker 26 indicating the speed limit or the like. In this case, the optical guided vehicle 10 identifies the sign 26 and drives and controls the drive mechanism 14 according to the content indicated by the sign 26, for example, the speed instruction “20” km/h.

Although the line marker 22 and the sign 26 are illustrated here, the present invention is not limited to such a case, and may be a predetermined index (index image) used for drive control of the optical guided vehicle 10. The optical automatic guided vehicle 10 can specify various index images and drive-control the drive mechanism 14 according to the content indicated by the index images.

The line marker 22 and the sign 26 may be images of the existing line marker 22 and the sign 26 captured by the image capturing unit 12. However, in the present embodiment, as described later, the movement support device 100 virtually generates them. , Use the one that is expressed as if it were real.

<Movement support device 100>
FIG. 4 is a diagram illustrating a schematic configuration of the movement support device 100. The movement support device 100 assists image recognition of the optical automatic guided vehicle 10 and supports movement. The movement support apparatus 100 has a central processing unit (CPU), a ROM that holds a program, and a RAM that is used as a work area, and has an image acquisition unit 108, a position information generation unit 110, an index generation unit 112, and an image. It functions as the superposition unit 114.

The image acquisition unit 108 acquires the external image 20 captured by the image capturing unit 12. The position information generation unit 110 generates position information (including at least one of position, orientation, and azimuth) of the optical guided vehicle 10 based on the external image 20 acquired by the image acquisition unit 108. The index generation unit 112 generates an index image used for drive control of the optical guided vehicle 10 based on the position information. The image superimposing unit 114 superimposes the index image on the external image 20 to generate a superimposed image 28, and outputs the superimposed image 28 to the optical automated guided vehicle 10 as data. Then, the drive control unit 16 of the optical guided vehicle 10 specifies the index image from the superimposed image 28 acquired as data and controls the drive of the own vehicle. When tracing the line marker 22 where the optical automatic guided vehicle 10 actually exists, the drive control unit 16 may directly read the external image 20 without generating the superimposed image 28.

In the present disclosure, an example in which the optical guided vehicle 10 and the movement support device 100 are integrally formed is given, and the position of the optical guided vehicle 10 in the following description is the position of the movement support device 100. Can be replaced with

The position information generation unit 110 and the index generation unit 112 generate the position information and the index image using a three-dimensional map prepared in advance. Here, preprocessing such as generation of a three-dimensional map will be described.

5 to 7 are diagrams for explaining the preprocessing. As a preliminary process, the movement support device 100 generates or updates the three-dimensional map 30 using the external image 20 captured by the image capturing unit 12. Specifically, in the movement support device 100, as shown in FIG. 5, the optical automated guided vehicle 10 is at an arbitrary position 32 on the three-dimensional map 30, and has an arbitrary posture 34 and an arbitrary azimuth 36. The imaging unit 12 generates the external image 20 when the image is present. Note that, in FIG. 5, the three-dimensional map 30 is represented in two dimensions for convenience of description, but in reality, three-dimensional position information including the height direction is held. Further, here, a case where there is one image pickup element will be described, but a plurality of image pickup elements may be used to specify the distance (depth) in the depth direction, or a distance measuring laser may be used separately.

Next, the movement support device 100 extracts a plurality of feature points 38 from the external image 20 as shown in FIG. The movement support apparatus 100 specifies the three-dimensional positions of the plurality of feature points 38 based on the position information (position 32, posture 34, azimuth 36) of the optical guided vehicle 10, and specifies the three specified positions in the three-dimensional map 30. The feature point 38 is mapped to the dimensional position.

Such processing is continuously repeated a predetermined number of times while changing the position 32, the posture 34, and the azimuth 36. Then, the feature point 38 is specified in correspondence with the plurality of different positions 32, postures 34, and directions 36 (samples). In this way, the feature points 38 are properly mapped on the three-dimensional map 30.

Further, as a pre-processing, the movement support device 100 maps the three-dimensional position of the movement route 40 on the three-dimensional map 30 based on the movement route 40 of the optical guided vehicle 10, as shown in FIG. 7. As described above, after the feature points 38 are appropriately mapped on the three-dimensional map 30, by mapping the three-dimensional position of the movement route 40, the movement route 40 can be easily constructed and modified. The three-dimensional map 30 is used by the position information generation unit 110 and the index generation unit 112 when the optical automated guided vehicle 10 moves. Hereinafter, specific operations of the image acquisition unit 108, the position information generation unit 110, the index generation unit 112, and the image superposition unit 114 for moving the optical automated guided vehicle 10 after the preparation is completed will be described in detail.

FIG. 8 is a flowchart showing a process relating to movement support of the optical guided vehicle 10. Here, an example is shown in which the processing of the flowchart is executed each time a predetermined interrupt time elapses.

First, the imaging unit 12 captures an external environment in front of the optical guided vehicle 10 and generates an external image 20 (S200). The image acquisition unit 108 acquires the external image 20 imaged by the imaging unit 12 (S202).

The position information generation unit 110 generates the position information of the optical guided vehicle 10 based on the external image 20 acquired by the image acquisition unit 108 (S204). Based on the position information generated by the position information generation unit 110, the index generation unit 112 uses the three-dimensional map 30 generated in the preprocessing to generate an index image used for drive control in the optical guided vehicle 10. (S206).

9 and 10 are diagrams for explaining the operation of the position information generating unit 110, and FIG. 11 is a diagram for explaining the operation of the index generating unit 112. Specifically, the position information generation unit 110 extracts a plurality of feature points 38 from the external image 20 acquired by the image acquisition unit 108, as illustrated in FIG. 9. Then, the position information generation unit 110 performs pattern matching between the extracted feature points 38 and the feature points 38 included in the three-dimensional map 30.

In addition, in order to appropriately perform the pattern matching of the feature points 30, the imaging mode may be changed according to the environment. For example, a bifocal lens is adopted as the image pickup unit 12, and its focus is switched according to the environment. If fog occurs on the moving path 40 and the focus is in the distance, the feature point 38 may not be properly extracted due to the reflection of the fog. In this case, the position information generation unit 110 may switch the focus of the lens to the vicinity (or increase the specific gravity of the image in the vicinity of the focus) to reduce the influence of fog and extract the feature point 38. Further, here, the example in which the bifocal lens is adopted has been described, but an imaging unit having a plurality of focal points including a far imaging unit and a near imaging unit may be adopted, and both may be switched according to the environment. Good.

As illustrated in FIG. 10, the position information generation unit 110 extracts the position 32, the posture 34, and the azimuth 36 corresponding to the combination of the feature points 38 having a high degree of coincidence in pattern matching. In this way, the position 32, posture 34, and azimuth 36 of the optical guided vehicle 10 at the time of generation of the external image 20 are specified.

Next, the index generation unit 112 extracts the three-dimensional position of the movement route 40 based on the three-dimensional map 30. Subsequently, the index generation unit 112, based on the position 32, the posture 34, and the azimuth 36 of the optical guided vehicle 10 generated by the position information generation unit 110, as shown in FIG. 11, a virtual line that traces the movement route 40. The marker 42 is generated.

Here, the index generation unit 112 specifies at least one of the shape, size, width, color, and pattern regardless of the imaging state of the external image 20, for example, the change in environment due to the amount of light, reflection, shielding, or the like. The virtual line marker 42 is generated. Therefore, the index image (virtual line marker 42) that can be properly recognized by the optical guided vehicle 10 can be made to appear regardless of the external environment.

Further, here, the shape, size, width, color, and pattern of the index image are not formed as actual objects but are formed as images. Therefore, the degree of freedom in design is high and the display accuracy of the index image is high. Therefore, for example, it is possible to arrange the index image in the air according to the step on the moving route.

Further, since the index (virtual line marker 42) is generated as an image here, there is no possibility that the index image is physically soiled, missing, or deformed over time.

FIG. 12 is a diagram for explaining the operation of the image superimposing unit 114. The image superimposing unit 114 superimposes the virtual line marker 42, which is the index image generated by the index generating unit 112, on the external image 20 acquired by the image acquiring unit 108 to generate the superimposed image 28 (S208 in FIG. 8). In this way, it becomes possible to generate the superimposed image 28 in which the virtual line marker 42 physically exists. The superimposed image 28 thus generated is transmitted to the drive control unit 16 as data (electrical signal).

Then, the drive control unit 16 of the optical automated guided vehicle 10 identifies the index image (virtual line marker 42) from the superimposed image 28 transmitted as data from the index generation unit 112, and drives and controls the drive mechanism 14 (FIG. 8 S210). In this way, the drive control unit 16 can move the optical guided vehicle 10 while tracing the virtual line marker 42.

Here, the movement assisting device 100 temporarily generates the superimposed image 28 on which the virtual line marker 42 is superimposed, and causes the optical automated guided vehicle 10 to read the superimposed image 28. Therefore, the existing optical automated guided vehicle 10 can be used as it is, and the optical automated guided vehicle 10 can be moved appropriately.

Further, here, the position 32, the posture 34, and the azimuth 36 of the optical guided vehicle 10 are derived by image recognition. Therefore, the optical unmanned guided vehicle 10 can be appropriately moved even in a situation where the index is in the shadow of a building or the like and the GPS (Global Positioning System) radio wave does not reach.

Further, even if an abnormality occurs in the image recognition of the drive control unit 16 of the optical automated guided vehicle 10, if the abnormality (for example, a partial lack of a sensor that receives each of the three primary colors) can be specified, the index generation can be performed. The section 112 adjusts the shape, size, width, color, and pattern of the index so as to compensate (cancel) the abnormality so that the drive control section 16 of the optical automated guided vehicle 10 operates normally. You can

In addition, here, the example in which the index generation unit 112 simply generates the index image based on the movement route 40 has been described, but the invention is not limited to such a case, and the index generation unit 112 causes the drive control unit 16 within The required index image may be generated. Specifically, the index generation unit 112 identifies which position 32 is currently on the movement route 40 based on the position information, and the movement route 40 for a predetermined distance or a predetermined time from the position 32. Extract (cut out) only. Then, the index generation unit 112 generates an index image (virtual line marker 42) based on the extracted movement route 40.

13 and 14 are explanatory diagrams for explaining the operation of the index generation unit 112. For example, as shown in the upper diagram of FIG. 13, there is a case where an arbitrary automatic guided vehicle 10a passes through the same point A at different times. In addition, the virtual line marker 42a of the arbitrary optical automated guided vehicle 10a and the virtual line marker 42b of the other optical automated guided vehicle 10b may intersect or overlap at an arbitrary point B.

Here, the index generator 112 generates only the virtual line marker 42 required by the drive controller 16 within a predetermined time. Therefore, as shown in the lower diagram of FIG. 13, the optical automatic guided vehicle 10a can refer only to the latest virtual line marker 42a, and the optical automatic guided vehicle 10b can refer only to the latest virtual line marker 42b. Even if the movement paths 40 overlap each other as described above, the optical automatic guided vehicle 10 can be appropriately moved.

In addition, when the movement routes 40 overlap each other, there are cases where the line markers cannot be escaped. For example, as shown in FIG. 14, the optical unmanned guided vehicle 10a displays the virtual line marker 42 where the physical line marker 22 of the optical unmanned guided vehicle 10b exists. In this case, at the point C, the physical line marker 22 and the virtual line marker 42 cannot escape from overlapping. At this time, the index generation unit 112 changes one of the shape, size, width, color, and pattern of the virtual line marker 42 so as to be different from the physical line marker 22, and the fact is transmitted to the drive control unit 16. To do. The drive control unit 16 drives and controls the drive mechanism 14 based on the changed virtual line marker 42.

Further, the image acquisition unit 108 determines whether or not an object that may be erroneously recognized as the virtual line marker 42 is reflected in the acquired external image 20, and if it is reflected, the object is deleted from the external image 20. Therefore, preprocessing such as applying a patch may be executed.

For example, inputting the external image 20 acquired from the image acquisition unit 108 to a simulation program simulating the drive control unit 16 (or the drive control unit 16 itself) and detecting whether or not the original drive control signal can be obtained. Thus, it is possible to determine whether or not an object that may be erroneously recognized is reflected. The drive control signal is not input to the actual drive mechanism 14 and is used only for detection.

Further, in order to reduce the processing time delay, the image acquisition unit 108 acquires the external image 20 acquired at a later time in time series from the external image 20 that is determined to include an object that may be erroneously recognized. Therefore, pre-processing for deleting the object may be executed.

Further, when the virtual line marker 42 and the physical line marker 22 are switched in the middle of the movement route, by adding an index image showing the switching point, the virtual line marker 42 and the physical line marker 22 are smoothly moved. It is possible to realize various switching.

In this way, the plurality of overlapping line markers are distinguished, and the optical guided vehicle 10 can refer only to the movement route 40 along which the own vehicle moves. Therefore, even if the movement routes 40 overlap, the optical guided vehicle 10 can be appropriately moved.

15 and 16 are explanatory diagrams for explaining another configuration of the movement support device 100. In the above-described embodiment, the image acquisition unit 108, the position information generation unit 110, the index generation unit 112, the optical unmanned guided vehicle 10 having the imaging unit 12, the drive mechanism 14, and the drive control unit 16. The configuration in which the movement support device 100 including the image superimposing unit 114 is added has been described. However, not limited to this case, as illustrated in FIG. 15, the imaging unit 12, the drive mechanism 14, the drive control unit 16, the image acquisition unit 108, the position information generation unit 110, the index generation unit 112, and the image. There is also provided a movement support device 100 integrally formed with the superposition unit 114.

Further, in the above-described embodiment, the image acquisition unit 108 of the movement support apparatus 100, the position information generation unit 110, the index generation unit 112, and the image superposition unit 114 are provided after the imaging unit 12 of the optical guided vehicle 10. An example of connecting and has been described. However, not limited to such a case, as illustrated in FIG. 16, the movement support device 100 includes the image capturing unit 116, the superimposed image in addition to the image acquisition unit 108, the position information generation unit 110, the index generation unit 112, and the image superimposition unit 114. May be included in the image display unit 118. In this case, the movement support apparatus 100 may be placed so that the image display unit 118 is located in front of the image pickup unit 12 of the existing optical guided vehicle 10.

Although one embodiment has been described with reference to the accompanying drawings, it goes without saying that the present disclosure is not limited to the above embodiment. It is obvious to those skilled in the art that various variations or modifications can be conceived within the scope of the claims, and it should be understood that they also belong to the technical scope of the present disclosure. To be done.

For example, in the above-described embodiment, the position information generation unit 110 has been described as an example in which the position 32, the posture 34, and the azimuth 36 of the optical guided vehicle 10 are specified using the three-dimensional map 30. However, not limited to such a case, the position and orientation of the optical guided vehicle 10 may be specified by GPS, and the attitude of the host vehicle may be specified by using INS (Inertial Navigation System) or the like.

Further, in the above-described embodiment, the position information generation unit 110 generates the position information of the optical guided vehicle 10 based on the one external image 20 acquired by the image acquisition unit 108, and the image superposition unit 114 The example in which the index image is superimposed on the first external image 20 to generate the superimposed image 28 has been described. However, the imaging range (angle of view) required for the position information generation unit 110 to generate the position information may be different from the imaging range (angle of view) of the superimposed image 28. Therefore, two image pickup units 12 are provided, or one image pickup unit 12 generates different external images 20 in two different image pickup ranges. Specifically, the image capturing unit 12 includes an external image 20 of the first image capturing range necessary for the position information generating unit 110 to generate the position information, and a second image capturing range different from the first image capturing range. Each external image 20 is generated.

In this case, the image acquisition unit 108 respectively acquires the external image 20 in the first imaging range and the external image 20 in the second imaging range. The position information generation unit 110 generates the position information of the optical guided vehicle 10 based on the external image 20 in the first imaging range. The image superimposing unit 114 superimposes the index image on the external image 20 in the second imaging range to generate the superimposed image 28.

Moreover, in the above-described embodiment, an example in which the index generating unit 112 generates the virtual line marker 42 that traces the movement route 40 based on the position 32, the posture 34, and the azimuth 36 of the optical guided vehicle 10 is given. Explained. However, not limited to such a case, the superimposed image 28 can be referred to from various positions as long as the three-dimensional map 30 is used. For example, a VR (Virtual Reality) head mount displays a virtual line marker 42 that traces a movement path 40 based on the position 32, posture 34, and azimuth 36 of the user wearing the VR head mount.

Therefore, the user can confirm the index image such as the virtual line marker 42 that does not actually exist in the virtual space through the VR head mount. Further, even when a plurality of optical automatic guided vehicles 10 move at once, by selecting the target optical automatic guided vehicle 10, only the index image of the optical automatic guided vehicle 10 is referred to. It becomes possible.

The present disclosure can be used for a movement support device.

10 optical unmanned guided vehicle 12 imaging unit 14 drive mechanism 16 drive control unit 20 external image 22 line marker 24 reference line 28 superimposed image 30 three-dimensional map 38 feature point 40 movement route 42 virtual line marker 100 movement support device 108 image acquisition unit 110 Position Information Generation Unit 112 Index Generation Unit 114 Image Superposition Unit

Claims (6)

An image acquisition unit that acquires an external image of the outside,
An index generation unit that generates an index image used for drive control,
An image superimposing unit that superimposes the index image on the external image to generate a superposed image, and outputs the superposed image as data to the transport vehicle.
A movement support device. The movement support device according to claim 1, further comprising a drive control unit that extracts the index image from the superimposed image and controls the drive of the vehicle. A position information generation unit that generates position information of the transport vehicle based on an external image acquired by the image acquisition unit;
The movement support device according to claim 1, wherein the index generation unit generates the index image based on the position information. The image acquisition unit acquires an external image of the first imaging range and an external image of the second imaging range, respectively.
The position information generation unit generates position information of the transport vehicle based on an external image of the first imaging range,
The movement assistance device according to claim 3, wherein the image superimposing unit superimposes the index image on an external image in the second imaging range. The movement support device according to claim 1, wherein the index generation unit extracts the index image required by the drive control unit within a predetermined time. 6. The index generation unit generates the index image in which at least one of shape, size, width, color, and pattern is specified regardless of the imaging state of the external image. The movement support device described in 1.

Images