JP6763656B2 - Driving guide system - Google Patents

Description

The present invention relates to a traveling guide system that guides an operator of a moving body such as a vehicle to a traveling position of the moving body and / or a position of a managed portion.

In order to prevent accidents, airports are required to quickly find and repair managed areas such as damaged areas on the runway surface, such as cracks and depressions. Conventionally, in this work, when the runway is not in use, a maintenance worker travels on the runway with a vehicle and visually discovers and repairs the managed part.

However, in some cases, the runway has a total length of 3000 meters or more and a width of about 60 meters. Therefore, it is a difficult task to visually find a small managed area on the runway. Therefore, in order to simplify the work, a method of traveling on the runway with a vehicle equipped with an imaging device for imaging the road surface of the runway and sequentially imaging the road surface during running to discover the management target location is considered. Has been done.

In this case, it is required to image the road surface while traveling on the runway with a vehicle while shifting the imaging position in parallel multiple times. However, since it is a human being who drives a vehicle, it is difficult to always drive in a straight line. In addition, traveling must be carried out at night, avoiding the daytime hours when the aircraft takes off and landing, and it becomes even more difficult to drive in a straight line because the landmarks cannot be seen. Therefore, when the vehicle is running, the vehicle may be displaced to the left or right, and there is a possibility that the images of the runway surface may be significantly duplicated or missing.

Therefore, a method is conceivable in which a cone (pylon) equipped with an LED that serves as a guide for the vehicle to travel is placed on the runway, and the vehicle travels along the cone (pylon).

In addition, there is a car navigation system as a system that guides the route to be traveled, and it is conceivable to use it. As an application example, there is also a system for navigating by projecting the direction in which a person should travel in a shaded position (Patent Document 1).

Japanese Unexamined Patent Publication No. 2012-2709

The simplest method is for the vehicle to travel along the LED-mounted cone placed on the runway, and since it can travel along the cone, there is no significant overlap or omission of the imaging range. However, the work of placing the cone on the runway must be performed by the person in charge of maintenance work, and it takes time to install and collect the cone. In addition, all cones must be collected after the work is completed, but if the cones are forgotten to be collected, the cones may come into contact with an airplane using the runway, which may lead to an accident. Therefore, there is a problem from the viewpoint of safety.

In the case of a car navigation system, the current position information is determined by GPS and the road to be traveled from the current point to the destination point is guided. Therefore, it is useful in terms of how to reach a certain destination along a road or lane (with one degree of freedom). However, when driving in a place where there is no map to drive, a place where there is no lane (when the degree of freedom is 2) such as a runway, or when specifying the position of a management target part, car navigation The system or the system described in Patent Document 1 cannot be used.

Therefore, the present inventor has made the following invention in view of the above problems.

The invention of claim 1 is a traveling guide system that guides an operator of a moving object to a traveling position of the moving object, and the traveling guide system is a position information acquisition processing unit that acquires position information from a position information acquisition device. A virtual space corresponding to a part or all of the real space including the direction information acquisition processing unit for acquiring the direction information and the range for guiding the traveling position to the operator of the moving body, and the operator moves. A traveling guide indicating a traveling position where the body should be moved is arranged in the virtual space. The virtual space information storage unit for storing the virtual space information, and the acquired position information and the direction information are used in the virtual space. It is an image of the virtual space including the traveling guide from the virtual viewpoint in the virtual space position conversion processing unit that converts the information of the position and direction of the virtual viewpoint and the virtual space stored in the virtual space information storage unit. By referring to the management target location storage unit using the virtual viewpoint image generation processing unit that generates the virtual viewpoint image, the management target location storage unit that stores the position information of the management target location, and the acquired position information. , A management target location identification processing unit that specifies a management target location within a predetermined range, and a guide image generation processing unit that calculates the distance and / or direction to the specified management target location and generates a guide image to guide. , The virtual space information storage unit has an output processing unit that outputs the generated virtual viewpoint image and / or guide image from the projection device or the display device , and the virtual space information storage unit uses the virtual space as the virtual space information. Part or all of the above is divided into the length direction and the width direction at predetermined intervals, the length direction is represented by the distance from the reference position in the length direction, and the width direction is represented by the lane number from the reference position in the width direction. At the intersection of the distance and the lane number, a distance marker indicating the distance and the lane number is arranged in the virtual space, and the virtual viewpoint image generation processing unit is the traveling guide and the traveling guide from the virtual viewpoint. This is a driving guide system that generates virtual viewpoint images including distance markers .
The invention of claim 2 is a traveling guide system that guides the operator of the moving object to the traveling position of the moving object, and the traveling guide system is a position information acquisition process for acquiring position information from the position information acquisition device. A virtual space corresponding to a part or all of a real space including a unit, a direction information acquisition processing unit for acquiring direction information, and a range for guiding a traveling position to an operator of the moving body, and the operator The virtual space using the virtual space information storage unit that stores the virtual space information in which the traveling guide indicating the traveling position where the moving body should be moved is arranged in the virtual space, and the acquired position information and direction information. In the virtual space position conversion processing unit that converts the information of the position and direction of the virtual viewpoint in the above and the virtual space stored in the virtual space information storage unit, the image of the virtual space including the traveling guide from the virtual viewpoint is used. It has a virtual viewpoint image generation processing unit that generates a certain virtual viewpoint image and an output processing unit that outputs the generated virtual viewpoint image from a projection device or a display device. The virtual space information storage unit has. As the virtual space information, a part or all of the virtual space is divided into a length direction and a width direction at predetermined intervals, the length direction is represented by a distance from a reference position in the length direction, and the width direction is defined as a width. It is represented by the lane number from the reference position in the direction, and at the intersection of the distance and the lane number, a distance marker indicating the distance and the lane number is arranged in the virtual space, and the virtual viewpoint image generation processing unit is the virtual. This is a traveling guide system that generates a virtual viewpoint image including the traveling guide and the distance marker from a viewpoint .

By configuring as in these inventions, an operator of a moving body such as a vehicle can appropriately move the moving body according to a running guide even in a space having two or more degrees of freedom such as a runway. It becomes. Therefore, it is not necessary to install a cone to control the traveling position of the moving body, the working time can be shortened, and safety can be ensured. Further, by configuring as in these inventions, not only the traveling guide but also the distance marker is output. Therefore, the operator can recognize the positions in the length direction and the width direction through the distance and the lane number.

Further, in the invention of claim 1 , the moving body can be moved to a desired management target location from the management target locations such as damages specified in advance. Therefore, maintenance work becomes easy.

The invention of claim 3 is a traveling guide system that guides the operator of the moving object to the traveling position of the moving object. The traveling guide system is a position information acquisition processing unit that acquires position information from a position information acquisition device. A virtual space corresponding to a part or all of the real space including the direction information acquisition processing unit for acquiring the direction information and the range for guiding the traveling position to the operator of the moving body, and the operator moves. A traveling guide indicating a traveling position where the body should be moved is arranged in the virtual space. The virtual space information storage unit for storing the virtual space information, and the acquired position information and the direction information are used in the virtual space. It is an image of the virtual space including the traveling guide from the virtual viewpoint in the virtual space position conversion processing unit that converts the information of the position and direction of the virtual viewpoint and the virtual space stored in the virtual space information storage unit. By referring to the management target location storage unit using the virtual viewpoint image generation processing unit that generates the virtual viewpoint image, the management target location storage unit that stores the position information of the management target location, and the acquired position information. , A management target location identification processing unit that specifies a management target location within a predetermined range, and a guide image generation processing unit that calculates the distance and / or direction to the specified management target location and generates a guide image to guide. , The output processing unit that outputs the generated virtual viewpoint image and / or guide image from the projection device or display device , and the direction information acquisition processing unit has the moving speed of the moving body and a predetermined value. If it is determined that the threshold has been exceeded by comparing with the threshold, the calculation process of the direction information using the position information is stopped, or the calculation process of the direction information is performed by the moving average of the position information in the predetermined time. It is a system.
The invention of claim 4 is a traveling guide system that guides the operator of the moving body to the traveling position of the moving body, and the traveling guide system is a position information acquisition process for acquiring position information from the position information acquisition device. A virtual space corresponding to a part or all of a real space including a unit, a direction information acquisition processing unit for acquiring direction information, and a range for guiding a traveling position to an operator of the moving body, and the operator The virtual space using the virtual space information storage unit that stores the virtual space information in which the traveling guide indicating the traveling position where the moving body should be moved is arranged in the virtual space, and the acquired position information and direction information. In the virtual space position conversion processing unit that converts the information of the position and direction of the virtual viewpoint in the above, and the virtual space stored in the virtual space information storage unit, the image of the virtual space including the traveling guide from the virtual viewpoint. It has a virtual viewpoint image generation processing unit that generates a certain virtual viewpoint image and an output processing unit that outputs the generated virtual viewpoint image from a projection device or a display device , and the direction information acquisition processing unit has. When the moving speed of the moving body is compared with a predetermined threshold and it is determined that the threshold is exceeded, the calculation process of the direction information using the position information is stopped, or the direction is determined by the moving average of the position information in the predetermined time. This is a driving guide system that calculates information.

By configuring as in these inventions, an operator of a moving body such as a vehicle can appropriately move the moving body according to a running guide even in a space having two or more degrees of freedom such as a runway. It becomes. Therefore, it is not necessary to install a cone to control the traveling position of the moving body, the working time can be shortened, and safety can be ensured. In addition, when positioning position information, errors may occur randomly due to the limit of sensor accuracy. In that case, if the direction information is specified from the change in the position information and an image (virtual viewpoint image) to be output is generated using the direction information, the direction will be randomly distorted. Therefore, when the moving body is being moved at high speed, the operator may see the output image and feel that the moving body is swaying in the rotation direction, which may cause a phenomenon of being dazzled. Therefore, by configuring as in these inventions, it is possible to suppress fluctuations in the direction of the output virtual viewpoint image. Further, in the invention of claim 3, the moving body can be moved to a desired management target location from the management target locations such as damages specified in advance. Therefore, maintenance work becomes easy.

In the above-described invention, the virtual space information storage unit is configured like a traveling guide system in which the traveling guide is arranged in front of the driver of the moving body as the virtual space information. You can also do it.

When driving on a normal road, white lines such as guide lanes can be seen on both sides of the vehicle, and the driver travels straight with that as a guide, but from the driver's point of view, the white line is the driver's line of sight. Since there is an offset from the line of sight to the left and right instead of the front, it cannot be said that it is the optimum way to give visual information in order to drive accurately in the center of the lane. However, as in the present invention, if the driving guide is arranged so as to be in front of the driver in the virtual space information, the driving guide is projected and displayed in front of the driver's line of sight. Therefore, the driver only has to drive so that the driving guide always comes in front of him / her, and more accurate driving becomes possible.

In the above-described invention, the virtual viewpoint image generation processing unit further acquires three-axis angle information from the three-axis inertial sensor provided in the moving body or the projection device, and obtains the elevation angle and the horizontal rotation direction of the virtual viewpoint. , It can be configured like a traveling guide system that generates the virtual viewpoint image by reflecting it as a twist angle.

The posture of the projection device installed on the moving body changes depending on the unevenness of the place where the moving body moves and the acceleration and deceleration of the moving body, and the projection position of the virtual viewpoint image to be projected becomes unstable. .. Therefore, by configuring as in the present invention, the projection position of the virtual viewpoint image can be displayed at substantially the same position.

In the above-described invention, the management target location storage unit further stores the management target image in association with the position information of the management target location, and the output processing unit stores the acquired position information and the management. By comparing with the position information of the target location, it is determined whether the moving body has entered the predetermined range from the management target location, and when it is determined that the moving body has entered the predetermined range, the management target location is determined. It can be configured like a driving guide system that extracts and outputs the managed image of the managed part from the storage unit.

With the configuration as in the present invention, when the moving body approaches the management target location, it is possible to output a management target image in which one or more of the size, position, direction, etc. are roughly matched to the management target location. Therefore, the person in charge of maintenance work can identify the approximate position of the managed part by looking at it and proceed with the maintenance work promptly.

The invention of claim 1 can also be realized by executing the program of the present invention on a computer. That is, it is a virtual space corresponding to a part or all of the real space including the range for guiding the traveling position to the operator of the moving body, and the traveling guide indicating the traveling position where the operator should move the moving body is the virtual space. A computer that can access a storage device that stores virtual space information arranged in a virtual space, a position information acquisition processing unit that acquires position information from the position information acquisition device, a direction information acquisition processing unit that acquires direction information, and the above. A virtual space position conversion processing unit that converts information on the position and orientation of a virtual viewpoint in the virtual space using the acquired position information and direction information, and a virtual space stored in the storage device from the virtual viewpoint. Refer to the virtual viewpoint image generation processing unit that generates a virtual viewpoint image that is an image of the virtual space including the traveling guide, and the management target location storage unit that stores the position information of the management target location using the acquired position information. By doing so, the management target location identification processing unit that identifies the management target location within a predetermined range, the distance and / or direction to the specified management target location is calculated, and the guide image generation process that generates a guide image to guide is generated. A traveling guide program that functions as a unit, an output processing unit that outputs the generated virtual viewpoint image and / or guide image from a projection device or a display device, and is a part of the virtual space or a part of the virtual space as the virtual space information. All are divided into the length direction and the width direction at predetermined intervals, the length direction is represented by the distance from the reference position in the length direction, and the width direction is represented by the lane number from the reference position in the width direction. At the intersection of the lane numbers, a distance marker indicating the distance and the lane number is arranged in the virtual space, and the virtual viewpoint image generation processing unit includes the traveling guide and the distance marker from the virtual viewpoint. This is a driving guide program that generates a virtual viewpoint image .
The invention of claim 2 can also be realized by executing the program of the present invention on a computer. That is, it is a virtual space corresponding to a part or all of the real space including the range for guiding the traveling position to the operator of the moving body, and the traveling guide indicating the traveling position where the operator should move the moving body is the virtual space. A computer that can access a storage device that stores virtual space information arranged in a virtual space, a position information acquisition processing unit that acquires position information from the position information acquisition device, a direction information acquisition processing unit that acquires direction information, and the above. A virtual space position conversion processing unit that converts information on the position and orientation of a virtual viewpoint in the virtual space using the acquired position information and direction information, and a virtual space stored in the storage device from the virtual viewpoint. virtual viewpoint image generation processing unit, a virtual viewpoint image which the generated travel guide to function as an output unit, output from the projection device or a display device for generating a virtual viewpoint image wherein an image of the virtual space including the travel guide In the program, as the virtual space information, a part or all of the virtual space is divided into a length direction and a width direction at predetermined intervals, and the length direction is represented by a distance from a reference position in the length direction. The width direction is represented by the lane number from the reference position in the width direction, and at the intersection of the distance and the lane number, a distance marker indicating the distance and the lane number is arranged in the virtual space, and the virtual viewpoint image generation process is performed. The unit is a traveling guide program that generates a virtual viewpoint image including the traveling guide and the distance marker from the virtual viewpoint .

The invention of claim 3 can also be realized by executing the program of the present invention on a computer. That is, it is a virtual space corresponding to a part or all of the real space including a range for guiding the traveling position to the operator of the moving body, and the traveling guide indicating the traveling position where the operator should move the moving body is the virtual space. A computer that can access a storage device that stores virtual space information arranged in a virtual space is a position information acquisition processing unit that acquires position information from the position information acquisition device, a direction information acquisition processing unit that acquires direction information, and the above. A virtual space position conversion processing unit that converts information on the position and orientation of a virtual viewpoint in the virtual space using the acquired position information and direction information, and a virtual space stored in the storage device from the virtual viewpoint. Refer to the virtual viewpoint image generation processing unit that generates a virtual viewpoint image that is an image of the virtual space including the traveling guide, and the management target location storage unit that stores the position information of the management target location using the acquired position information. By doing so, the management target location identification processing unit that identifies the management target location within a predetermined range, the distance and / or direction to the specified management target location is calculated, and the guide image generation process that generates a guide image to guide is generated. A traveling guide program that functions as a unit, an output processing unit that outputs the generated virtual viewpoint image and / or guide image from a projection device or a display device , and the direction information acquisition processing unit is a movement of the moving body. When the speed is compared with a predetermined threshold and it is determined that the threshold is exceeded, the direction information calculation process using the position information is stopped, or the direction information calculation process is performed by the moving average of the position information at the predetermined time. This is a driving guide program.
The invention of claim 4 can also be realized by executing the program of the present invention on a computer. That is, it is a virtual space corresponding to a part or all of the real space including the range for guiding the traveling position to the operator of the moving body, and the traveling guide indicating the traveling position where the operator should move the moving body is the virtual space. A computer that can access a storage device that stores virtual space information arranged in a virtual space is a position information acquisition processing unit that acquires position information from the position information acquisition device, a direction information acquisition processing unit that acquires direction information, and the above. A virtual space position conversion processing unit that converts information on the position and orientation of a virtual viewpoint in the virtual space using the acquired position information and direction information, and a virtual space stored in the storage device from the virtual viewpoint. virtual viewpoint image generation processing unit, a virtual viewpoint image which the generated travel guide to function as an output unit, output from the projection device or a display device for generating a virtual viewpoint image wherein an image of the virtual space including the travel guide In the program, the direction information acquisition processing unit compares the moving speed of the moving body with a predetermined threshold, and if it is determined that the threshold is exceeded, the direction information calculation process using the position information is stopped. This is a travel guide program that calculates direction information based on the moving average of position information at a predetermined time .

In the traveling guide system of the present invention, the traveling position is projected on the road surface or displayed on the display position by the projection device mounted on the moving body, and the operator only needs to drive the moving body such as a vehicle accordingly. Because there is, it is easy for the operator. In addition, since it is not necessary to place a cone or the like on the runway on the runway, the working time can be shortened and the safety is not impaired.

It is a conceptual diagram which shows typically an example of the concept of the system structure of the traveling guide system of this invention. It is a conceptual diagram which shows an example of the hardware configuration of a management computer schematically. It is a flowchart which shows typically an example of the processing process of the traveling guide system of this invention. It is a figure which shows typically an example of the state in which a vehicle equipped with a traveling guide system is towing a towing carriage equipped with an image pickup device. It is a figure which shows typically an example of the state which takes an image with an image pickup apparatus when a vehicle is towing a towing carriage. It is a figure which shows an example of the imaging range by an imaging apparatus schematically. It is a figure which shows typically an example of the state which the projection device is attached to a vehicle. It is a figure which shows typically an example which represented the virtual space information as a figure. It is a figure which shows typically the width of a lane. It is a figure which shows an example of the traveling path by a vehicle schematically. It is a figure which shows typically an example of the visual recognition situation from a driver's seat when an image is projected by a projection device. It is a bird's-eye view when an image is projected by a projection device. It is a figure from above which shows the projection range of a traveling guide and a projection device arranged in a virtual space. It is a figure which shows an example of the image of the virtual space to be projected schematically. It is a figure which shows an example of the image of the virtual space to be projected schematically. It is a figure which shows an example of each area of low-speed running and high-speed running of a running path schematically. It is a figure which shows typically an example of the concept of the system structure of the traveling guide system in Example 4. It is a figure which shows an example of the generated guide image schematically. It is a flowchart which shows typically an example of the processing process of the traveling guide system in Example 4. It is a figure which shows typically an example of the case where the management target image is projected on the road surface when approaching a management target place. It is a figure which showed typically the runway in a virtual space. It is a figure which shows typically the camera position (virtual viewpoint) on the runway in a virtual space. It is a figure which shows typically the process which the virtual space position conversion processing unit 23 converts the position information and direction information of a real space into the position information and direction information of a virtual viewpoint in virtual space information.

The traveling guide system 1 of the present invention is mounted on a moving body such as a vehicle 10. In the following description, the case of the vehicle 10 as the moving body will be described, but other moving bodies may be used. Moreover, since the moving body is a vehicle, the operator is a driver. FIG. 1 schematically shows a conceptual diagram of an example of the system configuration of the traveling guide system 1. Further, FIG. 2 schematically shows an example of the hardware configuration of the management computer 2 constituting the traveling guide system 1.

The travel guide system 1 includes a management computer 2, a GPS device 3, a projection device 4 and / or a display device 5. The management computer 2 includes an arithmetic unit 70 such as a CPU that executes arithmetic processing of a program, and a storage device 71 such as a RAM or a hard disk that stores information. In some cases, a communication device for transmitting and receiving the processing result of the arithmetic unit 70 and the information stored in the storage device 71 via a network such as the Internet or LAN may be provided.

Although FIG. 1 shows a case where the traveling guide system 1 is realized by one computer, the functions may be distributed and realized by a plurality of computers.

Each means in the present invention has only a logical distinction in its function, and may form the same area physically or substantially.

FIG. 4 schematically shows an example of a state in which the towing carriage 11 is attached to the vehicle 10 as a moving body. In FIG. 4, a towing carriage 11 equipped with an image pickup device 6 is towed behind the vehicle 10, and the image pickup device 6 of the towing carriage 11 photographs the road surface. FIG. 5 is a diagram schematically showing a state in which an image pickup device 6 captures an image when the vehicle 10 is towing the towing carriage 11.

The towing carriage 11 is formed of a metal frame 9 such as iron or stainless steel, and wheels are attached to the rear left and right sides of the towing carriage 11. Further, a hanger 8L to which a lighting device 7 such as an LED lamp for illuminating an image pickup range by the image pickup device 6 is attached and a hanger 8C to which the image pickup device 6 is attached are attached to the frame body 9. Further, a GPS device 3 is attached above the image pickup device 6. The image pickup device 6 is attached to the hanger 8C so that the lens is directed toward the road surface so as to image the road surface, and the lighting device 7 is also attached to the hanger 8L so as to irradiate the road surface with light. In the example of FIG. 4, two image pickup devices 6a and 6b are attached in the left-right direction of the towing carriage 11. In the following description, the image pickup device 6a on the left side of the towing carriage 11 in FIG. 4 may be referred to as a first image pickup device, and the image pickup device 6b on the right side may be referred to as a second image pickup device.

The lighting device 7 of the tow truck 11 illuminates the road surface with illumination when taking an image, and each of the image pickup devices 6a and 6b takes an image of the road surface for a certain period of time, for example, every 1/30 second. Then, since the vehicle 10 moves at a constant speed while towing the towing carriage 11, the road surface can be imaged as shown in FIG. At this time, it may be attached to the towing carriage 11 so that the imaging ranges of the first imaging device 6a and the second imaging device 6b partially overlap. This is because the captured images partially overlap, and by detecting the overlap, adjacent images can be accurately combined.

In addition, the time interval for imaging may be such that the previous imaging range and the current imaging range partially overlap. This is also because the captured images partially overlap, and by detecting the overlap, adjacent images can be accurately combined. Since the imaging interval also depends on the traveling speed of the vehicle 10, it is possible to select and set an appropriate value in association with the traveling speed of the vehicle 10. It should be noted that the vehicle speed sensor may be configured to receive information on the traveling speed of the vehicle 10 and automatically set the imaging interval from the traveling speed.

Information is transmitted and received between the management computer 2 attached to the vehicle 10 and each device attached to the towing carriage 11 as appropriate by wire or wirelessly. Further, the management computer 2 may be attached to the towing carriage 11.

In the embodiment of the present invention, the case of the vehicle 10 and the towing carriage 11 will be described, but they may be composed of one vehicle. Further, the configuration of the towing carriage 11 may be any configuration as long as the road surface can be imaged by the imaging device 6.

The projection device 4 may be installed at any position as long as it can project the front in the traveling direction of the vehicle 10. For example, the roof of the vehicle 10, the vicinity of the ceiling of the driver's seat or the passenger's seat, and the vicinity of the windshield. It may be installed in. FIG. 7 schematically shows an example of a state in which the projection device 4 is attached to the vehicle 10. FIG. 7A schematically shows an example of a state in which the projection device 4 is attached to the roof of the vehicle 10, and FIG. 7B is a state in which the projection device 4 is attached to the passenger seat of the vehicle 10. An example is schematically shown.

It is preferable that the GPS device 3 is attached to the position of the projection device 4, but when the attachment positions of the projection device 4 and the GPS device 3 are different, the positions of the projection device 4 and the GPS device 3 are moved back and forth. It is more accurate to manage the amount of offset in each of the left and right directions, correct the position information acquired by the GPS device 3 with the above-managed offset, and calculate the position of the projection device 4. The position information of the projection device 4 can be acquired.

A projector can be used as the projection device 4. For example, a business projector "EB-S03" (brightness 2600 lm (lumen), number of pixels 800 x 600) manufactured by Seiko Epson can be used, but the present invention is not limited thereto. With the performance of the current relatively inexpensive imaging device 6 and lighting device 7, it is preferable that the vehicle 10 travels at a speed of 30 km / h or less in order to image the road surface well. In that case, if the projection device 4 can project up to about 30 meters ahead, sufficient feedback information can be provided to the driver. Therefore, it is preferable that the performance of the projection device 4 is such that it can project up to about 30 meters away. Further, when projecting in a dark place such as a runway at night, the projection device 4 can project up to 30 meters even if an inexpensive projector having a brightness of 3000 lumens or less is used. ..

The installation position and projection angle of the projection device 4 are adjusted so that the projection device 4 can be projected onto a projection target, for example, a road surface. Projectors usually project diagonally upwards when installed horizontally, and project only a small angle below horizontal. Therefore, when a projector is used as the projection device 4, it is preferable to install the projector upside down. As a result, it is possible to project downward (toward the road surface) from the horizontal position of the projector, so that a wide projection range by the projector can be secured.

The display device 5 may be mounted at a position that can be visually recognized by the driver of the vehicle 10. When a transmissive display or a transmissive screen is used as the display device 5, a sun installed near the upper part of the dashboard or on the roof of the driver's seat so that the display device 5 is located in front of the driver's line of sight. It can be installed like a head-up display below the visor.

The management computer 2 of the traveling guide system 1 includes a position information acquisition processing unit 20, a direction information acquisition processing unit 21, a virtual space information storage unit 22, a virtual space position conversion processing unit 23, a virtual viewpoint image generation processing unit 24, and an output processing unit. It has 25 and.

The position information acquisition processing unit 20 acquires latitude and longitude information as position information by a device that acquires position information such as a GPS device 3. As the position information acquisition device, in addition to the latitude and longitude information obtained by the GPS device 3, various sensors such as a gyro sensor may be used to acquire the position from the traveling start position.

The direction information acquisition processing unit 21 acquires information on the traveling direction of the vehicle 10 by the GPS device 3. Information on the traveling direction of the vehicle 10 can be acquired by various sensors such as a gyro sensor and an acceleration sensor, but the traveling direction is acquired by a change in the position information acquired by the position information acquisition processing unit 20. You may. The various sensors may be installed in a device such as the GPS device 3, or the sensors installed in the vehicle 10 may be used.

The virtual space information storage unit 22 has modeled a part or all of the real space in which the driver of the vehicle 10 outputs an image by the output processing unit 25 described later to guide the traveling in two or three dimensions. Store as spatial information (virtual space information). It is preferable that the scale of the real space and the virtual space is the same magnification (1x), but it is modeled as a virtual space with a magnification different from that of the real space, and the magnification is changed during output processing to correspond to the real space. It may be output so as to.

The virtual space information further indicates to the driver the position on the virtual space corresponding to the position where the vehicle should travel in the real space, the position where the vehicle 10 should travel, and the position of the management target location which is the destination. Guides are arranged as objects. For example, when the virtual space information is an airport runway, the runway in the imaging range is modeled as a virtual space, and on the road surface of the runway in the virtual space, a triangular mark, which is a travel guide, is an object in the real space. The vehicle 10 is arranged so as to correspond to the position of the runway on which the vehicle 10 should travel. A triangular mark that can indicate the direction of travel is preferable as a traveling guide, but the traveling guide is not limited to that, and any symbol such as a line, an arrow, or a circle, a character, a number, or a figure may be used. .. Since the traveling guide is displayed at the place to be projected by the projection device 4, it is preferable that the traveling guide is arranged on the road surface of the runway even in the virtual space.

The runway in the virtual space is divided into length and width directions at predetermined intervals, and the length direction is represented by the distance from the predetermined position of the runway imaging range (for example, 0 to 3000 in meters), and the width direction is defined as width. It is represented by a lane number for each predetermined width in consideration of the shooting range from the reference position in the direction (for example, the center lane number 0 is used as a reference, and the left and right lane numbers are -10 to +10). At the intersection of the distance and the lane number, a distance marker indicating the distance and the lane number is arranged as an object on the road surface of the runway.

FIG. 8 shows an example of virtual space information when a runway in a real space is three-dimensionally modeled as a virtual space and a travel guide and a distance marker are arranged as objects on the road surface of the runway. FIG. 8A is a bird's-eye view of a part of the virtual space information, and FIG. 8B is a part of the virtual space information shown from above. Note that FIG. 8B shows a case where the distance between the lanes in the virtual space is 3 meters and the distance between the distance markers in the length direction of the virtual space is 5 meters, but the present invention is not limited to this.

The width of the lane is preferably slightly narrower than the imaging range of the image pickup device 6 attached to the towing carriage 11. This is schematically shown in FIG. By determining that they are adjacent images based on the position information associated with the captured image and detecting the overlap between the determined images, the adjacent images are combined adjacent to one image.

The distance of 3 meters between the lanes in FIGS. 8 (b) and 9 is a case where the total imaging range of the two imaging devices 6a and 6b installed on the towing carriage 11 is 4 meters. In this case, if the vehicle 10 reciprocates with the lanes at intervals of 3 meters, the imaging range overlaps with that of traveling in the adjacent lane by 1 meter. Therefore, if the blurring when the vehicle 10 is driven is within 50 cm to the left and right, the imaging range will not be missed.

In the virtual space, it is sufficient to have a driving guide, which is the minimum information necessary for the driver to drive the vehicle 10, but in addition to that, objects such as the above-mentioned distance markers and targets are arranged as virtual space information. It may have been done.

The virtual space position conversion processing unit 23 virtually uses the position information acquired by the position information acquisition processing unit 20, the direction information acquired by the direction information acquisition processing unit 21, and the height information of the projection device 4 set in advance. The conversion process is performed on the position information of the viewpoint (virtual viewpoint) in the virtual space information stored in the spatial information storage unit 22 and the information of the direction of the viewpoint.

In the virtual space position conversion processing unit 23, the process of converting the position information and the direction information of the real space into the position information and the direction information of the virtual viewpoint in the virtual space information can be realized by performing the following processing, for example. ..

As an example, in the virtual space information storage unit 22, a runway (width 60 meters, distance 3000 meters) in the real space and a runway (width 60 meters, distance 3000 meters) whose scale is 1 times are three-dimensionally modeled. ing. Then, the traveling guide and the distance marker are arranged as objects on the runway in the virtual space. It is assumed that such virtual space information is stored in the virtual space information storage unit 22. FIG. 21 shows an example of a runway modeled as a virtual space. In FIG. 21, the runway has an X-axis in the width direction and a Y-axis in the distance direction. Then, of the width of 60 meters, the center (position 30 meters from the edge) is set to X = 0.

When the projection device 4 is installed on the roof of the vehicle 10, the height of the projection device 4 does not change when the output processing unit 25 performs the projection process. Therefore, the height of the projection device 4 in the real space corresponds to the Z coordinate in the virtual space. Therefore, the parameters that control the virtual viewpoint (camera position and orientation) in the virtual space are the offset of the camera position from the origin (x1, y1) and the orientation θ of the camera. This is schematically shown in FIG. When the height of the projection device 4 changes in the real space, the height information acquired from the GPS device 3 may be used as the Z coordinate in the virtual space.

Based on such a premise, the virtual space position conversion processing unit 23 executes the conversion processing shown in FIG. 23. First, when the position information acquisition processing unit 20 acquires latitude and longitude information as position information, it is converted to a plane orthogonal coordinate system (x, y) based on, for example, 2002 Ministry of Land, Infrastructure, Transport and Tourism Notification No. 9. To do. Since there is a deviation in the coordinate system between the plane orthogonal coordinate system and the virtual space coordinate system, the coordinates of the plane orthogonal coordinate system (x, y) are further added to the coordinates of the virtual space based on the angle φ of the deviation. Convert to a system. The plane orthogonal coordinate system (X1, Y1) of the start point on the runway is calculated in advance based on the latitude and longitude information as the position information.

After the coordinate system conversion processing is performed as described above, the conversion is performed to the position (xc, yc) of the virtual viewpoint in the virtual space information. By performing such processing, the position information acquired by the position information acquisition processing unit 10 can be converted into the position of the virtual viewpoint.

That is, the position information conversion process in the virtual space position conversion processing unit 23 is
xc = (X1-x) x cos (φ)-(y-Y1) x sin (φ)
yc = (X1-x) × sin (φ) + (y−Y1) × cos (φ)
Can be converted to the position of the virtual viewpoint (xc, yc) by calculating.
The X-axis of the plane orthogonal coordinate system is the axis that coincides with the meridional line at the origin of the coordinate system, the value toward the true north is positive, and the Y-axis of the coordinate system is orthogonal to the X-axis of the coordinate system at the origin of the coordinate system. The axis is set, and the value toward the true east is set as positive. Therefore, the calculation formula is such that X and Y are interchanged with the normal coordinate system.

The orientation information in the virtual viewpoint can be converted by adding the offset θ with the direction of 0 degrees in the real space to the direction of the runway in the real space.

When the display is performed by the display device 5 instead of the projection by the projection device 4, the height information and the position information of the projection device 4 described above are read as the height information and the position information of the display device 5.

The virtual viewpoint image generation processing unit 24 is based on the position information and orientation information (direction information) of the virtual viewpoint converted by the virtual space position conversion processing unit 23, and the virtual space image (virtual viewpoint image) that can be seen from the position information. To generate. That is, the virtual viewpoint image is an image of the virtual space that is transferred when the virtual space is imaged from the camera installed in the virtual viewpoint in the virtual space information stored in the virtual space information storage unit 22.

The output processing unit 25 projects the virtual viewpoint image generated by the virtual viewpoint image generation processing unit 24 on the road surface by the projection device 4 or displays it on the display device 5.

Projection can be performed by the projection device 4 at night, but cannot be projected from the ambient brightness during the daytime. Therefore, in the daytime, the output processing unit 25 may provide a display device 5 in the vehicle 10 and the output processing unit 25 may display the display device 5 instead of using the projection device 4.

When the vehicle 10 has to move backward for some reason, the original traveling direction and the direction may be recognized in the opposite direction, and the output processing unit 25 may project an image in the opposite direction to the original traveling direction. is there. Therefore, when it is determined that the direction is opposite to the direction of travel up to that point (when the direction information in the direction opposite to the direction information acquired by the direction information acquisition processing unit 21 is acquired, or when the direction information built in the GPS device 3 is acquired. When the direction of the magnetic needle is compared with the direction information acquired by the direction information acquisition processing unit 21 and it is determined that the vehicle 10 is moving backward, the output processing unit 25 does not perform the output processing. You may.

Further, the output processing unit 25 may be configured to stop the output processing, and the imaging devices 6a and 6b may be configured to stop the imaging processing.

Next, an example of the processing process using the traveling guide system 1 of the present invention will be described with reference to the flowchart of FIG.

First, the projection device 4 in the traveling guide system 1 is installed at a predetermined position of the vehicle 10, for example, on the roof. At this time, the height information of the projection device 4 from the road surface is measured in advance and input to the traveling guide system 1. Further, a towing carriage 11 on which the imaging device 6 and the lighting device 7 are installed is attached to the vehicle 10 so that it can be towed.

The vehicle 10 starts traveling in advance from a predetermined lane on the runway, for example, the leftmost lane (S100). For example, as shown in FIG. 10, the running starts from the starting point on the runway. At the same time as the start of traveling, the GPS device 3 starts positioning of position information and direction information. Further, since the lighting device 7 attached to the tow truck 11 irradiates the road surface, the image pickup device 6 starts imaging.

The position information acquisition processing unit 20 acquires latitude and longitude information as position information acquired by the GPS device 3 (S110). Similarly, the direction information acquisition processing unit 21 acquires the direction information acquired by the GPS device 3 (S110). Using the position information and direction information acquired in this way and the height information of the projection device 4, the virtual space position conversion processing unit 23 converts the information into information on the camera position and camera direction in the virtual space. That is, it is converted into the X coordinate, the Y coordinate, the Z coordinate, and the camera direction as the camera position in the virtual space (S120). This camera position corresponds to the X-axis, Y-axis, and Z-axis coordinate information of the virtual viewpoint in the virtual space corresponding to the projection device 4 in the real space, and the camera direction corresponds to the projection device 4 in the real space. ， Corresponds to the projection direction in virtual space.

Then, based on the information of the camera position (X coordinate, Y coordinate, Z coordinate in the virtual space) and the camera direction in the virtual space converted in S120, the virtual viewpoint image generation processing unit 24 determines the camera position and the camera in the virtual space. From the direction, an image of the virtual space when the runway of the virtual space including the object of the traveling guide and the distance marker is photographed is generated (S130). As a result, an image to be projected (virtual viewpoint image) can be generated.

Then, the output means projects (outputs) the image generated by the virtual viewpoint image generation processing unit 24 from the projection device 4 (S140). By this projection, the traveling guide and the distance marker in the virtual space are projected toward the road surface in the real space. 11 to 13 show an example when the virtual viewpoint image is projected on the runway by the projection device 4. FIG. 11 schematically shows an example of the visual recognition situation from the driver's seat when the virtual viewpoint image is projected by the projection device 4. The center line in FIG. 11 is a line originally drawn from the runway. FIG. 12 is a bird's-eye view when a virtual viewpoint image is projected by the projection device 4. FIG. 13 is a view from above showing the projection range of the traveling guide and the projection device 4 arranged in the virtual space.

The above processes S110 to S140 are performed, for example, every 1/10 second until the vehicle 10 stops traveling (S100). The projection interval can be arbitrarily determined by the acquisition interval of the position information and the like by the GPS device 3, and is not limited to the 1/10 second interval, and can be shorter or longer than that.

The driver may drive along the traveling guides (triangular marks shown in FIGS. 11 and 12) projected by the projection device 4. Further, even when the vehicle 10 is traveling, the GPS device 3 acquires position information and the like at 1/10 second intervals and changes the projected image sequentially, so that the traveling direction of the vehicle 10 is changed. A traveling guide suitable for the vehicle is displayed. Then, even if the vehicle 10 is displaced to the left or right from the traveling guide, the traveling guide is projected at substantially the same position as if it were painted on the road surface, so that the driver can see that the vehicle 10 is the traveling guide. The vehicle 10 can be returned to the original running position by immediately returning the steering wheel so as to return to the running position. Furthermore, by displaying the distance marker, it becomes easier to accurately grasp one's own running position even on a runway without a target.

While the vehicle 10 is traveling, the image pickup device 6 sequentially images the road surface of the runway at predetermined intervals. The time interval for imaging can be arbitrarily determined. It may be the same time interval as the time interval for acquiring the position information or the like in the GPS device 3, or it may be a different time interval. In addition, the mileage may be sensed and images may be taken at regular intervals. At this time, it is preferable that the image of the road surface of the runway is stored in association with the position information acquired by the GPS device 3. Thereby, the position of the captured image information can be specified. When the image pickup device 6 and the GPS device 3 are installed at distant positions, their offsets are managed in advance, and the offset is calculated based on the position information acquired by the GPS device 3 to correct the image pickup device 6. It is preferable to specify the exact position of the captured image information after calculating the position.

14 and 15 schematically show an example of an image of the virtual space to be projected. FIG. 14 is a virtual viewpoint image when traveling around 40 meters in the 0th lane, and FIG. 15 is a virtual viewpoint image when traveling near 150 meters in the 0th lane. Therefore, the virtual viewpoint image of FIG. 14 is generated by the virtual viewpoint image generation processing unit 24 near 40 meters when traveling in the 0th lane, and the output processing unit 25 projects the virtual viewpoint image from the projection device 4. To do. Further, in the vicinity of 150 meters when traveling in the 0th lane, the virtual viewpoint image of FIG. 15 is generated by the virtual viewpoint image generation processing unit 24, and the output processing unit 25 projects the virtual viewpoint image from the projection device 4. To do. The generated virtual viewpoint image is projected by the output processing unit 25 via the projection device 4, but the driver can see the image of the area actually projected on the road surface.

In FIG. 14, since the driving guide is displayed in front of the driver's seat (driver), the driver is driving the vehicle 10 at the correct driving position, but in FIG. 15, the vehicle 10 is displaced to the right side of the driving guide. ing. Therefore, the driver is required to turn the steering wheel to the left and correct the driving position so that the driving guide is in front of the driver's seat (driver).

Since the traveling guide system 1 of the present invention captures the road surface with the imaging device 6, it is an object of the present invention to appropriately guide the driver to what position the vehicle 10 should travel. For example, in FIG. It will travel on the route shown. At this time, high-speed driving is possible in a straight line, but low-speed driving is inevitable in a place where the traveling direction is changed. This is schematically shown in FIG.

In the GPS device 3, since the direction (direction information) is usually determined from the change in the position information, the direction determination accuracy is not high and the accuracy of the direction information is limited. Therefore, since an error occurs randomly for each measurement, an image (virtual viewpoint image) to be output is generated by using the direction information as it is, and the generated image is projected on the road surface by the projection device 4 or displayed by the display device 5. Then, the direction is randomly distorted. Therefore, in the region where the vehicle 10 is driven in a straight line at high speed, the driver sees the projected image and feels that the vehicle 10 is swaying in the rotation direction, which causes a phenomenon of being dazzled.

Therefore, in this embodiment, in order to alleviate this phenomenon, the traveling guide system 1 generates a virtual viewpoint image that processes and outputs the position information and the direction information separately. In this case, the position information is updated at a frequency of 1/10 second, for example, but the direction information is updated while the vehicle is traveling at high speed, such as when the vehicle speed exceeds the threshold of 20 km / h. , The calculation process of the direction information using the two position information of the GPS device 3 of the direction information acquisition processing unit 21 is stopped, or instead of the two position information of the GPS device 3, for example, for a predetermined time (for example, the latest 1 second). It may be configured to perform the calculation process of the direction information by the moving average.

By changing the direction information calculation process according to the vehicle speed in this way, it is possible to suppress fluctuations in the direction of the output image.

By using the processing of this embodiment, the driver will not receive accurate feedback of the direction information or will be delayed (even if the direction is deviated, it will not be immediately reflected in the virtual viewpoint image), but the position information Is updated at the same frequency as normal processing, for example, at a frequency of 1/10 second, so the left and right deviations from the lane where the vehicle 10 is supposed to run are fed back to the driver in real time to ensure accurate straight-line driving. Enough to do.

If the assumed lane in which the vehicle 10 travels is curved instead of straight, some feedback on the directional information is required, so the directional information acquisition processing unit 21 cancels the directional information calculation process. It is preferable to use a processing method in which the direction information is calculated by the moving average of the latest predetermined time interval rather than the method.

Further, in order to ensure the stability of the image projected by the traveling guide system 1 of the present invention, a 3-axis inertial sensor may be provided.

That is, the posture of the projection device 4 installed on the vehicle 10 changes depending on the condition of the unevenness of the road surface and the conditions of acceleration and deceleration of the vehicle 10, and the projection position of the projected image becomes unstable. For example, when the vehicle 10 accelerates or decelerates, the projection device 4 may face upward or downward than usual, or when traveling in a curved lane, the projection device 4 may tilt more than usual due to centrifugal force or the like. In order to alleviate such a state and display at the same position as much as possible, a 3-axis inertial sensor may be further installed in the vehicle 10.

In this case, the traveling guide system 1 obtains the angle information of the pitch axis, the yaw axis, and the roll axis from the 3-axis inertial sensor at the frequency of image projection (once every 1/10 second in the above embodiment). , Each parameter is reflected in the elevation angle of the camera in the virtual space, the direction in the horizontal rotation direction, and the twist angle (the camera is photographed diagonally instead of horizontally), and the virtual viewpoint image generation processing unit 24 generates a virtual viewpoint image. , The output processing unit 25 projects from the projection device 4, so that the projection position of the virtual viewpoint image can be stabilized.

In this embodiment, a case where a traveling guide is provided to a managed part to be managed such as a damaged part of a facility or a road surface will be described. In this case, as the traveling guide system 1, an image (management target image) obtained by extracting the management target location from the image captured by the image pickup device 6 or the like is stored, and the management target location is used by using the management target image. This is a case where the driver of the vehicle 10 is configured to be guided to a place. An example of the configuration of the traveling guide system 1 in this embodiment is shown in FIG. The managed image in the present specification means that a part to be managed such as a damaged part is recognized by means such as image recognition from the image captured by the image pickup apparatus 6 or the like, and the recognized part is two. It is an image such as a CAD drawing that has been converted into a value image or a vector image.

The traveling guide system 1 of this embodiment further includes a management target location storage unit 26, a management target location identification processing unit 27, and a guide image generation processing unit 28. In addition to the functions of the first to third embodiments in the traveling guide system 1, each of the above-mentioned functions may be provided, but the virtual space information storage unit 22, the virtual space position conversion processing unit 23, and the virtual viewpoint image generation It is not necessary to have a part or all of the functions of the processing unit 24 and the like.

The management target location storage unit 26 stores the management target image and its position information (latitude, longitude information, direction information, scale information). It is preferable that the managed image and the position information are stored in association with the identification information that identifies them. When the management target image is superimposed and displayed on the management target location on the road surface, the management target location may be further cut out from the management target image and managed as an enlarged / reduced or rotated image.

The management target location identification processing unit 27 identifies a management target location within a predetermined range from the position information (current position information of the vehicle 10) acquired by the position information acquisition processing unit 20 by the GPS device 3. .. Specifically, the position information acquired by the position information acquisition processing unit 20 is compared with the position information stored in the management target location storage unit 26, and the management within a predetermined range from the acquired position information, for example, within a radius of 100 meters. Identify the identification information of the target location.

The guide image generation processing unit 28 calculates the distance and direction from the current position to the specified management target location based on the identification information of the management target location specified by the management target location identification processing unit 27, and guides the distance and direction. Generate a guide image. An example of the generated guide image is shown in FIG.

In the guide image shown in FIG. 18, three concentric circles are drawn centering on the position information acquired by the position information acquisition processing unit 20. When the first concentric circle, the second concentric circle, and the third concentric circle are defined from the inside, the third concentric circle means 100 meters, which is a predetermined range from the above-mentioned acquired position information, and each concentric circle is a distance of 1/3. Means. The arrows extending in each direction from the third concentric circle are the directions in which the management target location exists, the "#" symbol attached to the arrow is the identification information indicating the management target location image, and the number next to it is the relevant. The distance to the management target location. The length of the arrow is displayed in proportion to the distance. The black circles in the concentric circles indicate the distance and direction to each management target location when the origin of the concentric circles is the position information acquired by the position information acquisition processing unit 20. That is, when 100 meters is set as a predetermined range, it serves as a guide that roughly indicates in which direction and position the management target location is located.

The guide image is not limited to the display method shown in FIG. 18, and may be any display method as long as the traveling direction is displayed at least.

Next, an example of the processing process of the traveling guide system 1 in this embodiment will be described with reference to the flowchart of FIG. In the following description, it is assumed that the management target image, the position information, the damage level, the identification information, and the like are stored in the management target location storage unit 26.

In the vehicle 10 equipped with the traveling guide system 1 in this embodiment, the position information acquisition means acquires the position information by the GPS device 3 (S210). Then, based on the acquired position information, the management target location identification processing unit 27 identifies from the acquired location information whether there is a management target location within a predetermined range, for example, 100 meters (S220).

When the management target location identification processing unit 27 specifies the management target location within the predetermined range, the guide image generation processing unit 28 compares the position information acquired in S200 with the location information of the specified management target location. The direction and distance are calculated (S230). Then, a guide image is generated by calculating the direction and distance to each specified management target location (S240).

The guide image generated by the guide image generation processing unit 28 is projected by the output processing unit 25 by the projection device 4 (S250).

In this way, the processes from S210 to S250 are repeated until the vehicle 10 finishes traveling. The frequency of repetition may be, for example, the frequency of acquiring position information by the GPS device 3, for example, every 1/10 second, but is not limited thereto.

As a result, the driver of the vehicle 10 can move to a desired management target location by driving while looking at the guide image projected on the road surface.

In the above processing, when the vehicle 10 approaches a specific management target location, for example, within 10 meters from the damage, the output processing unit 25 is managed from the management target image storage unit instead of the guide image. The image may be extracted and the managed image may be projected on the road surface. When switching from the display of the guide image, the switching timing can be adjusted based on the positional relationship between the projection device 4 and the managed image. That is, when the projection device 4 projects on the road surface, the distance to the management target location of the projection device 4, which roughly matches the management target location when the management target image is projected, is determined in advance. Therefore, a predetermined distance from the management target location can be set as the switching timing.

FIG. 20 shows an example of a case where the management target image is projected on the road surface when approaching the management target location. In FIG. 20, in addition to the management target image, the identification information of the management target location and the level indicating the degree of damage are projected. In the case of this embodiment, it is preferable that the management target location can be accurately specified, but an error may occur. Therefore, even if it cannot be projected accurately, if the approximate position can be specified, it can actually be sufficiently dealt with. Therefore, since the image to be managed can be projected as in the present invention, the vehicle 10 can be quickly and easily guided to a desired location to be managed. Then, after the vehicle 10 reaches the management target location, a maintenance worker or the like performs maintenance work.

At present, maintenance workers bring a printed drawing of the management target drawing, and at night, while illuminating the drawing with a light on hand or a light attached to a helmet, what seems to be the management target of the road surface and the management target drawing By collating, the parts to be managed to be confirmed and repaired are identified. However, if this embodiment is used, the managed image is superimposed and displayed on the managed location, so that reference and collation are extremely easy and efficient. In addition, since repair instructions and the like are usually written in the image to be managed, the information can be easily and surely transmitted to the person in charge of maintenance work.

Furthermore, using this example, when maintaining a facility buried underground in a road, the underground facility cannot be visually confirmed, so guide the facility to the vicinity of the facility with a traveling guide. It can be used when performing maintenance work repeatedly while projecting a design drawing on the road surface and grasping the rough position. For example, if the pipe buried in the basement of the road is the management target location, the piping diagram and its position information are stored as the management target image. Then, when maintaining the piping, by using the traveling guide system 1 in this embodiment, it is possible to easily reach the place where the piping to be maintained is located, and to project a managed image such as a piping diagram on the road surface. Therefore, maintenance work can be easily performed.

In the description of each of the above-described embodiments, the traveling guide system 1 of the present invention has been described as an example of traveling on a runway, but the present invention is not limited thereto. It can be used to guide the driver for the purpose of management, maintenance, maintenance, or playing in a place where the vehicle 10 can move with "degree of freedom 2" such as a desert, a golf course, or a beach.

Further, the projection device 4 does not have to be installed in the vehicle 10, and may be installed in an air vehicle (such as a helicopter or a UAV such as a drone) as a moving body. In this case, the height of the aircraft, that is, the distance to the object in the height direction and the elevation angle of the projection device 4 will change, but the height information is acquired by the GPS device 3 and as in the third embodiment. A 3-axis inertial sensor is installed on the flying object, and the angle information of the pitch axis, yaw axis, and roll axis is obtained by the frequency of image projection, and each parameter is set to the height, elevation, and horizontal of the camera in the virtual space. The direction of rotation and the twist angle (shooting the camera diagonally instead of horizontally) are further reflected, the virtual viewpoint image generation processing unit 24 generates a virtual viewpoint image, and the output processing unit 25 projects it from the projection device 4. Therefore, the projection position of the virtual viewpoint image can be stabilized. When the projection device 4 is mounted on the UAV, it can be used for the purpose of projecting a traveling guide from the sky in order to guide the traveling or walking movement of the vehicle 10.

In the traveling guide system 1 of the present invention, the traveling position is projected on the road surface by the projection device 4 or the like mounted on the moving body or displayed at the display position 5, and the operator causes the moving body such as the vehicle 10 to travel accordingly. It is easy for the operator because all he has to do is do it. In addition, since it is not necessary to place a cone or the like on the runway, the working time can be shortened and the safety is not impaired.

In addition, by using the travel guide system in Example 4, when maintaining a facility buried underground in a road, a travel guide guides the user to the vicinity of the underground facility, which cannot be visually confirmed, and then draws a design drawing or the like. Since maintenance work can be performed repeatedly while projecting onto the road surface and grasping the rough position, the work time can be shortened for the maintenance work person, and safety is not impaired.

1: Travel guide system 2: Management computer 3: GPS device 4: Projection device 5: Display device 6: Imaging device 7: Lighting device 8: Hanging tool 9: Frame body 10: Vehicle 11: Tow truck 20: Position information acquisition processing Unit 21: Direction information acquisition processing unit 22: Virtual space information storage unit 23: Virtual space position conversion processing unit 24: Virtual viewpoint image generation processing unit 25: Output processing unit 26: Management target location storage unit 27: Management target location identification processing Unit 28: Guide image generation processing unit 70: Arithmetic device 71: Storage device

Claims (11)

It is a traveling guide system that guides the operator of a moving body to the traveling position of the moving body.
The traveling guide system is
The location information acquisition processing unit that acquires location information from the location information acquisition device, and
Direction information acquisition processing unit that acquires direction information, and
A virtual space corresponding to a part or all of the real space including a range for guiding the traveling position to the operator of the moving body, and the traveling guide indicating the traveling position where the operator should move the moving body is the virtual space. A virtual space information storage unit that stores virtual space information arranged in space,
A virtual space position conversion processing unit that converts the acquired position information and direction information into information on the position and orientation of the virtual viewpoint in the virtual space.
In the virtual space stored in the virtual space information storage unit, a virtual viewpoint image generation processing unit that generates a virtual viewpoint image that is an image of the virtual space including the traveling guide from the virtual viewpoint.
A management target location storage unit that stores the location information of the management target location,
The management target location identification processing unit that identifies the management target location within a predetermined range by referring to the management target location storage unit using the acquired position information, and the management target location identification processing unit.
A guide image generation processing unit that calculates the distance and / or direction to the specified management target location and generates a guide image to guide.
It has an output processing unit that outputs the generated virtual viewpoint image and / or guide image from the projection device or display device.
The virtual space information storage unit
As the virtual space information, a part or all of the virtual space is divided into a length direction and a width direction at predetermined intervals, the length direction is represented by a distance from a reference position in the length direction, and the width direction is defined as a width. It is represented by the lane number from the reference position in the direction, and at the intersection of the distance and the lane number, a distance marker indicating the distance and the lane number is arranged in the virtual space.
The virtual viewpoint image generation processing unit
Generate a virtual viewpoint image including the traveling guide and the distance marker from the virtual viewpoint.
A driving guide system that features this. It is a traveling guide system that guides the operator of a moving body to the traveling position of the moving body.
The traveling guide system is
The location information acquisition processing unit that acquires location information from the location information acquisition device, and
Direction information acquisition processing unit that acquires direction information, and
A virtual space corresponding to a part or all of the real space including a range for guiding the traveling position to the operator of the moving body, and the traveling guide indicating the traveling position where the operator should move the moving body is the virtual space. A virtual space information storage unit that stores virtual space information arranged in space,
A virtual space position conversion processing unit that converts the acquired position information and direction information into information on the position and orientation of the virtual viewpoint in the virtual space.
In the virtual space stored in the virtual space information storage unit, a virtual viewpoint image generation processing unit that generates a virtual viewpoint image that is an image of the virtual space including the traveling guide from the virtual viewpoint.
It has an output processing unit that outputs the generated virtual viewpoint image from a projection device or a display device.
The virtual space information storage unit
As the virtual space information, a part or all of the virtual space is divided into a length direction and a width direction at predetermined intervals, the length direction is represented by a distance from a reference position in the length direction, and the width direction is defined as a width. It is represented by the lane number from the reference position in the direction, and at the intersection of the distance and the lane number, a distance marker indicating the distance and the lane number is arranged in the virtual space.
The virtual viewpoint image generation processing unit
Generate a virtual viewpoint image including the traveling guide and the distance marker from the virtual viewpoint.
A driving guide system that features this. It is a traveling guide system that guides the operator of a moving body to the traveling position of the moving body.
The traveling guide system is
The location information acquisition processing unit that acquires location information from the location information acquisition device, and
Direction information acquisition processing unit that acquires direction information, and
A virtual space corresponding to a part or all of the real space including a range for guiding the traveling position to the operator of the moving body, and the traveling guide indicating the traveling position where the operator should move the moving body is the virtual space. A virtual space information storage unit that stores virtual space information arranged in space,
A virtual space position conversion processing unit that converts the acquired position information and direction information into information on the position and orientation of the virtual viewpoint in the virtual space.
In the virtual space stored in the virtual space information storage unit, a virtual viewpoint image generation processing unit that generates a virtual viewpoint image that is an image of the virtual space including the traveling guide from the virtual viewpoint.
A management target location storage unit that stores the location information of the management target location,
The management target location identification processing unit that identifies the management target location within a predetermined range by referring to the management target location storage unit using the acquired position information, and the management target location identification processing unit.
A guide image generation processing unit that calculates the distance and / or direction to the specified management target location and generates a guide image to guide.
It has an output processing unit that outputs the generated virtual viewpoint image and / or guide image from the projection device or display device.
The direction information acquisition processing unit
When the moving speed of the moving body is compared with a predetermined threshold value and it is determined that the threshold value is exceeded, the calculation process of the direction information using the position information is stopped, or the direction is determined by the moving average of the position information in the predetermined time. Performs information calculation processing,
A driving guide system characterized by this. It is a traveling guide system that guides the operator of a moving body to the traveling position of the moving body.
The traveling guide system is
The location information acquisition processing unit that acquires location information from the location information acquisition device, and
Direction information acquisition processing unit that acquires direction information, and
A virtual space corresponding to a part or all of the real space including a range for guiding the traveling position to the operator of the moving body, and the traveling guide indicating the traveling position where the operator should move the moving body is the virtual space. A virtual space information storage unit that stores virtual space information arranged in space,
A virtual space position conversion processing unit that converts the acquired position information and direction information into information on the position and orientation of the virtual viewpoint in the virtual space.
In the virtual space stored in the virtual space information storage unit, a virtual viewpoint image generation processing unit that generates a virtual viewpoint image that is an image of the virtual space including the traveling guide from the virtual viewpoint.
It has an output processing unit that outputs the generated virtual viewpoint image from a projection device or a display device.
The direction information acquisition processing unit
When the moving speed of the moving body is compared with a predetermined threshold value and it is determined that the threshold value is exceeded, the calculation process of the direction information using the position information is stopped, or the direction is determined by the moving average of the position information in the predetermined time. Performs information calculation processing,
A driving guide system that features this. The virtual space information storage unit
As the virtual space information, the traveling guide is arranged so as to be located in front of the driver of the moving body .
The traveling guide system according to any one of claims 1 to 4, wherein the traveling guide system is characterized in that. The virtual viewpoint image generation processing unit further
The virtual viewpoint image is generated by acquiring the angle information of the three axes from the three-axis inertial sensor provided in the moving body or the projection device and reflecting it as the elevation angle, the horizontal rotation direction direction, and the twist angle of the virtual viewpoint.
The traveling guide system according to any one of claims 1 to 5, wherein the traveling guide system is characterized in that. The management target location storage unit further
The management target image is stored in association with the position information of the management target location.
The output processing unit
By comparing the acquired position information with the position information of the management target location, it is determined whether the moving body is within a predetermined range from the management target location.
When it is determined that the range is within the predetermined range, the management target image of the management target location is extracted from the management target location storage unit and output.
The traveling guide system according to claim 1 or 3, wherein the traveling guide system is characterized in that. A virtual space corresponding to a part or all of the real space including a range for guiding the traveling position to the operator of the moving body, and the traveling guide indicating the traveling position where the operator should move the moving body is the virtual space. A computer that can access the storage device that stores the virtual space information located in
Location information acquisition processing unit that acquires location information from the location information acquisition device,
Direction information acquisition processing unit that acquires direction information,
A virtual space position conversion processing unit that converts the acquired position information and direction information into information on the position and orientation of the virtual viewpoint in the virtual space.
A virtual viewpoint image generation processing unit that generates a virtual viewpoint image that is an image of the virtual space including the traveling guide from the virtual viewpoint in the virtual space stored in the storage device.
The management target location identification processing unit that identifies the management target location within a predetermined range by referring to the management target location storage unit that stores the location information of the management target location using the acquired location information.
A guide image generation processing unit that calculates the distance and / or direction to the specified management target location and generates a guide image to guide.
A traveling guide program that functions as an output processing unit that outputs the generated virtual viewpoint image and / or guide image from the projection device or display device.
As the virtual space information, a part or all of the virtual space is divided into a length direction and a width direction at predetermined intervals, the length direction is represented by a distance from a reference position in the length direction, and the width direction is defined as a width. It is represented by the lane number from the reference position in the direction, and at the intersection of the distance and the lane number, a distance marker indicating the distance and the lane number is arranged in the virtual space.
The virtual viewpoint image generation processing unit
Generate a virtual viewpoint image including the traveling guide and the distance marker from the virtual viewpoint.
A driving guide program characterized by this. A virtual space corresponding to a part or all of the real space including a range for guiding the traveling position to the operator of the moving body, and the traveling guide indicating the traveling position where the operator should move the moving body is the virtual space. A computer that can access the storage device that stores the virtual space information located in
Location information acquisition processing unit that acquires location information from the location information acquisition device,
Direction information acquisition processing unit that acquires direction information,
A virtual space position conversion processing unit that converts the acquired position information and direction information into information on the position and orientation of the virtual viewpoint in the virtual space.
A virtual viewpoint image generation processing unit that generates a virtual viewpoint image that is an image of the virtual space including the traveling guide from the virtual viewpoint in the virtual space stored in the storage device.
A traveling guide program that functions as an output processing unit that outputs the generated virtual viewpoint image from a projection device or a display device.
As the virtual space information, a part or all of the virtual space is divided into a length direction and a width direction at predetermined intervals, the length direction is represented by a distance from a reference position in the length direction, and the width direction is defined as a width. It is represented by the lane number from the reference position in the direction, and at the intersection of the distance and the lane number, a distance marker indicating the distance and the lane number is arranged in the virtual space.
The virtual viewpoint image generation processing unit
Generate a virtual viewpoint image including the traveling guide and the distance marker from the virtual viewpoint.
A driving guide program characterized by this. A virtual space corresponding to a part or all of the real space including a range for guiding the traveling position to the operator of the moving body, and the traveling guide indicating the traveling position where the operator should move the moving body is the virtual space. A computer that can access the storage device that stores the virtual space information located in
Location information acquisition processing unit that acquires location information from the location information acquisition device,
Direction information acquisition processing unit that acquires direction information,
A virtual space position conversion processing unit that converts the acquired position information and direction information into information on the position and orientation of the virtual viewpoint in the virtual space.
A virtual viewpoint image generation processing unit that generates a virtual viewpoint image that is an image of the virtual space including the traveling guide from the virtual viewpoint in the virtual space stored in the storage device.
The management target location identification processing unit that identifies the management target location within a predetermined range by referring to the management target location storage unit that stores the location information of the management target location using the acquired location information.
A guide image generation processing unit that calculates the distance and / or direction to the specified management target location and generates a guide image to guide.
A traveling guide program that functions as an output processing unit that outputs the generated virtual viewpoint image and / or guide image from the projection device or display device.
The direction information acquisition processing unit
When the moving speed of the moving body is compared with a predetermined threshold value and it is determined that the threshold value is exceeded, the calculation process of the direction information using the position information is stopped, or the direction is determined by the moving average of the position information in the predetermined time. Performs information calculation processing,
A driving guide program characterized by this. A virtual space corresponding to a part or all of the real space including a range for guiding the traveling position to the operator of the moving body, and the traveling guide indicating the traveling position where the operator should move the moving body is the virtual space. A computer that can access the storage device that stores the virtual space information located in
Location information acquisition processing unit that acquires location information from the location information acquisition device,
Direction information acquisition processing unit that acquires direction information,
A virtual space position conversion processing unit that converts the acquired position information and direction information into information on the position and orientation of the virtual viewpoint in the virtual space.
A virtual viewpoint image generation processing unit that generates a virtual viewpoint image that is an image of the virtual space including the traveling guide from the virtual viewpoint in the virtual space stored in the storage device.
A traveling guide program that functions as an output processing unit that outputs the generated virtual viewpoint image from a projection device or a display device.
The direction information acquisition processing unit
When the moving speed of the moving body is compared with a predetermined threshold value and it is determined that the threshold value is exceeded, the calculation process of the direction information using the position information is stopped, or the direction is determined by the moving average of the position information in the predetermined time. Performs information calculation processing,
A driving guide program characterized by this.

Images