JP2004046875A - Transportation automatic guiding device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a transportation automatic guiding device capable of automatically recognizing temporally changing ambient conditions of transportation such as an automobile, and automatically guiding a travel path such as a road. <P>SOLUTION: In the transportation automatic guiding device 1, image data, sound data or the like regarding the ambient conditions of the running transportation acquired from an input device 2 is compared with image data prestored in a database by a comparing device 4. When the image data, sound data or the like matches as a result of comparison, contents of the data are recognized by a recognition device 5, and a recognized result is notified to an operator or the like by an output device 6 by characters, image, sound or the like. <P>COPYRIGHT: (C)2004,JPO

Description

The present invention automatically recognizes the situation of the operating transportation system, notifies the operator of the situation by letters, images, sounds, etc., guides the operation route, and is suitable for operation. The present invention relates to a transportation automatic guidance device for instructing various actions.

Conventionally, signs and signs installed on roads are automatically read and their meanings are recognized, and the situation of translational vehicles, overtaking vehicles, oncoming vehicles, etc. traveling on the road is recognized, There is no device that automatically informs the driver or passenger of the situation and provides guidance on the road. Therefore, the passenger sitting in the passenger seat reads the signs and signs installed on the road, informs the driver of the contents, and looks at the translation vehicles, overtaking vehicles, oncoming vehicles, etc. running on the road. The driver was informed of the situation, and in some cases, instructed an operation such as stepping on a brake.
On the other hand, in recent years, with the advance of the autopilot technology, a device that detects a white line attached to a road shoulder and automatically drives an automobile along the white line has been put into practical use. In addition, a device that detects the distance from the preceding vehicle by some means and automatically drives the vehicle while maintaining a predetermined distance, a device that automatically drives the vehicle according to a beacon installed on the road, and the like have been studied. It is approaching (for example, see Patent Document 1).
In any case, a device that recognizes signs, signs, etc. installed on the road, and recognizes a translational vehicle, an overtaking vehicle, an oncoming vehicle, etc., traveling on the road, and automatically guides the road on which the automobile travels, Did not exist until.
Such conventional technologies are not limited to vehicles running on the road surface such as automobiles, but also vehicles running on tracks such as trains, and further, airframes such as aircraft, hulls navigating the sea such as ships, etc. The same was true for.

JP-A-10-162285

Here, signs and signs installed on the road, and also automatically recognizes the situation of own vehicle such as preceding vehicle, following vehicle, translation vehicle, overtaking vehicle, oncoming vehicle, etc. In this method, the situation around the vehicle must be captured over time as images, sounds, and the like, and the captured image data, sound data, and the like must be processed at high speed.
However, in order to automatically recognize the situation around the vehicle that changes every moment and automatically guide the road, it is necessary to process a huge amount of image data, audio data, etc. It was difficult to process. Moreover, it was impossible to process in real time.
Such problems are not limited to vehicles traveling on road surfaces such as automobiles, vehicles traveling on tracks such as trains, and furthermore, aircraft traveling in space such as aircraft, and hulls traveling on the sea such as ships. And so on.

The present invention has been made in view of such conventional problems, and is capable of processing an enormous amount of image data, audio data, and the like at a high speed, and is capable of processing an ever-changing transportation system such as an automobile. It is an object of the present invention to provide a transportation automatic guidance device capable of automatically recognizing surrounding conditions and automatically guiding an operation route such as a road.

In order to achieve the above object, a transportation automatic guidance device of the present invention includes an input device that acquires the surroundings of an operating transportation as an image, a sound, and the like, and an image data and a voice data related to the transportation in advance. The result of comparing the stored database, the image data acquired by the input device, the audio data, etc. with the image data, the audio data, etc. stored in the database and the comparison device, which compared the image data, the audio data, etc., were in agreement. In this case, the apparatus is characterized by comprising a recognition device for recognizing and specifying the content of the data and an output device for recognizing the specified result by the recognition device and notifying the operator of the result by text, image, voice, or the like. And

Further, when the image data corresponding to the object acquired by the input device, audio data, and the like do not exist in the database, the image data corresponding to the new object, the audio data, and the like are defined as a position on the map. A storage device to be newly stored in the database in correspondence with the image data, audio data, and the like stored in the database; It is preferable to have a data updating device that updates the data and the like and stores the updated data in the database.
In addition, there is also provided a determination device that makes some determination based on the items recognized or specified by the recognition device and notifies the operator or the like of an instruction based on the determination result by text, image, voice, or the like by the output device. Is preferred.
Further, the determination device makes some determination based on the items recognized or specified by the recognition device, and further instructs the output device to perform a predetermined operation based on the determination result, and controls the braking device, It is preferable that the device automatically operates.

In the transportation automatic guidance device, one or more devices constituting the device may be connected to another device via a communication line.

Furthermore, a plane image conversion device that converts perspective image data about the surroundings of the transportation system acquired by the input device into plane image data that eliminates perspective, and a plane image converted by the comparison device. Based on the result of comparing the data with the image data stored in the database, a planar image recognition device that recognizes and specifies the contents of the data, and an object recognized and identified by the planar image recognition device And an image content measuring device for measuring various spatial physical quantities, and a plane development processing device may be additionally provided.
Further, the plane image conversion device may have a function of converting 360-degree omni-directional image data on the surroundings of the transportation system acquired by the input device into plane image data.

Furthermore, a traffic information detecting device that acquires the surroundings of the transportation facility as image data, measurement data, and the like can be installed on an operation route of the transportation facility, and image data, measurement data, and the like acquired by the traffic information detection device can be received. You may do so.
Here, it is preferable that the traffic information detection device has a graphic device for performing computer graphics based on the acquired image data and measurement data.

Further, an image acquisition unit for acquiring a video by an input device mounted on a transportation facility, an image temporary recording unit for recording the acquired video for a certain period, and a cue point for automatically extracting a cue point for obtaining a corresponding point in the image. An automatic extraction unit, a corresponding point detection unit that extracts two or more images having a distance difference and obtains corresponding points of a plurality of clue points in each image, and a position and a direction of the input device from the detected corresponding points. And a measurement scale conversion unit that converts the relative distance value of the obtained three-dimensional coordinates of the input device position into an absolute distance value using the measurement value. A recognition device may be provided.

Hereinafter, a preferred embodiment of a transportation automatic guidance device of the present invention will be described in detail with reference to the drawings.
As shown in FIG. 1, the transportation automatic guidance device 1 of the present invention includes an input device 2, a database 3, a comparison device 4, a recognition device 5, an output device 6, a determination device 7, and a storage device 8. And a data updating device 9.

The input device 2 is for acquiring a situation around the transportation as an image, a sound or the like when operating the transportation. For example, images are acquired by a video camera, and sound is acquired by a microphone.

The database 3 stores in advance, in the case of a car, a road sign, a road sign, a traffic information board, and the like, and image data, audio data, and the like regarding the car and the like. In the case of a train, a railroad crossing surface, a railroad crossing signal, a moving obstacle such as a car, a platform, and the like, and image data, voice data, and the like regarding the train and the like are stored. In the case of an aircraft, an airport shape, an airport runway, a taxiway, a structure such as a control tower, a moving obstacle such as a car, a moving obstacle such as an automobile, and image data and audio data related to an aircraft are stored. In the case of a ship, the port shape, port quay, wharf, seaway sign, image data, sound data, and the like regarding the ship are stored.

The comparison device 4 compares the image data, audio data, and the like acquired by the input device 2 with the image data, audio data, and the like stored in the database 3.

(4) The recognition device 5 recognizes and specifies the content of the data based on the result of comparing the image data, the audio data, and the like.

The output device 6 notifies the operator of the result recognized and recognized by the recognition device 5 using characters, images, sounds, or the like.

The storage device 8 stores the image data, the audio data, and the like corresponding to the new object when the image data, the audio data, and the like corresponding to the target acquired by the input device 2 do not exist in the database 3 in the first place. It is newly stored in the database 3 in association with the position on the map.

The data updating device 9 is different from the image data, the audio data, and the like, in which the image data, the audio data, and the like corresponding to the object are stored in the database 3 even if the target exists at a predetermined position on the map. In such a case, the data is updated to new image data, audio data, and the like and stored in the database 3.

The judging device 7 makes some judgment based on the items recognized or specified by the recognizing device 5 and notifies the operator based on the judgment result by text, image, sound, or the like by the output device 6. It is.
Further, the determination device 7 may instruct the output device 6 to perform a predetermined operation based on the determination result, and automatically operate a braking device such as a brake and a steering device such as a steering wheel.

Next, when the transportation is an automobile, the automatic transportation apparatus 1 automatically recognizes the situation where the vehicle is placed and notifies the driver or passenger of the situation by text, image, sound, or the like. Then, an example of guiding the road and instructing an appropriate operation when traveling on the road will be specifically described.

As shown in FIG. 2, when a road sign 12 having a speed limit of 50 km is acquired as image data by the input device 2, the road sign 12 is first compared with the image data stored in the database 3 by the comparison device 4, and the recognition device 5. Is recognized and specified as the road sign data at the speed limit of 50 km.
Next, based on the recognition result, the output device 6 notifies the driver or the fellow passenger that the speed limit is 50 km by text, image, sound, or the like.

As described above, according to the automatic transportation apparatus 1, even if the driver or the passenger misses the road sign 12, it is possible to surely know that the speed limit is 50 km on the currently traveling road. It is possible to prevent traffic violations due to excessive speed, and to prevent rear-end collisions and the like.

As shown in FIG. 2, when another vehicle 13 is acquired as image data on the right side of the own vehicle 11 by the input device 2, first, the comparison device 4 compares the image data with the image data stored in the database 3 and performs recognition. The device 5 recognizes that the other vehicle 13 is an overtaking vehicle.
Next, based on the recognition result, the determination device 7 determines that the other vehicle 13 does not accelerate during the overtaking, and the output device 6 instructs the driver or the fellow passenger to not accelerate the vehicle. The notification is made by an image, a sound, or the like.

As described above, according to the automatic transportation apparatus 1, even when the driver or the passenger does not notice that the other vehicle 13 exists on the right side of the own vehicle 11, the other vehicle 13 is overtaking. In addition, by giving an instruction not to accelerate, it is possible to prevent a collision accident or the like without any panic.

As shown in FIG. 2, when the input device 2 acquires a railroad crossing signal 14 in front of which the red light is blinking and the alarm is sounding as image data and audio data, first, the comparison device 4 Is compared with the image data and the audio data stored in the database 3, and the recognition device 5 recognizes that the red light of the front crossing signal 14 is blinking and the alarm is sounding.
Next, based on the recognition result, the determination device 7 makes a determination that the vehicle must stop, and further, considering that it is dangerous to enter a railroad crossing and is prohibited by law, the output device 6 Instruct the driver to apply the brake. As a result, the vehicle is automatically stopped at a safe position before the railroad crossing because the output device 6 operates the brake.

As described above, according to the transportation automatic guidance apparatus 1, even when the driver or the passenger does not notice that the red light of the railroad crossing signal 14 is flashing ahead and the alarm is sounding, Even if you do not take appropriate measures even if you notice, the red light of the level crossing signal 14 will flash, the alarm will sound, and it will be notified that entering the level crossing is dangerous, Therefore, it is possible to instruct and execute an appropriate action of stopping the vehicle, thereby preventing the occurrence of an accident or the like that causes enormous damage.

Next, when the transportation is a train, the automatic transportation guidance device 1 automatically recognizes the situation where the own vehicle is placed, and notifies the driver or passenger of the situation by text, image, voice, or the like. An example of guiding a track and instructing a proper operation when traveling on the track will be specifically described.

As shown in FIG. 3, when the platform 21 is obtained as image data in front of the host vehicle by the input device 2, first, the comparison device 4 compares the image with the image data stored in the database 3, and It is recognized that the stop station is approaching.
Next, based on the recognition result, the determination device 7 determines that the vehicle is decelerating because the stopping station is approaching, and the output device 6 instructs the driver or a passenger to decelerate, and outputs a character, image, or voice. And so on.

As described above, according to the automatic transportation apparatus 1, even when the driver or the passenger does not notice that the platform 21 exists in front of the own vehicle, the driver or the passenger can surely know that the stop station is approaching. In addition, by giving an instruction to decelerate, it is possible to prevent a mistake of passing a stop station due to an error.

As shown in FIG. 3, when the level crossing surface 22, the level crossing signal 23, and the moving obstacle 24 such as a car are acquired as image data and audio data in front of the own vehicle by the input device 2, first, the comparison device 4 Is compared with the image data and the audio data stored in the database 3, and the red light of the forward crossing signal 23 flashes and the alarm sounds by the recognition device 5, and the vehicle is displayed on the crossing surface 22. It is recognized that there is a moving obstacle 24 such as.
Next, based on the recognition result, the determination device 7 determines that an emergency stop must be performed, and furthermore, considering that it is dangerous to enter the railroad crossing surface 22, immediately activates the brake on the output device 6. To do so. As a result, the output device 6 immediately activates the brake, so that the host vehicle is automatically suddenly stopped at a safe position before the railroad crossing surface 22.

As described above, according to the automatic transportation apparatus 1, the driver or the fellow passenger blinks the red light of the level crossing signal 23 in front, sounds the alarm, and moves the vehicle or the like to the level crossing surface 22. The red light of the railroad crossing signal 23 flashes and the alarm sounds even when the driver does not notice that the obstacle 24 is present, or even when he does not take appropriate measures even if he does. Since it is possible to reliably notify that entering the railroad crossing surface 22 is dangerous, and to instruct and execute an appropriate measure of automatically stopping the vehicle suddenly, the occurrence of a railroad crossing accident or the like that causes enormous damage can be prevented. It can be prevented before it happens.

Next, in the case where the transportation means is an aircraft, the automatic transportation means 1 automatically recognizes the situation where the own body is placed, and notifies the situation to the pilot or the passenger by text, image, sound, or the like. An example of guiding the navigation route and the inside of the airport, and navigating the navigation route and instructing an appropriate operation when traveling in the airport will be specifically described.

As shown in FIG. 4, when moving obstacles such as another body 32 and a car 33 are acquired as image data and audio data in front of the own body 31 by the input device 2, first, the comparison device 4 uses the database 3. It is compared with the image data and the audio data stored therein, and the recognizing device 5 recognizes that there is a moving obstacle such as the other body 32 or the automobile 33 in front.
Next, based on the recognition result, the determination device 7 determines that an emergency stop must be performed, and further considers that it is dangerous to move forward, so that the output device 6 immediately activates the brake. To instruct. As a result, the output device 6 immediately activates the brake, so that the own body 31 automatically stops suddenly at a safe position in front of the moving obstacle.

As described above, according to the automatic transportation apparatus 1, even when the pilot or the passenger does not notice that there is a moving obstacle such as the other body 32 or the car 33 in front of the own body 31, furthermore, Is to ensure that even if you are aware and do not take appropriate action, there is a moving obstacle ahead and that it is dangerous to move forward Since such measures can be instructed and executed, it is possible to prevent the occurrence of an aircraft accident or the like that causes enormous damage.

Next, in the case where the transportation is a ship, the transportation automatic guidance device 1 automatically recognizes the situation where the own hull is placed, and notifies the pilot or passenger of the situation by text, image, sound, or the like. An example of guiding a navigation channel and a harbor and instructing an appropriate operation when navigating the navigation channel and a harbor will be specifically described.

As shown in FIG. 5, when the waterway, the quay 43, and the quay 44 are acquired as image data by the input device 2 when the ship 41 sails to berth in the port 45, first, the comparison device 4 The image data and audio data stored in the database 3 are compared with each other, and the waterway, the pier 43, and the quay 44 are recognized by the recognition device 5.
Next, based on the recognition result, the determination device 7 determines the navigation route from the docking position of the quay 44, the current position of the ship 41, and the relevant laws and regulations such as the Port Regulation Law, the Marine Traffic Safety Law, and the like. The instruction about the navigation route is notified to the pilot or the passenger by characters, images, sounds, or the like.

As described above, according to the transportation automatic guidance device 1, even when the pilot or the passenger is not informed about the shape of the port 45, the navigation route to the berth 44 at the berth position can be easily obtained. Berth 44 can be smoothly berthed, and congestion of ships in the harbor 45 can be reduced.

As shown in FIG. 5, when the other device 42, the pier 43, the quay 44, the shallow water, and the like are acquired as image data and audio data in front of the own ship 41 by the input device 2, first, the comparison device 4 uses the database. The image data and the audio data stored in the storage device 3 are compared with each other, and the recognition device 5 recognizes that the other ship 42, the pier 43, the quay 44, the shallow water, and the like exist.
Next, based on the recognition result, the determination device 7 makes a determination that an emergency stop or a change in route is necessary, and further considers that it is dangerous to sail as it is, so that the ship 41 is output to the output device 6. Instruct emergency stop or route change immediately. As a result, the output device 6 causes the ship 41 to perform an emergency stop or route change, so that the ship 41 automatically stops at a safe position or moves to a safe position.

As described above, according to the automatic transportation apparatus 1, even when the pilot or the passenger does not notice that the other ship 42, the pier 43, the quay 44, the shallows, and the like exist in front, the driver or the passenger further notices. Even if the ship does not take appropriate measures, it is informed that other vessels 42, wharves 43, quays 44, shallows, etc. are present and that it is dangerous to sail as it is Since appropriate measures such as stopping and changing the route can be instructed and executed, it is possible to prevent the occurrence of marine accidents or the like that cause enormous damage.

In the transportation automatic guidance device 1, any one of the input device 2, the database 3, the comparison device 4, the recognition device 5, the output device 6, the determination device 7, the storage device 8, and the data update device 9 constituting the device is provided. The device or a plurality of devices may be connected to other devices constituting the automatic transportation guidance device 1 via a communication line.
According to such a configuration, it is also possible to remotely control the transportation at a central control center or the like.

As described above, according to the transportation automatic guidance device of the present invention, a huge amount of image data, audio data, and the like can be processed at high speed, and the situation around the transportation that changes every moment can be automatically detected. And automatically guide the operating route.

平面 Further, the automatic transportation guide device 1 of the present invention may be provided with a plane development processing device 51.

  The plane development processing device 51 converts the perspective image acquired by the input device 2 with respect to the situation around the operating transportation system into a plane image in which perspective is eliminated by performing plane development, and based on this, the distance, As shown in FIG. 6, the measurement processing of the area and the like is performed. The measurement processing includes a planar image conversion device 52, a planar image recognition device 53, a planar image combining device 54, and an image content measuring device 55. You.

When operating a transportation facility, the situation around the transportation facility is acquired by the input device 2 as a perspective image.
The plane image conversion device 52 converts the acquired perspective image into a plane image in which perspective is eliminated by expanding the plane into a plane.

The perspective image acquired by the video camera as the input device 2 is flattened by the following equations (1) and (2) using, for example, the variables shown in FIG. Is converted.

y = v · 2 ^1/2 · h · cos (π / 4−θ) · cos (β−θ) /
(F · sinβ) (1)
x = u · h · cos (β−θ) / (f · sinβ) (2)

Here, θ is the angle between the optical axis of the camera and a plane such as a road surface, f is the focal length of the camera, h is the height of the camera, β is the point at a distance of h + y from directly below the camera and the camera. The angle between the connecting line segment and a plane such as a road surface, v is the vertical coordinate from the origin on the camera projection surface, u is the horizontal coordinate from the origin on the camera projection surface, and y is the road surface etc. The coordinates in the vertical direction with the point advanced h from directly below the camera on the plane as the origin, and x are the coordinates in the horizontal direction on a plane such as a road surface.

The plane image recognition device 53 recognizes and specifies the contents of the converted plane image data based on the result of comparison by the comparison device 4 with the image data stored in the database 3. .

The plane image combining device 54 appropriately combines the converted plane image data to generate large-screen plane image data.

The image content measuring device 55 measures various spatial physical quantities of the object recognized and specified by the planar image recognizing device 53. Examples of the spatial physical quantity include a position, a distance, an area, a height, a depth, a volume, a speed, an acceleration, and the like.

Next, an example in which a predetermined spatial physical quantity is measured and processed by the plane development processing device 51 when the transportation means is a vehicle running on a road surface such as an automobile will be specifically described.

As shown in FIG. 8A, when the traveling lane 56 extending in front of the host vehicle is acquired as the perspective image data by the input device 2, first, the plane image conversion device 52 converts the traveling lane 56 into FIG. As shown in the figure, the image data is converted into planar image data viewed from above vertically without perspective. Here, the plane image recognition device 53 recognizes and specifies the contents of the converted plane image data, and recognizes and specifies that the traveling lane 56 exists in the plane image.
Next, as shown in FIG. 8B, various spatial physical quantities such as the lane width and the parallelism of the lane are measured by the image content measuring device 55, as shown in FIG. 8B. If the perspective image data is acquired in real time by the input device 2, the traveling speed of the host vehicle can be measured by measuring the moving distance per time.

Next, an example in which a predetermined spatial physical quantity is measured and processed by the plane development processing device 51 in a case where the transportation means is a vehicle running on a track such as a train will be specifically described.

As shown in FIG. 9A, when the input device 2 acquires the line 57 extending in front of the own vehicle as perspective image data, first, the plane image converter 52 shown in FIG. 9B. As described above, the image data is converted into the planar image data viewed from above vertically without the perspective. Here, the contents of the converted plane image data are recognized and specified by the plane image recognition device 53, and the existence of the line 57 in the plane image is recognized and specified.
Next, as shown in FIG. 10, various spatial physical quantities such as a rail width, a rail parallelism, and the like are measured by the image content measuring device 55 as shown in FIG. 10. If the perspective image data is acquired in real time by the input device 2, the traveling speed of the host vehicle can be measured by measuring the moving distance per time.

As described above, according to the plane development processing device 51, it is possible to acquire various spatial physical quantities related to the situation around the transportation when operating the transportation, accompanying the transportation automatic guidance device 1. it can.
Further, based on the acquired spatial physical quantity, the automatic transportation guide device 1 can issue more appropriate notification and instruction.

The plane development processing device 51 may have not only a function of converting a perspective image to a plane image but also a function of converting a 360-degree omnidirectional image (spherical image) to a plane image.
The 360-degree all-around image is an image that captures the entire periphery of the operating transportation system, that is, the front, rear, left, right, up, down, and all directions. For example, 1) combining images captured by a plurality of cameras, 2) A curved mirror is installed in front of the camera to capture images reflected on the curved mirror. 3) One camera is rotated to combine images captured at each position. 4) A fisheye lens is attached to the camera. Then, the image can be obtained by a method of processing an image captured in a wide range.

Assuming that the 360-degree omnidirectional image is an image pasted on a virtual spherical surface as shown in FIG. 11, the converted plane image is an image projected on a plane viewed from a desired viewpoint. It can be regarded as.
FIG. 12 shows an example in which a situation around a car traveling on a road surface is acquired as a 360-degree omnidirectional image by the above-described method, and is converted into a plane image from the viewpoint of the top of the car.

In the above description, a vehicle running on a road surface such as an automobile and a vehicle running on a track such as a train have been described as examples of transportation. However, the present invention can be similarly applied to an aircraft and a ship.

Furthermore, a traffic information detection device 61 may be separately installed on the route of transportation, and image data, measurement data, and the like acquired by the traffic information detection device 61 may be transmitted to the automatic transportation device 1.

The traffic information detecting device 61 includes the input device 2, the database 3, the comparing device 4, the recognizing device 5, the storage device 8, the data updating device 9, the plane development processing device 51, and the graphic device 62 in the automatic transportation guide device 1. Consists of

The graphic conversion device 62 converts the image data and measurement data obtained from the input device 2 into computer graphics (CG).

Next, an example of transmitting image data, measurement data, and the like acquired by the traffic information detection device 61 installed on the operation route to the transportation automatic guidance device 1 when the transportation is an automobile will be specifically described.

In the case of a car, the traffic information detecting device 61 is installed at a gate 63, a lighting lamp 64, and the like attached to a road, for example, as shown in FIG. Then, as shown in FIG. 14, the image data, measurement data, and the like acquired by the traffic information detection device 61 are first transmitted to the base station 65 installed for centrally managing the data, and then transmitted to an appropriate location. The information is distributed to a plurality of terminal stations 66 installed and transmitted from the terminal station 66 to each vehicle equipped with the automatic transportation apparatus 1.

Image data, measurement data, and the like distributed from the traffic information detection device 61 to each vehicle equipped with the automatic transportation system 1 via the base station 65 and the terminal station 66 are converted into computer graphic images (CG), Since the information is transmitted together with numerical values and the like, in the automatic transportation guidance device 1 of each vehicle, as shown in FIG.

On the computerized output screen, the coordinates of the own vehicle are fixed at appropriate positions, and the surroundings of the running own vehicle, that is, the position or display of road signs, road signs, traffic information boards, other vehicles, etc. Each time it passes through the traffic information detection device 61, it changes every moment.
It should be noted that what is displayed on the output screen is not limited to a planar computer graphic image, and may be a three-dimensional computer graphic image (3DCG), which is synthesized with a real image developed on a plane. Is also good.

The output screen shown in FIG. 15 shows a case where there is another vehicle that infringes or is likely to infringe the safety area of the own vehicle that has been set in advance. Is notified to the driver or passenger.

Recognition of traffic information by transmission from the traffic information detection device 61 to the automatic transportation guide device 1 is, as shown in FIG. 16, particularly when the left and right roads are blind from the own vehicle at an intersection or the like. The traffic conditions on the left and right roads can be easily grasped, which is extremely useful for traffic safety measures.
The output screen shown in FIG. 16 is a case where another vehicle is located at a position where the left and right roads are blind spots from the own vehicle, and is notified to the driver or passenger by characters, voice, etc., such as "there is a right turn ahead". You.

In the above description, an example has been described in which a vehicle that travels on a road surface is used as a means of transportation, but the present invention can be similarly applied to a vehicle that travels on a track such as a train, an aircraft, a ship, and the like.

Furthermore, if a device capable of accurately recognizing the positional relationship between the operating transportation system and the surroundings is provided in the automatic transportation system 1, the transportation system can be automatically guided more effectively.
In the following, a description will be given of a positional relationship recognition device 101 that can accurately recognize the positional relationship between a running transportation system and the surroundings in the case of a vehicle running on a road surface such as an automobile as a transportation system.

As a device for recognizing the positional relationship of a running vehicle and guiding the vehicle, a device that detects a white line attached to a road from an image acquired by a mounted camera and guides the vehicle so as not to protrude from the white line. Has already been developed.
In this device, the vehicle is positioned only in relation to the white line, and is limited to a special object such as the white line. Therefore, on roads without white lines, the position recognition function and the automatic guidance function are stopped. However, if the white line is bent, there is a risk of malfunction. In addition, when traveling along a white line such as an intersection does not always meet the purpose, the position recognition function and the automatic guidance function have no meaning at all. Furthermore, there is a risk that something other than the white line may be erroneously recognized as a white line. Even if the vehicle is running along the white line, if an obstacle exists, it cannot be avoided.

The positional relationship recognition device 101 according to the present invention recognizes the three-dimensional position of the vehicle using not only the white line attached to the road, but also all the objects in the image acquired by the camera, and furthermore, recognizes the surroundings. In this situation, the vehicle can be positioned three-dimensionally.
That is, in the positional relationship recognition apparatus 101, not only the white line attached to the road, but also all the objects existing in the video are clues for the three-dimensional measurement. Also, not the target object itself but a part in the target object that is easily extracted by image processing is used as a clue. In addition, a plurality of portions that are likely to be clues for three-dimensional measurement in the object are extracted. Another feature is that a detected object is used as a clue instead of specifying the object from the beginning.

The cue part in the video is automatically tracked by applying an image processing technique such as a matching method in each of the continuous images generated by the traveling of the vehicle. Here, there are some which succeed in tracking, and others which miss due to failure in tracking, but only the clue which succeeded in tracking may be subjected to three-dimensional measurement.
Then, based on a plurality of clues successfully tracked while the vehicle travels an appropriate distance, a predetermined calculation is performed to detect and recognize the three-dimensional position and direction of the own vehicle.

In addition, multiple cameras are mounted so that their fields of view overlap, detect corresponding points in the overlapping fields of view, perform three-dimensional measurement, and place them in three-dimensional coordinates composed of surrounding clues. Alternatively, the three-dimensional position and direction of the vehicle are located in the surrounding three-dimensional image acquired by another method.

According to the positional relationship recognition device 101 of the present invention,
1) Since a large parallax can be obtained, the measurement accuracy of a distant object is improved.
2) Accumulation of errors can be reduced.
3) Since the camera position can be measured by calculation, the device configuration is simplified, and the setting of the camera is extremely simplified.
4) Since three-dimensional measurement can be performed from past video data captured by a camera, past video data can be used.
5) The position of the own vehicle can be obtained three-dimensionally from the data of the characteristic points obtained by traveling in the past, using the characteristic points as clues.
6) The position and direction of the vehicle in an area not shown in the video can be obtained by calculation.
As described above, according to the positional relationship recognition device 101, it is possible to exhibit various functions that cannot be performed by detecting the position of the vehicle using only the white line.
Then, it can judge the surrounding traffic situation and transmit it to the operator, or control the vehicle directly based on it, and realize more advanced automatic guidance, effectively contributing to traffic safety. be able to.

The positional relationship recognition device 101 of the present invention is based on image information of a road surface, surrounding objects, and the like captured by an input device such as a video camera mounted on a vehicle. An image acquisition unit 102 for detecting a position and recognizing a positional relationship with a road surface, for acquiring an image by an input device mounted on a vehicle, and an image for recording the acquired image for a certain period of time. A temporary recording unit 103, a cue point automatic extraction unit 104 for automatically extracting cue points for obtaining corresponding points in an image, and corresponding points of a plurality of cue points in each image from two or more images having a distance difference. A corresponding point detecting unit 105 for detecting the position of the input device from a plurality of detected corresponding points, an input device position / direction measuring unit 106 for calculating the position and direction of the input device, and three-dimensional coordinates of the calculated position of the input device. The relative distance that a measured scale converter 107 for converting the absolute distance using measured values, characterized in that it is composed of.

According to the image acquisition unit 102 and the image temporary recording unit 103, it is possible to record a video from an input device such as a video camera mounted on the own vehicle. A changing image is obtained. In other words, for a stationary object, the same result is obtained as when a plurality of cameras are arranged and photographed on the road on which the vehicle is traveling. In this way, even if only one camera is used, a plurality of parallaxes can be obtained by moving the camera.

Next, the automatic cue point extraction unit 104 extracts the contour of the image or divides it into color regions in order to make a characteristic portion in the image a cue point, and so on. Automatically extract parts.
Here, the clue point means a feature point in the image that is the corresponding point in order to find a corresponding point between images captured at different points. The clue point may be the target object itself, but is often a part of the target object. In addition, the feature point here may be a feature point for a computer that performs image processing, not a feature point for a human, and thus image processing is extremely advantageous.
It is not necessary to specify a key point to be detected from the beginning, and an object or a part thereof that can be easily detected at the site at that time can be set as a key point. Further, it is not necessary to specify the cue points to be tracked from the beginning for the plurality of cue points detected, and only the cue points that can be tracked can be set as the cue points to be calculated in the three-dimensional measurement. With these features, the positional relationship recognition device 101 can realize the positional relationship recognition.

Next, the corresponding point detecting unit 105 and the input device position / direction measuring unit 106 detect a plurality of corresponding points by an image matching method or the like, and calculate the position of the input device such as a camera based on the coordinates of each corresponding point. Find the direction.
Here, by continuously obtaining the position and the direction of the input device, the speed of the input device, that is, the vehicle speed, the vehicle acceleration, and the vehicle traveling direction can be obtained.
Some of the corresponding points may be hidden with the movement of the vehicle, but if a sufficient number of clue points are selected and finally only a few clue points are left, the calculation can be performed sufficiently. is there.
Also, the object fixed in the stationary coordinate system must be selected as a clue point, but a sufficient number of clue points are selected, statistical processing is performed on them, and those indicating abnormal values are deleted. By leaving only effective clue points, three-dimensional measurement becomes possible.

The camera position data obtained by the calculation is a relative distance, and in order to convert the data into an absolute distance, one or more known distances are required for the corresponding point coordinates or the vehicle position coordinates to be measured. However, as a value that does not change the distance due to the running of the vehicle, calibration may be performed using, for example, the installation height of the camera or a known distance from the beginning in the image. When a plurality of cameras are installed, the distance between the cameras can be a known distance. Furthermore, if any of the objects in the image has a known measured value, the measured value can be converted into an absolute distance by the measured scale conversion unit 107 as a known distance.
As described above, the camera position and direction can be measured three-dimensionally.

A corresponding point three-dimensional measuring unit that performs three-dimensional measurement of a plurality of cue points from corresponding points in each image of the plurality of cue points, and obtains a relationship between the plurality of cue points and a camera position as three-dimensional coordinates. 108 may be added.

According to the corresponding point three-dimensional measuring unit 108, the three-dimensional data of the clue point is also obtained at the same time in the calculation process of obtaining the three-dimensional data of the position and direction of the camera. This makes it possible to arrange camera positions in the three-dimensional positions, three-dimensional arrays, and three-dimensional distributions of a plurality of clue points. That is, the position and direction of the vehicle can be three-dimensionally arranged in the same three-dimensional coordinate system as the three-dimensional distribution of a plurality of surrounding clue points.
Therefore, in the stationary object in the surrounding stationary coordinate system, in other words, in the three-dimensional coordinates including the surrounding buildings, telephone poles, roads, etc., the own vehicle, to be precise, the camera mounted on the own vehicle. The position can be positioned three-dimensionally.

{Circle around (3)} A three-dimensional data recording unit 109 for recording the three-dimensional coordinates of the corresponding points obtained by the corresponding point three-dimensional measuring unit 108 may be added.

By recording and storing the three-dimensional data of the clue points obtained by the calculation, the clue points for which the three-dimensional coordinates have already been acquired when traveling around the area later are used as indices for calculating the position and direction of the vehicle. As a reference.
In practice, by driving a vehicle equipped with a dedicated device in advance and collecting more accurate three-dimensional data of clue points, those vehicles can be used later by many vehicles.

Further, the three-dimensional data reading unit 110 reads out the three-dimensional data of the clue points accumulated in the three-dimensional data recording unit 109 from the three-dimensional data recording unit 109 when the vehicle travels around the next time. A corresponding point comparison unit 111 that compares the obtained three-dimensional data with the three-dimensional data obtained at that time to obtain a coincidence point and improves the calculation accuracy of the vehicle position may be added.

The three-dimensional data is read out from the three-dimensional data recording unit 109 by the three-dimensional data reading unit 110, and if there is a change in the coordinates of the previous clue point every time the vehicle travels thereafter, the coordinates are updated and new data is re-created. It is good to record.
By updating the data of the three-dimensional data recording unit 109 and adding new data as the vehicle travels near the same place, the number of clue points increases or the position accuracy improves, and the calculation accuracy of the position and direction of the vehicle is improved. Also improve.
By generating high-precision three-dimensional data using a dedicated device in advance and using it later by another vehicle, the positional accuracy of the clue point becomes extremely high. Furthermore, if three-dimensional map data in which the clue points are extended to pixel units is generated in advance by a dedicated device, a three-dimensional space around the traveling road is formed, and the position and direction of the vehicle can be arranged therein.

An absolute coordinate conversion unit 112 that selects an object whose absolute coordinates are known as a corresponding point, and gives absolute coordinates to the three-dimensional data acquired by the input position direction measuring unit 106 and the corresponding point three-dimensional measuring unit 108; A coordinate integrating unit 113 that integrates three-dimensional coordinates of clue points existing in a certain area into an absolute coordinate system may be added.

According to the absolute coordinate conversion unit 112, for example, the absolute coordinates of the camera position are acquired by GPS, or the three-dimensional data obtained as the relative distance is converted into the absolute coordinates by using the object whose latitude and longitude are already known as a clue. can do.
Further, according to the coordinate integrating unit 113, the coordinates of several clue points can be unified and displayed in a common absolute coordinate system, and the three-dimensional data immediately acquires the absolute coordinates.
If the clue point obtains the absolute coordinates, it can be used as common three-dimensional data of the clue point after the next time or in another vehicle.
The site data that matches the three-dimensional data in which the clue points are collected matches the position, arrangement, and distribution of the clue points, and thus the absolute coordinates are obtained. From there, the camera position, that is, the absolute coordinates of the vehicle, can be obtained immediately.
Furthermore, by dropping the three-dimensional coordinates of the clue points in the map of a certain area, the map and the clue points are combined, a new database is created, and many vehicles use common data as the position and direction of the vehicle as common data. You will be able to get a map to record the data.

A name attribute adding unit 114 that records and saves the name and attribute of the clue point in association with the position data of the clue point, and adds the name and attribute of the object to which the clue point belongs to the coordinate data of the clue point; The database 115 for writing the coordinates, names, and attributes of the added clue points in the map in association with each other and recording and storing the coordinates may be added.

If the name and attribute of the object to which the clue point belongs are known in advance, or if the name is known by image recognition, the name of the object and its general properties and properties specific to the object should also be acquired as attribute data. Can be.
That is, if the clue point at the time of measurement is associated with the clue point in the database, the name or property of the object at the clue point can be read.

A display unit 116 for displaying and informing the driver or the like of the various calculation results as appropriate may be added.

(4) If the acquired data, the position and direction of the vehicle, the names and attributes of the objects to which the clue points belong, and the like are displayed on the display unit 116, the driver and the administrator can observe them to determine the situation.

Based on the calculation result, a situation determination unit 117 that automatically determines the situation of the vehicle from the positional relationship between the surroundings and the own vehicle and the positional relationship between the road and the own vehicle, and automatically matches the purpose of the vehicle using the result of the situation determination. An automatic vehicle control unit 118 that automatically performs other operations (brake operation, speed control operation, steering wheel operation, alarm operation, etc.) may be added.

According to the situation determination unit 117, the position and direction of the vehicle, the names and attributes of the objects to which the clue points belong, the names and attributes of the clue points, the positional relationship between the road surface and the vehicle, signs, road markings, and the like are comprehensively determined. , The situation where the vehicle is placed can be determined.
Thereby, the vehicle can be operated automatically or semi-automatically via the automatic vehicle control unit 118. Further, it is possible to determine the positional relationship between the surrounding situation and the vehicle and transmit the vehicle position information to the driver or the like.

In addition, a plurality of cameras are installed to capture an image, a plurality of camera image acquisition units 119 in which a part or all of the field of view of each camera are overlapped, and a three-dimensional distance calculated from a parallax based on a moving distance by a single camera. A combination of measurement and three-dimensional distance measurement calculated from camera-to-camera parallax by multiple cameras, and using the three-dimensional distance data of clue points obtained by the overlapped view of multiple cameras as a reference length, a single camera And a calibration unit 120 for converting the three-dimensional distance data obtained by the moving distance parallax method based on the movement of the moving object to an absolute distance.

If multiple cameras are installed around the vehicle so that the surroundings of the vehicle can be observed, and the field of view of each camera partially overlaps the field of view of other cameras, the camera-to-camera parallax can be reduced at the overlapped portion. Can be generated.
The greatest feature of the three-dimensional distance measurement by the multiple camera view overlapping method is that it can measure a moving object. In addition, according to the multiple camera visual field overlapping method, high accuracy can be obtained in the measurement of a short distance.
On the other hand, according to the single camera moving distance method, the distance between cameras in the multiple camera method can be made substantially long, which is advantageous when measuring a long distance three-dimensional distance. For example, the distance between cameras in a vehicle-mounted camera is at most about 1 m, but the distance between cameras in the camera moving distance method can be not only 1 m, 10 m, and 100 m but also 1 km and 10 km.
Therefore, the short distance is measured three-dimensionally by the overlapping visual field method, and the long distance is measured three-dimensionally by the camera moving distance method that can obtain a large parallax.
Basically, the principle of measurement is the same for both singular and multiple cases.However, as a feature of the multiple camera method with overlapping visual fields, the distance between cameras can be taken as the reference length in measurement by parallax. It can measure absolute distance, and it can calibrate the measurement distance by single camera movement method with the distance data of intermediate distance measurement obtained from distance measurement by parallax, so accurate measurement from short distance to long distance Becomes possible. That is, the position, speed, acceleration, and traveling direction of the vehicle traveling ahead can be measured.

Also, the three-dimensional shape of the object to which the clue point belongs is represented and arranged in a correct position in the coordinate system defined in the display screen by three-dimensional computer graphics, and the name, attribute, Another object, the own vehicle, is displayed on its display screen, a shape coordinate attribute display unit 120 that reproduces a three-dimensional space, and three-dimensional computer graphics showing various objects represented on the display screen. Touching the image with the hand, clicking with the mouse, or displaying only the real image, touching the target object with the hand on the displayed real image, clicking with the mouse, etc. By specifying the target object, the attributes such as the name, coordinates, shape, and other related data of the target object are displayed, and data related to the specified target object is input, and the specified target object is input. for It is possible to instruct various operations and actions, and the user interface unit 121, may be added.

If the three-dimensional coordinates, names, and attributes of the objects to which a plurality of clue points belong can be acquired, these clue points mean that they correspond to the respective objects to which they belong.
Therefore, by storing the three-dimensional shape and three-dimensional coordinates of the object in a database in advance, the vehicle and its surroundings are represented in 3DCG (three-dimensional computer graphics) and displayed in a three-dimensional coordinate system. can do. Of course, other attributes, and even an object for which a clue point cannot be found, can be displayed because the shape and position coordinates are known.
Situation determination can be performed more appropriately from the reconstructed three-dimensional space and information such as the positional relationship, names, attributes, and the like of the respective objects arranged therein.
Also, by specifying the object on the displayed 3D CG screen by clicking with the mouse or touching it with the hand, or by directly instructing by voice, understanding the contents of the instruction with the voice recognition device, The attributes of the target object can be read out from the database and displayed.
Furthermore, the two-dimensional data of the real image and the two-dimensional projected image data of the three-dimensional computer graphics are configured to overlap each other so that the shapes match in layers, and only the real image is displayed. If the target object is specified by clicking the target image with the mouse or touching it with the hand, etc., the 3D CG image of the 3D CG image is obtained from the data structure in which the real image and the 3D CG image coincide with each other. By specifying the data of the corresponding object, the name, coordinates, attributes, and other related information relating to the object can be called from the database or written.

外部 Also, an external communication unit 122 connected to another vehicle or another communication point via a communication line for transmitting and receiving information may be added.

If it is connected to another vehicle or another communication point via a communication line, it is automatically or instructed via a user interface, and the three-dimensional information and the moving object information of the clue point generated by the own vehicle are provided. Or the other vehicle can receive the three-dimensional information and the moving object information of the clue point generated by the other vehicle at a different position in the same manner as generated by the own vehicle.
Also, clue point information analyzed from a video acquired by a camera installed at a fixed point in the surroundings, position information of the vehicle including the own vehicle, speed information, and information of the result of determining the situation, for example, traffic jam information, The fixed station transmits accident information and the like, and the own vehicle receives and displays the information, or receives and displays information that cannot be acquired by the own vehicle, and further displays the information acquired by the own vehicle. The situation can be determined by adding the received information, and highly accurate information can be displayed.

Next, an example in which the positional relationship recognizing device 101 recognizes the positional relationship between the operating vehicle and the surroundings when the vehicle is a vehicle running on a road surface such as an automobile will be specifically described.

As shown in FIG. 19, four super-wide-angle cameras equipped with a fisheye lens are installed on the roof of the vehicle so that the fields of view partially overlap, and the overlapping part of the camera's field of view is used to measure the distance of the vehicle. Speed measurement was performed. In addition, for a short cue point, three-dimensional measurement is performed by parallax between a plurality of cameras, and for a long cue point, a corresponding point of the cue point is detected by moving a single camera, and a cubic measurement is performed based on the moving distance. Original measurement was performed.
The calculation based on the parallax between the cameras and the calculation based on the motion parallax due to the movement of the camera are basically the same type of calculation, but which is an unknown number is different. In the case of the motion parallax, the moving distance of the camera is an unknown number. However, in the case of the parallax between the cameras, the distance between the cameras can be measured in advance, so that the known number can be used.

Also, if the three-dimensional measurement data obtained by the calculation is recorded, the next time the vehicle travels in the same area, the previously obtained clue points become known numbers, and the three-dimensional measurement can be performed with further improved accuracy. It becomes possible. Examples of clue points are indicated by circles in FIG.
It is advantageous for the subsequent processing that the acquired image is developed into a spherical coordinate format attached to the spherical surface. As for the positional relationship between the cameras, it is desirable to accurately measure the distance and direction between the cameras. However, the distance between the cameras can also be obtained by calculation as an unknown value, so that the positions may be appropriately set. The inter-camera distance can also be used as a reference for actual value conversion.
The direction of the camera can also be a known number, but if the direction is expected to shift slightly due to vibration or the like depending on the method of fixing the camera, the direction of the camera can be treated as an unknown number.
If there are many unknowns, many clues can be taken and the unknowns can be obtained by calculation. In principle, the clue point can be increased to the number of pixels.

Here, any one or more clue points are accurately measured by the parallax of the camera overlapping the visual field and used as a reference value, and converted to an actual measurement scale, or the height of the camera from the road surface as a reference value is used as an actual measurement scale. Can be converted. Further, an object having a known length in the image may be used as the reference length.
Since the parallax is obtained at the overlapping part of the visual fields, the corresponding point of the clue point can be obtained by image recognition in the video of each camera taken simultaneously with the inter-camera distance as a known reference length. The cue point can be automatically extracted in real time from the image by performing contour processing of the image. In order to make the image recognition real-time and to facilitate the calculation, a cross intersection, a triangular intersection, a quadrangular intersection, and the like in the contoured image are extracted from the image. Since real-time processing is not required for data collection of clue points, a considerable number of clue points can be calculated by off-line processing. It is also possible to extend to the whole pixel.

In general, some of the many clue points are set as characteristic points as clue points, and corresponding points are obtained by an image matching method or the like. When the corresponding point is determined, the three-dimensional distance to the clue point can be obtained by calculation. At the same time, the camera position and direction are also calculated.
All clue points are not always found as corresponding points. It is preferable to extract a little more automatically, assuming that the corresponding point cannot be found halfway.
If a clue point is found, post-processing may be performed. Therefore, it is preferable that the three-dimensional position of the clue point be associated with the map, the name and attribute of the object to which the clue point belongs be linked, and recorded in a database.
As a result, in the subsequent travel, the name of the corresponding point to which the key point belongs and the attribute of the object can be known only by finding the key point.

By using the database, it is possible to recognize the object to which the clue point belongs, to understand the situation in which the vehicle is placed, and to determine the next action to be taken simply by finding the position of the clue point. Then, based on the result of the determination, the vehicle direction, speed, and the like can be appropriately controlled, and the vehicle can be automatically guided appropriately.
In addition, if the three-dimensional measurement result or the determination result is output on a screen, by voice, or the like, the driver or the administrator can be notified as shown in FIG. In addition, if information from another vehicle or information from a fixed camera installed around the vehicle is integrated, the situation around the own vehicle can be displayed as shown in FIG. 16 to notify the driver or the administrator. . In any case, the display screen functions as a user interface, and can specify a target vehicle, start communication, transmit and receive data, and clearly indicate the vehicle.

In the above description, a vehicle running on a road surface such as an automobile has been described as an example of transportation. However, the same applies to a vehicle running on a track such as a train, an aircraft, a ship, a spacecraft, and the like. Can be.

It is a lineblock diagram of a transportation automatic guidance device of the present invention. It is explanatory drawing which shows the effect | action of the transportation automatic guidance apparatus of this invention when a transportation is a motor vehicle. It is explanatory drawing which shows the effect | action of the transportation automatic guidance apparatus of this invention when a transportation is a track vehicle. It is explanatory drawing which shows the effect | action of the transportation automatic guidance apparatus of this invention when a transportation is an aircraft. It is explanatory drawing which shows the effect | action of the transportation automatic guidance apparatus of this invention when a transportation is a ship. It is a lineblock diagram of a plane development processing device attached to a transportation automatic guidance device of the present invention. FIG. 4 is a diagram illustrating variables used in a formula for developing a perspective image acquired by a video camera into a plane image and converting the perspective image into a planar image in which perspective is eliminated. (A) is a perspective image acquired by the input device, and (B) is a plane image converted by the plane image conversion device. (A) is a perspective image acquired by the input device, and (B) is a plane image converted by the plane image conversion device. It is explanatory drawing which shows the spatial physical quantity measured by the image content measuring device. It is explanatory drawing which shows the concept which expands a 360-degree omnidirectional image into a plane and converts it into a plane image. It is an example of a plane image converted from a 360-degree omnidirectional image. It is explanatory drawing which shows the example of installation of the traffic information detection apparatus in a road. It is explanatory drawing which shows the example of transmission from a traffic information detection apparatus to a transportation automatic guidance apparatus. 7 is an example of an output screen displayed as a computer graphic image when a vehicle is traveling on a road. It is an example of an output screen displayed as a computer graphic image when a vehicle enters an intersection. 1 is a configuration diagram of a positional relation recognition device attached to a transportation automatic guidance device of the present invention. FIG. 18 is a diagram illustrating an example of a target part selected as a clue point and a part of the target part in the positional relationship recognition device of FIG. 17. It is a conceptual diagram which shows the overlapping state of the visual field by several vehicle-mounted cameras.

Explanation of reference numerals

DESCRIPTION OF SYMBOLS 1 Transportation automatic guidance apparatus 2 Input device 3 Database 4 Comparison device 5 Recognition device 6 Output device 7 Judgment device 8 Storage device 9 Data update device 11 Own vehicle 51 Plane development processing device 52 Plane image conversion device 53 Plane image recognition device 54 Plane Image combining device 55 Image content measuring device 61 Traffic information detecting device 62 Graphical device 101 Positional relationship recognizing device 102 Image acquiring unit 103 Temporary recording unit 104 Automatic clue point extracting unit 105 Corresponding point detecting unit 106 Input device position and direction measuring unit 107 Actual scale conversion unit 108 Corresponding point three-dimensional measuring unit 109 Three-dimensional data recording unit 110 Three-dimensional data reading unit 111 Corresponding point comparing unit 112 Absolute coordinate converting unit 113 Coordinate integrating unit 114 Name attribute adding unit 115 Database 116 Display unit 117 Situation judgment Part 118 automatic vehicle control part 119 Multiple input device image acquisition unit 120 Calibration unit 121 Shape coordinate attribute display 122 User interface unit 123 External communication unit

Claims (24)

An input device for acquiring the surroundings of the operating transportation system as images, sounds, etc., an image data relating to the transportation system, a database pre-stored with audio data, etc., and image data, audio data, etc. obtained by the input device. A comparison device that compares the image data, audio data, and the like stored in the database, and a recognition device that recognizes and specifies the content of the data when the result of comparison of the image data, audio data, and the like matches, An automatic transportation guide device comprising: an output device that recognizes and specifies a result by a device and notifies a driver or the like of the result by characters, images, sounds, or the like. If the image data, sound data, etc. corresponding to the object acquired by the input device does not exist in the database, the image data, sound data, etc., corresponding to the new object are made to correspond to the position on the map. A storage device newly stored in the database, and when the image data, audio data, etc. corresponding to the object are different from the image data, audio data, etc. stored in the database, new image data, audio data, etc. The automatic transportation apparatus according to claim 1, further comprising a data updating device that updates the data and stores the updated data in the database. It has a determination device that makes some determination based on the items recognized or specified by the recognition device, and notifies the operator based on the determination result by text, image, voice, or the like by the output device. The transportation automatic guidance device according to claim 1 or 2. The determination device performs some determination based on the items recognized or specified by the recognition device, and further instructs the output device to perform a predetermined operation based on a result of the determination, such as a braking device, a steering device, or the like. The automatic transportation apparatus according to claim 3, wherein the automatic operation is performed. The transportation automatic guidance device according to claim 1, wherein one or a plurality of devices constituting the transportation automatic guidance device are connected to another device via a communication line. The operating transportation system is a vehicle that runs on the road surface such as an automobile. An input device that acquires the surroundings of the own vehicle as images and sounds, and images related to automobiles such as road signs, road signs, and traffic information boards. A database in which data, audio data, and the like are stored in advance, a comparison device that compares image data, audio data, and the like acquired by the input device with image data, audio data, and the like stored in the database; image data, audio data, and the like And a recognition device for recognizing and specifying the content of the data when the result of the comparison matches, and an output for recognizing the specified result and notifying the specified result to a driver or a passenger by text, image, voice, or the like. A transportation automatic guidance device composed of a device. The operating transportation system is a vehicle that runs on a track such as a railway train, an input device that acquires the situation around the own vehicle as an image, a voice, etc., a track mark, a track mark display, a track mark guide plate, etc. A database which previously stores image data, voice data, and the like regarding railway vehicles and the like; a comparison device that compares image data, voice data, and the like acquired by the input device with image data, voice data, and the like stored in the database; When the result of comparison of data, voice data, and the like matches, the content of the data is recognized and identified by a recognition device that recognizes and specifies the character, image, and voice to a driver or a passenger. And an output device for notifying by means of a transportation means. The operating transportation system is a hull or airframe, such as a ship or an aircraft, that navigates two or three dimensions without orbit, and an input device that acquires the surrounding conditions as images and sounds, signs, displays, and ports. Shapes, airport shapes, etc., image data relating to ships or aircraft, etc., a database pre-stored with audio data, etc., image data acquired by the input device, image data stored in the database with voice data, audio data, etc. When the comparison result of the comparison device and the image data, the sound data, etc. match, the contents of the data are recognized and identified. The recognition device that identifies and identifies the recognition device, and the identified result is identified by the pilot or passenger. A transportation automatic guidance device comprising an output device for notifying a person by text, an image, a sound, or the like. The perspective image data about the surroundings of the transportation system acquired by the input device, a planar image conversion device that converts the perspective image data into planar image data that eliminates perspective, and the comparison device, the converted planar image data And a planar image recognizing device for recognizing and specifying the contents of the data based on the result of comparison with the image data stored in the database. 9. A transportation automatic guidance device according to claim 1. 10. A plane development processing device, comprising: an image content measuring device for measuring various spatial physical quantities with respect to an object recognized and specified by the planar image recognition device. A transportation automatic guidance device according to 1. The said plane image conversion apparatus has the function which converts 360 degree omnidirectional image data about the surroundings of the transportation acquired by the input device into plane image data. A transportation automatic guidance device according to 1. A traffic information detection device that acquires the surroundings of a transportation facility as image data, measurement data, or the like is installed on an operation route of the transportation facility so that image data, measurement data, and the like acquired by the traffic information detection device can be received. 12. A transportation automatic guidance device according to claim 1, wherein: 13. The automatic transportation guidance device according to claim 12, wherein the traffic information detection device includes a graphic device for performing computer graphics based on the acquired image data and measurement data. An image acquisition unit for acquiring an image by an input device mounted on a transportation system, an image temporary recording unit for recording the acquired image for a certain period of time, and a cue point automatic extraction for automatically extracting a cue point for obtaining a corresponding point in the image Unit, a corresponding point detection unit that extracts two or more images having a distance difference and calculates corresponding points of a plurality of clue points in each image, and calculates the position and direction of the input device from the detected corresponding points. An input device position and direction measurement unit, and a relative position recognition device including an actual measurement scale conversion unit that converts the relative distance of the obtained three-dimensional coordinates of the position of the input device into an absolute distance using an actual measurement value. The transportation automatic guidance device according to claim 1, wherein the transportation automatic guidance device is provided. In the positional relationship recognition device, three-dimensional measurement of a plurality of clue points from corresponding points in each image of the plurality of clue points, a corresponding point three-dimensional measuring unit for obtaining the relationship between them and the position of the input device as three-dimensional coordinates The automatic transportation apparatus according to claim 14, further comprising: 16. The automatic transportation guide according to claim 15, wherein a three-dimensional data recording unit that records three-dimensional coordinates of the corresponding point obtained by the corresponding point three-dimensional measurement unit is added to the positional relationship recognition device. apparatus. The three-dimensional data reading unit that reads the three-dimensional data of the clue points accumulated in the three-dimensional data recording unit obtained by operation from the three-dimensional data recording unit at the time of the next and subsequent peripheral operations. And a corresponding point comparison unit that improves the calculation accuracy of the position of the transportation system by comparing the data with the image data acquired at the time of the next and subsequent operations to obtain a coincidence point. The transportation automatic guidance device according to claim 16, characterized in that: By providing the positional relationship recognition device with a target whose absolute coordinates are known as a corresponding point, absolute coordinates are given to the three-dimensional data acquired by the input device position / direction measuring unit and the corresponding point three-dimensional measuring unit. 18. The automatic transportation apparatus according to claim 17, wherein an absolute coordinate conversion unit and a coordinate integration unit that integrates three-dimensional coordinates of clue points existing in a certain area into an absolute coordinate system are added. In the positional relationship recognition device, the names and attributes of the clue points are recorded and stored in association with the position data of the clue points, and the names and attributes of the objects to which the respective clue points belong are added to the coordinate data of the clue points. 19. The automatic transportation system according to claim 18, wherein an attribute adding unit and a database for writing, recording, and storing the coordinates, names, and attributes of the added clue points in a map are added. . 20. The automatic transportation system according to claim 19, wherein a display unit for appropriately displaying and notifying the various calculation results to an operator or the like is added to the positional relationship recognition device. The positional relationship recognition device, based on the results of the various calculations, from the positional relationship between the surroundings and the operating transportation system, a status determination unit that automatically determines the status of the transportation system, and automatically uses the result of the status determination to automatically select the transportation system. 21. The automatic transportation apparatus according to claim 20, further comprising an automatic control unit for automatically performing an operation suitable for the purpose of the invention. In the positional relationship recognition device, a plurality of input devices are installed to capture an image, a plurality of input device image acquisition units in which part or all of the field of view of each input device are overlapped, and input device movement by a single input device Cubic points obtained by the overlapping visual field method of multiple input devices using both 3D distance measurement calculated from parallax based on distance and 3D distance measurement calculated from input device parallax using multiple input devices A calibration unit that converts three-dimensional distance data obtained by a moving distance parallax method based on movement of a single input device by using the original distance data as a reference length, thereby converting the three-dimensional distance data into an absolute distance; A transportation automatic guidance device according to claim 21. In the positional relationship recognition device, the three-dimensional shape of the object to which the clue point belongs is represented at the correct position in the coordinate system defined in the display screen by arranging it in three-dimensional computer graphics, and if necessary. A shape coordinate attribute display unit that reproduces a three-dimensional space in which the name, attribute, other objects, and the own vehicle can also be displayed in the display image, and various objects represented in the display image Touching the image of the three-dimensional computer graphics with the hand, clicking with the mouse, or displaying only the real image and touching the target object with the displayed real image By specifying the target object by clicking with the mouse, the attributes such as the name, coordinates, shape, and other related data of the target are displayed, and the data related to the specified target object is displayed. Enter and instruct With respect to the object, transportation automatic guide apparatus according to claim 22 in which the user interface unit can instruct various operation and behavior, characterized in that adding to. 24. The transportation system according to claim 23, wherein an external communication unit connected to another transportation system or another communication point and transmitting / receiving information is added to the positional relationship recognition device via a communication line. Automatic guidance device.