JP4311426B2 - Display system, in-vehicle device, and display method for displaying moving object - Google Patents

Description

  The present invention captures road traffic conditions including moving objects (vehicles, two-wheeled vehicles, people, etc.) with a camera installed at a place with poor visibility such as an intersection or an in-vehicle camera of another vehicle, and images obtained from the road are communicated with each other. Alternatively, a vehicle display system and an in-vehicle device that provide an image of a moving body that is received by a host vehicle entering from a different road to an intersection or the like through inter-vehicle communication so that the driver can easily understand a blind spot from the host vehicle. The present invention relates to an image transmission apparatus and a display method.

Conventionally, as a display device for preventing head-on collision of vehicles, a vehicle detector is installed on the priority road, a warning display board is installed at the intersection of the priority road, and a vehicle traveling on the priority road is detected by the vehicle detector. There is a device that displays a warning on the warning display board (see Patent Document 1). As a result, the driver can be notified of the possibility of a head-on accident at the intersection.
In addition, there is a device that installs a camera at an intersection, captures an image of another vehicle approaching the intersection from a right-angle road that is a blind spot from the own vehicle, and receives the image by the own vehicle (Patent Document). 2). According to this apparatus, an actual image of another vehicle viewed from the camera can be observed in the own vehicle.
JP-A-5-28400 JP 2003-109199 A

  In the technology of the above-mentioned Patent Document 1, the own vehicle approaching the intersection can recognize that some vehicle approaches from the priority road, but at what position the vehicle is now and at what speed, I do not know detailed information such as. Therefore, the warning on the warning display board may be neglected, and there is a possibility that the vehicle will proceed without stopping, which is not perfect for preventing accidents.

  In the technique of Patent Document 2, the own vehicle approaching the intersection can recognize the other vehicle approaching from the priority road on the screen. However, this technology only displays the image viewed from the camera on the screen, so it is difficult to understand the relative position and relative speed of the host vehicle and the other vehicle, and where the displayed image is. It is difficult for the driver to instantly determine what is dangerous. Therefore, it may cause misrecognition and illusion.

  Therefore, the present invention provides a display system for a moving body, an in-vehicle device, which allows the driver of the own vehicle to easily grasp the relative relationship with the moving body when providing the driver with an image of the moving body that is a blind spot from the own vehicle. An object is to provide an image transmission apparatus and a display method.

A mobile object display system according to the present invention is a mobile object display system for displaying a mobile object on a screen in a vehicle, the camera for imaging a traffic situation of a road including the mobile object, and the imaging by the camera. An image transmission device that transmits transmission information including the image and the position and orientation of the camera, and an in-vehicle device mounted on the vehicle,
A receiving unit that receives the image and the position and orientation of the camera, and a display unit that simultaneously displays the image received by the receiving unit and the position and orientation of the camera on the screen in the vehicle; A calculation unit that calculates position and orientation data when viewed from a driver on the vehicle based on the position and orientation of the camera, and the display unit displays the calculated position and orientation data. It comprises a head-up display that displays on the windshield surface of the vehicle .

In this display system, the display unit of the in-vehicle device displays not only the image of the traffic situation of the road including the moving body imaged by the camera, but also the position and direction of the camera in a display format that can be understood by the driver. Therefore, it is easy for the driver in the vehicle to determine which image is displayed on the screen and what is dangerous in the traffic situation shown in the image.
Therefore, when the moving body is shown in the image, the driver of the host vehicle can easily grasp the relative relationship with the moving body, and the driver can be more surely supported for safe driving.

  In the present invention, the direction of the camera means a direction that allows the driver to substantially understand where the camera is imaging, for example, the imaging direction of the camera (the optical axis of the lens). The direction of the camera can be expressed by displaying the (direction) and the imaging area of the camera (area within the angle of view range) on the screen in the vehicle. The direction of the camera can also be expressed by directly presenting the area to be imaged by the camera on the map.

In the display system of the present invention, the transmission information transmitted by the image transmission device includes warning information that identifies a type of risk that can cause an accident that is assumed from the traffic situation of the road when viewed from the vehicle. It is preferable that the vehicle-mounted device includes an output unit that outputs the warning information received by the receiving unit to the driver of the vehicle in a recognizable manner (claim 2).
The above warning information varies depending on the camera installation position, etc., and is specified corresponding to the purpose of imaging the traffic situation. For example, right-hand accident warning, encounter warning, curve blind spot warning, jumping blind spot warning, road entry Consists of information such as blind spots.

In this case, since the output means of the in-vehicle device outputs the warning information recognizable to the driver, the risk of the image displayed on the screen is coupled with the display of the camera position and the imaging direction on the screen of the display unit. Can be detected more reliably by the driver, and safe driving assistance for the driver is further ensured.
Note that the output by the output means can be performed by displaying warning information by characters using a display unit such as a head-up display, or by notifying the warning information by voice using voice means such as a speaker.

In the display system according to the aspect of the invention, it is preferable that the image transmission device includes a recognition unit for the moving body and transmits the image and the transmission information when the recognition unit recognizes the moving body. (Claim 3).
In this case, the image and the transmission information are transmitted only when the moving body is recognized, and when there is no moving body that can cause an accident, the information is not transmitted. For this reason, it is possible to prevent the driver from being confused by continuing to transmit images and the like to the in-vehicle device even though it is not dangerous.

In the display system of the present invention, it is preferable that the display system further includes an approach sensor that detects that the mobile body or the vehicle on which the vehicle-mounted device is mounted or both of them are approaching the camera. Item 4).
In this case, if the image transmission device transmits the image and the transmission information when the proximity sensor detects the mobile body, the vehicle, or both (Claim 4), the mobile body Images and transmission information are transmitted only when a vehicle is detected, and when there is no moving object that can cause an accident, such information is not transmitted. For this reason, it is possible to prevent the driver from being confused by continuing to transmit images and the like to the in-vehicle device even though it is not dangerous.

Furthermore, in the display system of the present invention, as the image transmission device, a device that transmits the image and the transmission information when a transmission request from the in-vehicle device is received may be employed.
In this case, since the image transmission device transmits the image and the transmission information based on the transmission request from the in-vehicle device, compared to the case where the captured image of the traffic situation is continuously transmitted regardless of the presence or absence of the transmission request, The amount of data transmission in communication and inter-vehicle communication can be reduced.

Further, in the display system of the present invention, the in-vehicle device, based on the position and orientation of the camera have a calculator for position and calculating the orientation data when viewed from the driver on board the vehicle, the display unit is constituted of a head-up display for displaying the position and orientation data the operational front glass surface of the vehicle (claim 1).
In this case, the driver in the vehicle can grasp the actual camera position and orientation from the camera position and orientation data displayed on the windshield.

An in-vehicle device of the present invention is an in-vehicle device for displaying a moving body on a screen in a vehicle, and an image obtained by imaging a traffic situation of a road including the moving body, and the position and orientation of the camera. A receiving unit that receives transmission information including: a display unit that simultaneously displays the image received by the receiving unit and the position and orientation of the camera on the screen in the vehicle; and a position and orientation of the camera. A calculation unit that calculates position and orientation data when viewed from a driver on the vehicle, and the display unit displays the calculated position and orientation data on a windshield surface of the vehicle. It consists of a head-up display (Claim 6 ).
If this in-vehicle device is used, the display unit of the in-vehicle device displays not only the image of the traffic situation of the road including the moving body imaged by the camera but also the position and orientation of the camera in a display format understandable by the driver. Therefore, the driver in the vehicle can easily determine which image is displayed on the screen and what is dangerous in the traffic situation shown in the image.

According to another aspect of the present invention, there is provided a method for displaying a moving body, in which a traffic situation of a road including the moving body is captured by a camera, and an image captured by the camera and transmission information including the position and orientation of the camera are transmitted to the vehicle-mounted device. The position and orientation data as seen from the driver on board the vehicle is calculated based on the image received by the in-vehicle device and the position and orientation of the camera, and the computed position and orientation a method of displaying simultaneously on the screen data composed of a front glass surface of the head up display (claim 7).
By this method, not only an image of a road traffic situation including a moving body imaged by a camera, but also the position and orientation of the camera can be displayed on a screen in the vehicle. This makes it easier for the driver in the vehicle to determine which image is displayed on the screen and what is dangerous in the traffic situation shown in the image. Therefore, accidents such as head-on collisions can be prevented more reliably.

In the present invention, the image of the traffic situation of the road including the moving body and the position and orientation of the camera are simultaneously displayed on the screen in the vehicle. Therefore, the image transmission apparatus captures and transmits the image of the traffic situation. And a function for transmitting transmission information including the position and orientation of the camera that has been calculated or stored.
However, in this image transmission device, the position and orientation data when viewed on the road map is calculated based on the position and orientation of the camera, and the calculated position and orientation data are superimposed on the road figure of the road map. It is also possible to adopt a system configuration in which after a composite graphic is created, transmission information including the traffic situation image and the composite image is transmitted to the in-vehicle device.
In this case, since it is not necessary to perform the calculation of the data and the creation of the composite figure with the in-vehicle device, there is an advantage that the operation load of the in-vehicle device is reduced and the device can be made more inexpensive.

  As described above, according to the present invention, not only the image of the moving object that becomes a blind spot from the host vehicle but also the position and orientation of the camera that captured the image are displayed, so the driver recognizes the moving object without a sense of incongruity or illusion. It becomes possible to do. As a result, accidents such as head-on collisions (collisions with crossing vehicles) and right-straight collisions (collisions with straight-ahead motorcycles that become blind spots when turning right) are more reliably prevented or suppressed, and support for safe driving is more reliably ensured. Become.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
In the following description, the display unit includes a head-up display that displays the position and orientation data of the camera as viewed from the driver on the vehicle on the windshield of the vehicle (FIGS. 11 to 14). And FIG. 21) belong to the present invention, but the display unit is composed of an in-vehicle display that displays the position and orientation data of the camera when viewed on the road map superimposed on the road graphic of the road map (FIGS. 15 to 18, 20 and 22) do not belong to the present invention.
1-4 has shown the whole structure of the display system which concerns on embodiment of this invention which displays the image of a moving body on the display of the own vehicle.
[Schematic configuration of the system]
1 to 4, the display system 1 shown in FIG. 1 is for preventing a so-called right-straight accident, and is an intersection 5 of an opposite two-lane road 4 having an own lane 2 and an opposite lane 3. Is displayed on the driver of the host vehicle 6 who is going to turn right at the vehicle 7 (particularly, the two-wheeled vehicle 7B such as a straight-ahead motorcycle that causes a right-straight accident).

Therefore, the display system 1 includes a camera 10 installed in the vicinity of the intersection 5 to image the moving body 7 traveling in front of the intersection 5 in the oncoming lane 3, and images captured by the camera 10 (still images and moving images). Any of these may be used). The image transmission device 11 transmits the image data by road-to-vehicle communication, and the in-vehicle device 12 that receives the transmission data from the image transmission device 11 and displays the captured image on the driver of the host vehicle 6. Yes.
In FIG. 1, an opposite two-lane road 4 having an own lane 2 and an opposite lane 3 is illustrated, but an opposite one lane or an opposite three lane or more road 4 may be used.

In the example of FIG. 1, the camera 10 is installed on an upper portion of a utility pole 13 erected on the sidewalk portion of the opposite lane 3 at the intersection 5. In addition to the utility pole 13, the installation location of the camera 10 may be an arbitrary location on the ground facility such as an overpass, a footbridge and a building. Alternatively, the camera 10 may be suspended from a wire so as to capture an image from directly above.
The installation position of the camera 10 is a position where the front and left side portions of the oncoming vehicles 7A and 7B traveling in the oncoming lane 3 toward the intersection 5 can be imaged. Therefore, it can be installed not only on the utility pole 13 shown in FIG. 1 but also on the ground facility 15 located on the intersection road 14 side of the utility pole 13.

The display system 1 shown in FIG. 2 is for preventing a head-on accident at an intersection 5, traveling to the driver of the own vehicle 6 traveling on the own lane 2 toward the intersection 5, and traveling on the intersection road 14 toward the intersection 5. The moving body 7 (in particular, a vehicle 7A such as a truck or a passenger car) is displayed.
Accordingly, the display system 1 includes a camera 10 installed in the vicinity of the intersection 5 to image the moving body 7 traveling on the intersection road 14, and an image captured by the camera 10 (either a still image or a moving image may be used). Is transmitted through road-to-vehicle communication, and an in-vehicle device 12 that receives transmission data from the image transmission device 11 and displays a captured image on the driver of the host vehicle 6.

The installation position of the camera 10 is a position where the vehicle 7A traveling on the intersection road 14 toward the intersection 5 can be imaged from the front.
Therefore, also in the display system 1 of FIG. 2, the camera 10 can be installed not only on the utility pole 13 but also on other ground facilities close to the intersection 5. For example, in addition to the utility pole 13, any place on ground facilities such as an overpass, a footbridge and a building may be used. Alternatively, the camera 10 may be suspended from a wire so as to capture an image from directly above.

The display system 1 shown in FIG. 3 is for preventing a curve accident, and a driver of the host vehicle 6 traveling toward the curved portion 8 of the own lane 2 travels on the opposite lane 3 toward the curved portion 8. The moving body 7 (particularly, a vehicle 7A such as a truck or a passenger car) is displayed.
Therefore, the display system 1 includes a camera 10 installed on the curve portion 8 to image the moving body 7 traveling in the oncoming lane 3 toward the curve portion 8, and images captured by the camera 10 (still images and moving images). Or the vehicle-mounted device 12 that receives transmission data from the image transmission device 11 and displays a captured image on the driver of the host vehicle 6. ing.

The installation position of the camera 10 is a position where the vehicle 7A traveling toward the curved portion 8 of the road 4 can be imaged from the front.
Therefore, also in the display system 1 of FIG. 3, the camera 10 can be installed not only on the utility pole 13 but also on other ground facilities close to the curved portion 8. For example, in addition to the utility pole 13, any place on ground facilities such as an overpass, a footbridge and a building may be used. Alternatively, the camera 10 may be suspended from a wire so as to capture an image from directly above.

The display system 1 shown in FIG. 4 is for preventing a jump-out accident, and causes the driver of the own vehicle 6 traveling in the own lane 2 to become a blind spot by the front vehicle 9 traveling or stopping in front of the own vehicle 6. The moving body 7 (particularly, a pedestrian 7C or a bicycle 7D hidden in the front vehicle 9) is displayed.
Therefore, the display system 1 includes a camera 10 mounted on the forward vehicle 9, an image transmission device 11 that transmits an image captured by the camera 10 (either a still image or a moving image) by inter-vehicle communication, It is comprised from the vehicle-mounted apparatus 12 which receives the transmission data from the image transmission apparatus 11, and displays a captured image on the driver of the own vehicle 6. FIG.

FIG. 5 is a functional block diagram of the display system 1 of the present embodiment.
〔camera〕
Of the devices 10, 11, and 12 constituting the display system 1 of the present embodiment, the camera 10 captures the traffic situation of the road including the moving body 7 and repeatedly captures still images of the traffic situation. Digital cameras that capture the video of the traffic situation (moving images). The camera 10 may be a monocular camera or a stereo camera.

[Image transmission device]
The image transmission device 11 transmits an image captured by the camera 10 and the position and imaging direction of the camera 10 to the in-vehicle device 12. The image transmission device 11 is housed in the same housing as the camera 10 and is connected to the utility pole together with the camera 10. 13 or the like and installed in the forward vehicle 9. However, in the case of road-to-vehicle communication (FIGS. 1 to 3), the image transmission device 11 may be installed separately from the camera 10 and in another place. In this case, the image transmission device 11 is connected to the camera 10 through a communication line. It is.
The image transmission device 11 includes a processing computer 17 that performs recognition processing of the moving body 7 from an image from the camera 10, and a communication device 18 that performs road-to-vehicle communication or vehicle-to-vehicle communication based on a command from the processing computer 17. I have. Hereinafter, processing contents performed by the processing computer 17 of the image transmission apparatus 11 will be described.

[Processing contents of processing computer]
(1) Capture and transmission of image The processing computer 17 includes a frame buffer in the storage area, and always captures an image of the traffic situation captured by the camera 10 in this buffer.
The processing computer 17 performs a mobile object recognition process, which will be described later. When the presence of a desired mobile object 7 is recognized in this recognition process, the processing computer 17 transmits the image data captured in the frame buffer to the communication device 18 in real time. .

(2) Camera Position and Imaging Direction Moreover, the processing computer 17 always knows the position and imaging direction of the camera 10. In the case of the display system 1 by road-to-vehicle communication shown in FIGS. 1 to 3, the position and imaging direction of the camera 10 are stored in advance in the storage area of the processing computer 17. The position of the camera 10 can be set in advance in the apparatus or obtained by a position detection function such as GPS, and is composed of absolute position data of the longitude and latitude of the camera 10. The camera imaging direction can be set in advance in the apparatus or can be automatically obtained by recognizing a road or the like by image recognition and comparing it with a road map or the like. Consists of
On the other hand, in the case of the inter-vehicle communication shown in FIG. 4, since the camera 10 is mounted on the front vehicle 9, the processing computer 17 determines the position and imaging direction of the camera 10 that changes from moment to moment as the GPS of the front vehicle 9. Acquired by a function or the like and transmitted to the communication device 18.

Note that the imaging direction of the camera 10 is usually defined by the optical axis direction of the camera lens. However, as long as an image of a desired traffic situation is obtained, it is not necessary to exactly match the optical axis direction.
If the position (coordinates) of the camera 10 and its imaging direction (vector) can be specified as described above, the camera viewed from the host vehicle 6 can be identified by specifying the position of the host vehicle 6 using the position detection function of the host vehicle 6 or the like. The relative position of 10 and the relative direction of imaging can be calculated, and the moving body 7 can be displayed on the display (display unit) of the in-vehicle device 12 of the host vehicle 6.

(3) Recognition of moving object by feature points FIG. 8 is a diagram illustrating a method of recognition processing of a moving object by feature points and feature lines.
Here, assuming that a stereo camera is used as the camera 10, it is possible to correlate the feature points and feature lines of the vehicle in the left and right images and calculate the three-dimensional position thereof. In this case, for example, recognition data including feature points and feature lines as shown in FIG. 8B is obtained from an actual image of the moving object C as shown in FIG.
For this reason, the extracted vehicle is a passenger car, a truck or the like, a two-wheeled vehicle such as a bicycle or a motorcycle, or a pedestrian by comparing the recognition data with a separately prepared vehicle type template. Can be determined.

  In the example illustrated in FIG. 8, the feature points a to i are feature points associated with each corner of the moving body C, and the line segment ae, the line segment cg, the line segment di, the line segment eh, and the like are moved. This is a feature line associated with the outline of the body C, whereby the three-dimensional position of the moving body C can be specified. This feature point is a point that represents the moving object C, and its position coordinates are three-dimensional in the traveling direction of the road 4, the horizontal direction perpendicular to the road 4 and the horizontal direction and the height direction. However, only the traveling direction of the road 4 or only the traveling direction and the crossing direction of the road may be used.

In addition, the feature point of the moving body C is not used, and a simple method for obtaining the vehicle type from the size on the image of the vehicle, or a method for obtaining the vehicle type of the moving body C from the result of this and template matching can be considered.
In these cases, for example, an appropriate left end point and right end point (points a and h in FIG. 8B) of the cut-out portion near the ground of the image of the moving object C are used as feature points. What is necessary is just to obtain | require the position coordinate of the advancing direction and a cross direction.

(4) Transmission of Captured Image and Transmission Information If the presence of the moving body 7 including the vehicle 7A or the like is found in the traffic situation image by the above recognition process, the processing computer 17 captures the captured image obtained by the camera 10. And the position and imaging direction of the camera 10 are transmitted to the communication device 18 by road-to-vehicle communication or vehicle-to-vehicle communication.
Further, the processing computer 17 stores warning information that identifies the type of danger that can be caused by an accident, which is assumed from the traffic situation of the road 4 when viewed from the host vehicle 6, and uses this warning information as the transmission information. Including the data to be transmitted to the communication device 18.

The warning information stored in the processing computer 17 is specified in accordance with the purpose of imaging the traffic situation by the camera 10, which varies depending on the installation position and imaging direction of the camera 10. In the case of the display system 1 for use, for example, it is composed of character information “right-hand accident warning”.
Further, in the case of the display system 1 for preventing a head-on accident shown in FIG. 2, the warning information is composed of character information such as “attention to encounter” and “attention to blind spots when entering a road”, for example, and the curve accident shown in FIG. In the case of the display system 1 for prevention, for example, it is composed of character information “Curve Blind Spot Caution”, and in the case of the display system 1 for prevention of popping accident shown in FIG. Consists of.

[Information communication system]
FIG. 9 is a conceptual diagram showing a communication method in the case of road-to-vehicle communication by the image transmission device 11.
The communication method shown in FIG. 9A shows a mid-range wireless communication method such as a wireless LAN in which the vicinity of the intersection 5 is a sensing area. In this case, from the communication device 18 of the image transmission device 11, A communication radio wave 19 is transmitted to a certain area including the intersection 5.
However, when such a wireless communication method is adopted, a function for identifying from which image transmission device 11 the transmission information received by the host vehicle 6 entering the intersection 5 from the host lane 2 is transmitted. The in-vehicle device 12 needs to have.

On the other hand, the communication system shown in FIG. 9B shows a narrow-area wireless communication system such as a narrow-area radio such as DSRC or a spot communication system such as an optical beacon.
In this case, a plurality of narrow-area communication devices 20 such as DSRC are arranged in parallel along the traveling direction before the intersection 5 in the own lane 2, and the mobile unit 7 is connected via the plurality of narrow-area communication devices 20. The transmission information including the image is transmitted.
In this case, it is necessary to connect the image transmission device 11 and the narrow area communication device 20 so that they can communicate with each other by, for example, a wired connection.

[In-vehicle device]
Returning to FIG. 5, the in-vehicle device 12 displays the image of the moving body 7 received from the communication device 18 of the image transmission device 11 on the driver, and is imaginary so that the position and imaging direction of the camera 10 included in the transmission information can be visually recognized. Is displayed to notify the driver of the host vehicle 6 on which the in-vehicle device 12 is mounted to the presence of the moving body 7.
The in-vehicle device 12 includes an in-vehicle computer 26 having an image processing function and a man-machine function, a clock device 27 controlled by the computer 26, a position / orientation detection device 28, a communication device 29, a display 30, a speaker 31, and the like. It is configured.

The communication device 29 receives data (transmission information) transmitted from the image transmission device 11, and a demodulation method and a decoding method correspond to the communication device 18 of the image transmission device 11. The timepiece device 27 is composed of, for example, a radio timepiece that accurately obtains the current time, but is not limited thereto.
If the time data is sent from the image transmission device 11, this time data may be referred to, and if the time is known from other GPS, broadcast media, public communication media, etc., refer to this. May be.

The position / orientation detection device 28 is a device for detecting the position and orientation of the vehicle. For example, GPS can be used for position detection of the in-vehicle device 12. In addition to using GPS, various techniques used in in-vehicle navigation, such as map matching on the assumption that the vehicle travels on the road, in-vehicle sensors, may be used.
Alternatively, the current position may be detected by acquiring an absolute position from a roadside device that has just communicated, such as an optical beacon, and using the distance meter of the own vehicle. There are two types of representations of position: absolute representation based on longitude and latitude, and relative representation such as distance from the intersection 5.

Further, the position / orientation detection device 28 detects the direction of the vehicle together with the position of the vehicle. As the vehicle orientation, a technology used in in-vehicle navigation such as a geomagnetic orientation sensor and a gyro sensor can be used. Further, the azimuth may be detected by taking the difference between the current position and the most recently detected position.
The display 30 includes an in-vehicle display 30A that constitutes an image display unit of an in-vehicle navigation device or a television device, or a front display (or head-up display) 30B that imaginarily displays a figure on the windshield surface of the host vehicle 6. . The display 30 is a display unit that displays the image of the moving body 7 and its imaging position and imaging direction so that the driver can see the image. The display 30 functions as a notification unit that notifies the driver of the presence of the moving body 7.

The head-up display 30B displays an image such as a figure on the glass near the driver's seat using an image projection device. A technique for displaying an image such as a figure on a windshield of a vehicle is known (see, for example, JP-A-8-108773).
The speaker 31 is used for announcing warning information included in the image transmission device 11 to the driver by voice.
The in-vehicle computer 26, based on the warning information transmitted from the image transmission device 11, in order to make the driver aware of the presence of the moving body 7 as necessary, screen display data by the display 30 of the warning information, Data for voice output by the speaker 31 is created.

Hereinafter, processing contents performed by the in-vehicle computer 26 of the in-vehicle device 12 will be described.
[Processing content of in-vehicle computer]
(1) Position evaluation of own vehicle and display start timing of moving body When displaying the position of the camera 10 on the display 30, the relative position with respect to the position of the own vehicle 6 needs to be known.
Therefore, when the position of the camera 10 transmitted from the image transmission device 11 is an absolute position such as longitude and latitude, the in-vehicle computer 26 evaluates the position of the host vehicle 6 at the time of display. In other words, the in-vehicle computer 26 of the host vehicle 6 always accurately and accurately evaluates the absolute position by the position / orientation detection device 28.

On the other hand, in the case of mid-range wireless communication such as a wireless LAN, the in-vehicle computer 26 displays an image of the moving body 7 by the display 30 when it arrives at a predetermined position such as immediately before the intersection 5 or a right turn position. Start display. In the case of spot communication, the display of the moving body 7 may be started when data is received, or the display may be started when it arrives at a predetermined position.
Even if the position detection accuracy of the host vehicle 6 by the position / orientation detection device 28 is poor, in the case of the spot communication, if the position of the two-wheeled vehicle 7B can be obtained as a relative position from the spot communication position, at that time Display may be started.

(2) Creation of Display Data for Camera Position and Imaging Direction The in-vehicle computer 26 of the host vehicle 6 is based on the position of the host vehicle 6 and the position and imaging direction of the camera 10 received by the communication device 29. The position and direction data when the position and direction are displayed on the display 30 are calculated.
For example, when the display 30 is the head-up display 30B, the in-vehicle computer 26 determines the display position (w ′, h ′) of the camera 10 on the windshield surface 32 and the camera as shown in FIG. The display vector α ′ for the 10 imaging directions is calculated as follows.

Here, as shown in FIG. 10A, when the height of the driver's eyes from the ground is h, the distance to the windshield surface (display surface) is x, and the distance to the front motorcycle is y, the camera The height (vertical position) h ′ of the display surface 10 is h ′ = (y−x) · h / y. Since y >> x, h ′ = h may be set. Since h varies depending on the driver, an average numerical value may be entered in the system so that it can be easily changed by a man machine during use.
When the host vehicle 6 is traveling in the road direction, as shown in FIG. 10B, from the position of the host vehicle 6, the position of the camera 10, and the road direction, The horizontal position w and the horizontal position w ′ on the windshield surface can be calculated. If the traveling direction of the vehicle can be detected by a sensor or the like mounted on the vehicle, this may be used.

Further, the in-vehicle computer 26 performs the above-described coordinate conversion on the end point of the vector α as well as the coordinates (x, y) of the start point of the vector α in the imaging direction, and the start point and end point of the vector α are calculated. From the coordinates of the display position, a display vector α ′ for displaying the imaging direction of the camera 10 on the windshield surface 32 is calculated.
On the other hand, when the display 30 is the in-vehicle display 30A, the in-vehicle computer 26 converts the position and imaging direction of the camera 10 received from the image transmission device 11 into the coordinate system of the road map displayed on the display 30A, The position and direction data are calculated.

[Display example on display]
(1) When using a head-up display FIG. 11 shows a display example using a head-up display in the case of a display system for preventing a right hand accident (FIG. 1).
In this case, the in-vehicle computer 26 displays the captured image 34 obtained by the camera 10 on the in-vehicle display 30A, and also displays the patterned camera position display 35 and imaging direction display 36 on the windshield constituting the head-up display 30B. The image is displayed at a predetermined display position on the surface 32.

As described above, the driver on the own vehicle 6 can grasp the position and the imaging direction of the camera 10 by the virtual displays 35 and 36 displayed on the windshield surface 32, and thus the captured image 34 of the camera 10 It becomes easy to determine what is dangerous in the traffic situation shown in the captured image 34 as to whether it is an image, and the right-handed accident can be prevented more reliably.
In the display example of FIG. 11, not only the virtual displays 35 and 36 relating to the camera 10 but also warning information for the driver (in this case, a warning display 37 including “right-hand accident warning”) is displayed on the front by the in-vehicle computer 26. Displayed on the glass surface 32. Furthermore, the in-vehicle computer 26 simultaneously outputs the above-mentioned alert statement from the speaker 31 by voice. Note that the captured image 34 of the camera 10 may also be displayed at an arbitrary position on the windshield 32 as indicated by a virtual line in FIG.

FIG. 12 shows a display example using a head-up display in the case of a display system for preventing head-on accidents (FIG. 2), and FIG. 13 shows a head-up display in the case of a display system for preventing curve accidents (FIG. 3). FIG. 14 shows a display example using a head-up display in the case of a pop-up accident prevention display system (FIG. 4).
The display examples shown in FIGS. 12 to 14 are the same as the display example of FIG. 11 except for the imaging purpose and the imaging purpose. Therefore, detailed description will be omitted by assigning the same reference numerals to FIGS.

(2) When using a vehicle-mounted display FIG. 15 shows a display example using a vehicle-mounted display in the case of a display system for preventing a right-handed accident (FIG. 1).
In this case, the in-vehicle computer 26 displays the captured image 34 obtained by the camera 10 on the left side portion of the in-vehicle display 30A, and displays the patterned camera position display 35 and imaging direction display 36 on the road map of the in-vehicle display 30A. Display above.

Also in this case, the driver on the own vehicle 6 can grasp the position and the imaging direction of the camera 10 by the virtual displays 35 and 36 adjacent to the captured image 34. Therefore, where the captured image 34 of the camera 10 is. It is easier to determine what is dangerous in the traffic situation shown in the captured image 34, and the right-handed accident can be prevented more reliably.
In the display example shown in FIG. 15, a vehicle display 38 in which the own vehicle 6 is patterned is also displayed on the road map, which makes it easier for the driver to grasp the relative position between the own vehicle 6 and the camera 10. ing.

FIG. 16 shows a display example using a vehicle-mounted display in the case of a display system for preventing head-on accidents (FIG. 2), and FIG. 17 uses a vehicle-mounted display in the case of a display system for preventing curve accidents (FIG. 3). FIG. 18 shows a display example using a vehicle-mounted display in the case of a display system for preventing a jump-out accident (FIG. 4).
The display examples shown in FIGS. 16 to 18 are the same as the display example in FIG. 12 except for the imaging purpose and the imaging purpose. Therefore, detailed description is omitted by assigning the same reference numerals to FIGS.

Next, a periodic processing flow performed by the processing computer 17 of the image transmission device 11 and the in-vehicle computer 26 of the in-vehicle device 12 will be described with reference to FIGS. 6 and 7.
[Cyclic flow of processing computer]
The processing computer 17 of the image transmission apparatus 11 performs the processing flow shown in FIG. 6 at a predetermined processing cycle.
As shown in FIG. 6, the processing computer 17 first captures an image of the traffic situation imaged by the camera 10 (S1 in FIG. 6), and moves the vehicle or the like by subjecting the subject imaged therein to the feature point processing described above. The body 7 is recognized (S2 in FIG. 6).

  Next, when the processing computer 17 recognizes the moving body 7 (Yes in S2 of FIG. 6), the communication device 18 sends the image captured by the camera 10, the position and direction of the camera, and warning information corresponding to the imaging purpose. (S3 in FIG. 6). If the processing computer 17 does not recognize the moving body 7, the processing computer 17 ends the processing without transmitting the information (No in S2 of FIG. 3).

[Cyclic flow of in-vehicle computer]
On the other hand, the in-vehicle computer 26 of the in-vehicle device 12 performs the processing flow shown in FIG. 7 at a predetermined processing cycle.
As shown in FIG. 7, first, the in-vehicle computer 26 checks whether there is a captured image in the transmission data from the image transmission device 11 (S1 in FIG. 7). In S2, Yes, the relative position and direction of the camera 10 viewed from the own vehicle 6 from the position of the own vehicle 6 and the position and imaging direction of the camera included in the transmission information (the display position (w ', H') and the display vector α ') (S3 in FIG. 7), and the relative position and direction of the captured image 34 of the traffic situation are displayed on the display 30 (S4 in FIG. 7).

The in-vehicle computer 26 determines whether or not the warning information corresponding to the imaging purpose is included in the transmission information from the image transmission device 11 (S5 in FIG. 7). Information is reported to the driver of the host vehicle 6 in a recognizable manner using the display 30 or the speaker 31 or both (S6 in FIG. 7).
As described above, according to the display system 1 according to the present embodiment, not only the traffic situation image of the road 4 including the moving body 7 imaged by the camera 10 but also the position and imaging direction of the camera 10 are determined based on the host vehicle 6. Therefore, the driver of the host vehicle 6 can easily grasp the relative relationship with the moving body 7, and safe driving support for the driver can be more reliably performed.

  Further, in the display system 1 of the present embodiment, warning information corresponding to the purpose of imaging by the camera 10 is output to the driver so as to be recognizable through the display 30 and the speaker 31 of the in-vehicle device 12, so that the position of the camera 10 and the imaging direction Is displayed on the screen of the display 30, the driver can more reliably detect the danger of the image displayed on the screen, and the driver is further surely supported for safe driving.

Furthermore, in the display system 1 of the present invention, when the image transmission device 11 recognizes the moving body 7, the image captured by the camera 10 and transmission information are transmitted. Information is not sent. For this reason, it is possible to prevent the driver from being confused by continuing to transmit images and the like to the in-vehicle device even though it is not dangerous.
In addition, in this case, there is an advantage that the amount of data transmission in road-to-vehicle communication and vehicle-to-vehicle communication can be reduced as compared with a case where a captured image of traffic conditions is continuously transmitted.

FIG. 19 shows a display system 1 according to another embodiment of the present invention.
In addition to the camera 10, the image transmission device 11, and the in-vehicle device 12, the display system 1 of the present embodiment includes a first approach sensor 40 that detects that the moving body 7 is approaching the camera 10, and the in-vehicle device 12. And a second approach sensor 41 for detecting that the host vehicle 5 on which the vehicle 5 is approaching the camera 10 is provided.
These sensors 40 and 41 are all communicably connected to the processing computer 17 of the image transmission apparatus 11 by wire, and the processing computer 17 is configured such that both or one of these sensors 40 and 41 moves. When the body 7 and the host vehicle 6 are detected, a captured image and transmission information are transmitted.

Therefore, only when the moving body 7 and the host vehicle 6 are detected, the image and the transmission information are transmitted from the image transmission device 11, and when there is no moving body 7 that can cause an accident, the information is not transmitted. For this reason, it is possible to prevent the driver from being confused by continuing to transmit images and the like to the in-vehicle device 12 without danger. Also in the present embodiment, there is an advantage that the amount of data transmission in road-to-vehicle communication and vehicle-to-vehicle communication can be reduced compared to the case where a captured image of traffic conditions is continuously transmitted.
In the example of FIG. 19, two types of proximity sensors 40 and 41 for detecting the moving body 7 and for detecting the own vehicle 6 are installed on the road, but only one of these is provided. Also good.

In the embodiment described above, the imaging direction of the camera 10 is displayed on the display 30 in order to make the driver understand the orientation of the camera 10, but the orientation of the camera 10 can also be presented to the driver in other expression formats. . 20 to 22 show other expression formats of the orientation of the camera 20, respectively.
Of these, FIG. 20 shows a display example when the imaging area of the camera 10 is displayed on the in-vehicle display 30A, and the imaging area display 43 in this case corresponds to the angle of view of the camera 10 from the camera position display 35. It has a fan shape that spreads at an opening angle.

FIG. 21 shows a display example when the imaging area of the camera 10 is displayed on the head-up display 30B. The imaging area display 43 in this case corresponds to the angle of view of the camera 10 from the camera position display 35. It has a conical shape with an opening angle.
Furthermore, the orientation of the camera 10 can also be expressed by directly presenting an area that is an imaging target of the camera 10 on a map.
For example, FIG. 22 shows a display example in which an imaging target of the camera 10 is displayed as it is on a simple road figure 44 using VICS.

  In FIG. 22, among the roads included in the road graphic 44, an imaging target display 45 expressed by coloring, shading, or the like is applied to a road portion that is an imaging target of the camera 10. The orientation of the camera 10 is presented to the driver by drawing the imaging target display 45 along with the camera position display 35 on the road graphic 44.

The present invention is not limited to the embodiments described above.
For example, in each of the above embodiments, there may be a plurality of image transmission apparatuses 11 instead of one. Moreover, the recognition accuracy of the mobile body 7 may be improved by installing a plurality of cameras 10 on the same road and using a plurality of imaging data.
Further, as other places where the display system 1 of the present invention can be applied, there are places where the parking lot comes from the main road to the main road, and conversely, there are places where the main road enters the parking lot.

Furthermore, as the image transmission device 11, a device that transmits an image and transmission information when a transmission request from the in-vehicle device 12 is received may be adopted. In this case, there is an advantage that the amount of data transmission in road-to-vehicle communication and vehicle-to-vehicle communication can be reduced as compared with a case where a captured image of traffic conditions is continuously transmitted regardless of whether there is a transmission request.
Further, in each of the above-described embodiments, the calculation of the position and orientation data of the camera 10 and the synthesis to the road graphic are all performed by the in-vehicle device 12, but a system configuration in which this is performed by the image transmission device 11 is also possible. it can.

That is, the image transmission device 11 of this system configuration calculates position and orientation data when viewed on a road map based on the position and orientation of the camera 10, and calculates the calculated position and orientation data of the road map. After creating a composite graphic superimposed on the road graphic, transmission information including the traffic situation image and the composite image is transmitted to the in-vehicle device 12.
Therefore, in this case, since it is not necessary to perform the calculation of the data and the creation of the composite figure in the in-vehicle device 12, the calculation load on the in-vehicle device 12 is reduced and the device can be made more inexpensive.

It is a perspective view which shows the whole structure of the display system for right-right accident prevention. It is a perspective view which shows the whole structure of the display system for a head-on accident prevention. It is a perspective view which shows the whole structure of the display system for a curve accident prevention. It is a perspective view which shows the whole structure of the display system for popping out accident prevention. It is a functional block diagram of the said display system. It is a flowchart which shows the processing flow of an image transmission apparatus. It is a flowchart which shows the processing flow of a vehicle-mounted apparatus. It is a figure which shows the recognition method of the moving body by a feature point and a feature line. It is a conceptual diagram which shows the communication method of road-to-vehicle communication. (A) is a figure which shows the height position of the camera in a windshield surface, (b) is a figure which shows the horizontal direction position of the camera in a windshield surface, (c) is the front where the camera was displayed. It is a front view of a glass surface. It is a figure which shows the example of a display with respect to a windshield surface in the case of the display system for right-handed accident prevention. It is a figure which shows the example of a display with respect to a windshield surface in the case of the display system for a head-on accident prevention. It is a figure which shows the example of a display with respect to a windshield surface in the case of the display system for a curve accident prevention. It is a figure which shows the example of a display with respect to a windshield surface in the case of the display system for popping out accident prevention. It is a figure which shows the example of a display with respect to a vehicle-mounted display in the case of the display system for right-handed accident prevention. It is a figure which shows the example of a display with respect to a vehicle-mounted display in the case of the display system for a head-on accident prevention. It is a figure which shows the example of a display with respect to a vehicle-mounted display in the case of the display system for curve accident prevention. It is a figure which shows the example of a display with respect to a vehicle-mounted display in the case of the display system for popping out accident prevention. It is a perspective view which shows the whole structure of the display system which concerns on other embodiment. It is a figure which shows the example of a display with respect to a vehicle-mounted display in the case of displaying the imaging area of a camera. It is a figure which shows the example of a display with respect to a head-up display in the case of displaying the imaging area of a camera. It is a figure which shows the example of a display with respect to a vehicle-mounted display in the case of displaying the imaging target of a camera.

DESCRIPTION OF SYMBOLS 1 Display system 2 Own lane 3 Opposite lane 4 Road 5 Intersection 6 Own vehicle 7 Moving body 8 Curve part 9 Front vehicle 10 Camera 11 Image transmission apparatus 12 Car-mounted apparatus 17 Processing computer (recognition means)
18 Communication device (transmission means)
26 Onboard computer (calculation unit)
28 Position / Orientation Detection Device 29 Communication Device (Reception Unit)
30 Display (display unit, output means)
30A In-vehicle display 30B Head-up display 31 Speaker (output means)
32 Windshield surface 40 First approach sensor 41 Second approach sensor

Claims (7)

A mobile object display system for displaying a mobile object on a screen in a vehicle,
A camera for imaging the traffic situation of the road including the moving body;
An image transmission device that transmits an image captured by the camera and transmission information including a position and an orientation of the camera;
An in-vehicle device mounted on the vehicle,
The in-vehicle device includes a receiving unit that receives the image and the position and orientation of the camera, and a display unit that simultaneously displays the image and the position and orientation of the camera received on the screen in the vehicle. A calculation unit that calculates position and orientation data when viewed from a driver boarding the vehicle based on the position and orientation of the camera;
The display unit includes a head-up display that displays the calculated position and orientation data on a windshield surface of the vehicle . The transmission information transmitted by the image transmission device includes warning information that identifies the type of danger that can cause an accident that is assumed from the traffic situation of the road when viewed from the vehicle,
2. The mobile display system according to claim 1, wherein the in-vehicle device includes an output unit that warns the warning information received by the receiving unit to a driver of the vehicle.   3. The display system for a moving body according to claim 1, wherein the image transmitting apparatus includes a recognition unit for the moving body, and transmits the image and the transmission information when the recognition unit recognizes the moving body. . The vehicle further equipped with the moving body or the vehicle-mounted device, or both of them are further provided with an proximity sensor that detects that the camera is approaching,
The said image transmission apparatus is a display system of the mobile body of Claim 1 or 2 which transmits the said image and the said transmission information, when the said proximity sensor detects the said mobile body, the said vehicle, or both.   The mobile image display system according to claim 1, wherein the image transmission device transmits the image and the transmission information when receiving a transmission request from the in-vehicle device. An in-vehicle device for displaying a moving body on a screen in a vehicle,
An image obtained by imaging the traffic situation of the road including the moving body, a receiving unit that receives transmission information including the position and orientation of the camera, and
Viewed from the display unit that simultaneously displays the image received by the receiving unit and the position and orientation of the camera on the screen in the vehicle, and a driver boarding the vehicle based on the position and orientation of the camera A calculation unit for calculating the position and orientation data of the case,
The in-vehicle device , wherein the display unit includes a head-up display that displays the calculated position and orientation data on a windshield surface of the vehicle. A method for displaying a moving object for displaying the moving object on a screen in a vehicle,
Imaging the traffic situation of the road including the moving body with a camera,
Transmitting the image captured by the camera and transmission information including the position and orientation of the camera to the in-vehicle device of the vehicle;
Based on the image received by the in-vehicle device and the position and orientation of the camera, the position and orientation data when viewed from the driver on the vehicle is calculated,
A method for displaying a moving body, wherein the calculated position and orientation data are simultaneously displayed on the screen made of a windshield surface of a head-up display .

Images