JP2006240383A - Cruising assist on-vehicle information system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a cruising assist on-vehicle information system capable of successively or selectively obtaining the information of the outside of a vehicle, the inside of the vehicle, and traveling, and machining that to indicate to a driver and the like. <P>SOLUTION: An image of the outside of a vehicle is obtained by a plurality of super wide-angle cameras 1a to 1d. A recording/replying device can record/replay the output of each wide-angle camera as a super wide-angle image. A distortion correcting device cuts out a part of the super wide-angle image and corrects distortion in actual time. A situation detecting means detects vehicle speed or a traveling mode or the behavior of the driver. A display band is structured by two or more display devices having a single or same level display area arranged around the driver. A presentation controlling means controls the presentation to the display device by selecting or composing the output of each correcting device on the basis of judging output of the situation detecting means. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a cruising support in-vehicle information system for extending a driver's field of view. More specifically, by having a plurality of cameras, etc. and a video recording / playback device, editing and processing video information according to the state of driving and the driver, and presenting information desired by the driver on a plurality of displays The present invention relates to a cruising support in-vehicle information system that provides a "surrounding sensation" related to the ability to grasp the driving environment and realizes an atmosphere of safe and rich cruising (driving for fun).

Improvements in driving safety and advancement of in-vehicle information devices provide drivers with more information, such as expansion of field of view. In-vehicle information such as a camera attached to the rear of the vehicle or under the side mirror to support the driver's safe driving, expand visual recognition ability, and display the camera image on the display of the navigation device during a predetermined operation A system has been developed.
In addition, with the advancement of intelligent and multifunctional vehicles, the technology for detecting the driver's drowsiness and the recognition of the vehicle surroundings has progressed, and in-vehicle information systems have been developed to realize safe driving. . The invention described in Patent Document 1 relates to an alarm device, a driver's seat having a vibrating body, and a moving body in which the alarm device is placed on a vehicle, and is intended to awaken the driver.
The invention described in Patent Document 2 relates to an information providing device and provides information and sound regarding the relative positional relationship between a moving body and an obstacle.
JP 2001-199296 A JP2004-251769

Cars are being developed with various functions such as various electronic and car navigation systems, and development of individual elemental technologies and sensing functions for in-vehicle devices. Recently, higher performance of in-vehicle devices using cameras and displays has been promoted. However, the current situation is that the number of accidents during the operation of in-vehicle devices and mobile phone calls continues to increase, and research and development from a wide and integrated perspective based on human cognition and emotional mechanisms has been sufficiently conducted. Not.
In the present invention, it is considered as a means that can be moved to various places that are safe, free, and enjoyable for human beings, and it is not the realization of “individual systems” that support individual driving, but cruising that matches human cognitive and emotional functions (comfortable comfort) We decided to change the problem to realize an in-vehicle information system that supports driving.
An object of the present invention is to provide a cruising support in-vehicle information system that can acquire information on outside, in-vehicle, and traveling sequentially or selectively, process the information, and present it to a driver or the like.

In order to achieve the above object, a cruising assistance vehicle-mounted information system according to the present invention includes:
For example, a plurality of super-wide-angle cameras (cameras with a super-wide-angle lens such as a fisheye lens or a curved mirror) that captures the front, rear, left and right of the vehicle body, means for recording and reproducing the output of the super-wide-angle camera as super-wide-angle camera images, It comprises means for cutting out part of the video and correcting distortion in real time, means for detecting vehicle speed / travel mode / driver behavior, and a plurality of displays arranged around the driver.
The output of one camera is used as it is or the output of two or more ultra-wide-angle lens cameras is extracted and distorted, and when driving at low speed, the image of the outside of the vehicle with a large depression angle is taken with the area to be cut down, and the depression angle is It is characterized by presenting a small outside image. In addition, it is possible to realize that the sensor for detecting stroking is arranged in a circular shape or a rod shape, and the content of the super-wide-angle video can be intentionally controlled.

The cruising assistance vehicle-mounted information system according to claim 1 according to the present invention comprises:
Multiple ultra-wide-angle cameras that capture images outside the vehicle,
A recording / reproducing apparatus capable of recording / reproducing the output of each ultra-wide-angle camera as an ultra-wide-angle image;
A distortion correction device that cuts out a part of the super wide-angle video and corrects the distortion in real time;
Situation detection means for detecting vehicle speed or driving mode or driver behavior;
A display band having a band-shaped display area composed of two or more display devices having a display area at a single or the same height arranged around the driver;
It comprises display control means for controlling the presentation to the display device by selectively or synthesizing the outputs of the correction devices based on the judgment output of the situation detection means.
The cruising assistance vehicle-mounted information system according to claim 2 according to the present invention includes:
The cruising support in-vehicle information system according to claim 1,
Outputs of two or more ultra-wide-angle cameras are cut out by the distortion correction device and synthesized, and presented on the display band.
The cruising assistance vehicle-mounted information system according to claim 3 according to the present invention,
The cruising support in-vehicle information system according to claim 2,
The system further includes an angle-of-view determining unit that determines an angle of view by obtaining a vehicle speed, a direction indicator, a driving mode, a surrounding environment, and the like, and presents an image outside the vehicle with a depression angle depending on a situation on a display band. Is.

The cruising assistance vehicle-mounted information system according to claim 4 according to the present invention includes:
One or more cameras with variable imaging directions for capturing images outside the vehicle;
A recording / reproducing apparatus capable of recording / reproducing the output of the camera;
A situation detection means for detecting a shooting direction of the camera, a vehicle speed or a driving mode or a driver's behavior;
A display band having a band-shaped display area composed of two or more display devices having a display area at a single or the same height arranged around the driver;
Presentation control means for controlling the presentation to the display device by selectively or synthesizing the output of the recording / reproducing apparatus based on the judgment output of the situation detection means;
It is composed of
The cruising assistance vehicle-mounted information system according to claim 5 according to the present invention comprises:
One or more ultra-wide-angle cameras;
A distortion correction device that cuts out a part of the ultra-wide-angle video of the camera and corrects the distortion in real time;
A situation detection means for detecting a shooting direction of the camera, a vehicle speed or a driving mode or a driver's behavior;
A display band having a band-shaped display area composed of two or more display devices having a display area at a single or the same height arranged around the driver;
Presentation control means for presenting images outside the vehicle with different depression angles based on the judgment output of the situation detection means;
It is composed of

A cruising support vehicle-mounted information system according to claim 6 according to the present invention,
The cruising assistance vehicle-mounted information system according to claim 3 or claim 4 or claim 5,
The presentation control means is characterized by always presenting a part of the vehicle body and giving a sense of distance from other objects presented at the same time.
The cruising assistance vehicle-mounted information system according to claim 7 according to the present invention includes:
The cruising assistance vehicle-mounted information system according to claim 3 or claim 4 or claim 5,
The system is provided with a handle in which a position on the handle corresponds to a driver's viewing angle, and a sensor that can detect that the driver has touched and stroked is arranged in a circle,
The situation detection means further detects the fact that the position on the steering wheel corresponds to the viewing angle of the driver and the driver touches and strokes,
By using the handle to indicate the area to be cut out when the video of each ultra-wide-angle camera recorded in the recording / reproducing device is cut out and combined by the distortion correction device,
The line of sight of the recorded video is changed and presented to the presenting means.
The cruising assistance vehicle-mounted information system according to claim 8 according to the present invention comprises:
The cruising assistance vehicle-mounted information system according to claim 3, claim 4 or claim 5,
The system further comprises a rod-shaped sensor in which sensors capable of detecting touch and stroke are arranged in a rod shape on the right or / and left side of the driver in parallel with the direction of travel to serve as a driver's operation means. is there.

A cruising assistance vehicle-mounted information system according to claim 9 according to the present invention,
A cruising support in-vehicle information system according to claim 8,
The position of the rod sensor corresponds to the time,
The rod-shaped sensor indicates the time of a frame to be cut out when the video of each ultra-wide-angle camera recorded in the recording / reproducing apparatus is cut out and combined by the distortion correction device.
A cruising assistance vehicle-mounted information system according to claim 10 according to the present invention,
The cruising support vehicle-mounted information system according to claim 9, wherein the voice comment and the voice recognition result are temporally matched using the voice recognition device in correspondence with the real time and the time of the recorded super wide-angle video. And a means for storing the contents as a video content with a voice comment, and a means for reproducing and presenting the video content with a voice comment.
A cruising assistance vehicle-mounted information system according to claim 11 according to the present invention,
A cruising support in-vehicle information system according to claim 8,
The output of the recording / playback device or the distortion correction device of the system can be recorded in conjunction with the car navigation system, and after setting the route to the destination, the cursor is moved from the starting point by operating the bar sensor back and forth. It is possible to present a recorded video related to the position of the cursor by moving back and forth along the route to the destination.

A cruising assistance vehicle-mounted information system according to claim 12 according to the present invention,
The cruising assistance vehicle-mounted information system according to claim 3 or claim 4 or claim 5,
The system further comprises:
A plurality of distance sensors arranged in either the front, rear, left or right direction of the vehicle body outside the vehicle;
A sounding body corresponding to the distance sensor is arranged around the driver,
A reference sound is generated from these sounding bodies, and the sound for the object detected by the distance sensor is configured to sound a reference sound delayed according to the distance to the object detected by the distance sensor.
A cruising support vehicle-mounted information system according to claim 13 according to the present invention,
A cruising support in-vehicle information system according to claim 12,
The sound emitted from the plurality of sounding bodies can be heard from the approximate direction of the object detected by the distance sensor.
A cruising assistance vehicle-mounted information system according to claim 14 according to the present invention,
14. A cruising assistance vehicle-mounted information system according to claim 13, wherein a voice or video presentation device is placed symmetrically with respect to the driver and information symmetrical to the vehicle body is presented.

The driver's visual cognitive ability can be expanded to give a “surrounding sensation”, and images desired by the driver and passengers can be presented. In addition, by recording and playing back wide-angle video, video content can be produced, transmitted, and shared between the driver and passengers. Driving and moving cars are not limited to safety-oriented driving support. In the sense of enjoying this, an in-vehicle information system capable of cruising support that matches the mechanism of human cognition and emotion can be realized.
In addition, the effect of the present invention is not only to improve safety with an information system that uses a camera or display in a vehicle, but also to focus on the mechanism of human cognition, emotion, memory, and behavior, and `` fun, comfort, To satisfy "comfort and satisfaction", the vehicle (car) itself, spatial and temporal, using multiple wide-angle cameras, multiple displays, storage and playback devices, various input / output devices, and various sensor information processing technologies -The point is to design comprehensively from an aesthetic and functional viewpoint.
In addition to improving the comfort and enjoyment for drivers and passengers while driving, the recorded video and sound information content is not only great for cruising, It has a great effect of creating new value as an excursion, as a memory of excursions and life, and as digital content for enjoying conversations with fellow passengers, friends and acquaintances.

  Embodiments of an apparatus according to the present invention will be described below with reference to the drawings. The system according to the present invention has a variety of support modes and various camera arrangements.

(Camera arrangement example in an embodiment using a plurality of wide-angle cameras)
FIG. 1 is a plan view and a side view showing an example of the arrangement of wide-angle cameras in a cruising assistance in-vehicle information system according to the present invention. In this embodiment, a plurality of super-wide-angle cameras (referred to as cameras having a super-wide-angle lens such as a fisheye lens or a curved mirror) 1a to 1d for imaging the front, rear, left and right of the vehicle body 101 are arranged. A recording / playback apparatus to be described later can record and play back the outputs of the super-wide-angle cameras 1a to 1d without changing the super-wide-angle video. The distortion correction device cuts out a part of the super wide-angle video and corrects the distortion in real time. The driving condition detecting means detects the vehicle speed, the driving mode, or the behavior of the driver. FIG. 2 shows an example of arrangement of display bands in the vehicle in the cruising assistance in-vehicle information system.
The image is displayed on a display band having a band-shaped display area constituted by two or more display devices 2a to 2d (see FIG. 2) having a display area at a single or the same height arranged around the driver.

FIG. 3 is a block diagram for explaining the concept of the cruising support in-vehicle information system having the above arrangement. In this embodiment, the output of the wide-angle camera group 501 arranged so as to surround the vehicle outside the vehicle is connected to the input of the distributor 502.
One of the outputs of the distributor 502 is connected to the input of the recording / reproducing apparatus 503, and the other of the distributor 502 and the output of the recording / reproducing apparatus 503 are connected to the input (upper and lower) of the selector 504, respectively.
The output of the selector 504 is connected to the screen composition device through the distortion correction device 505. When the selector 504 is switched to the upper side, the camera video is connected, and when the selector 504 is switched to the lower side, the playback video of the recording / playback device 503 is connected to the distortion correction device 505. Super-wide-angle images are like fisheye lenses and cannot be used as they are. Decide the center point where you want to see and correct the distortion. An image with a normal angle of view is cut out from the super-wide-angle image by the center coordinates input to the distortion correction device 505. The plurality of cut out videos are combined by the image combining device 506 and displayed on the display group 507.

  When the input of the selector 504 is set upward, the video imaged by the camera group 501 is always recorded in the recording / reproducing apparatus 503 through the distributor 502. Then, it is sent to the distortion correction device 505 through the selector 504. The super wide-angle image sent to the distortion correction device 505 is separated into an image with a normal angle of view based on the center point instructed from the viewpoint calculation device, and sent to the image composition device 506. The images combined inside the image composition apparatus are sent to each display group 507.

FIG. 4 is an explanatory diagram for explaining the distortion correction. The determination method of various coordinates is illustrated.
Here, the coordinate determination device obtains the vehicle speed, the direction indicator, the driving mode, and the surrounding environment, and the center coordinates (x _a , y _a ) (x _z ) of the image obtained by correcting the distortion of each super wide-angle camera image. , y _z ) and global coordinates (X _a , Y _a ) to (X _z , Y _z ) and scale factors (magnification ratios) S _{a to} S used for pasting the respective images after distortion correction. Determine _z .
The images subjected to the distortion correction are pasted in the image composition device, and the pasted images are cut out and displayed on each display.
In this embodiment, the region to be cut out by distortion correction is taken downward when driving at low speed and upward when driving at high speed, so that a video outside the vehicle with a large depression angle at low speed driving and a vehicle outside image with a small depression angle at high speed driving are presented on the display band. I have to.

(Still another embodiment using a single movable camera and vehicle speed)
FIG. 5 is an explanatory diagram of a mode in which cruising is supported using a single movable camera, vehicle speed, and the like. One movable camera 200 is attached to the electric pan head 201 and is installed from the left front of the vehicle body, and the angle (the depression angle) is controlled by a servo system. The main body of the camera 200 is connected to the display device 203 for image output. The vehicle speed is obtained from the vehicle speed sensor 205, and the table held by the angle-of-view determining device 204 is drawn to control the servo of the depression angle of the electric pan head 201. The azimuth angle and the depression angle may be defined in advance as mathematical tables or may be determined by mathematical formulas, and this example is based on the former. It may be defined as a multidimensional table that takes into account not only the change in vehicle speed but also the angle of view due to changes in other factors such as the shift lever. The electric pan head 201 is controlled vertically and horizontally by the output of the angle of view determination device 204, and the camera is directed at an angle corresponding to the vehicle speed.
By registering the angle at which a part of the vehicle body is shown in this numerical table, it is possible to always show the vehicle and an object outside the vehicle at the same time. For example, when the vehicle speed is low, the depression angle is set large, and when the vehicle speed is high, the depression angle is set small so that the start can be displayed and an enlarged image of the front can be presented at high speed.

(Still another embodiment using a single wide-angle camera and vehicle speed)
FIG. 6 is an explanatory diagram of a mode in which cruising is supported using a single wide-angle camera and vehicle speed or the like. Super-wide-angle camera 300 is disposed at the same position as in the above-described embodiment. An angle-of-view determination device 301, a distortion correction device 302, and a display device 303 are provided. However, in this example, since the camera acquires ultra-wide-angle data, there is a problem in which part is taken out.
The video output of the super wide-angle camera 300 is connected to the display device 303 via the distortion correction device 302. The angle-of-view determination device 301 obtains the vehicle speed and determines the corresponding angle of view. The angle of view is sent to the distortion correction device 302 and converted into a normal image around the center point of the super-wide angle image corresponding to the angle of view. This embodiment shows a state of reproduction in real time, and does not depend on a form of recording and reproduction once. An image taken from the super-wide-angle camera 300 in the case of using the super-wide-angle lens appears on the display device 303, and wheels are shown. The video on the left side is distorted as seen from the entrance door scope, and this is presented to the driver on the display device 303 through the above-described distortion correction device 302.

(Embodiment for supporting passenger's editing entertainment)
The most important feature of the cruising support in-vehicle information system is that it uses a number of cameras, displays, and recording and playback functions, enabling not only the driver but also the passengers of the vehicle, including passengers, to enjoy comfortable and fun driving. To create value.
FIG. 7 is a block diagram for explaining the relationship between the stroke sensor and other main components, and FIG. 8 is a schematic diagram showing the sensor and display arrangement in the vehicle when the sensor is used.
As shown in the schematic configuration diagram of the present embodiment shown in FIG. 7, a camera image outside the vehicle during traveling and a stroking type sensor that transmits the human intention are operated in the time axis direction, and in the vehicle It is characterized by the fact that video, voice comments, and text are associated with each other while driving and are stored in the content DB by synergistic effects with human voice.

FIG. 8 shows an example of a mounting position of a sensor (bar-shaped sensor) stroked by a driver and a passenger, and a mounting position of a plurality or a single display for displaying images taken by a plurality of cameras outside the vehicle.
The operation of the time axis by the boiled bar sensors 701 and 701 attached to the door side is well aligned with the traveling direction and is easy to understand. Camera images outside the vehicle are constantly recorded and are played back while being recorded. When the boiled rod sensor is operated, the variable videoTime increases or decreases, and the video at the time videoTime is displayed on the display 704. If the bar sensor is not operated, the variable videoTime is the current time.

The driver who is driving pays attention to the surroundings so as not to cause an accident, and cannot afford to enjoy beautiful scenery and unusual scenes and things like a passenger.
It can be said that when there was no car navigation system, passengers were driving together (cruising) with assistance to the driver while looking at the map. However, the passengers are currently free, and can lead the creation of a memorable video album when, for example, cruising for sightseeing or leisure.

If you encounter a beautiful scene, flora and fauna, or something unusual or interesting with a video taken by multiple cameras outside the car while driving, if there is an image you want to leave behind, stroke the rod sensor Then, the passenger reverted the image in time, and the passenger adjusted the time by turning on the microphone in the car, adding a voice comment, and utilizing voice recognition. Voice comment + text can be associated. Memories remain, and search is facilitated with text entered by voice recognition.
Specifically, multiple camera images are displayed on the front display, and the direction of travel and the time axis are made to correspond to each other using a bar-shaped sensor installed on the driver's seat and front passenger seat, and also on the door side of the rear seat. Control of the relative time axis is possible. Control the video temporally while stroking, and input a voice comment at an appropriate timing. At this time, a voice comment is recorded by a specific input method to the stroke sensor, voice command recognition, or input to the touch panel on the display. Although speech recognition is incomplete, text editing facilitates later editing and video search.

Another voice comment input format allows you to record conversations with drivers and passengers in-house while associating live real-time video with voice conversations and utterances while looking out the window while driving. is there. At the same time, voice recognition can be used to make text. Even if there is an error in text conversion from speech, it can be corrected by post-editing. A simple response is important in the car.
The example of the combination of the rod-shaped sensor, voice signal input, voice recognition text input, and voice command recognition as in this embodiment is a novel combination idea, and it is possible to enhance the multimodal video content DB. It is.

FIG. 9 shows a flowchart of the process.
If there is an input to the boiled sensor (bar-shaped sensor), the data will be other than 0 and the content production will be started.
Assign the data to the variable Data1, read the data from the sensor (bar-shaped sensor) that is stroked again, and assign the data to the variable Data2.
If Data1 is larger than Data2, it means that it has been stroked in the direction of travel of the car, so add the time variable videoTime.
Conversely, if Data1 is smaller than Data2, it means that the vehicle has been stroked in the reverse direction of the car, so the time variable videoTime is subtracted.
The video at time videoTime is displayed on the display.
Since the recording of the voice comment is started by performing a specific operation, the time adjustment is continued until the specific operation is performed.
When a specific operation comes, voice recognition starts. Here, the voice is recorded while performing voice recognition, and when the voice comment input is completed, the text of the voice recognition result and the recorded voice data are tagged at the time videoTime obtained by the stroke sensor operation, and saved in the content DB. .

(Embodiment in cooperation with car navigation)
An embodiment in cooperation with car navigation will be described with reference to FIG.
In this embodiment, the driver refers to a video from the starting point to the current location and a video from the current location to the destination by stroking a bar sensor characterized by being installed in parallel with the traveling direction. be able to. The operating principle of this system is as follows.
After the route is determined, the horizontal and vertical movement on the road can be simplified to the movement back and forth along the route. When the driver sets a destination on the car navigation, the car navigation determines a route from the departure point to the destination and displays the current location with a cursor.
The cursor can be moved back and forth along the route by stroking the rod sensor in the front-rear direction, and car navigation searches the content database 503 based on the position information of the latitude and longitude corresponding to the cursor and corresponds to the position information If there is content to be played, playback is started. Whether or not to pass through the distortion correction device is selected depending on whether the content is a super wide-angle video or not. When going through the distortion correction device, the video range is cut out instead of cutting out the time. The cut-out area can be moved by a driver's behavior sensor and a stroke sensor installed on the steering wheel. After setting the route from the departure point to the destination on the car navigation, the driver can move the cursor along the route by stroking the rod-shaped sensor back and forth.

(Embodiment for Reproducing Past Cruising Record)
In FIG. 3, when the input of the selector 4 is set to the lower side, the super wide-angle playback video from the recording / playback device 3 is guided to the distortion correction device 5.
In the present embodiment, the driver can view the images taken from the departure point to the present location by stroking a rod sensor characterized by being installed in parallel with the traveling direction in the front-rear direction.
Further, the front end and the rear end of the bar-shaped sensor correspond to the current location and the departure location, respectively, and the top of the recorded video is cued by touching the bar. At this time, touching the stick indicates the ratio from the starting point to the present location. For example, if you touch the center of the bar, you can cue and refer to the center of the recorded video from the starting location to the current location. The center is designed to be the center of time. It can also be designed to be in the middle of the distance.
Alternatively, the front end and the rear end of the bar sensor can be designed to correspond to the current location and the destination, respectively. At this time, the video from a certain point on the bar (current location) to the front end is naturally not recorded, but according to the planned route from the current location to the destination obtained by this embodiment, it is related to the longitude and latitude corresponding to the touched part. Information to be searched can be retrieved from the content database 503 and referenced.
Furthermore, a display device can be provided on the rod-shaped sensor to display the time and distance scales and the current car position to facilitate the touching operation. When a convenience store is searched for by car navigation, if the convenience store to the destination is lit up and displayed on the bar-shaped sensor, it is possible to determine where to stop at the convenience store. Searching for a gas station can be linked to the remaining amount of gasoline.
By making the touch correspond to absolute designation such as distance and time, and stroking to relative designation, and displaying scales, the operation can be facilitated.
Since the video is recorded as a super wide-angle video, the angle of view can be changed in any direction during playback. This is done using a handle (not shown). The feature is that it keeps shooting at an ultra-wide angle, and it is possible to take out images at any time and in any direction.

(Embodiment in which a voice guide is provided to a cruising support in-vehicle information system)
A plurality of distance sensors are arranged in one of the front, rear, left and right directions of the vehicle body outside the vehicle. A speaker, which is a sounding body corresponding to the distance sensor, is arranged around the driver.
A reference sound is emitted from these speakers, and the sound for the object detected by the distance sensor is configured to sound a reference sound delayed according to the distance to the object detected by the distance sensor.
In the cruising assistance in-vehicle information system, cruising assistance is performed by arranging the sound emitted from the plurality of speakers so as to be heard from the approximate direction of the object detected by the distance sensor.

  FIG. 11 is a schematic diagram illustrating an arrangement example of the speakers 901 to 904, and FIG. 12 is a schematic diagram illustrating an arrangement example of the plurality of distance sensors 981 to 986. FIG. 13 is a circuit block diagram of an embodiment to which an audio guide is added.

  The distance information of the distance sensors 981 to 986 is input to the control unit 1001. In the control unit 1001, the reference sound is generated by the reference sound generation unit 1003, and the reference sound is delayed by the delay time control unit 1004 using the information obtained by converting the distance information from the distance information to the time information by the arithmetic processing unit 1102. The approximate sound source direction calculation processing unit 1105 uses the detected object approximate direction information processed by the operation processing unit 1102 to calculate the reference sound delayed by the delay time control unit 1004 as if there is a sound source in the approximate direction of the detected object. After being processed and added to the reference sound by the adders 1111 to 1114, the sound is amplified by the amplifiers 1121 to 1124 and output from the speakers 991 to 994.

In the cruising assistance vehicle-mounted information system, the cruising assistance is performed by the rough sound source direction calculation processing unit 1105 of FIG. 13 described above by arranging the sounds emitted from the plurality of speakers so that they can be heard from the approximate direction of the object detected by the distance sensor. This can be realized if the processing is not performed (the output of the delay time control unit 1004 is output as it is).
As a modification of the thirteenth aspect, a part of the speaker shown in FIG. 13 can be a diaphragm speaker or a diaphragm. As an example of arrangement, the backrest or leg of the seat may be used.

(Embodiment of cruising support for proximity obstacle)
Support at the start and end of Kruzink is also necessary to support a comfortable Kruzink. FIG. 14 is an explanatory diagram showing an embodiment that supports driving for nearby obstacles such as garage entry. The configuration will be described along with the operation in relation to garage entry. In the vehicle, acoustic presentation devices (speakers) 1204a and 1204b arranged symmetrically with respect to the driver are arranged.
First, a reference sound is simultaneously generated, and the delay exaggeration device 1205 (arrangement is at an arbitrary position in the vehicle) is based on the measured left and right distances based on the distance from the vehicle outer wall to the vehicle outer object measured by the distance sensors 1203a and 1203b. Exaggerate the time delay and sound a confirmation sound with the delay.
The confirmation sound delay is determined by the distance from the outer wall of the car.
If the left and right obstacles are equidistant from the outer wall of the car, the left and right confirmation sounds will sound at the same time after a short delay after the reference sound sounds at the same time.
If the driver's seat is on the right side of the car, the distance from the driver's seat to the left obstacle is actually long, but according to this method, if the distance from the vehicle body to the left and right obstacles is the same Since the confirmation sound is heard at the same time, it is possible to grasp the state from the left and right sides of the vehicle to the obstacle, which is important when entering the garage. According to this embodiment, even if the driver's seat is arranged on either the left or right side, it is possible to give the driver an auditory assistive sensation as if he is sitting in the center of the car.
FIG. 14 is an example in which images captured by the cameras 1201a to 1201d attached to the outer wall are respectively presented to the image presentation devices 1202a to 1202d arranged on the left and right sides of the driver's seat. Since the image of the outer wall is presented symmetrically with respect to the driver's left and right, it is possible to give the driver a visual assistance feeling as if he is sitting in the center of the car.

The importance of the cruising support in-vehicle information system according to the present invention will surely increase with the high functionality and low price of the car navigation system and the image equipment system.
Various automobiles have evolved from simple means of movement to “moving space” for enjoying driving, running, and even communication. What is needed is not only to improve driving safety, but also to human safety. The pursuit of “fun, richness, comfort and satisfaction” based on cognition, emotion and behavior. The development of “cruising (comfortable driving) support” or “fast driving support” using a plurality of cameras and displays according to the present invention is certainly important industrially. The information system according to the present invention has developed into an industrially important technical field and is very likely to be widely used.

It is the top view and side view which show the example of arrangement | positioning of the camera in the cruising assistance vehicle-mounted information system by this invention. The example of arrangement | positioning of the display band used with the said cruising assistance vehicle-mounted information system is shown. It is a block diagram for demonstrating the concept of the cruising assistance vehicle-mounted information system by this invention. It is a figure for demonstrating the relationship of the distortion correction and the coordinate between images. It is explanatory drawing of the aspect which assists cruising using a single movable camera, vehicle speed, etc. FIG. It is explanatory drawing of the aspect which supports cruising using a single wide-angle camera, a vehicle speed, etc. FIG. It is a block diagram for demonstrating the relationship with the other main structure relevant to the boiled sensor. It is the schematic which shows arrangement | positioning of the sensor in a vehicle, a display, etc. in the case of using the sensor which strokes. The flowchart of a process is shown. A description will be given with reference to an embodiment in cooperation with car navigation. It is a schematic diagram showing an example of arrangement of a plurality of distance sensors. It is the schematic which shows the example of arrangement | positioning of a speaker. It is a circuit block diagram of a system to which a voice guide is added. It is explanatory drawing which shows the embodiment which assists the driving | operation with respect to the obstacle which adjoins, such as garage storage.

Explanation of symbols

1 Super Wide Angle Camera 2a ・ ・ 2d (201) Display

Claims (14)

Multiple ultra-wide-angle cameras that capture images outside the vehicle,
A recording / reproducing apparatus capable of recording / reproducing the output of each ultra-wide-angle camera as an ultra-wide-angle image;
A distortion correction device that cuts out a part of the super wide-angle video and corrects the distortion in real time;
Situation detection means for detecting vehicle speed or driving mode or driver behavior;
A display band having a band-shaped display area composed of two or more display devices having a display area at a single or the same height arranged around the driver;
Presentation control means for controlling the presentation to the display device by selectively or synthesizing the outputs of the correction devices based on the judgment output of the situation detection means;
Cruising support in-vehicle information system composed of The cruising support in-vehicle information system according to claim 1,
A cruising assistance vehicle-mounted information system characterized in that outputs of two or more ultra-wide-angle cameras are cut out and synthesized by the distortion correction device and presented on the display band. The cruising support in-vehicle information system according to claim 2,
The system further includes an angle-of-view determining unit that determines an angle of view by obtaining a vehicle speed, a direction indicator, a driving mode, a surrounding environment, and the like, and presents an image outside the vehicle with a depression angle depending on a situation on a display band. Cruising support in-vehicle information system. One or more cameras with variable imaging directions for capturing images outside the vehicle;
A recording / reproducing apparatus capable of recording / reproducing the output of the camera;
A situation detection means for detecting a shooting direction of the camera, a vehicle speed or a driving mode or a driver's behavior;
A display band having a band-shaped display area composed of two or more display devices having a display area at a single or the same height arranged around the driver;
Presentation control means for controlling the presentation to the display device by selectively or synthesizing the output of the recording / reproducing apparatus based on the judgment output of the situation detection means;
Cruising support in-vehicle information system composed of One or more ultra-wide-angle cameras;
A distortion correction device that cuts out a part of the ultra-wide-angle video of the camera and corrects the distortion in real time;
A situation detection means for detecting a shooting direction of the camera, a vehicle speed or a driving mode or a driver's behavior;
A display band having a band-shaped display area composed of two or more display devices having a display area at a single or the same height arranged around the driver;
Presentation control means for presenting images outside the vehicle with different depression angles based on the judgment output of the situation detection means;
Cruising support in-vehicle information system composed of The cruising assistance vehicle-mounted information system according to claim 3 or claim 4 or claim 5,
The cruising support vehicle-mounted information system, wherein the presentation control means always presents a part of the vehicle body and gives a sense of distance to other objects presented at the same time. The cruising assistance vehicle-mounted information system according to claim 3 or claim 4 or claim 5,
The system is provided with a handle in which a position on the handle corresponds to a driver's viewing angle, and a sensor that can detect that the driver has touched and stroked is arranged in a circle,
The situation detection means further detects the fact that the position on the steering wheel corresponds to the viewing angle of the driver and the driver touches and strokes,
By using the handle to indicate the area to be cut out when the video of each ultra-wide-angle camera recorded in the recording / reproducing device is cut out and combined by the distortion correction device,
A cruising support in-vehicle information system characterized by changing the line of sight of a recorded video and presenting it to the presenting means. The cruising assistance vehicle-mounted information system according to claim 3, claim 4 or claim 5,
The system further provides a cruising support in which a rod-like sensor in which a sensor capable of detecting touch and stroke is arranged in a rod-like shape is arranged on the right or / and left side of the driver in parallel with the direction of travel to operate the driver. In-vehicle information system. A cruising support in-vehicle information system according to claim 8,
The position of the rod sensor corresponds to the time,
A cruising support vehicle-mounted information system configured to indicate, with the rod-shaped sensor, the time of a frame to be cut out when the video of each ultra-wide-angle camera recorded in a recording / reproducing device is cut out and combined by the distortion correction device. A cruising support in-vehicle information system according to claim 9,
Corresponding to real time and time of recorded super wide-angle video, using voice recognition device, add voice comment and voice recognition result in time correspondence and record as video content with voice comment A cruising support on-vehicle information system having means for playing back and presenting the video content with voice comments. A cruising support in-vehicle information system according to claim 8,
The output of the recording / playback device or the distortion correction device of the system can be recorded in conjunction with the car navigation system, and after setting the route to the destination, the cursor is moved from the starting point by operating the bar sensor back and forth. A cruising support in-vehicle information system that can move back and forth along the route to the destination and present the recorded video related to the cursor position. The cruising assistance vehicle-mounted information system according to claim 3 or claim 4 or claim 5,
The system further comprises:
A plurality of distance sensors arranged in either the front, rear, left or right direction of the vehicle body outside the vehicle;
A sounding body corresponding to the distance sensor is arranged around the driver,
A cruising assistance vehicle-mounted information system configured to sound a reference sound from these sounding bodies, and to sound a sound for an object detected by the distance sensor delayed according to a distance to the object detected by the distance sensor. A cruising support in-vehicle information system according to claim 12,
A cruising assistance vehicle-mounted information system configured to be able to hear sounds from the plurality of sounding bodies from a general direction of an object detected by a distance sensor.   14. The cruising assistance vehicle-mounted information system according to claim 13, wherein a voice or video presentation device is placed symmetrically with respect to the driver and information symmetrical to the vehicle body is presented.

Images