JP4329525B2 - Outside-vehicle information display device - Google Patents

Description

  The present invention relates to an image display device mounted on a vehicle, and more particularly to an outside information display device that displays a transmission image of a vehicle body.

2. Description of the Related Art Conventionally, when a driver of an automobile visually recognizes the surroundings of a vehicle, there is a blind spot area around the vehicle that is unrecognizable because the vision is blocked by the shadow of the vehicle body. Conventionally, a configuration has been proposed in which a blind spot area is acquired by an image acquisition device such as a camera as an assisting driving operation and an image is displayed on a monitor in a vehicle. For example, a configuration has been proposed in which a camera that displays an area close to the rear of the vehicle is installed at the rear of the vehicle, and a rear captured image is automatically displayed on the in-vehicle monitor when the vehicle moves backward.
JP2002-262280

  As represented by Patent Document 1, a monitor on which an image is projected is fixed to the vehicle side. For this reason, when the direction to be confirmed with the naked eye does not coincide with the installation position of the monitor, it is necessary to perform troublesome operations such as confirming the outside of the vehicle with the naked eye and checking the blind spot area with the face facing the monitor.

  For example, in a back monitor that displays the blind spot area behind the vehicle, the steering is performed with the face facing the back seat in order to check the rear with the naked eye when reversing, and when checking the blind spot area, You need to look at the monitor placed in front. In this way, when retreating, there is a problem that the face must be frequently turned back and forth, which makes the operation difficult.

  In addition, since the monitor is fixed in the conventional device, in order to see what the driver wants to see, the camera is switched from the menu screen using a joystick or touch panel device installed on the main body or remote control, or I took measures such as panning and tilting the camera. The driver needs to scroll the camera image or switch the camera image by these means, which is very troublesome for the driver.

  An object of the present invention is to provide a vehicle information display device that can display a vehicle exterior situation in an arbitrary direction on the user's line of sight as an image viewed from the user's viewpoint position.

The present invention for solving the above problems has the following configuration.
(1) imaging means for imaging the surroundings of the vehicle;
A display having an image display surface;
Position detecting means for detecting a positional relationship between a predetermined starting point and the display;
Image extracting means for extracting an image around the vehicle existing on an extension line of a line from the reference point toward the display direction based on the positional relationship detected by the position detecting means from the image picked up by the image pickup means;
An outside information display device comprising: display means for displaying an image extracted based on the image extraction means on the display.

(2) The vehicle exterior information display device according to (1), wherein the reference point is a position virtually assuming a driver's viewpoint position.

(3) The vehicle outside information display device according to ( 1) or ( 2), wherein the starting point is a holding position of the display.

(4) imaging means for imaging the surroundings of the vehicle;
A display having an image display surface;
Position detecting means for detecting a positional relationship of the display with respect to a predetermined starting point;
Viewpoint position specifying means for specifying the viewpoint position of the user using the display based on the positional relationship detected by the position detecting means;
Direction detecting means for detecting a direction from the viewpoint toward the display from the viewpoint specified by the viewpoint position specifying means and the position of the display;
Image extracting means for extracting an image around the vehicle corresponding to the direction detected by the direction detecting means from the image taken by the imaging means;
A vehicle outside information display device comprising: display means for displaying an image extracted based on the image extraction means on the display.

  (5) The above-described (2) or (2), further comprising: viewpoint conversion means for performing viewpoint conversion processing on the image extracted by the image extraction means to an image when the position of the display where the position is specified is visually recognized from the viewpoint position. (4) The vehicle information display device according to (4).

(6) an angle detection means for detecting an inclination and a rotation angle of the display surface in the vehicle,
The vehicle information display apparatus according to (1) or (4), wherein the image extraction unit further extracts the image based on the angle detected by the angle detection unit.

(7) vehicle information storage means for storing the contour of the vehicle;
Contour forming means for forming a vehicle contour when viewed from a viewpoint position based on vehicle contour information stored in the vehicle information storage means;
The vehicle exterior information display device according to any one of the above (1) to (6), further comprising: an image synthesis unit that synthesizes the contour of the vehicle formed by the contour formation unit with an image displayed on the display unit.

(8) A plurality of the imaging means are provided in different directions,
A selection unit that selects one of the plurality of imaging units, the imaging unit in a direction in which the angle formed by the line from the viewpoint position to the display position and the optical axis of the imaging unit is the smallest,
The vehicle information display device according to any one of (1) to (7), wherein the image extraction unit extracts an image from an image acquired by the selected imaging unit.

(9) imaging means for imaging the surroundings of the vehicle;
A display having an image display surface;
Angle detection means for detecting the tilt and rotation angle of the display surface;
Image extracting means for extracting an image facing the back surface of the display surface from the image captured by the imaging means based on the tilt and rotation angle of the display detected by the angle detecting means;
An outside information display device comprising: display means for displaying an image extracted based on the image extraction means on the display means.

According to the first aspect of the present invention, since the image around the vehicle existing on the extended line in the display direction is displayed on the display starting from the reference point, the situation of the area that becomes the blind spot behind the vehicle is accurately determined. I can grasp it.
According to the invention described in claim 2, by setting the reference point to a position virtually assuming the driver's viewpoint position, the orientation of the displayed image matches the orientation of the user. In addition, it can be recognized as if a perspective image of the vehicle is displayed on the screen, and an image that matches the sense of direction and distance of the user can be displayed.

According to the third aspect of the present invention, since the reference point and the viewpoint position are defined starting from the holding position where the normal display is held, the coordinate display becomes simple and the reference point and the viewpoint position can be easily calculated. It becomes.
According to the fourth aspect of the invention, the relative position of the display with respect to the starting point is defined based on the starting point, the viewpoint position is specified based on the specified relative position, and then the direction from the viewpoint position toward the display is determined. To detect. Since the vehicle surrounding image in the direction from the viewpoint position toward the display is extracted, the blind spot area in the line-of-sight direction in which the user is looking at the display is displayed on the display. For this reason, it is easy to sensuously grasp the relative position from the location where the user is located about the contents of the displayed image.

  According to the fifth aspect of the present invention, the image acquired by the imaging unit is converted from the position of the imaging unit that displayed the image into the viewpoint position of the user, and the converted image is converted to a display. The displayed image can be recognized as if it has been projected through the vehicle body, and the situation around the vehicle can be visually grasped as if it were actually viewed with the naked eye. Can do.

According to the sixth aspect of the present invention, since the image to be extracted is determined based on the tilt and rotation angle of the display surface, the user operates the tilt and rotation angle of the display, and the vehicle body transmission image in an arbitrary direction. Can be displayed on the display.
According to the seventh aspect of the present invention, since the outline of the vehicle is combined with the image and displayed, it is easy to grasp the positional relationship between the situation around the vehicle displayed on the display and the vehicle.

According to the eighth aspect of the present invention, since the image pickup unit that is directed in the direction closest to the direction of the line of sight from the viewpoint position toward the display is selected and the image of the image pickup unit is used, image processing is easy, Image distortion caused by processing can also be suppressed.
According to the ninth aspect of the present invention, since the image to be extracted is determined based on the tilt or rotation angle of the image display surface, the user operates the tilt or rotation angle of the display, and the positional relationship between the viewpoint position and the display. Without changing, the transmission image of the vehicle body in any direction can be projected on the display.

  Next, a preferred embodiment of the vehicle exterior information display device 1 of the present invention will be described. The vehicle exterior information display device 1 of the present invention is mounted on an automobile and includes an image acquisition device 1A and an image display device 1B. FIG. 1 is a block diagram showing the configuration of the image acquisition device 1A. The image acquisition device 1A includes a plurality of cameras 3F to 3B, which are imaging units that are attached to the vehicle 2 and image the surroundings of the vehicle, an arithmetic processing device 4A that performs arithmetic and control operations such as image processing, a memory 5A, and an image display A communication device 61A that communicates with the device 1B, a connection unit 62A that connects the image display device 1B, and a system bus 11A that mediates communication of these devices are provided.

  As shown in FIG. 2, the cameras 3F, 3R, 3L, and 3B are provided on the side surface of the vehicle 10 toward the outside. In the vehicle 10, the camera 3F is provided at the center of the front of the vehicle body, the camera 3B is provided at the center of the rear of the vehicle body, the camera 3R is provided at the side of the driver's right side of the vehicle, and the camera 3L is provided at the side of the driver's left side of the vehicle. ing. Each camera in the present embodiment is a fisheye lens and has an angle of view of 180 °. The cameras 3R and 3L provided on the side surfaces are provided not on the center of the vehicle body but on the side of the driver's seat. This is because it is often the driver who uses the apparatus, and is set to such a position in order to bring the photographing positions of the cameras 3R and 3L closer to the viewpoint position of the user (driver). This is because, when performing viewpoint conversion processing described later, the burden of the conversion processing is reduced and distortion of the image after conversion can be suppressed.

Each of the cameras 3F, 3R, 3L, and 3B uses a fish-eye lens, and has a configuration that converts a light (image) signal into an electrical signal using, for example, a CCD (Charge-Coupled Devices). The cameras 3F, 3R, 3L, and 3B are connected to the system bus 11A via A / D converters 31F, 31R, 31L, and 31B, respectively. The camera 3F directs the optical axis of the lens in front of the vehicle 10 and acquires the surrounding scenery as an image. The camera 3R directs the optical axis of the lens to the right side of the vehicle 10 and acquires the surrounding scenery as an image. The camera 3L directs the optical axis of the lens toward the left side of the vehicle 10 and acquires the surrounding scenery as an image. The camera 3B directs the optical axis of the lens to the rear of the vehicle 10 and acquires the surrounding scenery as an image.

Image signals output from the cameras 3F, 3R, 3L, and 3B are converted into digital signals by A / D converters 31F, 31R, 31L, and 31B, respectively.
Further, when the connected cameras 3F, 3R, 3L, and 3B can output digital signals, they are not necessary for the A / D converters 31F, 31R, 31L, and 31B. In the present embodiment, as described above, the imaging unit has four cameras using fisheye lenses, but any camera may be used as long as an image of 360 ° around the vehicle can be acquired. Absent. For example, a plurality of five or more cameras may be installed toward the outside of the vehicle, and one camera that can acquire 360 ° images may be used.

The arithmetic processing unit 4A is a central processing unit [CPU (Central Processing Unit).
] 41A, read only memory [ROM] 42A, and random access memory [RAM] 43A. For example, the central processing unit 41 calculates a relative position between the image capturing device 1A and the image display device 1B, or synthesizes a 360 ° image around the vehicle body 10 based on images acquired from the cameras. Alternatively, an image to be sent to the image display device 1B is cut out (extracted) from the combined image, and the cut image is converted into the viewpoint position of the user, and the contour of the vehicle is combined with the cut image. Alternatively, processing and calculation such as data transmission / reception with the image display device 1B are performed.
The ROM 42 stores, for example, software for the central processing unit 41 to perform image processing, and the RAM 43 is used as a working area as a place for performing various image processing, for example.

  The memory 5A includes camera image buffers 51F, 51R, 51L, and 51B that continuously record images taken by the cameras 3F, 3R, 3L, and 3B over time, a drawing image buffer 52, and an image processing information memory. 53, a vehicle body data memory 54 serving as vehicle information storage means, and a work memory 55.

  The drawing image buffer 52 stores an image generated by performing image processing on the camera image. The image processing information memory 53 stores variables necessary for executing image processing such as viewpoint conversion, distortion correction, and composition. For example, setting values can be input and changed as appropriate via a menu screen configured with a GUI. The vehicle body data memory 54 stores contour information necessary for creating a vehicle body transmission image superimposed on the display image. In the contour information, 3D modeling data representing the contour of the vehicle body, the tire position, etc., or bitmap data drawn so as to be visible in 3D are stored. The data stored in the vehicle body data memory 54 includes mask data used for combining with a camera image or a camera image after image processing.

The communication device 61A receives sensor data from the image display device 1B and transmits drawing image data to the image display device 1B. As communication means, there are wired and wireless, but in the case of wireless, wireless LAN (IEEE802.11a / b / g), Bluetooth,
th, Infrared (IrDA), low power FM transmitter, UWB (Ultra Wide Band), spread spectrum (SS), etc. are conceivable.

  62 A of connection parts are provided with a connection terminal and connect with the connection terminal provided in the connection part 62B of the image display apparatus 1B. By connecting to the connection unit 62B of the image display device 1B, the same operation as that of the communication device 61A is performed. The connecting portion 62A functions as a holding position for connecting to the image display device 1B, for example.

On the other hand, the image display device 1B includes an arithmetic processing device 4B that performs arithmetic processing and control such as image processing, a communication device 61B, a connection unit 62B, a memory 5B, a position / angle detection device 7, and an A / D conversion device. 71, a display device 8, a D / A conversion device 81, and a system bus 11B that mediates communication of these devices. FIG. 3 is a block diagram showing a configuration of the image display device 1B.

The arithmetic processing unit 4B is a central processing unit [CPU (Central Processing Unit)].
] 41B, read-only memory [ROM] 42B, and random access memory [RAM] 43B. The central processing unit 41B, for example, writes and calculates position and angle data (sensor data) from the position / angle detection device 7 to the memory 5B, performs data transmission / reception control, image drawing, and the like. The communication device 61B transmits sensor data to the image capturing device 1A and receives drawing image data from the image display device 1A. The connection unit 62B includes a connection terminal, and operates in the same manner as the communication device 61B by connecting to the connection terminal provided in the connection unit 62A of the image acquisition device 1A.

  The position / angle detection device 7 which is the position detection means and the angle detection means of the present invention detects the position and angle from the reference point of the image display device 1B. The position / angle detection device 7 includes a gyro sensor, an acceleration sensor, a geomagnetic sensor, and the like. These sensors are used to calculate the relative distance (positional relationship) and the relative angle (direction) between the image acquisition device 1A and the image display device 1B. If the relative distance and the relative angle can be calculated, One or more of the above sensors may be used. Furthermore, the present invention is not limited to the above-described sensor, and any sensor may be used as long as it can detect the relative distance and the relative angle.

  The orientation of the image display device 1B is detected by the gyro sensor and the geomagnetic sensor, and the moving distance and angle (elevation angle / depression angle) are detected by the acceleration sensor. Further, a method of measuring the position of the image display device 1B from three survey points provided in the vehicle by radio waves, ultrasonic waves, etc., and detecting the position is adopted. Specifically, position detection is performed using a combination of position detection by the sensors and position detection by three-point surveying. In this position detection calculation, the calculation based on the data detected by the sensors and the calculation based on the three-point survey are performed by the central processing unit 1A of the image acquisition device 1A. Further, the attitude of the image display device 1B can be detected by detecting the positions of the three points on the image display device 1B side, as in the case of GPS and radio wave surveying.

The A / D converter 71 converts the sensor output, which is an analog output, into digital data so that the central processing unit 1B can handle it. When the output from the position / angle detection device 7 is a digital output in advance, the A / D converter 71 is not necessary.

  The display device 8 as the display means of the present invention displays the received drawing image data. The display device 8 includes a display screen, and for example, a liquid crystal panel or an EL panel is used as the display screen. When the display device 8 is an analog input, the D / A converter 81 converts digital data handled by the central processing unit 1B into analog data. If the display device 8 is compatible with digital input, it is not necessary. The memory 5B is used for buffering sensor data and received image data.

The operation of the vehicle exterior information display apparatus 1 of the present invention configured as described above will be described based on the flowchart shown in FIG.
When the system is activated, first, initialization processing of the image capturing device 1A and the image display device 1B is performed (steps S101 and S201). System activation may be performed by ignition ON, switch operation, or the like, and operation restrictions such as operation according to the shift position or the vehicle speed may be provided. Or it is good also as a structure which detects the operation | movement performed as a preliminary | backup operation | movement which changes the direction of a vehicle, and starts a system. Further, the system may be activated when the image display device 1B is detached from the holding position (connection portion 62A), that is, when the connection between the connection portion 62A and the connection portion 62B is disconnected.

As initialization processing of the image capturing device 1A, the camera is initialized such as turning on the camera power and setting the brightness / contrast and preparing for image capturing. Further, parameters necessary for image processing are set by a method of reading a default value stored in the memory 5A or prompting user input. Further, as an initialization process of the image display device 1B, sensors used for position / angle detection are initialized, and a starting point that is a measurement reference point is set to a holding position.

Communication line connection processing is performed between the image capturing device 1A and the image display device 1B (steps S103 and S203). When the line is established, the image capturing device 1A starts image capturing from each camera (step S105). The captured image data is stored in buffers 51F, 51R, 51L, 51B secured in the memory 5A as needed. Further, in the image display device 1B, the position / angle detection device 7 acquires information on the position / angle of the image display device 1B (step S205). The acquired values are values relating to the elevation angle or depression angle of the display screen, the screen orientation (horizontal rotation angle with respect to the vehicle), the movement distance, and the like. The acquired data is transmitted to the image capturing device 1A via the communication device 61B (step S207). On the image capturing device 1A side, sensor data transmitted from the image display device 1B is received via the communication device 61A (step S107). The received sensor data includes data on the display screen angle (elevation angle or depression angle) detected by the gyro sensor and the screen rotation angle (horizontal rotation angle with respect to the vehicle), and the direction of the display screen by the geomagnetic sensor ( There are data relating to the direction of the line L1 obtained by projecting a straight line perpendicular to the display screen into the horizontal plane.

Image extraction processing is performed based on the supplied data (step S109). This image extraction process will be described based on a subroutine shown in FIG. Based on the position / angle detection device 7, the three-dimensional position of the image display device 1B is detected by three-point surveying (step S301). As shown in FIG. 5, the three-dimensional position of the image display device 1B is received in step S107 with the holding position 101 of the image display device 1B as the starting point (X, Y, Z = 0, 0, 0). The position is specified by calculating the relative position using position information from the position / angle detection device 7 of the image display device 1B or three-point surveying (X, Y, Z = x1, y1, z1). . This step S3
01 constitutes a position detecting means. The starting point does not have to be the holding position, and may be any of the four corners of the interior space, the center of the interior space, the position of the steering wheel, the viewpoint position of the driver, the position of the rearview mirror, and the like.

Further, the attitude of the image display device 1B is specified based on values from the gyro sensor, acceleration sensor, and geomagnetic sensor provided in the image display device 1B (step S).
302). The posture is the direction in which the display screen of the image display device 1B is facing, and the elevation angle or depression angle (tilt) and rotation angle of the display screen are specified by specifying the posture.

A viewpoint position is specified from the position and orientation determined in steps S301 and S302 (step S303). In this embodiment, the viewpoint position is predetermined for each seat, and coordinates as the viewpoint position are stored in the image processing information memory 53. For example, the viewpoint 111 is stored in advance in the driver's seat 111 as coordinates (xe1, ye1, ze1). In the passenger seat 112, the viewpoint position E2 has coordinates (x
e2, ye2, ze2), the viewpoint positions E3, E4, E5 are determined in advance for the other three-seat rear seat 113, and the viewpoint positions E6, E7, E8 are determined in advance for the seat 114, respectively. Coordinates are stored. An area corresponding to each viewpoint position is determined in advance, and it is determined in which area the image display device 1B is included, and the viewpoint position corresponding to the area is determined.

This area is set for each seat and is an area ahead of the backrest in the first row of seats, corresponding to the driver's seat and front passenger seat. In this area, the viewpoint positions E1 and E2 are set. Each corresponds.
It is an area provided between the backrest of the seat 113 in the second row and the backrest of the seats 111 and 112 in the first row, and is provided in three areas provided according to the position where the person who gets on the seat sits. Respectively correspond to the viewpoint positions E3, E4, and E5.
There are three areas provided between the backrest of the seat 114 in the third row and the backrest of the seat 113 in the second row, which are provided in accordance with the position where the person on board is sitting, The viewpoint positions E6, E7, and E8 correspond to each other.

When the image display device 1B is located at the boundary of each region, the viewpoint position on the display side of the display 8 as the display device is selected. Step S303 having such an action constitutes the viewpoint position specifying means. Thus, when the position and orientation of the image display device 1B are specified, the viewpoint position is determined. In this embodiment, it becomes the viewpoint position E1 of the driver of the driver's seat.

  As described above, as a method for specifying the viewpoint position, the coordinates of the viewpoint position corresponding to the seat position are determined in advance and stored, and the display screen 8 of the image display device 1B whose position has been specified is stored. A point located on a vertical line (axis line) passing through the center of the display and located at a predetermined distance (for example, a predetermined distance within a range of about 35 to 50 cm) from the display screen 8 is assumed as a viewpoint position, You may take the method of calculating the coordinate of the position. As another method, the viewpoint position closest to the image display device 1B on the side where the display screen 8 is provided may be selected from the eight viewpoint positions.

  As described above, when the viewpoint position E1 is specified, a straight line connecting the viewpoint position E1 and the center of the display is specified, and the direction from the viewpoint position E1 toward the display 8 on the extended line of the straight line is the image. Is specified as the direction to acquire (step S304). This step S304 constitutes a direction detecting means.

Next, an angle α (rotation angle) formed by the straight line L1 when a straight line connecting the viewpoint position E1 and the display 8 is projected onto a plane and the vehicle axis (Y axis) is calculated (step S).
305). Steps S302 and S305 constitute the angle detection means of the present invention.

Next, based on the angle α, a camera that captures an image from which an extracted image is extracted is selected (step S307). This step S307 constitutes selection means.
As shown in FIG. 2, with respect to the angle α, centering on the intersection of the optical axes of the camera lenses, the front is set to 0 °, the rear is ± 180 °, the left is set to a negative value, and the right is set to a positive value. . When -25 ° <α <25 °, the front camera 3F, when -160 ° <α <-25 °, the left camera 3L, when 25 ° <α <160 °, When the right side camera 3R is in the case of -160 ° <α <-180 ° or 160 ° <α <180 °, the rear camera 3B is selected. In this manner, the image area to be cut out, the images acquired by the cameras 3R and 3L on the side of the vehicle are wider, and the images acquired by the cameras 3F and 3B before and after the vehicle are narrower. This is because the image of the side camera is useful to catch a bicycle or motorcycle running in parallel with the vehicle. In other words, the cameras 3R and 3L on the side are used in principle, and in a region that cannot be captured by these cameras, that is, in a region such as right in front or right behind, the camera F provided in the front and the camera B provided in the rear are Use.

  In addition, as another embodiment of the selection means configured in step S307, the optical axis (vector) of the camera lens directed in the imaging direction and an extension line (vector) from the identified viewpoint position toward the image display device 1B It is also possible to select a camera that has the smallest intersection angle with. Since the camera whose optical axis and line of sight are closest to each other is selected, distortion of the image subjected to viewpoint conversion is suppressed, and a clearer image can be obtained. Such a selection method is particularly useful in the case where a large number of cameras that do not have fisheye lenses are provided and these camera images are connected to form an image.

  Furthermore, as another method, when the straight line L1 intersects the contour line of the side surface of the vehicle body, the cameras 3R and 3L provided on the side surface are selected, and when the straight line L1 intersects the contour of the front surface of the vehicle body, When the camera 3F is selected and intersects with the contour of the rear surface of the vehicle body, the rear camera 3B may be selected. Moreover, it is good also as a structure which selects the camera of the direction which the display is facing with respect to a vehicle instead of the angle (alpha) detected by step S305.

Next, the extraction position of the image to be extracted is determined within the fisheye image of the selected camera (step S309). As shown in FIG. 7, the extracted image Pc2 is determined from the selected camera image (fisheye image Pc1). Specifically, the angle α used for camera selection is converted into horizontal coordinates of the fisheye image Pc1. The vertical coordinate of the fisheye image is made to correspond to the elevation angle or depression angle of the image display device. For example, if α = −70 ° and the depression angle data of the sensor is −20 °, the position in the fisheye image is 20 ° in the horizontal direction (= 90 ° −70 °) and −20 ° in the vertical direction. (However, when the center of the fisheye image is 0 °, the right and up directions from the center are + values, and the left and down directions are -values.)
The center point P0 of the image extracted in this way is determined (step S309). Then, an extracted image Pc2 centered on this center point P0 is extracted (step S311). These steps S301 to S311 constitute an image extraction means.

  Next, image processing of the extracted image is performed (step S111). FIG. 8 is a subroutine showing the contents of the image processing in step S111. Distortion is corrected by performing normal image conversion processing on the extracted image Pc2 (step S401). Next, viewpoint conversion processing (step S403) is performed, and image composition processing (step S405) for compositing vehicle body data is performed.

The viewpoint position determined in step S303 (the viewpoint position E1 of the driving vehicle in FIG. 5) is set as the reference point. Based on this viewpoint position, the image processed in step S401 is subjected to image processing so as to become a viewpoint image from the viewpoint position determined from the camera viewpoint image by affine transformation (step S403). Thereby, the acquired image is converted into an image when viewed from the specified viewpoint position. This step S403 constitutes viewpoint conversion means.

  After performing the viewpoint conversion process as described above, the image composition process is performed (step S405). FIG. 9 is a flowchart showing the contents of the image composition process. Car body contour data, tire installation position data, and the like stored in the car body data memory 54 are read (step S501). These data are data necessary for imaging the vehicle body contour, tires, and the like to be displayed on the screen. Using these data, based on the image extraction direction data obtained in steps S301 to S309, a vehicle body contour image in the same line-of-sight direction as the image extracted in step S311 is generated (step S503).

  Specifically, the vehicle body outline image includes, for example, a wire frame image showing the outline of the vehicle body, or a translucent image showing the inside of the vehicle body. These images do not have to represent the entire vehicle body, and may be a part of the vehicle body. It is sufficient that the positional relationship between the vehicle body and the external situation can be grasped. For example, one or more wire frames (or a translucent image) such as a window frame, a wheel, a tail lamp, and a head lamp may be used.

  A viewpoint conversion process is performed on the generated vehicle body contour image based on the viewpoint position determined in step S403 (step S505). The conversion method is the same as the conversion method performed in step S403. Steps S501 to S505 constitute the contour forming means.

  Further, the contour forming unit may store the image of the wire frame for each user (for each seat), and in that case, the contour forming unit corresponds to the user (viewpoint position) specified in step S303. Read the wire frame and synthesize it. By doing so, it is possible to reduce the image processing load.

  The vehicle body contour image obtained in step S505 is combined with the vehicle exterior image in which the viewpoint conversion processing (step S403) is completed (step S507). The image synthesized in this way is an image in which the outline of the vehicle body can be grasped and the outside situation that can be seen through the vehicle body can be grasped simultaneously. Step S507 constitutes an image composition unit.

  Steps S401 to S405 in the image processing are not necessarily performed in the above order, and the order of processing is not particularly limited as long as these three processes are performed. For example, after performing viewpoint conversion and image synthesis, normal image conversion may be performed, or viewpoint conversion may be performed first or last.

The drawing data generated as described above is transmitted to the image display device 1B via the communication device 61A (step S113). The image display device 1B receives the drawing data transmitted from the image capturing device 1A via the communication device 61B (step S).
209). The received drawing data is displayed as an image on the display screen 8 (step S211). This step S211 constitutes display means. As shown in FIG. 11, the image 82 displayed on the display screen 8 has a contour Bd of the vehicle body on the image outside the vehicle on the extension line of the line of sight for the user who is viewing the display screen 8. It is displayed overlaid. That is, it is possible to grasp the situation outside the vehicle that is a blind spot by the vehicle body, and to easily grasp the positional relationship with the vehicle body.

Thereafter, both the image capture device 1A and the image display device 1B determine whether an end trigger has been detected (steps S115 and S213). The end trigger includes switch OFF operation, ignition switch OFF, shift position and vehicle speed deviating from predetermined operating conditions, or setting to the holding position (connection portion 62A) of the image display device 1B, etc. There is. When these operations are performed, it is assumed that communication between the image capturing device 1A and the image display device 1B is no longer necessary, and the communication line between the communication device 61A and the communication device 61B is cut off (steps S117 and S215). .
When the end trigger is not detected, the image capturing device 1A performs step S1.
07, the image display device 1B returns to step S207, respectively.
The image display operation is continued.

  As described above, the image display device 1B is configured to be detachable from the holding position. And in the state fixed to the holding position, while a clearer image can be displayed on an image display surface, only the image display apparatus 1B can also be used as a display screen of another apparatus.

It is a block diagram which shows the structure of an image capture device. It is a top view of a vehicle which shows a camera attachment position. It is a block diagram which shows the structure of an image display apparatus. It is a flowchart. It is a schematic diagram which shows the seat arrangement | positioning in a vehicle. It is a flowchart which shows the content of an image extraction process. It is an image figure of a fisheye lens. It is a flowchart which shows the content of image processing. It is a flowchart which shows the content of an image composition process. It is a display image figure.

Explanation of symbols

1A image capture device 1B image display device 3F, 3R, 3L, 3B camera 8 display device (display)

Claims (9)

Imaging means for imaging the surroundings of the vehicle;
A display having an image display surface;
Position detecting means for detecting a positional relationship between a predetermined starting point and the display;
Image extracting means for extracting an image around the vehicle existing on an extension line of a line from the reference point toward the display direction based on the positional relationship detected by the position detecting means from the image picked up by the image pickup means;
An outside information display device comprising: display means for displaying an image extracted based on the image extraction means on the display. The vehicle exterior information display device according to claim 1, wherein the reference point is a position virtually assuming a driver's viewpoint position. The origin is outside the information display apparatus according to claim 1 or 2, characterized in that the holding position of the display. Imaging means for imaging the surroundings of the vehicle;
A display having an image display surface;
Position detecting means for detecting a positional relationship of the display with respect to a predetermined starting point;
Viewpoint position specifying means for specifying the viewpoint position of the user using the display based on the positional relationship detected by the position detecting means;
Direction detecting means for detecting a direction from the viewpoint toward the display from the viewpoint specified by the viewpoint position specifying means and the position of the display;
Image extracting means for extracting an image around the vehicle corresponding to the direction detected by the direction detecting means from the image taken by the imaging means;
A vehicle outside information display device comprising: display means for displaying an image extracted based on the image extraction means on the display. The viewpoint conversion unit according to claim 2, further comprising: a viewpoint conversion unit that performs a viewpoint conversion process on the image extracted by the image extraction unit to an image in the case where the direction in which the display is located is visually recognized from the viewpoint position. Outside-vehicle information display device. Angle detection means for detecting the tilt and rotation angle of the display surface in the vehicle,
5. The vehicle exterior information display apparatus according to claim 1, wherein the image extraction unit further extracts the image based on the angle detected by the angle detection unit. Vehicle information storage means for storing the contour of the vehicle;
Contour forming means for forming a vehicle contour when viewed from a viewpoint position based on vehicle contour information stored in the vehicle information storage means;
The vehicle exterior information display device according to any one of claims 1 to 6, further comprising an image synthesis unit that synthesizes an outline of the vehicle formed by the contour formation unit with an image displayed on the display unit. The plurality of imaging means are provided in different directions, and the imaging means in the direction where the angle formed by the line from the viewpoint position to the display position and the optical axis of the imaging means is the smallest is the plurality of imaging means. A selection means for selecting one of the means;
The vehicle exterior information display device according to claim 1, wherein the image extraction unit extracts an image from an image acquired by the selected imaging unit. Imaging means for imaging the surroundings of the vehicle;
A display having an image display surface;
Angle detection means for detecting the tilt and rotation angle of the image display surface;
An image extracting means for extracting an image facing the back surface of the image display surface from the image captured by the image capturing means based on the inclination and rotation angle of the image display surface detected by the angle detecting means;
An outside information display device comprising: display means for displaying an image extracted based on the image extraction means on the display means.

Images