WO2017168953A1

WO2017168953A1 - Vehicle device, vehicle program, and filter design program

Info

Publication number: WO2017168953A1
Application number: PCT/JP2017/001217
Authority: WO
Inventors: 鈴木　孝光; 真之近藤; 祐司楠瀬
Original assignee: 株式会社デンソー
Priority date: 2016-04-01
Filing date: 2017-01-16
Publication date: 2017-10-05

Abstract

An ECU 2 serving as a vehicle device is provided with: an image generation unit 10a that generates in a predetermined reference coordinate system; a viewpoint position identification unit 10b that identifies the viewpoint position of the driver; a superimposed position identification unit 10c that identifies a display position for displaying an image (G1) in the driver's field of view; an image conversion unit 10d that generates a converted image (G2), which results from converting the image (G1) to a coordinate system based on the driver's viewpoint position; and a notification unit 10e that uses a projection device 7 to display a display image (G11), which fits into the converted image, so as to be overlaid in the driver's field of view.

Description

Vehicle device, vehicle program, filter design program

Cross-reference of related applications

This application is based on Japanese Application No. 2016-074383 filed on April 1, 2016, and Japanese Application No. 2016-218084 filed on November 8, 2016, the contents of which are described herein. Is used.

The present disclosure relates to a vehicle device, a vehicle program, and a filter design program.

There is a projection device that displays an image on a projection surface such as a windshield so that the image is superimposed on a landscape viewed by a driver. And in recent years, by using this projection device, for example, the direction of the route at a complicated intersection is displayed as an arrow in accordance with the road on which the vehicle actually travels, or the driver is alerted by framing the road sign. By doing so, a vehicle device that supports driving has also come out.

Now, the position of the driver's face is not necessarily a fixed position. For this reason, depending on the position of the driver's face, the superimposed image may be displayed deviated from the actual landscape, and the advantages of the projection device that superimposes and displays the image on the actual landscape may not be utilized. Moreover, the driver may be misunderstood by displaying an image at a shifted position. For this reason, for example, Patent Document 1 proposes that the image display method is devised to reduce the shift of the image depending on the viewpoint position of the driver.

JP 2005-69800 A

However, as a result of detailed examination by the inventor, the size of the projection surface is limited, and the position of the driver's eyes when the position of the driver's face is shifted up and down, front and rear, left and right, etc. Therefore, there has been a problem that the fundamental display position misalignment cannot be solved because the positional relationship between the driver's eye position and the projection plane is misaligned.

It is an object of the present disclosure to provide a vehicle device, a vehicle program, and a filter design program capable of accurately displaying an image projected from a projection device so as to overlap with an actual scenery viewed by a driver.

An apparatus for a vehicle according to an aspect of the present disclosure includes an image generation unit that generates an image to be notified to a driver using a predetermined reference coordinate system, and a driving detected by a viewpoint detection unit that detects the position of the driver's eyes. Based on the position of the driver's eyes, a viewpoint position specifying unit that specifies the viewpoint position indicating the position of the driver's eyes in the passenger compartment, and a superposition that specifies the display position when displaying the generated image in the driver's field of view A position specifying unit, an image converting unit that generates a converted image that is an image obtained by converting the generated image into a coordinate system based on the viewpoint position of the driver, and a projection device that overlaps the converted image with the driver's field of view Thus, the display unit is configured to include a notification unit that notifies the driver of information.

The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description with reference to the accompanying drawings. The drawing
FIG. 1 is a diagram schematically illustrating an electrical configuration of the ECU according to the first embodiment. FIG. 2 is a diagram schematically showing an example of the relationship between the viewpoint position of the camera and the viewpoint position of the driver. FIG. 3 is a diagram schematically illustrating an example of the relationship between the position of the driver's face and the visual field, FIG. 4 is a diagram showing a flow of processing of the vehicle program by the ECU. FIG. 5 is a diagram schematically showing an example of a mode in which images are superimposed and displayed. FIG. 6 is a diagram schematically showing an example of a mode in which images are superimposed and displayed. FIG. 7 is a diagram illustrating a flow of coordinate conversion filter generation processing by the ECU according to another embodiment. FIG. 8 is a first diagram illustrating a flow of a vehicle program by the ECU according to another embodiment. FIG. 9 is a second diagram illustrating the flow of the vehicle program by the ECU according to another embodiment.

Hereinafter, embodiments will be described with reference to FIGS. 1 to 6.
As shown in FIG. 1, a vehicle system 1 includes an ECU 2 (Electronic Control Unit) as a vehicle device, a camera 3 as an imaging unit, a millimeter wave radar 4, sensors 5, a viewpoint detection unit 6, a projection device 7, The speaker 8 and the microphone 9 are configured.

ECU2 is provided in the vehicle 20 (refer FIG. 2). The ECU 2 may be fixedly provided on the vehicle 20 or may be provided detachably from the vehicle 20. The ECU 2 includes a control unit 10, a storage unit 11, an operation switch 12, and the like. The control unit 10 is configured by a microcomputer including a CPU, a ROM, a RAM, and the like (not shown). For example, the control unit 10 controls the ECU 2 by executing a control program stored in the storage unit 11.

The storage unit 11 is composed of a recording medium that can read and write data, and stores the control program described above, a vehicle program described later, a program for image processing, and various data. That is, the storage unit 11 functions as a recording medium that stores the vehicle program. The storage unit 11 functions as a recording medium that stores a filter design program described later. However, the recording medium is not limited to the storage unit 11, and a recording medium that can be attached and detached from the ECU 2 can also be used.

The storage unit 11 also stores data such as the mounting position of the camera 3, the three-dimensional shape of the projection surface (C, see FIG. 2), and the driver's texture. The operation switch 12 inputs various user operations on the ECU 2.

The control unit 10 includes an image generation unit 10a, a viewpoint position specification unit 10b, a superimposed position specification unit 10c, an image conversion unit 10d, a notification unit 10e, a prediction unit 10f, and the like. In the present embodiment, each of these units (10a to 10e) is realized by software by causing the control unit 10 to execute a program. However, each unit (10a to 10e) can be provided by hardware, or can be provided by a combination of hardware and software.

The image generation unit 10a generates an image (G1, see FIG. 2) to notify the driver in a coordinate system based on the viewpoint position of the camera 3 that captures the traveling direction of the vehicle 20. That is, in the present embodiment, the reference coordinate system is set as a coordinate system based on the mounting position of the camera 3. The image generation unit 10 a generates an image as an image when displayed in the field of view of the camera 3 based on the attachment position of the camera 3.

Here, the image (G1) shown in FIG. 2 is actually projected onto the projection plane (C), but in FIG. 2, the driver seems to have an image at a certain distance. The image (G1) is shown at a position away from the vehicle 20 in order to schematically show that it is visible.

Further, FIG. 2 shows an example in which the image (G1) is a simple two-dimensional image for simplification of explanation, but of course, it is an image having a complicated shape that is three-dimensionally viewed from the driver. There may be. In that case, a three-dimensional stereoscopic image having a depth like a so-called 3D video is displayed to the driver without a sense of incongruity.

The viewpoint position specifying unit 10b specifies a viewpoint position indicating the position of the driver's eyes in the passenger compartment based on the position of the driver's eyes detected by the viewpoint detection unit 6. In other words, the viewpoint position specifying unit 10b specifies the positional relationship between the position of the driver's eyes and the projection plane (C).

Based on the viewpoint position of the camera 3 and the driver's viewpoint position, the superimposed position specifying unit 10c uses the display position when displaying the image on the projection plane (C), that is, the driver's viewpoint position as a reference. Specify the display position when displaying an image.

The image conversion unit 10d converts the image into a coordinate system based on the driver's viewpoint position based on the driver's viewpoint position, thereby converting the image (G2, see FIG. 2), that is, actually the projection plane (C ) Is generated for projection.

The alerting | reporting part 10e alert | reports a driver | operator by displaying the conversion image (G2) so that it may overlap with a driver | operator's visual field using the projection apparatus 7. FIG. At this time, the notification unit 103 displays an image in real time by following the change in the viewpoint position of the driver, as will be described later. Note that character information or the like may be displayed together.

The prediction unit (10f) predicts the driver's viewpoint position based on the movement history indicating the change of the driver's viewpoint position in time series.
The camera 3 is constituted by a CCD camera or a CMOS camera, and images a landscape in the traveling direction of the vehicle 20. The millimeter wave radar 4 radiates radio waves and detects the distance to the object based on the reflected wave reflected by the object. The sensors 5 are constituted by, for example, an infrared sensor or a proximity sensor, and detect surrounding objects.

The viewpoint detection unit 6 detects the position of the driver's eyes. Note that since various methods are known or known, the viewpoint detection unit 6 may use these methods. In that case, it is desirable to detect the eye position only by image processing without requiring special equipment from the driver.

The projection device 7 projects a virtual image on a transparent windshield 31 (see FIG. 2) or a combiner provided in the driver's field of view, and is also called a head-up display. It is. In the present embodiment, a part of the windshield 31 is a projection plane (C).

The speaker 8 notifies the response sound of the operation to the ECU 2 and a message to the driver by voice. It is good also as a structure which alert | reports with the speaker 8. FIG. The microphone 9 inputs an operation on the ECU 2 by voice, for example.

Further, the ECU 2 is connected to the position detection unit 13. The position detection unit 13 includes a GPS unit, a navigation device, and the like, and detects the current position of the host vehicle. Moreover, in order to implement | achieve a navigation function, the map data 13a is provided. The map data 13a may be stored in the storage unit 11.

Next, the operation of the above configuration will be described.
As described above, by using the projection device 7, various types of information are superimposed on an actual landscape, thereby providing a merit that a user may be aware of a danger that may not be noticed. If a position shift occurs between the landscape and the superimposed image, there is a possibility that the merit cannot be utilized or incorrect information is given.

Here, the difference between the viewpoint position of the camera 3 and the viewpoint position of the driver, and the necessity for obtaining the superimposed position will be described. As shown in FIG. 2, it is assumed that the camera 3 is attached to a position where an image of the front of the vehicle 20 can be captured in the vicinity of the ceiling inside the vehicle 20. The viewpoint position of the camera 3, that is, the center position of the visual field (A 1) of the camera 3 can be obtained from the mounting position of the camera 3.

The visual field (A2) of the driver (M) is specified by detecting the position of the eyes of the driver (M) by the viewpoint detection unit 6 provided in the passenger compartment.
When an actual landscape is captured by the camera 3 and an image (G1) to be superimposed on the landscape is generated by the image generation unit 10a, the image generation unit 10a displays the image (G1) as a field of view (A1) of the camera 3. ) To overlap the landscape. At this time, the position of the upper end of the image (G1) in the visual field (A1) of the camera 3 is referred to as an upper end position (Lc0) for convenience.

For example, it is assumed that the image (G1) generated based on the viewpoint position of the camera 3 is positioned below the center line (Lc1) of the camera 3 by a distance (L1). In other words, it is assumed that the image (G1) is located below the center in the coordinate system of the visual field (A1) of the camera 3.

On the other hand, this image (G1) is positioned above the center line (Lc2) by a distance (L2) from the view (A2) of the driver (M). That is, the image (G1) is located above the center in the driver (M) coordinate system.

Therefore, when the image (G2) is displayed in the coordinate system with the viewpoint position of the camera 3 as a reference, the projection plane (C) is provided between the driver (M) and the actual landscape. (G2) is displayed below the center line (Lc2) from the viewpoint of the driver (M). As a result, the image (G2) does not overlap with the scenery that the driver (M) is viewing.

This is the same when the position of the face of the driver (M) is shifted left and right as shown in FIG. For this reason, the visual field (A20 corresponding to A2 described above) when the face position of the driver (M) is the central position, the visual field (A21) when the face position is shifted to the right, Since the view seen through the projection plane (C) is different from the visual field (A22) when the position is shifted to the left, the image (G1) may not appropriately overlap the landscape. Hereinafter, the central position is also referred to as a normal position for convenience. Further, for example, a relatively tall driver and a relatively short driver have different face positions in the first place, and even if the driver is the same, the driving posture changes appropriately during driving. In addition, the position of the face may differ due to reclining of the seat.

Therefore, the ECU 2 can superimpose and display the image projected from the projection device 7 on the actual scenery viewed by the driver as follows. More specifically, the ECU 2 generates, converts, and displays an image in real time so as to overlap the driver's field of view, following changes in the scenery and changes in the driver's viewpoint position due to the movement of the vehicle 20.

The ECU 2 executes each process shown in FIG. These processes are performed by executing the vehicle program in the control unit 10. Each process is performed by each of the above-described units (10a to 10e), but for the sake of simplification of description, the ECU 2 will be mainly described below.

ECU2 first acquires the three-dimensional information in the visual field (A1) of the camera 3 (S10). That is, the ECU 2 captures a landscape in front of the vehicle 20 with the camera 3 and detects or extracts an object included in the image. Then, the ECU 2 generates an image image (G1) to notify the driver in a coordinate system based on the viewpoint position of the camera 3 (S11).

Subsequently, the ECU 2 specifies the driver's viewpoint position (S12), and converts the generated image (G1) into a coordinate image (G2, see FIG. 2) converted into a coordinate system based on the driver's viewpoint position. Generate (S13). That is, the ECU 2 performs conversion processing for converting the generated image (G1) into the converted image (G2). At this time, the ECU 2 does not perform the conversion process on the entire display screen, but performs the conversion process on the image notified to the driver. Thereby, the processing load is reduced.

Here, for example, a so-called view conversion method may be used as a coordinate conversion method when converting to a coordinate system based on the viewpoint position of the driver. Since this view conversion is a technique generally used in the graphics field that handles three-dimensional images, an outline thereof will be described here.

View conversion is a method for representing a figure generated in a state viewed from a certain viewpoint position in a state viewed from another viewpoint position in a three-dimensional space. Specifically, in view transformation, a parallel transformation matrix is used to translate the position of the image in the reference coordinate system that generated the figure so that it matches the coordinate system based on the viewpoint position of the person viewing the figure. Image conversion is performed by a combination of conversion and coordinate conversion that rotates a translated image toward a viewpoint position using a rotation conversion matrix.

In step S13 described above, the ECU 2 uses the view transformation projection to generate an image (G1) generated in the reference coordinate system with the camera 3 as a reference, and the viewpoint of the driver who will actually see the image. The image is converted into a converted image (G2) converted into a coordinate system based on the position. At this time, if the image is a three-dimensional image, the conversion is performed in a state where the image looks three-dimensional with a depth.

Now, when projecting the converted image (G2), it is necessary to consider the shape of the projection plane (C). For example, if the converted image (G1) is a square and is projected onto the projection surface (C) as it is, the image may be distorted depending on the surface shape of the projection surface (C) to form a trapezoid or a more complex curve. This is because it may look like a figure. Further, the image may be distorted depending on the viewpoint position of the driver, that is, depending on the positional relationship between the position of the driver's face and the projection plane (C).

Therefore, the ECU 2 acquires the three-dimensional shape of the projection surface (C), that is, the surface shape (S14), and corrects the converted image (G1) according to the surface shape and the viewpoint position of the driver (S15). As a result, a corrected converted image (G2), that is, an image without distortion as viewed from the driver is generated.

Then, the ECU 2 generates a frame image (G10; refer to FIG. 5, which corresponds to the boundary image) indicating the display range of the projection plane (C) (S16), the frame image (G10), and the corrected converted image (G2). More precisely, the display image (G11) that fits in the converted image (G2) is transmitted to the projection device 7 (S17). The transmitted frame image (G10) and display image (G11) are projected onto the projection plane (C) by the projection device 7 as shown in FIG.

As a result, the display image (G11) which is not distorted in the driver's visual field and accurately overlaps the driver's visual field is displayed.
By the way, when the size of the projection surface (C) is limited, there is a case where information desired to be notified to the driver, that is, the entire display image (G11) cannot be displayed. For example, in the case of FIG. 5, the display image (G11) that prompts the user to change the course is displayed, but the portion indicated by the broken line located on the left outer side of the frame image (G10) in the display image (G11) is displayed. Will not be.

Therefore, the ECU 2 determines whether or not to end the display (S18). In addition, the case where a display is complete | finished means complete | finishing the alerting | reporting to a driver | operator. For example, when the display image is displayed on the traffic guide map, when the vehicle passes the traffic guide map, or when it is confirmed that the driver has recognized the notified information, the image is displayed further. Since there is no meaning, the display of the image is terminated so as not to obstruct the driver's visual field.

And ECU2 will complete | finish a process, when it determines with complete | finishing a display (S18: YES). On the other hand, if the ECU 2 determines not to end the display (S18: NO), the ECU 2 proceeds to step S12 and repeats the process after the identification of the driver's viewpoint position. At this time, when the driver's viewpoint position changes, the process proceeds to step S12 in order to display a display image (G11) following the change. Thereby, the display image (G11) according to the change of the driver's viewpoint position is notified.

Specifically, when the left side of the display image (G11) is cut off, the driver turns his face to the right as shown in FIG. 6 in order to know what the left side of the display image (G11) is. I will send it. That is, when the driver wants to know what the left side of the display image (G11) is like, the driver brings his face to the opposite direction, that is, the right side.

This is easy to understand assuming that the frame image (G10) is a window frame. If a building has a window and you want to see the left side that is not visible, the person will shift his face to the right side. That is, when the position of the driver's face is shifted, the image to be displayed is in the direction opposite to the moving direction of the driver's face.

Therefore, the ECU 2 displays a frame image (G10) in order to inform the driver intuitively which part of the face that is not displayed can be seen by moving the face. Note that this is the same when the position of the face moves in the vertical direction as well as in the horizontal direction.

Note that the frame image (G11) is not limited to the rectangular frame illustrated in FIG. 5, and may be an L-shaped or cross-shaped figure indicating four corners, or may be a broken line or a double line instead of a solid line. There may be. That is, any image may be used as long as it can notify the boundary of the projection plane (C), that is, the limit of the display range.

In this way, the ECU 2 displays the image projected from the projection device 7 on the actual landscape that the driver is accurately seeing.
According to the embodiment described above, the following effects can be obtained.

The ECU 2 generates an image in a coordinate system based on the viewpoint position of the camera 3 that captures the traveling direction of the vehicle 20, a viewpoint position specifying unit 10b that specifies the viewpoint position of the driver, and an image (G1). ) In the driver's field of view, a superimposed position specifying unit 10c that specifies a display position, and a converted image (G2) that is an image obtained by converting the image (G1) into a coordinate system based on the viewpoint position of the driver. And an informing unit 10e that displays a display image (G11) that fits in the converted image using the projection device 7 so as to overlap the visual field of the driver.

Thereby, the display image (G1) generated so as to overlap the landscape in the visual field of the camera 3 is converted into a converted image (G2) overlapping the landscape in the visual field of the driver. And the display image (G11) which fits in a conversion image is displayed so that it may overlap with a driver | operator's visual field. Therefore, the image projected from the projection device 7 can be displayed in a superimposed manner on the actual landscape that the driver sees accurately.

That is, even when the scenery changes due to the movement of the vehicle 20 or the viewpoint position changes due to the driver's comfort, an image is generated, converted, and displayed in real time following the change. . As a result, the occurrence of displacement of the display position is suppressed, and the image with the displacement of the display position is not continuously displayed, and the image is displayed in a state where it is accurately superimposed on the actual scenery that the driver is looking at. can do. Therefore, the driver can be prevented from feeling uncomfortable.

In addition, by generating an image that can be superimposed on an actually viewed image even if the viewpoint is not at a fixed position, it is possible to stably provide the merit of superimposed display on an actual landscape.
At this time, when converting from the viewpoint position of the camera 3 to the viewpoint position of the driver, the conversion process is performed only on the target graphic to be superimposed, not on the entire display surface to be finally displayed. As a result, the processing load can be reduced.

ECU2 correct | amends the conversion image (G2) according to the shape of a projection surface (C), and displays the corrected conversion image (G2) so that it may overlap with a driver | operator's visual field. Thereby, even if the projection surface (C) is formed in a curved surface shape along the windshield 31, for example, an image without distortion can be displayed superimposed on the landscape when viewed from the driver.

ECU2 correct | amends conversion image (G2) according to a driver | operator's viewpoint position, and displays the corrected conversion image (G2) so that it may overlap with a driver | operator's visual field. Thereby, even if the position of the driver's face changes, an image without distortion when viewed from the driver can be displayed superimposed on the landscape.

ECU2 converts the image (G1) according to the movement of the driver's viewpoint position. As shown in FIG. 5 and FIG. 6 described above, when the face position is shifted, the appearance is also different. Therefore, the display image (G11) corresponding to the face position can be displayed by regenerating the image (G1). In this case, the generated image (G1) may be stored and reconverted according to the movement of the viewpoint position.

In this case, for example, when the display image is displayed on the traffic guide map, when the vehicle passes the traffic guide map, or when it is confirmed that the driver has recognized the notified information, the image is displayed. By ending, the possibility of obstructing the driver's field of view can be reduced.

ECU2 produces | generates the frame image (G10. Boundary image) which shows the display range of a projection surface (C). This makes it possible for the driver to intuitively understand that the face should be shifted to the right when, for example, the left side of the display image (G11) is cut off.

When the ECU 2 generates the image (G1) in a coordinate system based on the viewpoint position of the camera 3, the process for specifying the driver's viewpoint position, and the image (G1) is displayed in the driver's field of view. Processing for specifying a display position, processing for generating a converted image (G2) that is an image obtained by converting the image (G1) into a coordinate system based on the driver's viewpoint position, and a display image that fits in the converted image (G2) The image projected from the projection device 7 can be accurately superimposed and displayed on the actual scenery viewed by the driver, even with the vehicle program for executing the process of displaying (G11) so as to overlap the visual field of the driver. The same effects as the ECU 2 can be obtained.

(Other embodiments)
This indication is not limited to what was illustrated in above-mentioned embodiment, and can change arbitrarily or extend in the range which does not deviate from the gist.

In the embodiment, an example in which an image to be superimposed on an object in the field of view is generated and the image is converted is shown. However, an image to be superimposed on the entire object, that is, an image in which the inside is not filled, is shown. An image may be displayed. In this case, a contour image indicating the contour of the object may be generated in step S10 in FIG. 4, the contour image may be converted in step S13, and the converted image may be corrected in step S15. For example, when displaying an image over a traffic sign, it is easier to see the sign itself so that the image does not overlap, so by showing only the outline of the traffic sign, notification to the driver and visibility of the driver It is possible to achieve both improvement.

Further, when displaying an image whose interior is simply filled, in step S10, first, an outline image showing the outline of the image is generated, the outline image is converted in step S13, and the converted image is corrected in step S15. In step S17, the inside of the converted image may be filled and displayed. As a result, it is not necessary to perform conversion processing for the inside of the image, and it is only necessary to simply fill the inside when displaying, so that a reduction in processing load can be expected.

In the embodiment, the example in which the image (G1) is generated based on the viewpoint position of the driver is shown. However, in more detail, the image (G1) is displayed in FIG. 2 or FIG. You may generate | occur | produce from the driver | operator's mark as a starting point. Thereby, the coincidence ratio of the superimposition with the actually viewed image is increased, and an image without a sense of incongruity can be displayed. In this case, it is also possible to acquire a notch based on the break of the image that falls within the display range of the projection plane and the position of the face specified in step 12 of FIG. 4 with respect to the break.

In the embodiment, an example is shown in which an image is displayed based on the viewpoint position of the driver. However, a time-series change in the viewpoint position of the driver is stored as a movement history in the storage unit 11 or the like, and based on the movement history. The position of the driver's face may be predicted, and an image to be displayed may be corrected based on the prediction. That is, the image conversion unit 10d may perform image conversion and correction according to the shape of the projection plane (C) so as to correspond to the viewpoint position predicted by the prediction unit 10f.

Compared to the case where the change in the position of the face is detected and the image is converted or corrected in advance based on the prediction so that the change in the face position is detected and then the image is converted or corrected. , Processing delay can be reduced and image display delay can be improved.

In this case, based on not only the movement history of the driver's viewpoint position, but also the driver's face based on whether the portion of the display image (G11) that is not displayed is the top, bottom, left, or right of the projection plane (C). The movement of the position may be predicted. This is because when the image is out of the display range and is interrupted, the user is considered to move the position of the face to see the interrupted portion.

In the embodiment, the conversion image (G1) is corrected according to the surface shape of the projection surface (C) and the viewpoint position of the driver, but the surface shape of the projection surface (C) and the viewpoint position of the driver Depending on the positional relationship, the correction may be made according to either one of them.

In the embodiment, an example is shown in which the ECU 2 and the projection device 7 are provided separately, but they may be provided integrally. That is, the function of the ECU 2 may be mounted on the projection device 7. Alternatively, the function of the ECU 2 may be implemented in a drive recorder device or a navigation device that is already connectable to the camera 3, or a tablet terminal or a smartphone owned by the driver.

In the embodiment, the converted image is regenerated according to the movement of the viewpoint position of the driver. However, the converted image may be stored and the converted image may be recorrected according to the movement of the viewpoint position. Good.

In the embodiment, an example is shown in which only an image is displayed. However, characters and the like are also displayed without distortion by converting the shape in the same manner as in the conversion mode from the image (G1) to the display image (G10). Can do.

Although it is an image in the embodiment, the display image (G10) may be either a still image or a moving image.
With the configuration of the embodiment described above, it is possible to superimpose the landscape viewed from the position of the driver's eyes and the image projected on the projection plane (C), but the structure of the human eye, the method of recognizing the figure of the brain, etc. Because there are still unexplained factors, such factors may cause the image displayed on the projection plane (C) to feel shifted from the landscape or distorted. Conceivable.

In this case, by designing in advance a coordinate conversion filter in which a correction amount or the like is set so that a straight line can be correctly recognized as a straight line, for example, at a stage before the driver actually uses it, such as during a test of the ECU 2. Can be dealt with.

Specifically, the coordinate conversion filter is designed by executing the filter design program shown in FIG. In FIG. 7, there is a place where almost the same processing as the processing of FIG. 4 of the embodiment is performed, and thus detailed description thereof will be omitted.

ECU2 will acquire a three-dimensional information in Step S10 (S20), and will generate a standard picture (S21). This reference image is a relatively simple figure such as a lattice figure constituted by right angles. Subsequently, the ECU 2 specifies the viewpoint position of the tester sitting in the driver's seat (S2), and generates a converted image obtained by converting the reference image into a coordinate system based on the specified viewpoint position (S23).

When the converted image is generated, the ECU 2 acquires the shape of the projection plane (C) (S21), corrects the converted image according to the shape of the projection plane (C) and the viewpoint position (C25), and generates a frame image. Then, the corrected converted image and the frame image are transmitted to the projection device (S27), and the image is displayed on the projection plane (C).

The ECU 2 corrects distortion of the displayed image (S28). More precisely, in this step S28, the distortion of the image being displayed is corrected in accordance with an operation for correcting the tester so that it looks correct.

Subsequently, the ECU 2 acquires a correction amount when correcting the distortion of the image (S29), associates the correction amount as a correction amount for the current viewpoint position, and creates or updates a coordinate conversion filter (S30). . When the process shown in FIG. 7 is executed for the first time, a coordinate conversion filter is newly created in step S30, and when executed for the second time or later, the coordinate conversion filter is updated in step S30. . That is, a coordinate conversion filter that can specify a correction amount according to the viewpoint position is designed.

7 is repeated while changing the viewpoint position, the coordinate conversion filter in which the viewpoint position of the driver and the correction amount at the viewpoint position are associated with each other is updated.

This makes it possible to deal with the above-described unexplained factors and display an image that the driver feels easier to see. In this case, the coordinate conversion filter may be designed in advance, or may be generated after use is started by the driver.

This coordinate conversion filter is used during actual operation as follows.
The ECU 2 executes the vehicle program shown in FIG. In FIG. 8, the same steps as those in FIG. 4 of the embodiment are denoted by the same step numbers, and detailed description thereof is omitted.

The ECU 2 executes the processing from step S1 in the same manner as in the embodiment, and when the conversion image correction (S15) and the frame image generation (S16) are completed, the ECU 2 refers to the coordinate conversion filter described above, and finally the image. Is corrected (S40). In this step S40, based on the driver's viewpoint position, correction using the associated correction amount is performed. The process of step S40 is performed by the coordinate conversion unit 10d.

Subsequently, the ECU 2 transmits and displays the corrected image and the frame image (S17), and then determines whether the driver's line of sight is facing the displayed image (S41), and the line of sight is directed to the displayed image. If it is detected (S41: YES), the movement of the line of sight is detected (S42). At this time, the fluctuation of the line of sight is detected as a deviation amount between the center of the line of sight of the driver and the display position of the image. The line of sight can be specified by detecting the direction of the line of sight when the viewpoint position is specified by the viewpoint position specifying unit.

Then, when the detected shake is equal to or greater than the specified value (S43: YES), the ECU 2 reflects the detected shake on the coordinate conversion filter, assuming that the display position of the image is deviated from the driver's consciousness ( S44). Thus, the correction amount for the driver, that is, the coordinate conversion filter is learned. Note that if the line of sight is facing the image (S41: NO), or if the shake is less than the specified value (S43: NO), the process ends.

This makes it possible to determine that the display position does not match the driver's sensation if a gaze fluctuation is detected when the image is displayed. Then, by reflecting the fluctuation in the coordinate conversion filter, not only the general factors that cannot be clarified but also individual factors for each driver can be dealt with.

It should be noted that although it is naturally expected that the drivers will be different, it is naturally expected that the driver will not basically change. In the first place, the vehicle used when the coordinate conversion filter is designed in advance may be different from the vehicle in which the ECU 2 is actually provided. For this reason, the coordinate conversion filter may be newly created each time the vehicle is used to deal with different drivers, and the past driving state is saved to deal with the same driver. It is good also as a structure to keep. Of course, it is good also as a structure created newly, when there exists operation from a driver | operator.

In the embodiment, an example of generating an image based on a three-dimensional space specified based on an image captured by the camera 3 has been described. However, high-precision data that can specify a landscape or the like at the current position in the map data 13a. May be stored, and a three-dimensional space based on the absolute space specified from the map data 13a may be used as a reference. That is, the reference coordinate system may be set based on the map data 13a. In this case, as shown in FIG. 9, it can be realized by acquiring three-dimensional information based on the map data 13a in step S50 and performing the processing from step S12 onward as in the embodiment. Even with such a configuration, similarly to the embodiment, the image projected from the projection device 7 can be accurately superimposed and displayed on the actual scenery viewed by the driver.

Although the present disclosure has been described according to the embodiment, it is understood that the present disclosure is not limited to the embodiment or the structure. The present disclosure includes various modifications and modifications within the equivalent range. In addition, various combinations and forms, as well as other combinations and forms including only one element, more or less, are within the scope and spirit of the present disclosure.

Claims

A vehicle device (2) for notifying a driver of information using a projection device (7) for displaying an image on a projection surface (C),
An image generation unit (10a) that generates an image to be notified to the driver in a predetermined reference coordinate system;
A viewpoint position specifying unit (10b) for specifying a viewpoint position indicating the position of the driver's eyes in the passenger compartment based on the position of the driver's eyes detected by the viewpoint detection unit (6) that detects the position of the driver's eyes. )When,
A superimposed position specifying unit (10c) for specifying a display position when displaying the image generated by the image generating unit (10a) in the field of view of the driver based on the viewpoint position of the driver;
An image conversion unit (10d) that generates a converted image that is an image obtained by converting the image generated by the image generation unit (10a) into a coordinate system based on the driver's viewpoint position based on the viewpoint position of the driver. When,
A notification unit (10e) for notifying the driver of information by displaying the converted image so as to overlap the driver's field of view using the projection device (7);
A vehicle apparatus comprising:
The image conversion unit (10d) corrects the converted image according to the shape of the projection plane (C),
The vehicle device according to claim 1, wherein the notification unit (10 e) displays the corrected converted image so as to overlap a driver's visual field.
The image conversion unit (10d) corrects the converted image according to the viewpoint position of the driver,
The vehicle device according to claim 1, wherein the notification unit (10 e) displays the corrected converted image so as to overlap a driver's visual field.
The vehicle apparatus according to any one of claims 1 to 3, wherein the image conversion unit (10d) reconverts the image generated by the image generation unit (10a) in accordance with movement of a viewpoint position of a driver. .
The image generation unit (10a) generates a boundary image that is an image to be displayed on the projection plane (C) and indicates a display range of the projection plane (C),
The vehicle device according to any one of claims 1 to 4, wherein the notification unit (10e) displays the boundary image together with the converted image so as to overlap a driver's visual field.
The device for a vehicle according to any one of claims 1 to 5, wherein the image conversion unit (10d) converts the image generated by the image generation unit (10a) using a driver's perception as a starting point.
A prediction unit (10f) for predicting the driver's viewpoint position based on a movement history indicating changes in the driver's viewpoint position in time series;
The said image conversion part (10d) transform | converts the image produced | generated by the said image generation part (10a) so that it may correspond to the viewpoint position corresponding to the position of the estimated face. Vehicle equipment.
The said image generation part (10a) produces | generates the image notified to a driver | operator using the coordinate system on the basis of the three-dimensional space specified based on the attachment position of an imaging part as said reference | standard coordinate system. The apparatus for vehicles as described in any one of from 7 to 7.
The said image generation part (10a) produces | generates the image notified to a driver | operator using the coordinate system on the basis of the three-dimensional space specified based on map data as said reference | standard coordinate system. The vehicle device according to any one of the above.
The image conversion unit (10d) converts the image generated by the image generation unit (10a) using a coordinate conversion filter designed based on how an actual human sees the image from the driver's seat. The apparatus for vehicles as described in any one of Claim 1 to 9 which corrects the said conversion image.
In the control unit (10) of the vehicle device (1) for notifying the driver of information using the projection device (7) that displays an image on the projection surface (C),
Processing for generating an image to notify the driver in a predetermined reference coordinate system;
A process of identifying a viewpoint position indicating the position of the driver's eyes in the passenger compartment based on the position of the driver's eyes detected by the viewpoint detection unit (6) that detects the position of the driver's eyes;
Based on the driver's viewpoint position, a process for specifying the display position when displaying the image generated by the image generation unit (10a) in the driver's field of view;
A process for generating a converted image that is an image obtained by converting the image generated by the image generation unit (10a) into a coordinate system based on the viewpoint position of the driver based on the viewpoint position of the driver;
A process of notifying the driver of information by displaying the converted image so as to overlap the visual field of the driver using the projection device (7);
A vehicle program that executes
In the control unit (10) of the vehicle device (1) for notifying the driver of information using the projection device (7) that displays an image on the projection surface (C),
Processing for generating a reference image to be notified in a predetermined reference coordinate system;
A process of identifying a viewpoint position indicating the position of the eyes of the person located in the driver seat based on the position of the eyes detected by the viewpoint detection unit (6) that detects the position of the eyes of the person located in the driver seat;
Based on the identified viewpoint position, a process of identifying a display position when displaying the image generated by the image generation unit (10a) in the field of view;
A process of generating a converted image that is an image obtained by converting the image generated by the image generation unit (10a) into a coordinate system based on the viewpoint position of a person located in the driver's seat based on the identified viewpoint position;
A process of displaying the converted image so as to overlap the visual field of the driver using the projection device (7);
Processing to correct distortion of the displayed image in accordance with the operation of the person located in the driver's seat;
A process of acquiring a correction amount when correcting image distortion;
A process of designing a coordinate transformation filter that can identify the correction amount at the viewpoint position by associating the acquired correction amount and the viewpoint position with each other;
Filter design program to execute