JP2013217808A - On-vehicle apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an "on-vehicle apparatus" capable of providing a composite image for which guide information is superimposed on a proper vehicle front image even when a portable terminal is not set inside a compartment so that a camera unit is turned to a proper direction in front of a vehicle.SOLUTION: The on-vehicle apparatus includes an on-vehicle camera unit for photographing a prescribed direction in front of a vehicle. A difference between a photographing direction of the camera unit set so as to photograph the vehicle front by the camera unit of a portable terminal and a photographing direction of the on-vehicle camera unit is detected to generate photographing direction difference information indicating the difference. Guide information generated in the portable terminal is acquired, a position in the vehicle front image obtained by the photographing by the on-vehicle camera unit of the guide information is determined on the basis of the photographing direction difference information, the guide information is superimposed on the determined position of the vehicle front image obtained by the photographing by the on-vehicle camera unit to generate a composite vehicle front image, and a display part is made to display the composite vehicle front image.

Description

  The present invention relates to an in-vehicle device to which a mobile terminal such as a smartphone provided with a camera unit that captures the outside is connected.

  Conventionally, an information providing method has been proposed in which information such as a mark is added to an image captured by a camera (actual image) (Patent Document 1). Such an information providing method superimposes an arrow indicating a traveling direction of a vehicle or a guidance mark (guidance information) such as a facility mark indicating a facility on a vehicle front image obtained by an in-vehicle camera that captures the front of the vehicle. It can be applied to AR (Augmented Reality) navigation displayed.

  In recent years, an application for realizing the above-described AR navigation in a mobile terminal such as a smartphone has also been proposed. In this case, the mobile terminal includes a camera unit for photographing the outside, a unit for detecting the current position of the mobile terminal (for example, GPS), and a unit for detecting the attitude and orientation of the mobile terminal (for example, a gyroscope) , Compass, etc.) and a database storing map information. Then, the application (AR navigation) is executed in a state in which the mobile terminal is set in the vehicle interior so that the camera unit can capture the front of the vehicle. By executing this application, on the basis of map information (database), roads and facilities in the camera field of view in the direction in which the camera unit is facing at the current position (detected based on the detected attitude and orientation of the mobile terminal) Etc. (environment) is generated, and a composite image generated by superimposing the guidance mark on the image obtained by the camera unit is displayed on the display screen of the portable terminal. As a result, the vehicle driver grasps information about the actual environment ahead of the vehicle (vehicle traveling direction, contents of the facility, etc.) based on the guidance mark superimposed on the vehicle forward image displayed on the mobile terminal. Driving operation can be performed.

JP 2003-216882 A

  However, when using a portable terminal equipped with an application as described above, the portable terminal is set in the passenger compartment so that the front of the vehicle can be properly captured by the camera unit so as not to interfere with the driver's field of view. It is difficult. When the mobile terminal is not set in the passenger compartment so that it can be properly photographed by the camera unit, the environment in front of the vehicle (position of roads, facilities, etc.) that the driver can visually recognize through the front window and the vehicle front image displayed on the mobile terminal Deviation occurs in the environment that appears. In this case, it becomes difficult to grasp which road, facility, etc. in the actual environment in front of the vehicle the guidance mark superimposed on the vehicle front image displayed on the mobile terminal.

  The present invention has been made in view of such circumstances, and even if the mobile terminal is not set in the vehicle interior so that the camera unit faces in an appropriate direction in front of the vehicle, guidance information is displayed in an appropriate vehicle front image. It is intended to provide an in-vehicle device capable of providing a composite image in which images are superimposed.

  The in-vehicle device according to the present invention has a camera unit for photographing the outside, and generates guide information corresponding to the environment in the field of view in the photographing direction of the camera unit at the current position based on map information, An in-vehicle device to which a portable terminal equipped with an application for generating and displaying a composite image by superimposing guidance information on an image obtained by the camera unit is connected, the display unit and a predetermined direction in front of the vehicle The difference between the in-vehicle camera unit to be photographed and the photographing direction of the camera unit set so as to photograph the front of the vehicle by the camera unit of the portable terminal and the photographing direction of the in-vehicle camera unit is represented. Direction difference detection means for generating photographing direction difference information, guidance information acquisition means for acquiring the guidance information generated by the portable terminal, and the guidance information Guidance information display position determining means for determining a position in the vehicle front image obtained by photographing with the mounted camera unit based on the photographing direction difference information, and the guidance information display of the vehicle forward image obtained by photographing with the in-vehicle camera unit Image processing means for generating a composite vehicle forward image by superimposing the guidance information at the position determined by the position determination means, and display control means for displaying the composite vehicle forward image obtained by the image processing means on the display unit. It becomes the composition which has.

  According to such a configuration, when the composite image is generated by superimposing the guidance information on the image obtained by the camera unit by executing the application on the mobile terminal, the shooting direction of the camera unit of the mobile terminal and the in-vehicle camera The position of the guidance information in the vehicle front image obtained by photographing of the in-vehicle camera unit is determined based on the difference from the shooting direction of the vehicle, and the guidance information is superimposed and synthesized on the determined position of the vehicle front image. A vehicle front image is generated, and the composite vehicle front image is displayed on the display unit.

  In the in-vehicle device according to the present invention, the direction difference detecting means includes means for obtaining azimuth information representing the orientation of the mobile terminal from the mobile terminal, vehicle moving direction detecting means for detecting the moving direction of the vehicle, It can be set as the structure which has the means to calculate the said imaging | photography direction difference information based on the said direction information about a portable terminal, and the moving direction of the said vehicle.

  With such a configuration, the moving direction of the vehicle is detected, and based on the azimuth information indicating the direction of the portable terminal acquired from the portable terminal and the detected moving direction of the vehicle, the photographing of the camera unit of the portable terminal is performed. The photographing direction difference information representing the difference between the direction and the photographing direction of the in-vehicle camera is obtained.

  Further, in the in-vehicle device according to the present invention, the vehicle movement direction detection unit includes a mobile terminal position acquisition unit that acquires position information from the mobile terminal at a plurality of timings, and a plurality of timings about the mobile terminal. A moving direction calculating means for calculating the moving direction of the vehicle based on the position information can be used.

  With such a configuration, the moving direction of the vehicle is detected based on position information obtained at a plurality of timings of the mobile terminal that moves as the vehicle travels.

  Further, in the in-vehicle device according to the present invention, the guide information acquisition unit may be configured to combine a monochrome background composite image obtained by superimposing a single color image and the guide information on the mobile terminal instead of an image obtained by the camera unit. A means for requesting, a means for acquiring the monochrome background composite image sent from the portable terminal in response to the request, and a guide information extracting means for extracting the guide information from the monochrome background composite image It can be.

  With such a configuration, the guide information is extracted from a single color background composite image synthesized by superimposing the guide information on the single color image instead of the image obtained by the camera unit that captures the front of the mobile terminal.

  Furthermore, in the in-vehicle device according to the present invention, the guide information acquisition means includes a composite image acquisition means for acquiring a composite image generated by superimposing the guide information on an image obtained by photographing with the camera unit from the portable terminal. And a guide information extracting means for extracting the guide information from the acquired composite image.

  With such a configuration, the guide information is extracted from the combined image of the image obtained by photographing with the camera unit of the mobile terminal and the guide information.

  According to the vehicle-mounted device according to the present invention, when the composite image is generated by superimposing the guidance information on the image obtained by the camera unit by executing the application on the mobile terminal, the shooting direction of the camera unit of the mobile terminal and Based on the difference with the shooting direction of the in-vehicle camera unit, the position of the guidance information in the vehicle front image obtained by the shooting of the in-vehicle camera unit is determined, and the guidance information is in the determined position of the vehicle front image. A composite vehicle forward image is generated by overlapping, and the composite vehicle forward image is displayed on the display unit, so even if the mobile terminal is not set in the vehicle interior so that the camera unit faces the appropriate direction in front of the vehicle, It is possible to provide a composite image in which guidance information is superimposed on an appropriate vehicle front image obtained by photographing with the in-vehicle camera unit.

It is a block diagram which shows the structure of the vehicle equipment which concerns on one Embodiment of this invention. It is a figure which shows an example of the home screen displayed on the display part of a smart phone (mobile terminal). It is a figure which shows the example which set the smart phone in the vehicle interior so that a vehicle front may be image | photographed with a camera unit. It is a figure which shows an example of the image obtained by imaging | photography with the said camera unit of the smart phone set in the vehicle interior so that a vehicle front may be image | photographed with a camera unit. It is a figure which shows an example of the vehicle front image obtained by imaging | photography with the vehicle-mounted camera unit provided in the vehicle interior so that the vehicle front might be image | photographed. It is a figure which shows an example of the synthesized image which synthesize | combined the image obtained by imaging | photography with a camera unit, and a guidance mark displayed on the display part of a smart phone. It is a flowchart which shows the procedure of the process for detecting imaging | photography direction difference (DELTA) (theta) showing the difference of the imaging | photography direction of the camera unit of a smart phone, and the imaging | photography direction of a vehicle-mounted camera unit. It is a flowchart which shows the procedure of the process for the production | generation and display of the synthetic | combination vehicle front image which synthesize | combined the vehicle front image obtained by imaging | photography with a vehicle-mounted camera unit, and a guidance mark. It is a figure explaining the method of extraction of the guidance mark produced | generated with the smart phone. It is a figure which shows the difference (DELTA) (theta) of the imaging | photography direction of the camera unit of a smart phone, and the imaging | photography direction of a vehicle-mounted camera unit, and the relationship with a display screen. It is a figure which shows the position which overlaps in the vehicle front image obtained by imaging | photography with a vehicle-mounted camera unit of the guidance mark synthesize | combined with the image obtained by imaging | photography with the camera unit with a portable terminal. It is a figure which shows an example of the synthetic | combination vehicle front image obtained by a synthesis | combination with the vehicle-mounted front image obtained by imaging | photography with a vehicle-mounted camera unit, and a guide mark.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  An in-vehicle device according to an embodiment of the present invention is configured as shown in FIG.

  In FIG. 1, the in-vehicle device 100 includes a processing unit 11, an operation unit 12, a display unit 13, a storage unit 14, an output circuit 15, a speaker 16, and an in-vehicle camera unit 17. A smart phone 20 (mobile terminal) can be connected to the in-vehicle device 100 via a connector 18. With the smartphone 20 connected to the connector 18, the processing unit 11 and the smartphone 20 can communicate information (data) with each other. The processing unit 11 executes various processes according to operations on the operation unit 12 and causes the display unit 13 to display information obtained by the processes. An audio signal based on the message information from the processing unit 11 is supplied from the output circuit 15 to the speaker 16, and the message information is output from the speaker 16. Information used for processing in the processing unit 11 is stored in the storage unit 14, and information obtained by processing in the processing unit 11 is stored.

  The in-vehicle camera unit 17 is installed in a predetermined part of the front window (for example, a part on the back side of the room mirror) so that an environment in the forward movement direction of the vehicle can be photographed. As the in-vehicle camera unit 17, not only a dedicated camera unit, but also, for example, in order to obtain the inter-vehicle distance with the preceding vehicle, the white line indicating the boundary of the traveling lane, and other environmental information in front of the vehicle as vehicle driving support information. A camera unit for photographing can be used. The processing unit 11 generates image data representing the environment ahead of the vehicle from the image signal output from the in-vehicle camera unit 17 and executes image processing. The processing unit 11 can display a vehicle forward image on the display unit 13 based on image data obtained by photographing with the in-vehicle camera unit 17.

  Although not shown, the smartphone 20 (mobile terminal) is a camera unit that captures the outside, a unit (for example, GPS) for detecting the current position of the smartphone 20, and a posture and orientation of the smartphone 20. Unit (for example, gyroscope, compass, etc.) and a database storing map information. Further, the smartphone 20 is based on map information (database), and roads and facilities in the camera field of view of the camera unit at the current position (detected based on the detected posture and orientation of the smartphone 20). AR navigation application that generates a guidance mark corresponding to the environment (environment), overlays the guidance mark on the image obtained by the camera unit, generates a composite image, and displays the composite image on the display unit (Application software) is installed.

  In the smartphone 20, for example, as illustrated in FIG. 2, a home screen on which selection buttons (icons) of installed applications are arranged is displayed on the display unit 201. On this home screen, for example, an AR navigation application is activated by performing a selection operation of a selection button (AR Navi) corresponding to the AR navigation. For example, as shown in FIG. 3, the smartphone 20 is set on a dash panel in the vehicle interior so that the camera unit (not shown) faces the front of the vehicle and the display unit 201 faces the vehicle interior. . The smartphone 20 can be set at a position that does not interfere with the driver's forward view.

  FIG. 4 shows, for example, an image in front of the vehicle (hereinafter referred to as a terminal front-captured image) Isph that can be displayed on the display unit 201 by photographing with the camera unit of the smartphone 20. Further, when the terminal front photographed image Isph is obtained, an image in front of the vehicle (hereinafter referred to as a device front photographed image) Ic displayed on the display unit 13 by photographing with the in-vehicle camera unit 17 at the same timing is, for example, FIG. As shown. The difference between the appearance of the terminal front captured image Isph shown in FIG. 4 and the appearance of the device front captured image Ic shown in FIG. 5 is that the shooting direction of the camera unit of the smartphone 20 and the shooting direction of the in-vehicle camera unit 17 are different. This is due to the deviation.

  By executing the above-described AR navigation application on the smartphone 20, based on the map information, for example, as shown in FIG. 6, it corresponds to a road on which the vehicle in the camera field of view in the direction of the camera unit at the current position travels. An arrow M1 indicating the travel direction thus generated, a facility mark M2 corresponding to the bank A, and a facility mark M3 corresponding to another bank B are generated. Then, the composite image Isph / M is generated by superimposing these arrows M1 and facility marks M2, M3 (guidance marks / guidance information) on the terminal front photographed image Isph (same as shown in FIG. 4).

  The processing unit 11 performs information communication with the smartphone 20 that executes the application of the AR navigation, and implements the AR navigation function in the in-vehicle device 100 using the device front captured image Ic obtained by imaging with the in-vehicle camera unit 17. In order to do so, processing is performed according to the procedure shown in FIGS. The processing of the procedure shown in FIG. 7 (direction difference detection means) obtains a shooting direction difference Δθ that is a deviation (difference) between the shooting direction of the camera unit of the smartphone 20 and the shooting direction of the in-vehicle camera unit 17. The processing of the procedure shown in FIG. 8 is performed by superimposing guide marks (M1, M2, M3) on the device front photographed image Ic obtained by photographing with the in-vehicle camera unit 17 using the photographing direction difference Δθ. M is generated and displayed on the display unit 13 (AR navigation function).

  In FIG. 7, when the processing unit 11 confirms that information can be mutually communicated with the smartphone 20 connected to the connector 18 (S11), the AR navigation application (AR navigation application) is activated for the smartphone 20. A request is made (S12), and a response from the smartphone 20 to the activation request is waited (S13). In response to the activation request, the smartphone 20 activates the AR navigation application and returns a response signal to the in-vehicle device 100. When the processing unit 11 receives the response signal from the smartphone 20 after making the AR navigation application activation request (YES in S13), the processing unit 11 is obtained by executing the AR navigation application through information communication with the smartphone 20. Direction information (S14) indicating the direction in which the smartphone 20 faces and position information (latitude, longitude) indicating the current position of the smartphone 20 are acquired as first position information (S15). Thereafter, a waiting state for a predetermined time is entered (S16).

  When the waiting state for a predetermined time ends (YES in S16), the processing unit 11 acquires position information (latitude, longitude) representing the current position of the smartphone 20 as second position information through information communication with the smartphone 20 ( S17). Due to the traveling of the vehicle, the first position information acquired before and the second position information acquired after the predetermined time are different, and the processing unit 11 determines the difference between the first position information and the second position information. The moving direction of the vehicle is calculated from (S18). Then, the processing unit 11 determines the difference between the direction in which the smartphone 20 is represented and the moving direction of the vehicle represented by the acquired azimuth information (see S14) as the shooting direction of the camera unit in the smartphone 20 (the direction in which the smartphone 20 is directed). The shooting direction difference Δθ, which is a difference between the shooting direction of the vehicle-mounted camera unit (corresponding to the moving direction of the vehicle) and the vehicle-mounted camera unit, is calculated (S19).

  When the photographing direction difference Δθ is obtained in this way, the processing unit 11 executes image processing for AR navigation according to the procedure shown in FIG.

In FIG. 8, the processing unit 11 replaces an image obtained by the camera unit with respect to the smartphone 20 that executes the AR navigation application by information communication with the smartphone 20, and _displays a guide mark (M1, M2, M3) is overlaid and a request for a monochrome background composite image is made (S21). The smartphone 20 that has received this request, for example, stops sending image signals from the camera unit, thereby replacing the terminal front image Isph as shown in FIG. 4 with a monochrome (for example, black) monochrome background image I _BG. Is generated. And the smart phone 20 performs the process which synthesize _{| combines} with this monochromatic background image _IBG and the guidance mark (M1, M2, M3) mentioned above, for example, as shown in FIG. 9, monochromatic background image _IBG (for example, A single-color background composite image I _BG / M for one screen in which the guide marks (M1, M2, M3) are superimposed on the black image) is generated. The smartphone 20 sends the image data representing the monochrome background composite image I _BG / M to the in-vehicle device 100 together with the response to the request.

After making the request for the monochrome background composite image, when the processing unit 11 that enters the waiting state (S22) confirms the response from the smartphone 20 (YES in S22), the monochrome background composition that has been sent together with the response. Image data of the image I _BG / M is acquired (S23). Thereafter, the processing unit 11 identifies, for example, the guide marks (M1, M2) for the acquired image data while identifying the background (black) portion and other portions of the monochrome background composite image I _BG / M. , M3) is cut out (extracted) from the background portion (S24). Thereby, image data representing the guide marks (M1, M2, M3) distributed and arranged in one screen is obtained (S21 to S24: guide information acquisition means).

  The processing unit 11 has a shooting direction different from the shooting direction of the camera unit of the smartphone 20 of the guide marks (M1, M2, M3) based on the shooting direction difference Δθ obtained as described above (see FIG. 7). For example, a position in an apparatus vehicle front photographed image Ic as shown in FIG. 5 obtained by photographing by the in-vehicle camera unit 17 is calculated (S25: guidance information display position determining means). The positions of the guide marks (M1, M2, M3) on the equipment vehicle front captured image Ic are calculated as follows.

  As shown in FIG. 10, the camera field of view Vsph by the camera unit of the smartphone 20 and the camera field of view Vc by the in-vehicle camera unit 17 are shifted according to the photographing direction difference Δθ. From this, the shooting direction difference Δθ, the camera unit of the smartphone 20 and the in-vehicle camera are determined from the position in the screen (corresponding to Vsph) of the mark M to be overlapped corresponding to the environment of the camera visual field Vsph by the camera unit of the smartphone 20. Based on the optical conditions of the unit 17 and the like, the position of the mark M in the screen (corresponding to Vc) to be superimposed corresponding to the environment of the camera visual field Vc by the in-vehicle camera unit 17 is calculated. In the example shown in FIG. 10, the mark M is located on the relatively right side in the screen of the terminal vehicle forward captured image Isph (corresponding to Vsph) captured by the camera unit of the smartphone 20, whereas the imaging direction difference Δθ. As a result, it is positioned relatively on the left side in the screen of the device vehicle front-captured image Ic (corresponding to Vc) obtained by photographing with the in-vehicle camera unit 17.

  As a result of the processing as described above, for example, as shown in FIG. 11, each guide mark M <b> 1 that is overlaid in correspondence with the environment (road, facility) in the terminal vehicle forward captured image Isph captured by the camera unit of the smartphone 20, The positions of M2 and M3 (indicated by broken lines in FIG. 11) in the screen of the device vehicle front captured image Ic captured by the in-vehicle camera unit 17 are determined as indicated by solid lines in FIG.

  Then, the processing unit 11 combines the guide marks M1, M2, and M3 at the positions determined as described above in the equipment vehicle front photographed image Ic (see FIG. 5) obtained by photographing by the in-vehicle camera unit 17. Then, image data representing the composite vehicle front image Ic / M is generated, and the composite vehicle front image Ic / M is displayed on the display unit 13 based on the image data (S26: image processing means, display control means). For example, as shown in FIG. 12, the composite vehicle forward image Ic in which the guide marks M1, M2, and M3 are superimposed on the display unit 13 so as to match the environment (road, facility) represented by the equipment vehicle forward captured image Ic, as shown in FIG. / M is displayed.

  Thereafter, the processing unit 11 repeats the above-described processing (S23 to S26), thereby executing the AR navigation application of the smartphone 20 on the device vehicle front photographed image Ic obtained by photographing with the in-vehicle camera unit 17 while the vehicle is moving. The composite vehicle forward image Ic / M on which the guide mark obtained by the above is superimposed is continuously displayed on the display unit 13.

  In the in-vehicle device as described above, the composite image Isph / M is generated by superimposing the guidance mark (M1, M2, M3) on the terminal vehicle forward image Isph obtained by the camera unit by executing the AR application on the smartphone 20. In this case, based on the difference (shooting direction difference) Δθ between the shooting direction of the camera unit of the smartphone 20 and the shooting direction of the in-vehicle camera unit 17, the shooting of the in-vehicle camera unit 17 of the guide marks (M1, M2, M3) is performed. The position in the device vehicle front captured image Ic obtained by the above is determined, and the guide mark (M1, M2, M3) is superimposed on the determined position of the device vehicle front captured image Ic to generate the composite vehicle front image Ic / M. Then, the composite vehicle forward image Ic / M is displayed on the display unit 13. Therefore, even if the smartphone 20 is not set in the vehicle interior so that the camera unit faces in an appropriate direction in front of the vehicle, a guidance mark is displayed on the appropriate device vehicle front-captured image Ic obtained by photographing with the in-vehicle camera unit 17. The composite vehicle forward image Ic / M on which (M1, M2, M3) is superimposed can be provided to the driver of the vehicle.

  Even if the in-vehicle device 100 does not have an AR navigation function, by connecting a mobile terminal such as the smartphone 20 equipped with the AR navigation application, a composite image by the AR navigation function can be displayed on the display unit 13. it can. Therefore, the driver can use the composite image by the function of the AR navigation through the relatively large screen of the display unit 13 of the in-vehicle device 100 instead of the relatively small screen of the display unit 201 of the smartphone 20.

In the process described above (see FIG. 8), the smartphone 20 is overlaid with the monochrome image I _BG (for example, a black image) and the guide marks (M1, M2, M3) instead of the image obtained by the camera unit. So that the single-color background composite image IBG / M synthesized is requested, and the guide marks (M1, M2, M3) are extracted from the single-color background composite image IBG / M sent from the smartphone 20 in response to the request. However, it is not limited to this. For example, the composite image Isph / M (see FIG. 6) generated by superimposing the guide marks (M1, M2, M3) on the device vehicle front photographed image Isph obtained by photographing with the camera unit from the smartphone 20 is acquired, and The guide marks (M1, M2, M3) may be extracted from the composite image Isph / M. In this case, the nature that the same color does not continue in the background image (device vehicle front photographed image Isph), and the nature that the same color such as a frame line or an arrow continues in the guidance mark (M1, M2, M3). Due to the difference, the guidance marks (M1, M2, M3) can be extracted from the device vehicle forward photographed image Isph as the background.

Further, the composite image Isph / M (see FIG. 6) generated by the smartphone 20 may be displayed on the display unit 201 of the smartphone 20 or may not be displayed. It is also possible to display a single color background and the background to a single color image IBG composite image I _{BG / M} (see FIG. 9) on the display unit 201 of the smartphone 20.

  As described above, the in-vehicle device according to the present invention superimposes the guidance information on the appropriate vehicle front image even if the mobile terminal is not set in the vehicle interior so that the camera unit faces the appropriate direction in front of the vehicle. It is useful as an in-vehicle device to which a portable terminal such as a smartphone provided with a camera unit for photographing the outside is connected.

DESCRIPTION OF SYMBOLS 11 Processing unit 12 Operation part 13 Display part 14 Memory | storage part 15 Output circuit 16 Speaker 17 Car-mounted camera unit 18 Connector 20 Smartphone (mobile terminal)
100 In-vehicle equipment

Claims (5)

It has a camera unit for photographing the outside, and generates guide information corresponding to the environment in the field of view in the shooting direction of the camera unit at the current position based on map information, and the guide information is generated by the camera unit. It is an in-vehicle device to which a portable terminal equipped with an application for generating and displaying a composite image superimposed on the obtained image is connected,
A display unit;
An in-vehicle camera unit that captures a predetermined direction in front of the vehicle;
The difference between the shooting direction of the camera unit set to capture the front of the vehicle by the camera unit of the portable terminal and the shooting direction of the in-vehicle camera unit is detected, and shooting direction difference information representing the difference is generated. Direction difference detection means;
Guidance information acquisition means for acquiring the guidance information generated by the portable terminal;
Guidance information display position determining means for determining the position of the guidance information in the vehicle front image obtained by photographing with the in-vehicle camera unit based on the photographing direction difference information;
Image processing means for generating a composite vehicle forward image by superimposing the guide information at a position determined by the guide information display position determining means of the vehicle forward image obtained by photographing by the in-vehicle camera unit;
Vehicle-mounted equipment having display control means for displaying the composite vehicle forward image obtained by the image processing means on the display unit. The direction difference detection means acquires direction information representing the orientation of the mobile terminal from the mobile terminal;
Vehicle moving direction detecting means for detecting the moving direction of the vehicle;
The in-vehicle device according to claim 1, further comprising means for calculating the photographing direction difference information based on the azimuth information about the portable terminal and a moving direction of the vehicle. The vehicle moving direction detecting means is
Mobile terminal position acquisition means for acquiring position information from the mobile terminal at a plurality of timings;
The in-vehicle device according to claim 2, further comprising a movement direction calculation unit that calculates a movement direction of the vehicle based on position information at the plurality of timings of the portable terminal. The guide information acquisition means requests the mobile terminal for a monochrome background composite image obtained by superimposing a single color image and the guide information instead of an image obtained by the camera unit;
Means for acquiring the monochrome background composite image sent from the mobile terminal in response to the request;
The in-vehicle device according to any one of claims 1 to 3, further comprising guide information extracting means for extracting the guide information from the monochrome background composite image. The guidance information acquisition means acquires a composite image generated by superimposing the guidance information on an image obtained by photographing with the camera unit from the portable terminal, and
The in-vehicle device according to claim 1, further comprising guide information extraction means for extracting the guide information from the acquired composite image.

Images