JP7282199B2 - Merging support device and merging support method - Google Patents

Description

The present invention relates to a merging support device that supports merging between a user and another person (merging partner).

In the automobile industry, with "sharing" as a keyword, the development and commercialization of services that allow service users to use vehicles owned by others, such as ride hailing, ride sharing, and dynamic shuttle, are progressing. . In these services, it is important to smoothly merge service users and vehicles.

For example, Patent Literatures 1 and 2 below propose a merging support device that supports merging by displaying a merging point (meeting place) on a map. In addition, according to Patent Document 3 below, the service user can correctly identify the reserved vehicle by providing in advance an authenticator for authenticating the reserved vehicle that will be the merging partner to the terminal device of the service user. A system is proposed.

JP 2019-053547 A JP 2016-085734 A JP 2019-28629 A

According to the merging support devices of Patent Documents 1 and 2, the user can confirm the position of the merging point on the map. However, it is difficult to match the map with the real world when the user is not good at reading the map, or when there is no feature (so-called "landmark") that serves as a landmark near the meeting point. The user may not be able to easily reach the meeting point.

SUMMARY OF THE INVENTION The present invention has been made to solve the problems described above, and an object of the present invention is to provide a merging support device capable of presenting the location of a merging point in the real world to a user.

A merging support device according to the present invention includes a merging point storage unit that stores a merging point between a user and another person, and an image that indicates the position of the merging point. a virtual object storage unit that stores a virtual junction object in a simulated display mode; a position information acquisition unit that acquires the position of the user; a display control unit for displaying at a position corresponding to the junction in the image of the real scene seen through the transparent screen or the real scenery around the user displayed on the screen , wherein the image of the virtual junction object is a plurality of different images. , and the user and the other person use the meeting support device, the user's meeting support device and the other person's meeting support device use a common virtual meeting point object. .

According to the present invention, it is possible to present a user with a virtual world in which a virtual junction object exists at a position corresponding to a junction in the real world. Therefore, the user can intuitively recognize the position of the meeting point in the real world.

Objects, features, aspects and advantages of the present invention will become more apparent from the following detailed description and accompanying drawings.

1 is a diagram showing a configuration of a merging support device according to Embodiment 1; FIG. FIG. 2 is a diagram showing a real-world example; 2 is a diagram showing an example of a virtual world provided by the merging support device according to Embodiment 1; FIG. FIG. 4 is a diagram showing an example of a real scene and a virtual meeting point object seen by a user; 4 is a flowchart showing the operation of the merging support device according to Embodiment 1; 3 is a diagram illustrating a hardware configuration example of a joining support device; FIG. 3 is a diagram illustrating a hardware configuration example of a joining support device; FIG. FIG. 5 is a diagram for explaining a modification of the merging support device according to Embodiment 1; FIG. FIG. 5 is a diagram for explaining a modification of the merging support device according to Embodiment 1; FIG. FIG. 5 is a diagram for explaining a modification of the merging support device according to Embodiment 1; FIG. FIG. 5 is a diagram for explaining a modification of the merging support device according to Embodiment 1; FIG. FIG. 5 is a diagram for explaining a modification of the merging support device according to Embodiment 1; FIG. FIG. 5 is a diagram for explaining a modification of the merging support device according to Embodiment 1; FIG. 7 is a flow chart for explaining a modification of the merging support device according to Embodiment 1; FIG. 5 is a diagram for explaining a modification of the merging support device according to Embodiment 1; FIG. FIG. 5 is a diagram for explaining a modification of the merging support device according to Embodiment 1; FIG. FIG. 5 is a diagram for explaining a modification of the merging support device according to Embodiment 1; FIG. FIG. 5 is a diagram for explaining a modification of the merging support device according to Embodiment 1; FIG. FIG. 10 is a diagram showing the configuration of a merging support device according to Embodiment 2; FIG. 3 is a diagram showing an example of a real scene image and a virtual junction object displayed on a display device; 9 is a flow chart showing the operation of the merging support device according to the second embodiment; FIG. 12 is a diagram showing the configuration of a merging support device according to Embodiment 3; FIG. 2 is a diagram showing a real-world example; FIG. 10 is a diagram showing an example of a virtual world provided by a merging support device according to Embodiment 3; FIG. FIG. 10 is a diagram showing an example of a real scene image and a virtual meeting point object displayed on the screen of the mobile terminal; 10 is a flowchart showing operations of a mobile terminal according to Embodiment 3; 14 is a flow chart showing a modification of the operation of the mobile terminal according to Embodiment 3; FIG. 11 is a diagram for explaining a modification of the merging support device according to Embodiment 3; FIG. 11 is a diagram for explaining a modification of the merging support device according to Embodiment 3; FIG. 11 is a diagram for explaining a modification of the merging support device according to Embodiment 3; FIG. 11 is a diagram for explaining a modification of the merging support device according to Embodiment 3; FIG. 12 is a diagram for explaining a modification of the mobile terminal according to Embodiment 3; FIG. 12 is a diagram for explaining a modification of the mobile terminal according to Embodiment 3; FIG. 12 is a diagram for explaining a modification of the mobile terminal according to Embodiment 3; FIG. 13 is a diagram showing the configuration of a merging support system according to Embodiment 4; FIG. 14 is a flow chart showing the operation of a merging support server in the merging support system according to Embodiment 4;

<Embodiment 1>
FIG. 1 is a diagram showing the configuration of a merging support device 10 according to Embodiment 1 of the present invention. As shown in FIG. 1, the merging support device 10 of Embodiment 1 is mounted on a vehicle 100. As shown in FIG. However, the merging support device 10 does not have to be permanently installed in the vehicle 100 and may be configured as a portable device that can be brought into the vehicle 100 .

In the present embodiment, the user of merging support device 10 is the driver of vehicle 100 . A “user” in the following description intends a driver of the vehicle 100 unless otherwise specified.

The merging support device 10 is connected to a head-up display (HUD) 21 , a display device 22 , an operation input device 23 , a positioning device 24 , a communication device 25 and a map information storage device 26 provided in the vehicle 100 .

The head-up display 21 and the display device 22 are means by which the merging support device 10 presents various kinds of information as images to the user (driver of the vehicle 100). In particular, the head-up display 21 uses the windshield of the vehicle 100 as a transparent screen to display an image on the actual scene that the user can see. The display device 22 is a general display device installed on the center panel or instrument panel of the vehicle 100, and displays an operation screen of the merging support device 10, for example. The display device 22 is configured by, for example, a liquid crystal display or an organic EL display (Organic ElectroLuminescence display).

Since the image displayed by the head-up display 21 is displayed on the actual scene, it looks as if it exists in the real world. Hereinafter, the image displayed by the head-up display 21 will be referred to as a "virtual object", and the world in which the virtual object exists will be referred to as a "virtual world".

The operation input device 23 is means for the user to input an operation to the merging support device 10 . The operation input device 23 may be a hardware key such as a push button, an operation lever, or a keyboard, or may be a software key displayed on the screen. For example, software keys as the operation input device 23 may be displayed on the display device 22, and in that case, the display device 22 and the operation input device 23 may be configured as one touch panel.

The positioning device 24 calculates the position of the vehicle 100 based on positioning signals received from GNSS (Global Navigation Satellite System) satellites such as GPS (Global Positioning System) satellites. Here, it is assumed that the position of the vehicle 100 is the same as the position of the user of the merging support device 10 . The communication device 25 is means for the merging support device 10 to communicate with the outside of the vehicle 100 .

The map information storage device 26 is a storage medium in which map information including position information such as roads and POIs (Points of Interest) is stored. The map information storage device 26 may be included in the navigation system of the vehicle 100 . Alternatively, the map information storage device 26 may be configured as a server installed outside the vehicle 100 , in which case the merging support device 10 acquires map information from the map information storage device 26 through the communication device 25 .

The meeting support device 10 includes a communication section 11 , a meeting point storage section 12 , a virtual object storage section 13 , a position information acquisition section 14 , a map information acquisition section 15 and a display control section 16 .

The communication unit 11 communicates with the outside of the vehicle 100 via the communication device 25 . The meeting point storage unit 12 stores a meeting point between the user and another person who is a meeting partner. A merging partner may be a pedestrian or a vehicle. The user can operate the operation input device 23 to perform direct or indirect communication with the merging partner via the communication device 25, and can predetermine the merging point with the merging partner. The point is stored in the merging point storage unit 12 . Here, indirect communication means communication between the user and the merging partner via another system (for example, an arbitration system at the merging point) composed of a server or the like, and direct communication , means that the user and the joining partner communicate without going through another system.

The virtual object storage unit 13 stores an image of a virtual junction object, which is a virtual object indicating the position of the junction.

The position information acquisition unit 14 acquires the position of the vehicle 100 (user's position) calculated by the positioning device 24 . Further, the position information acquisition unit 14 can calculate the direction of the vehicle 100 from the position history of the vehicle 100 . Note that the position information acquisition unit 14 may perform processing for increasing the accuracy of the position of the vehicle 100, such as map matching using map information and positioning by autonomous navigation using the speed sensor and direction sensor of the vehicle 100. good. Also, the orientation of the vehicle 100 may be measured using the orientation sensor of the vehicle 100 .

The map information acquisition unit 15 acquires map information around the position of the vehicle 100 acquired by the position information acquisition unit 14 from the map information storage device 26 .

Based on the position and orientation of the vehicle 100 acquired by the position information acquisition unit 14, the position of the junction stored in the virtual object storage unit 13, and the map information acquired by the map information acquisition unit 15, the display control unit 16 The position (relative position) of the meeting point seen from the user (driver of the vehicle 100) is calculated, and the head-up display 21 is used to place a virtual meeting point object at a position corresponding to the meeting point in the actual scenery seen by the user. indicate. More specifically, the display control unit 16 controls the virtual confluence at the intersection of the straight line connecting the user's eyes and the confluence and the windshield so that the user sees the virtual confluence as superimposed on the confluence. Display a point object. As a result, it appears to the user as if a virtual meeting point object exists at the meeting point. In other words, the display control unit 16 provides the user with a virtual world in which the virtual junction object exists at the junction in the real world.

For example, it is assumed that a point MP in the real world as shown in FIG. 2 is determined as the meeting point between the user and the meeting partner, and the vehicle 100 driven by the user reaches the vicinity of the meeting point MP. Information on the position of the merging point MP is stored in the merging point storage unit 12 of the merging support device 10 . In this case, the display control unit 16 uses the head-up display 21 to provide the user with a virtual world as shown in FIG. 3 in which the virtual meeting point object 200 exists at the meeting point MP. A user, who is the driver of the vehicle 100, sees a virtual junction object 200 at a position corresponding to the junction MP in the actual scene seen through the windshield 30 of the vehicle 100, as shown in FIG.

In FIG. 3 and subsequent figures, the virtual objects are hatched so that the real world and the virtual world can be easily distinguished. need not be hatched. Here, as an example of the virtual junction object, an image simulating a sign post installed at a public transportation stop (bus stop) is used, but any image may be used as the virtual junction object.

In this way, the merging support device 10 according to the first embodiment presents the location of the merging point to the user by presenting the virtual world in which the virtual merging point object appears to exist at the merging point in the real world. . Therefore, the user can intuitively recognize the location of the meeting point in the real world. As a result, it becomes easier for the user to reach the meeting point.

Here, the operation of the merging support device 10 will be described with reference to the flowchart of FIG. When the merging support device 10 is activated and a merging point is determined between the user and the merging partner, the merging point storage unit 12 stores the merging point (step S101).

Subsequently, the position information acquisition unit 14 acquires the position of the vehicle 100 calculated by the positioning device 24, that is, the position of the user (step S102). At this time, the position information acquiring unit 14 calculates the orientation of the vehicle 100 based on the position history of the vehicle 100 . The map information acquisition unit 15 acquires map information around the user's position from the map information storage device 26 (step S103).

The display control unit 16 displays the position of the user based on the position of the user acquired by the position information acquisition unit 14, the position of the meeting point stored in the meeting point storage unit 12, and the map information acquired by the map information acquisition unit 15. is near the junction (step S104).

The determination in step S104 may simply be based on whether or not the straight-line distance from the user to the junction is equal to or less than a predetermined threshold value (for example, 100 m). Even if the straight line distance from the user to the meeting point is short, it may take a long time for the user to reach the meeting point, for example, when the road crosses the intersection. Therefore, in the present embodiment, map information is used to calculate the route length (distance) from the user to the meeting point, and step S104 is determined depending on whether the route length is equal to or less than a threshold.

If the user's position is near the meeting point (YES in step S104), the display control unit 16 controls the position and orientation of the vehicle 100 acquired by the position information acquisition unit 14 and the meeting point stored in the virtual object storage unit 13. Based on the position and the map information acquired by the map information acquisition unit 15, the position of the meeting point seen from the user is calculated (step S105). Then, it is confirmed whether or not the position of the confluence seen from the user is within the display area of the head-up display 21 (step S106).

If the position of the meeting point as viewed by the user is within the display area of the head-up display 21 (YES in step S106), the display control unit 16 uses the head-up display 21 to display the virtual meeting point object as viewed by the user. It is superimposed and displayed on the position of the junction (step S107), and the process returns to step S102.

On the other hand, if the user's position is far from the meeting point (NO in step S104) and if the meeting point as seen from the user is outside the display area of the head-up display 21 (NO in step S106), the display The control unit 16 confirms whether or not the virtual junction object is displayed on the head-up display 21, erases it if it is displayed (step S108), and returns to step S102.

Regarding the determination in step S104, if the road on which the user is located and the road on which the confluence is located intersect, and there is a stairway or slope that allows walking between the two roads, the user can It is also possible to get off the vehicle 100 and walk to pick up the merging partner. As such, the determination of step S104 may be made based on the length of the path through the stairs or ramp. When such determination results in YES in step S104, it is preferable to notify the user to that effect.

In addition, even if there are stairs or a slope, if the arrival time to the merging point is later than using the stairs or the slope, the vehicle 100 can be used without taking the stairs or the slope into consideration. Preferably, the determination of S104 is made. Alternatively, the user may be notified that using the stairs or the slope will result in a later arrival time at the junction than driving the vehicle 100 along the road.

[Hardware configuration example]
6 and 7 are diagrams showing examples of the hardware configuration of the joining support device 10, respectively. Each function of the constituent elements of the merging support device 10 shown in FIG. 1 is implemented by, for example, a processing circuit 50 shown in FIG. That is, the merging support device 10 stores a merging point between the user and others, stores a virtual merging point object that is an image indicating the position of the merging point, acquires the user's position, and stores the user's position and the merging point. A processing circuit 50 for displaying a virtual junction object at a position corresponding to the junction in the real world seen by the user through the transparent screen, based on the position of . The processing circuit 50 may be dedicated hardware, or a processor (central processing unit (CPU: Central Processing Unit), processing device, arithmetic device, microprocessor, microcomputer, etc.) that executes a program stored in a memory. Also called a DSP (Digital Signal Processor)).

When the processing circuit 50 is dedicated hardware, the processing circuit 50 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination of these. Each function of the constituent elements of the joining support device 10 may be implemented by individual processing circuits, or these functions may be collectively implemented by one processing circuit.

FIG. 7 shows an example of the hardware configuration of the joining support device 10 when the processing circuit 50 is configured using a processor 51 that executes programs. In this case, the functions of the constituent elements of the joining support device 10 are implemented by software or the like (software, firmware, or a combination of software and firmware). Software or the like is written as a program and stored in the memory 52 . The processor 51 implements the function of each part by reading and executing the program stored in the memory 52 . Namely, when executed by the processor 51, the meeting support device 10 stores a process of storing the meeting point between the user and the other party, a process of storing a virtual meeting point object, which is an image indicating the position of the meeting point, A process of acquiring the position of the user and a process of displaying the virtual meeting point object at a position corresponding to the meeting point in the real world seen by the user through the transparent screen based on the user's position and the meeting point position are the results. A memory 52 is provided for storing the program to be executed on the fly. In other words, it can be said that this program causes a computer to execute the procedures and methods of operation of the components of the merging support device 10 .

Here, the memory 52 is, for example, a non-volatile or Volatile semiconductor memory, HDD (Hard Disk Drive), magnetic disk, flexible disk, optical disk, compact disk, mini disk, DVD (Digital Versatile Disc) and its drive device, etc., or any storage medium that will be used in the future. may

The configuration in which the functions of the constituent elements of the merging support device 10 are realized by either hardware or software has been described above. However, the present invention is not limited to this, and a configuration may be adopted in which a part of the constituent elements of the merging support device 10 are realized by dedicated hardware and another part of the constituent elements are realized by software or the like. For example, the processing circuit 50 as dedicated hardware implements the functions of some components, and the processing circuit 50 as a processor 51 executes programs stored in the memory 52 for some other components. Its function can be realized by reading and executing it.

As described above, the merging support device 10 can implement the functions described above by using hardware, software, etc., or a combination thereof.

[Modification]
Any method may be used to determine the meeting point. For example, an arbitration system may be built on the Internet for determining the location of the meeting point between the user of the meeting support device 10 and the meeting partner. In this case, the meeting point storage unit 12 of the meeting support device 10 acquires information on the determined meeting point from the arbitration system through the communication unit 11 and stores the information.

The image of the virtual meeting point object may be selected from a plurality of images by the user operating the operation input device 23 . The image of the virtual confluence point object may be an image specified by the confluence partner, or may be determined by consultation between the user of the confluence support device 10 and the confluence partner. For example, an arbitration system for arranging the image of the virtual meeting point object between the user of the meeting support device 10 and the meeting partner may be constructed on the Internet. In this case, the virtual object storage unit 13 of the meeting support device 10 acquires the image of the determined virtual meeting point object from the arbitration system through the communication unit 11 and stores it.

When the intention of the merging partner is taken into consideration when determining the image of the virtual merging point object, both the user of the merging support device 10 (the driver of the vehicle 100) and the merging partner can decide what kind of image the virtual merging point object should be. can recognize whether For example, when the vehicle 100 is a taxi, the driver of the vehicle 100 and the merging partner may not be acquainted with each other. If so, the images of the virtual meeting point objects can be confirmed by each other, so that mutual authentication can be performed, which contributes to the improvement of security.

Further, the display control unit 16 may provide the user with a virtual world including not only a virtual meeting point object indicating the position of the meeting point but also a virtual object indicating additional information for supporting the meeting. For example, as shown in FIG. 8, together with a virtual meeting point object 200, a virtual object 201 simulating a user arriving at the meeting point (in this case, the vehicle 100 driven by the user), and a virtual object 201 simulating a meeting partner arriving at the meeting point. A user may be presented with a virtual world in which the object 202 resides. In this case, the display control unit 16 uses the head-up display 21 to display a virtual object 201 simulating the vehicle 100 on the road in front of the virtual junction object 200 as shown in FIG. A virtual object 202 simulating a joining partner is displayed beside 200 .

Further, for example, as shown in FIG. 10, the user may be provided with a virtual world in which a virtual junction object 200 and a virtual object 203 indicating a stop position (parking space) when the vehicle 100 arrives at the junction exist. . In this case, the display control unit 16 uses the head-up display 21 to display a virtual object 203 indicating the stop position of the vehicle 100 on the road in front of the virtual junction object 200, as shown in FIG.

12, when the vehicle 100 has passed the confluence point MP, the position of the confluence point MP seen from the user is outside the display area of the head-up display 21. FIG. At this time, the display control unit 16 may display a virtual object 204 indicating that the meeting point MP is close to the vehicle 100 but not at a position where the virtual meeting point object 200 can be displayed, as shown in FIG. 13, for example. In FIG. 13, the virtual object 204 includes an image of the image of the virtual junction object 200 in an inconspicuous display mode (size, color, brightness, etc.) and an image of an arrow indicating the direction in which the junction MP is located. , the virtual object 204 may be just one of these two images. Also, an animation effect such as blinking the virtual object 204 may be used.

Note that the operation of the merging support device 10 when displaying a virtual object 204 indicating that the position of the merging point MP as seen from the user is outside the display area of the head-up display 21 as shown in FIG. shown in the flow chart. The flowchart of FIG. 14 is obtained by adding step S109, which is executed when NO is determined in step S106, to the flowchart of FIG.

That is, in the flowchart of FIG. 14, when the position of the meeting point seen by the user is outside the display area of the head-up display 21 (NO in step S106), the display control unit 16 displays the virtual object 204 indicating that fact. , is displayed using the head-up display 21 (step S109). Since other steps are the same as those in FIG. 5, descriptions thereof are omitted here.

In addition, the communication unit 11 acquires information on the location of the meeting partner and the planned passage route from the mobile terminal, navigation device, or the like of the meeting partner, and the display control unit 16 converts a virtual object indicating the information into a virtual meeting point object. 200 may be displayed. Specifically, the display control unit 16 displays a virtual object indicating the position of the merging partner at a position corresponding to the position of the merging partner as seen from the user, and also displays a virtual object indicating the position of the merging partner as seen from the user. A virtual object may be displayed at the corresponding position, which indicates the planned passage route of the merging partner.

For example, it is assumed that a point MP in the real world as shown in FIG. 15 is determined as the meeting point between the user and the meeting partner 101, and the vehicle 100 driven by the user and the meeting partner 101 arrive near the meeting point MP. Information on the merging point MP is stored in the merging point storage unit 12 of the merging support device 10, and the communication unit 11 sequentially acquires information on the position of the merging partner 101 and the planned passing route from the portable terminal held by the merging partner 101. do. In this case, the display control unit 16 uses the head-up display 21 to display a virtual confluence point object 200 indicating the confluence point MP, a virtual object 205 indicating the position of the confluence partner 101, and a planned passing route of the confluence partner 101. The user is provided with a virtual world as shown in FIG. 16 in which the virtual object 206 exists. As shown in FIG. 17, the user, who is the driver of the vehicle 100, can see a virtual merging point object 200, a virtual object 205 indicating the position of the merging partner 101, and a merging partner in the actual scene seen through the windshield 30 of the vehicle 100. A virtual object 206 indicating the planned passage route of 101 can be seen.

FIG. 17 shows a state in which the joining partner 101 is hidden behind a building and cannot be seen. . Therefore, the user can quickly find the joining partner 101 even in a busy place.

The information about the merging partner acquired by the communication unit 11 and displayed by the display control unit 16 is not limited to the information on the current position and the planned passage route. For example, the communication unit 11 may acquire information about the scheduled arrival time of the merging partner at the merging point, and display a character string such as "five minutes before arrival" as a virtual object.

<Embodiment 2>
FIG. 19 is a diagram showing the configuration of a merging support device 10 according to Embodiment 2 of the present invention. The merging support device 10 of Embodiment 2 is also mounted on the vehicle 100 , and the user of the merging support device 10 is the driver of the vehicle 100 . However, instead of the head-up display 21 shown in FIG. 1, an imaging device 27 is connected to the merging support device 10 of the second embodiment.

The photographing device 27 is a camera that photographs the actual scenery around the vehicle 100 (that is, around the user). Here, it is assumed that the photographing device 27 photographs the actual scene in front of the vehicle 100 .

The display control unit 16 of the merging support device 10 according to the second embodiment displays an image of the actual scene captured by the imaging device 27 (hereinafter referred to as “actual scene image”) on the screen of the display device 22 . Further, the display control unit 16 controls the position and orientation of the vehicle 100 acquired by the position information acquisition unit 14, the position of the junction stored in the virtual object storage unit 13, and the map information acquired by the map information acquisition unit 15. to calculate the relative position of the meeting point as seen from the user (driver of the vehicle 100), and display the virtual meeting point object at the position corresponding to the meeting point in the actual scene image displayed on the screen of the display device 22. . As a result, an image is displayed on the screen of the display device 22 as if a virtual meeting point object exists at the meeting point. In other words, the display control unit 16 provides the user with a virtual world in which a virtual meeting point object exists at the meeting point through the screen of the display device 22 .

For example, when the display control unit 16 provides the user with a virtual world similar to that shown in FIG. 3 described above, the display control unit 16 displays an image as shown in FIG. The image in FIG. 4 is obtained by superimposing a virtual junction object 200 at a position corresponding to the junction MP in the real scene image captured by the photographing device 27 .

In this way, the merging support device 10 according to the second embodiment presents the user, who is the driver of the vehicle 100, with a virtual world in which the virtual merging point object exists at the merging point in the real world. Present the location of the point. Therefore, the user can intuitively recognize the location of the meeting point in the real world, and can easily reach the meeting point.

Here, the operation of the merging support device 10 will be described with reference to the flowchart of FIG. 21 . When the merging support device 10 is activated and a merging point is determined between the user and the merging partner, the merging point storage unit 12 stores the merging point (step S201).

Subsequently, the display control unit 16 displays the image of the surroundings of the vehicle 100 taken by the imaging device 27 on the screen of the display device 22 (step S202). Then, the position information acquisition unit 14 acquires the position of the vehicle 100 calculated by the positioning device 24, that is, the position of the user (step S203). At this time, the position information acquiring unit 14 calculates the orientation of the vehicle 100 based on the position history of the vehicle 100 . The map information acquisition unit 15 acquires map information around the user's position from the map information storage device 26 (step S204).

The display control unit 16 displays the position of the user based on the position of the user acquired by the position information acquisition unit 14, the position of the meeting point stored in the meeting point storage unit 12, and the map information acquired by the map information acquisition unit 15. is near the junction (step S205). The determination in step S205 may be the same determination as in step S104 of FIG.

If the user's position is near the meeting point (YES in step S205), the display control unit 16 controls the position and direction of the vehicle 100, the position of the meeting point, and the map information to display the actual scene as seen by the user. is calculated (step S206).

Then, it is confirmed whether or not the position of the merging point seen by the user is within the range of the actual scene image captured by the image capturing device 27 (step S207). If the position of the meeting point as seen by the user is within the range of the actual scene image (YES in step S207), the display control unit 16 controls the position of the meeting point as seen by the user in the actual scene image displayed on the screen of the display device 22. , the virtual junction object is superimposed and displayed (step S208), and the process returns to step S202.

On the other hand, if the user's position is far from the meeting point (NO in step S205) and if the position of the meeting point seen by the user is outside the range of the actual scene image (NO in step S207), the display control unit 16 However, it is confirmed whether or not the virtual junction object is displayed on the screen of the display device 22, and if it is displayed, it is deleted (step S209), and the process returns to step S202.

[Modification]
Various modifications shown in the first embodiment can also be applied to the merging support device 10 of the second embodiment.

In the flowchart of FIG. 21, only the actual scene image is first displayed on the screen of the display device 22, and then the virtual junction object is displayed superimposed thereon. ), the actual scene image synthesized with the virtual junction object may be displayed on the display device 22 after going ahead.

Further, when the display device 22 has a superimpose function, the display device 22 displays on the screen a real scene image obtained directly from the photographing device 27 (without going through the display control unit 16), and the display control unit 16 A virtual meeting point object may be superimposed on the actual scene image displayed on the screen of 22 .

In Embodiment 2, an example in which the photographing device 27 photographs the actual scene in front of the vehicle 100 has been described, but the photographing device 27 may photograph the actual scene to the side, rear, or rear side of the vehicle 100. good. For example, when the photographing device 27 photographs an actual scene behind or behind the vehicle 100 , the photographing device 27 and the display device 22 function as an electronic mirror that electronically realizes the rear view mirror of the vehicle 100 . In other words, the merging support device 10 of the second embodiment is applicable to the electronic mirror system of the vehicle 100. FIG.

In addition, the merging support device 10 is connected to a plurality of photographing devices 27 with different photographing directions, and the display control unit 16 creates a surround image by synthesizing a plurality of real scene images photographed by the plurality of photographing devices 27, It may be displayed on the screen of the display device 22 .

Two of the technology using the head-up display 21 described in the first embodiment, the technology using the actual scene image in front of the vehicle 100 described in the second embodiment, the technology using the above-described electronic mirror, and the technology using the surround image. You can combine two or more. As a result, the area for displaying the virtual meeting point object is expanded, so the probability of displaying the virtual meeting point object increases. Also, when using a plurality of head-up displays 21 or display devices 22, the virtual junction object may be displayed only on some of the plurality of screens. At that time, the display control unit 16 may display a mark on the screen on which the virtual meeting point object is not displayed to notify the user on which screen the virtual meeting point object is displayed.

In Embodiment 2, vehicle 100 may be remotely controlled by a driver (user of merging support device 10) who is away from vehicle 100. FIG. In that case, the display device 22 is installed near the user at a remote location. When the vehicle 100 is remotely operated, the position of the vehicle 100 acquired by the position information acquisition unit 14 is not actually the position of the user. is regarded as the user's position. Then, the display control unit 16 displays the virtual junction object at the position corresponding to the junction of the images of the actual scenery around the vehicle 100 .

[Hardware configuration]
The merging support device 10 of the second embodiment can also be realized by a hardware configuration similar to that of FIGS. 6 and 7. FIG. For example, when the confluence support device 10 is realized by the processing circuit 50 shown in FIG. store the point object, obtain the position of the user, and based on the position of the user and the position of the meeting point, place the virtual meeting point object at the position corresponding to the meeting point of the images of the real world surrounding the user displayed on the screen; a processing circuit 50 for displaying the .

In addition, when the meeting support device 10 is composed of the processor 51 and the memory 52 shown in FIG. a process of storing a virtual meeting point object, which is an image indicating the position of the meeting point; a process of acquiring the position of the user; A program that results in the execution of a process of displaying at a position corresponding to the confluence of the displayed images of the actual scenery around the user is stored.

<Embodiment 3>
FIG. 22 is a diagram showing the configuration of a merging support device 10 according to Embodiment 3 of the present invention. The merging support device 10 according to the third embodiment is installed in a mobile terminal 300 such as a smart phone, for example. In other words, the merging support device 10 is implemented by executing a merging support application program (hereinafter referred to as a 'merging support application') stored in the memory by the processor of the mobile terminal 300 . Except for this, the configuration of the merging support device 10 of FIG. 22 is the same as that of FIG.

A user of the joining support device 10 is a user of the mobile terminal 300 . A “user” in the following description intends the user of the mobile terminal 300 unless otherwise specified. Here, the user is assumed to be a pedestrian, but the user can also bring the mobile terminal 300 into the vehicle. A merging partner may be a pedestrian or a vehicle, but is assumed to be a vehicle here. Also, the merging partner vehicle may be a vehicle without a driver (for example, an unmanned taxi) that travels by remote control or automatic operation.

The photographing device 27 of the portable terminal 300 photographs the actual surroundings of the user, but the photographing direction depends on the direction in which the user holds the portable terminal 300, and therefore does not necessarily match the traveling direction of the user. Therefore, the display control unit 16 of the merging support device 10 according to Embodiment 3 needs to recognize the orientation of the mobile terminal 300 in order to recognize the range of the actual scene image captured by the image capturing device 27 . Any method may be used to recognize the orientation of the mobile terminal 300 . The display control unit 16 may calculate the orientation of the mobile terminal 300 using, for example, an orientation sensor (not shown) of the mobile terminal 300, or the actual scene image captured by the imaging device 27 and the map information acquisition unit 15 may be The orientation of the mobile terminal 300 may be determined by comparing it with the acquired map information.

The display control unit 16 displays the actual scene image captured by the imaging device 27 on the screen of the display device 22 (also referred to as "the screen of the mobile terminal 300"). Further, the display control unit 16 controls the recognized orientation of the mobile terminal 300, the position of the mobile terminal 300 acquired by the position information acquisition unit 14, the position of the junction stored in the virtual object storage unit 13, and the map information acquisition unit 15. based on the acquired map information, calculates the relative position of the meeting point seen from the user, and displays the virtual meeting point object at the position corresponding to the meeting point in the actual scene image displayed on the screen of the display device 22. . As a result, an image is displayed on the screen of the display device 22 as if a virtual meeting point object exists at the meeting point. In other words, the display control unit 16 provides the user with a virtual world in which a virtual meeting point object exists at the meeting point through the screen of the display device 22 .

For example, it is assumed that a point MP in the real world as shown in FIG. 23 is determined as the meeting point between the user 301 and the meeting partner, and the user 301 reaches the vicinity of the meeting point MP. Information on the position of the merging point MP is stored in the merging point storage unit 12 of the merging support device 10 . In this case, the display control unit 16 uses the display device 22 to provide the user with a virtual world as shown in FIG. 24 in which the virtual meeting point object 400 exists at the meeting point MP. On the screen of the mobile terminal 300 of the user 301, as shown in FIG. 25, an image is displayed in which the virtual meeting point object 400 is superimposed at the position corresponding to the meeting point MP in the real scene image captured by the imaging device 27. FIG.

In this way, the merging support apparatus 10 according to Embodiment 3 presents to the user of the mobile terminal 300 a virtual world in which a virtual merging point object appears to exist at the merging point in the real world. present. Therefore, the user can intuitively recognize the location of the meeting point in the real world, and can easily reach the meeting point.

Here, the operation of the mobile terminal 300 will be described with reference to the flowchart of FIG. 26 . First, when the user of the mobile terminal 300 operates the operation input device 23 to start an application (meeting point determination application) for determining the meeting point with the meeting partner, the meeting point determination application activates the communication device 25. Direct or indirect communication is performed with the merging partner through which the merging point is arranged (step S301). The meeting point determined by the meeting point determination application is stored in the meeting point storage unit 12 (step S302).

After that, the user activates the merging support device 10 by activating the merging support application at an arbitrary timing (for example, when the user approaches the merging point to some extent) (step S303).

When the merging support device 10 is activated, the display control unit 16 displays an image of the actual surroundings of the user captured by the imaging device 27 on the screen of the display device 22 (step S304). Then, the position information acquisition unit 14 acquires the position of the mobile terminal 300 calculated by the positioning device 24, that is, the position of the user (step S305). The map information acquisition unit 15 acquires map information around the user's position from the map information storage device 26 (step S306).

The display control unit 16 displays the position of the user based on the position of the user acquired by the position information acquisition unit 14, the position of the meeting point stored in the meeting point storage unit 12, and the map information acquired by the map information acquisition unit 15. is near the junction (step S307). The determination in step S307 may be the same determination as in step S104 in FIG.

If the user's position is near the meeting point (YES in step S307), the display control unit 16 displays the actual scene seen by the user based on the position and orientation of the mobile terminal 300, the position of the meeting point, and the map information. The position of the upper junction point is calculated (step S308).

Then, the display control unit 16 confirms whether or not the position of the merging point seen by the user is within the range of the actual scene image captured by the image capturing device 27 (step S309). If the position of the meeting point as seen by the user is within the range of the actual scene image (YES in step S309), the display control unit 16 controls the position of the meeting point as seen by the user in the actual scene image displayed on the screen of the display device 22. , the virtual junction object is superimposed and displayed (step S310), and the process returns to step S304.

On the other hand, if the user's position is far from the meeting point (NO in step S307) and if the position of the meeting point seen by the user is outside the range of the actual scene image (NO in step S309), the display control unit 16 However, it is confirmed whether or not the virtual junction object is displayed on the screen of the display device 22, and if it is displayed, it is deleted (step S311), and the process returns to step S304.

[Modification]
If the position of the merging point is outside the range of the actual scene image captured by the image capturing device 27 (that is, if NO in step S309 of FIG. 26), the modification described in Embodiment 1 using FIGS. 12 to 14 is performed. As in the example, a virtual object indicating that the meeting point is outside the display area may be displayed on the screen of the mobile terminal 300 .

In the flowchart of FIG. 26, steps S304 to S311 are repeatedly executed, so the screen displayed on the display device 22 is updated sequentially. Therefore, even if the virtual meeting point object 400 is displayed on the screen of the display device 22, as soon as the user changes the direction in which the mobile terminal 300 is held, the meeting point is outside the real image area, and the virtual meeting point object 400 is displayed. Disappear. Therefore, the user may be allowed to pause the image displayed on the screen of the display device 22 . This makes it easier for the user to compare the image of the virtual world displayed on the display device 22 with the scenery of the real world.

Further, when the user rotates the mobile terminal 300 while photographing a real scene image and acquires a plurality of real scene images in different shooting directions, the display control unit 16 synthesizes the real scene images to generate a panorama image. , the virtual meeting point object may be displayed in the panorama image.

In the flowchart of FIG. 26, the meeting point determination application and the meeting support application are separate applications, but the function of the meeting point determination application may be incorporated in the meeting support application. An example of the operation of the mobile terminal 300 when the meeting point determination application has the function of the meeting support application will be described with reference to the flowchart of FIG.

When the user of the mobile terminal 300 operates the operation input device 23 to activate the joining support application (step S401), the joining support application directly or indirectly communicates with the joining partner via the communication device 25, A merging point is arranged with the merging partner (step S402). The determined meeting point is stored in the meeting point storage unit 12 (step S403).

After that, the position information acquiring unit 14 acquires the position of the mobile terminal 300 calculated by the positioning device 24, that is, the position of the user (step S404). The map information acquisition unit 15 acquires map information around the user's position from the map information storage device 26 (step S405). The display control unit 16 displays the position of the user based on the position of the user acquired by the position information acquisition unit 14, the position of the meeting point stored in the meeting point storage unit 12, and the map information acquired by the map information acquisition unit 15. is near the junction (step S406). If the user's position is near the meeting point (YES in step S406), the user is notified of this using, for example, the vibration function of the mobile terminal 300 (step S407).

The user who has received the notification can operate the operation input device 23 to instruct the rendezvous support device 10 to display the virtual rendezvous point object. If there is no instruction to display the virtual junction object for a certain period of time (NO in step S408), the process returns to step S404.

When the merging support device 10 receives an instruction to display the virtual merging point object from the user (YES in step S408), the display control unit 16 causes the image of the actual surroundings of the user captured by the imaging device 27 to be displayed on the display device 22. is displayed on the screen (step S409). Then, the position information acquisition unit 14 acquires the position of the mobile terminal 300 (user's position) calculated by the positioning device 24 (step S410). The map information acquisition unit 15 acquires map information around the user's position from the map information storage device 26 (step S411).

The display control unit 16 determines whether or not the user's position is near the meeting point by the same method as in step S406. , the position of the meeting point, and the map information, the position of the meeting point on the actual scene as seen from the user is calculated (step S413).

Then, it is confirmed whether or not the position of the meeting point seen by the user is within the range of the actual scene image captured by the image capturing device 27 (step S414). If the position of the junction as seen by the user is within the range of the actual scene image (YES in step S414), the display control unit 16 controls the position of the junction as seen by the user in the actual scene image displayed on the screen of the display device 22. , the virtual junction object is superimposed and displayed (step S415), and the process returns to step S409.

When the user moves away from the meeting point (NO in step S412), the display control unit 16 checks whether the virtual meeting point object is displayed on the screen of the display device 22, and displays it. If so, it is deleted (step S416) and the process returns to step S404. Further, when the position of the merging point seen by the user is outside the range of the actual scene image (NO in step S414), the display control unit 16 performs the same processing as in step S416 (step S417), and performs step S409. back to

In the third embodiment, as in the first embodiment, the user or the meeting partner selects an image to be the virtual meeting point object, and an arbitration system is used to determine the image of the virtual meeting point object. can be changed by

Further, the display control unit 16 may provide the user with a virtual world including not only a virtual meeting point object indicating the position of the meeting point but also a virtual object indicating additional information for supporting the meeting. For example, as shown in FIG. 28, along with a virtual meeting point object 400, there are a virtual object 401 imitating a user 301 arriving at the meeting point, and a virtual object 402 imitating a meeting partner (a vehicle in this case) arriving at the meeting point. A user may be provided with a virtual world that In this case, as shown in FIG. A virtual object 402 imitating a merging partner is displayed on the road.

In addition, the communication unit 11 acquires information on the location of the meeting partner and the planned passage route from the mobile terminal, navigation device, or the like of the meeting partner, and the display control unit 16 converts a virtual object indicating the information into a virtual meeting point object. 400 may be displayed. Specifically, the display control unit 16 displays a virtual object indicating the position of the merging partner at a position corresponding to the position of the merging partner as seen from the user, and also displays a virtual object indicating the position of the merging partner as seen from the user. A virtual object may be displayed at the corresponding position, which indicates the planned passage route of the merging partner.

For example, it is assumed that a point MP in the real world as shown in FIG. 30 is determined as the meeting point of the user 301 and the vehicle 302 of the meeting partner, and the user 301 and the vehicle 302 of the meeting partner arrive near the meeting point MP. . Information on the merging point MP is stored in the merging point storage unit 12 of the merging support device 10, and the communication unit 11 acquires information on the position of the vehicle 302 and the planned passage route from the navigation device of the vehicle 302 of the merging partner. . In this case, the display control unit 16 uses the screen of the mobile terminal 300 to display the virtual meeting point object 400 indicating the meeting point MP, the virtual object 403 indicating the position of the vehicle 302 of the meeting partner, and the vehicle 302 of the meeting partner. The user 301 is provided with a virtual world as shown in FIG. As shown in FIG. 31, on the screen of the portable terminal 300 of the user 301, a virtual merging point object 400, a virtual object 403 indicating the position of the vehicle 302 of the merging partner, and the planned passage route of the vehicle 302 are displayed in the actual scene image. An image in which the indicated virtual object 404 is superimposed on the real scene image is displayed.

The information about the merging partner acquired by the communication unit 11 and displayed by the display control unit 16 is not limited to the information on the current position and the planned passage route. For example, the communication unit 11 may acquire information about the scheduled arrival time of the merging partner at the merging point, and display a character string such as "five minutes before arrival" as a virtual object.

The mobile terminal 300 is configured to be able to switch between display of a real scene image (display of a virtual world) including a virtual junction object as shown in FIG. 25 and display of a map showing the position of the user and the junction as shown in FIG. may have been In that case, the map display area and the actual scene image display area on the screen of the mobile terminal 300 may be changed according to the orientation of the mobile terminal 300 .

For example, when the mobile terminal 300 is laid horizontally (this state is defined as an elevation angle of 0 degrees), a map such as that shown in FIG. This state is defined as an elevation angle of 90 degrees), a real scene image as shown in FIG. 25 may be displayed. Alternatively, when the elevation angle of portable terminal 300 is 90 degrees, both the actual scene image and the map may be displayed as shown in FIG. Also, when the user pauses the display of the actual scene image, both the actual scene image and the map may be displayed as shown in FIG. 33 regardless of the elevation angle of the mobile terminal 300 .

25 is displayed when the portable terminal 300 is oriented horizontally, and both the actual scene image and the map are displayed as shown in FIG. 34 when the portable terminal 300 is oriented vertically. good too. Furthermore, in a state where both the actual scene image and the map are displayed on the screen of the mobile terminal 300, the user can swipe the screen of the mobile terminal 300 to change the display area of the actual scene image and the map. ratio) may be changed.

<Embodiment 4>
In the fourth embodiment, the vehicle 100 equipped with the merging support device 10 according to the first embodiment (or the second embodiment) and the user of the mobile terminal 300 equipped with the merging support device 10 according to the third embodiment are allowed to merge. An example of a supporting merge system is shown.

35 is a diagram showing a configuration of a merging support system according to Embodiment 4. FIG. The merging support system includes a vehicle 100 and a mobile terminal 300 each having a merging support device 10 , and a merging support server 500 . Vehicle 100 and mobile terminal 300 can communicate with merging support server 500 through a communication network such as the Internet. Although one vehicle 100 and one mobile terminal 300 are shown in FIG. 35, there are a plurality of vehicles 100 and mobile terminals 300 .

The merging support server 500 provides a pick-up service using the vehicle 100 to the user of the mobile terminal 300, and a plurality of vehicles 100 that can be dispatched are registered in the merging support server 500 in advance. Hereinafter, the user of the mobile terminal 300 who wants to receive this service will be referred to as "service user".

The operation of the joining support server 500 will be described below with reference to the flowchart of FIG. The merging support server 500 receives a dispatch request from the mobile terminal 300 of the service user (step S501). The dispatch request from the mobile terminal 300 includes at least information about the place and time that the service user wishes to dispatch a vehicle, but may also include other information such as the number of passengers and the vehicle type desired to be dispatched. good. The merging support server 500, which has received the vehicle allocation request, asks for a vehicle that is closest to the service user's request from among the plurality of registered vehicles 100 (step S502).

After that, the merging support server 500 receives a response to the effect that the vehicle can be allocated from any of the vehicles 100 to which the request for vehicle allocation has been made (step S503). Then, the information on the vehicles 100 that can be dispatched is transmitted to the mobile terminal 300 of the service user (step S504). When there are a plurality of vehicles 100 that can be dispatched, information on the plurality of vehicles 100 is transmitted to the mobile terminal 300 of the service user, and the service user selects one of them.

The merging support server 500 receives information on the vehicle 100 selected by the service user and the service user's profile from the service user's portable terminal 300 (step S505). As a result, a pick-up contract is established between the service user and the selected vehicle 100 .

When the contract is established, the merging support server 500 notifies the vehicle 100 selected by the service user of the establishment of the contract and transmits an image of the virtual merging point object (step S506). The confluence support server 500 also notifies the service user's mobile terminal 300 of the establishment of the contract and transmits the image of the virtual confluence point object (step S507).

The operation of the joining support server 500 is now completed. After that, the merging support device 10 of the vehicle 100 performs the operation described in the first embodiment, and the merging support device 10 of the mobile terminal 300 performs the operation described in the third embodiment. support merging with

It is preferable that the same image as the image of the virtual meeting point object is transmitted to the vehicle 100 and the portable terminal 300 with which the contract has been concluded. Alternatively, the confluence support server 500 may operate as an arbitration system for arranging an image to be a virtual confluence point object between the service user and the driver of the vehicle 100 to be dispatched. When the same virtual meeting point object is used in the vehicle 100 and the mobile terminal 300, for example, when the service user joins the vehicle 100, the driver of the vehicle 100 can use the mobile terminal 300 on which the virtual meeting point object is displayed. The images of the virtual meeting point objects can be confirmed by each other by presenting the images to the virtual meeting point objects. As a result, mutual authentication can be performed, which contributes to an improvement in security.

[Modification]
In the first and third embodiments, it is determined whether or not the vehicle 100 has approached the junction (for example, the determination in step S104 in FIG. 5), or whether the user of the mobile terminal 300 has approached the junction (for example, 26) was performed by the merging support device 10 included in the vehicle 100 or the mobile terminal 300. FIG. However, merging support server 500 sequentially acquires the positions of vehicle 100 and mobile terminal 300 and makes determinations thereof, and notifies vehicle 100 or mobile terminal 300 when vehicle 100 or mobile terminal 300 approaches the merging point. You may

<Embodiment 5>
Embodiment 5 shows an example of a pick-up and drop-off system that uses a vehicle 100 equipped with a merging support device 10 to pick up and drop off an unspecified number of passengers having mobile terminals 300 equipped with the merging support device 10 .

In the pickup and drop-off system of the fifth embodiment, one or more vehicles 100 equipped with a merging support device 10 run on a specific patrol route like a patrol bus, and are carried at a plurality of boarding/alighting points along the patrol route. A service for getting on and off unspecified passengers with terminals 300 is provided. Hereinafter, the vehicle 100 that provides this service will be referred to as a "patrol vehicle", and the user of the portable terminal 300 who intends to receive this service will be referred to as a "service user".

The configuration of the pick-up system of Embodiment 5 may be the same as that shown in FIG. Information on the patrol route and each boarding/alighting point is distributed from the joining support server 500 to the patrol vehicle 100 and the mobile terminal 300 of the service user. A plurality of boarding/alighting points are stored as merging points in the merging point storage unit 12 of the merging support device 10 of the patrol vehicle 100 . Similarly, a plurality of boarding/alighting points are stored as merging points in the merging point storage unit 12 of the merging support device 10 of the service user's mobile terminal 300 . Therefore, the merging support device 10 of the patrolling vehicle 100 and the mobile terminal 300 provides the driver of the patrolling vehicle 100 and service users with a virtual world in which virtual merging point objects exist at positions corresponding to boarding/alighting points. Hereinafter, the virtual meeting point object indicating the position of the boarding/alighting point will be referred to as a "virtual boarding/alighting point object".

The driver of the patrol vehicle 100 and the service user can recognize the position of the boarding/alighting point by confirming the position of the virtual boarding/alighting point object in the virtual world provided by the merging support device 10 . Therefore, in the pick-up system, there is no need to actually install signposts on the patrol route. Therefore, in the pick-up system, it is possible to flexibly change the place of getting on and off and the patrol route.

For example, the merging support server 500 collects traffic information, weather information, distribution information of service users, etc., and, for example, changes the patrol route so as to avoid congested sections, and protects the boarding and alighting points from the weather during bad weather. By changing the location or adding areas with many service users to the patrol route, it is possible to provide a highly convenient pick-up service. Also, the merging support server 500 may change the image of the virtual boarding/alighting point object as necessary.

In addition, the merging support server 500 may receive a request from the service user for a desired boarding/alighting point, and determine the boarding/alighting point based on the request. For example, it is assumed that a place where a plurality of service users can reach within 10 minutes is defined as a boarding/alighting point. Furthermore, the merging support server 500 may temporarily increase the number of boarding/alighting points in response to a request from a service user who has difficulty walking, such as an elderly person or an injured person.

Information about changes in the images of the patrol route, boarding/alighting point, and virtual boarding/alighting point objects is distributed from the merging support server 500 to the patrol vehicle 100 and the mobile terminal 300 .

In addition, within the scope of the invention, each embodiment can be freely combined, and each embodiment can be appropriately modified or omitted.

While the present invention has been described in detail, the foregoing description is, in all its aspects, illustrative and not intended to limit the invention. It is understood that numerous variations not illustrated can be envisioned without departing from the scope of the invention.

10 confluence support device 11 communication unit 12 confluence point storage unit 13 virtual object storage unit 14 position information acquisition unit 15 map information acquisition unit 16 display control unit 21 head-up display 22 display device 23 operation input device, 24 positioning device, 25 communication device, 26 map information storage device, 27 photographing device, 30 windshield, MP meeting point, 100 user's vehicle, 101 meeting partner, 200, 400 virtual meeting point object, 201 to 206, 401 404 virtual object, 300 mobile terminal, 301 mobile terminal user, 302 merging partner vehicle, 500 merging support server.

Claims (22)

a meeting point storage unit that stores a meeting point between the user and others;
a virtual object storage unit for storing a virtual meeting point object, which is an image indicating the position of the meeting point and has a display mode simulating a substantial feature indicating the stopping position of the moving object;
a location information acquisition unit that acquires the location of the user;
Based on the position of the user and the position of the meeting point, the virtual meeting point object is the meeting point in an image of the real world seen by the user through a transparent screen or the image of the real world around the user displayed on the screen. a display control unit for displaying at a position corresponding to
In a merging support device comprising
The image of the virtual meeting point object is selected from a plurality of different images, and when the meeting support device is used by both the user and the other person, The virtual meeting point object is used in common with the meeting support device,
Merging support device. a meeting point storage unit that stores a meeting point between the user and others;
a virtual object storage unit for storing a virtual meeting point object, which is an image indicating the position of the meeting point and has a display mode simulating a substantial feature indicating the stopping position of the moving object;
a location information acquisition unit that acquires the location of the user;
Based on the position of the user and the position of the meeting point, the virtual meeting point object is the meeting point in an image of the real world seen by the user through a transparent screen or the image of the real world around the user displayed on the screen. a display control unit for displaying at a position corresponding to
with
The virtual object storage unit acquires an image to be the virtual meeting point object specified by the other person from an arbitration system that determines an image to be the virtual meeting point object between the user and the other person.
Merging support device. a meeting point storage unit that stores a meeting point between the user and others;
a virtual object storage unit for storing a virtual meeting point object, which is an image indicating the position of the meeting point and has a display mode simulating a substantial feature indicating the stopping position of the moving object;
a location information acquisition unit that acquires the location of the user;
Based on the position of the user and the position of the meeting point, the virtual meeting point object is the meeting point in an image of the real world seen by the user through a transparent screen or the image of the real world around the user displayed on the screen. a display control unit for displaying at a position corresponding to
with
the user is a vehicle driver,
The display control unit converts the virtual merging point object into a real scene seen by the driver through a transparent screen, or an image of a real scene around the vehicle displayed on the screen of a display device installed in the vehicle. Display at the position corresponding to the point,
The vehicle is a vehicle that travels on a patrol route that goes around a plurality of predetermined meeting points, and allows the unspecified other person to get on and off at the meeting point. A vehicle for which the meeting point is determined based on a request;
Merging support device. a meeting point storage unit that stores a meeting point between the user and others;
a virtual object storage unit for storing a virtual meeting point object, which is an image indicating the position of the meeting point and has a display mode simulating a substantial feature indicating the stopping position of the moving object;
a location information acquisition unit that acquires the location of the user;
Based on the position of the user and the position of the meeting point, the virtual meeting point object is the meeting point in an image of the real world seen by the user through a transparent screen or the image of the real world around the user displayed on the screen. a display control unit for displaying at a position corresponding to
with
one of said user and said other is a driver of a vehicle and the other is a passenger of said vehicle;
The vehicle is a vehicle that travels on a patrol route around a plurality of predetermined confluence points and allows unspecified passengers to get on and off at the confluence points,
The virtual junction object is an image simulating a signpost installed at a public transportation stop,
Merging support device. the user is a vehicle driver,
The display control unit converts the virtual merging point object into a real scene seen by the driver through a transparent screen, or an image of a real scene around the vehicle displayed on the screen of a display device installed in the vehicle. Display at the position corresponding to the point,
The merging support device according to claim 1 or 2 . The merging support device is mounted on a mobile terminal,
The display control unit displays the virtual meeting point object at a position corresponding to the meeting point in an image of the actual surroundings of the user displayed on the screen of the mobile terminal.
The merging support device according to claim 1 or 2 . The other person is a driverless vehicle that runs by remote control or automatic operation,
The merging support device according to claim 6 . The meeting point storage unit acquires information on the meeting point from an arbitration system that determines the location of the meeting point between the user and the other person.
The merging support device according to any one of claims 1 to 4 . The display control unit displays an image simulating the user or the other person arriving at the meeting point together with the virtual meeting point object.
The merging support device according to any one of claims 1 to 4 . The display control unit displays an image showing a stop position when the vehicle arrives at the junction together with the virtual junction object.
The merging support device according to claim 5 . further comprising a communication unit that obtains information on the position of the other person through communication;
The display control unit further displays an image indicating the position of the other person at a position corresponding to the position of the other person in the actual scene or the image of the actual scene.
The merging support device according to any one of claims 1 to 4 . further comprising a communication unit that acquires information on a planned route through which the other party will arrive at the meeting point through communication;
The display control unit further displays an image showing the planned passage route of the other person at a position corresponding to the position of the planned passage route of the other person in the actual scene or the image of the actual scene.
The merging support device according to any one of claims 1 to 4 . When the position of the meeting point is out of the range of the actual scene seen by the user through the transparent screen or out of the range of the image of the actual view around the user, an image indicating the direction in which the meeting point exists is displayed. do,
The merging support device according to any one of claims 1 to 4 . The patrol route and the plurality of meeting points are changed based on any one or more of distribution information, traffic information, and weather information of the other person using the vehicle,
5. The merging support device according to claim 3 or 4 . The mobile terminal is capable of displaying a map indicating the location of the user and the location of the meeting point, and displaying an image of the actual scene on which the virtual meeting point object is superimposed. , changing the area for displaying the map and the area for displaying the image of the actual scene on the screen;
The merging support device according to claim 6 . the vehicle is a vehicle remotely operated by the driver at a location remote from the vehicle;
The position information acquisition unit acquires the position of the vehicle as the position of the user,
The display control unit displays the virtual junction object at a position corresponding to the junction in the image of the actual scenery around the vehicle displayed on the screen.
The merging support device according to claim 3 or 5 . an image of the real scene containing the virtual junction object can be posed;
The merging support device according to any one of claims 1 to 4 . The image of the actual scene is a surround image obtained by synthesizing a plurality of images taken in different shooting directions,
The merging support device according to any one of claims 1 to 4 . a meeting point storage unit of the meeting support device stores a meeting point between the user and the other person;
The virtual object storage unit of the merging support device stores a virtual merging point object which is an image indicating the position of the merging point and has a display mode simulating a substantial feature indicating the stop position of the moving body,
a location information acquiring unit of the joining support device acquires the location of the user;
The display control unit of the meeting support device displays the virtual meeting point object on the basis of the position of the user and the position of the meeting point. displayed at a position corresponding to the confluence point in the image of the actual scene surrounding the
The image of the virtual meeting point object is selected from a plurality of different images, and when the meeting support device is used by both the user and the other person, The virtual meeting point object is used in common with the meeting support device,
Merge support method. a meeting point storage unit of the meeting support device stores a meeting point between the user and the other person;
The virtual object storage unit of the merging support device stores a virtual merging point object which is an image indicating the position of the merging point and has a display mode simulating a substantial feature indicating the stop position of the moving body,
a location information acquiring unit of the joining support device acquires the location of the user;
The display control unit of the meeting support device displays the virtual meeting point object on the basis of the position of the user and the position of the meeting point. displayed at a position corresponding to the confluence point in the image of the actual scene surrounding the
The virtual object storage unit acquires an image to be the virtual meeting point object specified by the other person from an arbitration system that determines an image to be the virtual meeting point object between the user and the other person.
Merge support method. a meeting point storage unit of the meeting support device stores a meeting point between the user and the other person;
The virtual object storage unit of the merging support device stores a virtual merging point object which is an image indicating the position of the merging point and has a display mode simulating a substantial feature indicating the stop position of the moving body,
a location information acquiring unit of the joining support device acquires the location of the user;
The display control unit of the meeting support device displays the virtual meeting point object on the basis of the position of the user and the position of the meeting point. displayed at a position corresponding to the confluence point in the image of the actual scene surrounding the
the user is a vehicle driver,
The display control unit converts the virtual merging point object into a real scene seen by the driver through a transparent screen, or an image of a real scene around the vehicle displayed on the screen of a display device installed in the vehicle. Display at the position corresponding to the point,
The vehicle is a vehicle that travels on a patrol route that goes around a plurality of predetermined meeting points, and allows the unspecified other person to get on and off at the meeting point. A vehicle for which the meeting point is determined based on a request;
Merge support method. a meeting point storage unit of the meeting support device stores a meeting point between the user and the other person;
The virtual object storage unit of the merging support device stores a virtual merging point object which is an image indicating the position of the merging point and has a display mode simulating a substantial feature indicating the stop position of the moving body,
a location information acquiring unit of the joining support device acquires the location of the user;
The display control unit of the meeting support device displays the virtual meeting point object on the basis of the position of the user and the position of the meeting point. displayed at a position corresponding to the confluence point in the image of the actual scene surrounding the
one of said user and said other is a driver of a vehicle and the other is a passenger of said vehicle;
The vehicle is a vehicle that travels on a patrol route around a plurality of predetermined confluence points and allows unspecified passengers to get on and off at the confluence points,
The virtual junction object is an image simulating a signpost installed at a public transportation stop,
Merge support method.

Images