WO2020230262A1

WO2020230262A1 - Image display device, image display system, and image display method

Info

Publication number: WO2020230262A1
Application number: PCT/JP2019/019112
Authority: WO
Inventors: 奥山　宣隆; 吉澤　和彦; 橋本　康宣; 眞弓中出
Original assignee: マクセル株式会社
Priority date: 2019-05-14
Filing date: 2019-05-14
Publication date: 2020-11-19
Also published as: JP7295224B2; JPWO2020230262A1; JP2023130347A

Abstract

An image display device 2 that is worn by a user and displays an image of a separate space differing from a real space in which the user is actually located, comprising: a position detection unit 7a for detecting the position of the user; a communication interface 9a for acquiring course information from the outside that includes the image data of the separate space and an imaged position; a memory 8a for preserving the image data and course data acquired by the communication interface; a display unit 12a for displaying the image of the separate space preserved in the memory; and a control unit 10a for controlling the display of an image of the separate space on the display unit. The control unit 10a maps a course 26a composed of a plurality of separate space points to a course 27a composed of a plurality of real space points on the basis of separate space course information and displays images 24a-24f of mapped separate space points in accordance with the user's position when the user moves along the real space course 27a.

Description

Image display device, image display system and image display method

The present invention relates to an image display device, an image display system, and an image display method for displaying an image in a different space whose position is different from the real space in which the user is actually located.

VR (Virtual Reality) technology that allows users to create and display image content of a virtual world on a computer, etc., and experience the feeling of being in that virtual world, or created on a computer in a real landscape AR (Augmented Reality) technology that superimposes and displays image contents is known, and has been put into practical use by terminals such as head-mounted displays (hereinafter referred to as HMDs) and smartphones.

As a game to which the above technology is applied, Patent Document 1 discloses an entertainment system in which a player actually moves in the real world and acquires actual information from an existing store or facility by using a mobile terminal. In this game, "a map of the area around the user is displayed on the user's mobile terminal, and the user selects an event spot. When the event spot is selected, a quiz or a mini game is presented to the user. The user relies on the map. Go to the actual event spot, get hints, play quizzes and mini games, and clear these to earn points. "

JP-A-2002-49681

According to VR technology and AR technology, while the user is in the space where he / she is actually located (hereinafter referred to as “real space”), he / she sees an image of a space (hereinafter referred to as “another space”) at a position different from the real space. By doing so, you can experience as if you were in another space.

As an image of another space, for example, when looking at the scenery of a place of memory that the user visited in the past or the scenery of a place of interest that the user has not visited yet, simply playing back the image of that place gives a sense of reality. I can't experience the place. In order to enhance the sense of presence, the real space and another space are fused, for example, the user actually moves to another place by seeing the image of another place while actually moving to the current place. You can experience other places as if you were doing it. Alternatively, when a user moves (walks or runs) a fixed course in real space, it may be boring just to see the usual fixed scenery in real space. In such a case, if an image of another space can be seen as the user moves, it is possible to give a fresh taste to walking and running and eliminate the feeling of boredom of the user.

In Patent Document 1, a user plays a game in virtual space using a mobile terminal, but the configuration is such that a user moves in real space and obtains a hint of a quiz by relying on a map displayed on the mobile terminal. It can be regarded as a game that combines space and real space. However, the configuration described in Patent Document 1 is not a fusion of a real space and another space, that is, a combination of a space actually located by the user and a space at a different position. Therefore, the user does not experience the image of another space with a sense of reality, and does not eliminate the feeling of boredom when moving in the real space.

An object of the present invention is to provide an image display device, an image display system, and an image display method that enable a user to view an image in another space with a more realistic feeling or eliminate the feeling of boredom when moving in a real space. It is in.

The present invention is an image display device that is worn by the user and displays an image in a different space whose position is different from the actual space in which the user is actually located. The position detection unit that detects the position of the user and the front direction of the user. The orientation / orientation detector that detects the orientation and orientation (the orientation angle and elevation angle in the user's line-of-sight direction), the communication interface that acquires course information including image data and shooting position in another space from the outside, and the communication interface. A memory for storing image data and course information, a display unit for displaying an image in another space stored in the memory, an audio output unit for outputting audio, and a display unit for controlling the display of an image in another space. It includes a control unit. Based on the course information in another space, the control unit associates a course consisting of multiple points in another space with a course consisting of multiple points in real space, and when the user moves the course in real space, the user It is characterized in that an image of a point in another space associated with the image is displayed according to the position.

According to the present invention, when a user moves on a course in a real space, a landscape at a corresponding position in another space can be viewed as a still image or a moving image. Therefore, there is an effect of experiencing the feeling of being in a different space as if it were actually moving, or eliminating the feeling of boredom when moving in a real space.

The figure which shows the structure of the image display system 1 which concerns on Example 1. FIG. The figure which shows the appearance of the user who attached the image display device 2 (head-mounted display 19). The figure which shows the example of the display screen of the image display device 2. The figure which shows the example which acquires the image of each point on the course of another space. The figure which shows the example which displays the course setting of a real space and the image taken in another space. The figure which shows the processing process which registers the course information of another space. The figure which shows the content of the course information registered by the process of FIG. 5A. The figure which shows the processing process which moves the course of a real space and displays the registered image. The figure which shows the example of the course of another space with the altitude difference which concerns on Example 2. The figure which shows the example of the flat course with no difference in altitude in the real space. The figure which shows the course of another space which concerns on Example 3. The figure which shows the course which goes around a track as a course of a real space. The figure which shows the user who attached the image display device 2 (smartphone 40). The figure which shows the example of the display screen of the smartphone 40. The figure which shows the example of the display screen of the smartphone 40. The figure which shows the example of the display screen of the smartphone 40.

Hereinafter, examples of the present invention will be described in detail with reference to the drawings. In principle, the same parts of the drawings are designated by the same reference numerals, and the repeated description thereof will be omitted.

In the first embodiment, an image display system 1 that displays an image taken in another space when the user moves to a corresponding position in the real space will be described.

[Image display system]
FIG. 1 is a diagram showing a configuration of an image display system 1 according to a first embodiment. In the image display system 1, the image display device 2, the image pickup device 3, and the Web server 4 are connected via a communication network 5. The communication network 5 includes a wireless communication network and the Internet. Here, a case where a head-mounted display (HMD) is used as the image display device 2 will be described. Hereinafter, the internal configuration of each device will be described.

The image display device 2 includes an image sensor 6a, a position detection unit 7a, a memory 8a, a communication interface 9a, a control unit 10a, a direction / orientation detection unit 11a, a display unit 12a, and an audio output unit 13. The image sensor 6a includes an optical lens (not shown) and is a portion for capturing a still image or a moving image, and the captured image data is stored in the memory 8a. The position detection unit 7a measures latitude, longitude, and altitude as position information using, for example, GPS (Global Positioning System). The azimuth / attitude detection unit 11a includes a geomagnetic sensor and a gyro sensor (not shown), and measures an azimuth angle (an angle measured clockwise from the north of the azimuth) and an elevation angle (an angle measured upward from a horizontal plane). The display unit 12a captures or generates an image taken in another space, a composite image with the image, a course, or the like, and displays the image. The communication interface 9a is connected to the communication network 5 and transmits / receives data to / from the Web server 4 and the image pickup device 3. The voice output unit 13 includes a speaker (not shown), generates a voice signal by synthesis or the like, and outputs the voice. The control unit 10a includes a CPU and controls all the functional blocks of the image display device 2. A display control application, a Web search application, a map application, and the like (not shown) are stored in the memory 8a.

Here, the image display device 2 is provided with the image pickup device 6a and is configured to be able to also serve as the photographing function of the image pickup device 3 described below, but when it is used only as the image display function, the image pickup element 6a may be omitted.

The image pickup device 3 includes an image pickup element 6b, a position detection unit 7b, a memory 8b, a communication interface 9b, a control unit 10b, a direction / orientation detection unit 11b, and a display unit 12b. The description of the functional block having the same name as the image display device 2 described above will be omitted because of the same configuration and function.

The image pickup device 3 is for taking a picture of another space, and the picture pickup element 6 takes a still image or a moving image, and stores these image data in the memory 8b. In addition, the position detection unit 7b and the orientation / attitude detection unit 11b acquire information (position, orientation, posture, etc.) at the time of shooting and store it in the memory 8b as attached information (metadata) of the image data. At that time, by pairing the image data and the attached information and saving the image data as one image file, the association becomes easy. The saved image file is transmitted to the Web server 4 and the image display device 2 via the communication interface 9b and the communication network 5. Alternatively, although not shown, it can be passed to the Web server 4 or the image display device 2 via a removable recording medium such as an SD card. The display unit 12b reproduces and displays an image at the time of shooting and stored data in the memory 8b.

The Web server 4 has a communication interface 9c, a control unit 10c, and a memory 8c, and an image file and a Web page document are stored in the memory 8c. The image file including the image data captured by the image pickup apparatus 3 and its accessory information is received via the communication interface 9c and taken into the memory 8c. The control unit 10c generates course information based on the acquired data and publishes it on the net as a Web page document. As a method of disclosing the course information, a method of directly indicating the latitude, longitude, and altitude as the point information on the course, a method of designating the shooting location as a point on the course from the attached information of the image file, and a combination thereof may be used. .. The above course is referred to as a course in another space in this embodiment. By publishing the course in another space as a Web page document in this way, the image data in another space can be used not only by the photographer but also by other interested users.

The outline of the operation of the image display system 1 of this embodiment is as follows. First, the image pickup device 3 captures an image at each point on the course in another space, and the image data and the attached information of the capture point are transmitted to the Web server 4 and registered. This registration process may be performed by the user who owns the image display device 2, or may be performed by another user. When the user himself / herself registers, the image sensor 6a in the image display device 2 can also be used, in which case the image sensor 3 becomes unnecessary. Next, when the user wears the image display device 2 and moves the course in the real space, the image of another space registered in the Web server 4 is displayed on the image display device 2 according to the moving position. At that time, when the user specifies a desired position (start point and end point) to move in the real space on the map, the course in the real space whose distance and direction are approximated by a predetermined geometric relationship with respect to the course in another space is obtained. The determined and registered image of another space is displayed at the moving point of the real space corresponding to the shooting point of another space.

[Image display device]
2A and 2B are diagrams showing an example of the appearance and display screen of the image display device 2. The image display device 2 is the case of the transmissive HMD19. FIG. 2A shows a state in which the user 20 wears the transmissive HMD 19 on the head. The display unit 12a of the transmissive HMD 19 has a configuration in which a semitransparent prism 22 is arranged in front of one eye (right eye) 21a, and the prism 22 is irradiated with a reproduced image of a side display element 23. Further, an image sensor 6a is arranged on the front surface of the transmissive HMD 19, and a control unit (drive unit) 10a is arranged on the upper surface.

FIG. 2B shows an example of a display screen (display unit 12a) when the user 20 looks at the prism 22. A landscape image 25 (indicated by a broken line) in front of the real space can be seen by superimposing a reproduced image 24 (indicated by a solid line) in another space by the display element 23 through the prism 22. When displaying the reproduced image 24, the display position (display direction) is, for example, the center of the display screen 12a. Furthermore, by comparing the orientation information of the image display device 2 and the image pickup device 3 with the direction /

orientation detection units

11a and 11b, the image pickup device 3 is photographed in a space different from the user's line-of-sight direction in the real space. It is also possible to display the corresponding reproduced image 24 when the orientation of the direction matches.

In the transmissive HMD19 of this example, the prism 22 and the display element 23 are arranged in one eye (right eye) 21a, but they can also be arranged in the other eye (left eye) 21b. .. Further, the transmittance of the prism 22 is appropriately set according to whether it is a monocular type or a binocular type so that both the landscape image 25 and the reproduced image 24 can be visually recognized in a well-balanced manner.

[Course in another space]
FIG. 3 is a diagram showing an example of acquiring an image of each point on a course in another space. The photographer (the user of the image display device 2 or another user) is moving around the course 26a (indicated by the broken line arrow) in another space, and the surroundings that should be recommended at each point on the course by the image pickup device 3. Take a picture of the scenery (a scenery that will be a memorable one for the user). In addition, the route that can be moved without duplication to shoot each recommended item is named the recommended course.

Specifically, at the starting point P111 of the recommended course 26a, the gate 28 as a mark is photographed, and at the point P112, the building 29 is photographed. At the branch point P113, a landscape including a branching road is photographed. At point P114, the tower 30 is photographed, and at the end point P115 of the course, the torii 31 is photographed. If there is another recommended item at a position outside the recommended course 26a (points P111 to P115), for example, the stone monument 32 is photographed at point P116. The image data taken at these points and the attached information (position information, etc.) are stored in the memory 8b as an image file and transmitted to the Web server 4. The data at each point is saved with a distinction as to whether or not it exists on the recommended course 26a.

[Real space course]
FIG. 4 is a diagram showing an example of displaying an image taken in a different space from the course setting in the real space. A course 27a (indicated by a solid arrow) is set in the real space, and points P211 to P215 on the course are shown. The outside of the course is indicated by

broken lines

39a and 39b. The procedure for setting the course will be explained. A user who intends to move in the real space displays a map of the real space on the display unit 12a of the image display device 2 using a map application or the like, and designates a start point P211 and an end point P215 of the course. As a result, the start point point P211 is associated with the start point point P111 in another space shown in FIG. 3, and the end point point P215 is associated with the point P115 in another space. Next, a course that can be moved in the real space is determined so as to have a predetermined geometric relationship (deviation of orientation and scale) approximately with the course of another space. As a matter of course, the conditions of roads and buildings are different between the real space and another space, so the shape of the course that the user can move in the real space does not exactly match the shape of the course in the other space. Therefore, set directions and intermediate points that have similar geometric relationships. Details of the course setting will be described later in PAD. As a result, the points P212 to P214 of the midpoint of the real space are determined so as to correspond to the points P112 to P114 of the midpoint of the course of another space shown in FIG. 3, and the course 27a of the real space connecting these is set. Will be done.

When the user moves on the course 27a in the real space and approaches each set point P211 to P215, the captured images 24a to 24e of the corresponding recommended objects (structures and scenery of the branch point) taken in another space are images. It is displayed on the display unit 12a of the display device 2. The orientation / posture detection unit 11a detects the user's line-of-sight direction, and when the direction of the registered recommended items is in the user's line-of-sight direction, these recommended items are displayed to further enhance the sense of presence. it can. Of these, point P213 corresponds to point P113, which is a branch point in another space. When approaching the branch point P213, not only the corresponding captured image 24c but also a branch guide map in which a pre-registered arrow 24c'indicating the branch direction is image-combined is displayed.

The intermediate points P112 and P114 on the separate space course 26a are not branch points, but at the corresponding points P212 and P214 of the real space course 27a, the course must be changed (turn right) at the immediately preceding intersection. Therefore, the position detection unit 7a of the image display device 2 detects that the intersection has been reached, and the voice output unit 13 gives a voice instruction to the user to turn right. As a result, the user can move without departing from the course 27a set in the real space. When the user moves the courses of the

broken lines

39a and 39b that deviate from the recommended course, the image 24f of the stone monument is displayed at the position of the point P216.

As a result, when the user walks through the course 27a in the real space, the landscape image of another space, which is a place different from the real space, can be seen according to the moving position, and the different space can be experienced with a sense of reality. Furthermore, at the branch point, the branch guide map combined with the landscape image can be seen, which has the effect of being able to prepare for the course at the branch point.

[Processing PAD and course information]
FIG. 5A is a diagram showing a processing process for registering course information in another space, and is shown by a PAD (Problem Analysis Diagram) in which the processing structure is layered. The control unit 10b of the image pickup apparatus 3 reads the captured image data and its accessory information from the memory 8b and transmits the captured image data to the Web server 4 via the communication interface 9b. Since a plurality of courses are registered in the Web server 4, a course number is assigned to each of them.

In step S11, the image data of another space taken by the imaging device 3 and its attached information (position information, etc.) are registered as an image file. At this time, an image file that is not on the recommended course may be included (for example, the data of the stone monument 32 in FIG. 3).

In step S12, the following processing is repeated for each point (P111 to P115) on the recommended course. In step S12a, the image files to be registered for the designated points (images 28 to 31 in P111 to P115 in FIG. 3) are selected. In step S12b, point information is set from the attached information of the selected image file. At this time, if there is shooting position information, it is used as point information, and if there is no shooting position information, it is specified from a map application or the like and the point information is registered. In addition, related figures / symbols to be displayed superimposed on the image such as the arrow of the branch guide are arbitrarily registered as one of the attached information. In the examples of FIGS. 3 and 4, the arrow symbol 24c'which serves as a route guide at the branch point P213 (P113) is registered as attached information.

FIG. 5B is a diagram showing the contents of the course information registered by the process of FIG. 5A. The course information 44 includes a course number, course point information, and image file information. The course number is a number for identifying the course. The course point information is position information (latitude, longitude, altitude) of points from the start point to the end point of the course. Altitude can be omitted if it is not covered as a function of the position detector. The image file information is a list (or link) of image files 45 of images taken on or around the course. Further, the image file 45 referred to from the above image file information includes image data 16 and its accessory information 17 (generally also referred to as metadata). The attached information 17 includes a shooting position, related figures / symbols, and the like.

FIG. 6 is a diagram showing a processing process of moving a course in real space and displaying a registered image, and shows the processing structure as a layered PAD. The control unit 10a of the image display device 2 takes in image data and its accessory information from the Web server 4 via the communication interface 9a, sets a course in real space, and displays the corresponding image according to the movement position.

Step S21 specifies a course in another space in which the image to be displayed is stored. When the user inputs the course number (for example, No. 1) in step S21a, the corresponding course information and the image file are fetched from the Web server 4 in step S21b.

In step S22, the user specifies a start point and a goal point in the real space. As a method of designating the position, there are a method of designating from the map application and a method of designating the current position of the user from the position detection unit 7a of the image display device 2.

In step S23, the following processing is repeated for each section defined between adjacent points on the course. That is, the end point (target point) of the section is set at the start point of the section, and when the user reaches the end point of the section, the registered image is displayed and the end point of the next section is set.

In step S23a, in order to set a course in a real space that approximates a course in another space, the end point of the section is determined based on the geometric relationship between the two courses. Therefore, in different space and real space, compare the linear orientation and distance when the end point Pg (goal point) of the course is expected from the start point Ps (start point at the first time) of the section, and compare the orientation difference and scale between the two. Ask. For example, the linear direction from the section start point Ps to the goal point Pg in another space is in the east direction, and the distance is 5 km. On the other hand, the linear direction from the section start point Ps'to the goal point Pg'in the real space is in the north direction, and the distance is 2 km. In this case, the directional difference in the real space with respect to another space is rotated 90 degrees counterclockwise, and the scale is 0.4 times. Using these geometric relationships (direction difference and scale), the section end point Pe'in the real space corresponding to the section end point Pe in another space is obtained. However, the position of the section end point Pe'obtained in this way is not necessarily a movable place in the real space, and may overlap with, for example, an area where a building exists. Therefore, with reference to the map application in the real space, a movable position (for example, on the road) close to the position Pe'is obtained and set as the target section end point Pe ”, and the section start point Ps is referred to with reference to the map application. Determine the route from'to section end point Pe'and set the course in real space.

In step S23b, it is determined whether or not the user has approached the section end point Pe ". The user's movement position in the real space is detected by the position detection unit 7a. Then, the movement position is near the section end point Pe", for example, the section. When it reaches within a range of 10% of the section distance from the end point Pe ", it is determined that the section end point Pe" has been approached.

As a result of the determination in step S23b, when the end point of the section is approached, the registered image at the corresponding point in another space is displayed on the display unit 12a in step S23c. After that, in step S23d, it is repeatedly determined whether or not the section end point Pe ”on the course in the real space has been reached. If it is determined that the section ends, that position is set as the start point of the next section in the real space in step S23. Return to.

Note that, as a condition for determining the end of a section in step S23d, if there is a branch point on the map application and the destination from the branch point is not a dead end, the branch point is regarded as the end point of the section. Point P213 in FIG. 4 corresponds to this case. Further, other conditions for determining the end of the section include a case where the section distance in the real space reaches a predetermined distance even if it is not a branch point, or a case where the point is a point specified by the user. Points P212 and P214 in FIG. 4 correspond to this case.

In addition, in the automatic creation of the course 27a in the real space, a simulated run may be performed on the map application. Further, in order to create a real space course 27a that suits the user's preference from the course 26a in another space, the real space course automatically created by the process of FIG. 6 is created before or during the movement. , You may fix it manually. For example, when arriving at point P213, a round-trip course 39a to point P216 outside the automatically generated course 27a may be added.

Further, when the directional difference between the traveling direction on the course 26a in another space and the traveling direction on the course 27a in the real space becomes equal to or more than a predetermined value, the voice output unit 13 notifies by voice or a message is sent to the display unit 12a. It may be displayed. As a result, the user can move the course while being aware that the difference between the course in another space and the course in the real space is large due to the geographical restrictions of the real space.

Furthermore, when it is detected that the position of the user deviates from the course 27a in the real space by a predetermined value or more, the voice output unit 13 emits an alarm sound or displays a warning message on the display unit 12a. You may. As a result, the user can move the course 27a in the real space without any mistake.

In the above description, the image captured by the imaging device 3 is registered in the Web server 4, and the image display device 2 is configured to capture and display the image registered from the Web server 4. On the other hand, when the user himself / herself shoots / displays, the image display device 2 can shoot / save. Therefore, the imaging device 3 and the Web server 4 can be omitted.

According to this embodiment, when the user moves the course in the real space, the scenery at the corresponding position in the other space can be seen as a still image or a moving image, so that it is as if the user is actually moving in the other space. You can experience it with a sense of reality. In addition, the course in the real space where the user moves is not only similar to the course in another space, but also flexibly set according to the geographical conditions such as the road in the real space, so that it is highly practical. It has become.

In the second embodiment, the setting of the course in the real space will be described when there is a difference in altitude between the courses in another space and the course in the real space is flat.

FIG. 7 is a diagram showing an example of a course in another space with different altitudes. The other space has undulations, and the altitude difference is represented by contour lines 34 (indicated by dotted lines), and the altitude [m] is described in each contour line. Course 26b (indicated by a solid arrow) in another space is a course with an altitude difference from the start point point P121 to the end point point P126. Point P122 has the highest altitude and point P123 has the lowest altitude. Points P124 and P125 are highlands. Therefore, the walking speed decreases in the uphill section (points P121 to P122, points P123 to P124). On the contrary, the walking speed increases in the downhill section (points P122 to P123, points P125 to P126). There is a hut 33 at the point P124, and the photographed image is registered. The same applies to the object to be photographed at other points, but the description thereof is omitted here.

The image data taken at each point is registered in the Web server 4 as in the case of the first embodiment, but as shown in the attached information 17 of FIG. 5B, the information of the shooting position obtained from the position detection unit 7b at the time of shooting. The altitude is added to. Altitude information can also be measured by altitude sensors. Alternatively, it is also possible to read the altitude value of the position from the map information.

FIG. 8 shows an example of a flat course 27b with no difference in altitude in the real space, and is associated with the course 26b in another space of FIG. Since there is no difference in altitude in the real space, the notation of contour lines is omitted. The course 27b in the real space is from the start point point P221 to the end point point P226, and the orientation of each section between adjacent points corresponds to the orientation of each section of the course 26b in another space with a predetermined orientation difference. .. However, the distance of each section in the real space is set in consideration of the fact that the walking speed changes due to the difference in altitude in another space. That is, the position information and the altitude information of each point of the course 26b in another space are acquired from the Web server 4, and the walking time of each section is assumed. Then, an intermediate point is set on the course 27b so that the walking time for each section of the course 27b in the real space corresponds to the walking time for each section of the course 26b in another space at a predetermined scale.

Specifically, the section corresponding to the uphill (points P221 to P222, points P223 to P224) increases the distance, and the section corresponding to the downhill (points P222 to P223, points P225 to P226) decreases the distance. And set. When the user approaches a point in each section (for example, point P224), the position detection unit 7a detects it, and similarly to the first embodiment, the registered image of another space (for example, 24 g) is displayed on the display unit 12a.

As described above, in the second embodiment, the required time is set by correcting the distance between the points in the real space so as to cancel the difference in the movement time of the user due to the altitude difference in consideration of the altitude difference in another space. In terms of fatigue and fatigue, you can move in the real space with a feeling closer to the course in another space.

Here, it is assumed that the course in another space has an altitude difference and the course in the real space is flat, but conversely, the course in another space is flat and the course in the real space has an altitude difference, or Even when the altitude difference between the two courses is different, the real space course can be set in consideration of the walking time.

In the third embodiment, an image display device 2 capable of running while looking at an image of a course in another space when running a circuit course in a real space will be described. Here, a case where the smartphone 40 is used as the image display device 2 will be described.

FIG. 9 is a diagram showing a course 26c in another space. Here, assuming a course used for road racing, point P131 is started and point P135 is the goal. A building 35 exists at the intermediate point P132, and a photographed image thereof is registered in the Web server 4.

FIG. 10 is a diagram showing a course 27c in real space. Here, it is a course that goes around a track for athletics. The shape of the course 27c in the real space has nothing to do with the shape of the course 26c in another space, and points are set at positions corresponding to the mileage. For example, the point P231 on the start line 38 is started, and the same point P235 that makes two laps of the track is set as the goal. The scale is calculated from the ratio of the total lengths of the course 26c in another space and the course 27c in the real space, and the passing point in the middle of the course 27c in the real space is the distance from the start of the course 26c in the other space to the passing point multiplied by the scale. It is automatically determined as the position of the distance. In this method of calculating the distance, it is necessary to measure the mileage of the user on the course 27c in the real space by the image display device 2 itself. As an alternative method, there is a method of installing a beacon at a passing point position on the course 27c in the real space and detecting that the passing point has been reached by the received signal from the beacon.

Here, the latter method using a beacon will be described. Assuming a race with multiple people, beacons 36a to 36d were placed at the start / goal and each passing point on the course 27c in the real space so that the position information could be measured accurately. The receivable range in which the signal from the beacon can be received above a certain threshold value is indicated by the broken line 37. The user starts the point P231 (beacon 36a), proceeds counterclockwise, passes the point P232 (beacon 36c), and then makes one round at the same point as the start (beacon 36a). Further, it passes through the point P233 (beacon 36b) and the point P234 (beacon 36d), and makes two laps at the point P235 (beacon 36a) drawn by the goal line 38 to reach the goal.

FIG. 11 is a diagram showing a state of a user wearing an image display device. Here, the smartphone 40, which is the image display device 2, is attached to the body of the user 20 by using the fixing device 41 so that the display screen can be seen hands-free. The smartphone 40 can detect the signals emitted by the beacons 36a to 36d and know the current position by using the Bluetooth (registered trademark) function as the position detection unit 7a. When the track is lapped a plurality of times as in this example, it is possible to determine which of the points P231 to P235 has been reached by combining with the number of times the beacon signal is received (that is, the number of laps).

On the smartphone 40, install an application that associates points P231 to P235 on the course 27c in the real space with points P131 to P135 on the course 26c in another space. Further, the image data on the course 26c in another space is fetched in advance from the Web server 4 via the communication network 5, or is in a state where it can be fetched from the Web server 4 at any time in response to a request from the application.

The case where a large number of users race on the course 27c in the real space will be specifically described. After launching the application of the smartphone 40, the first group of athletes waits inside the receivable range 37 of the signal from the beacon 36a in front of the point P231 of the starting point, and the race starts at 00 seconds per minute. To do. After this start, the second group of players also stands by in front of point P231. According to this method, it is possible to start a wave by dividing into groups of several players. The starter may simply signal the start according to the posting time of 00 seconds per minute or just watch over the athletes from flying.

When each player passes the points P233 and P234 in the real space and approaches the goal point P235, the smartphone 40 measures the time when the signal received from the beacon 36a becomes maximum as the goal time. The goal time of each athlete is transmitted to, for example, a race organizer's totaling device (not shown) via a wireless communication network 5, and is used for totaling the race ranking.

12A to 12C are diagrams showing an example of a display screen of the smartphone 40. Here, the display when the user approaches the point P232 in the real space corresponding to the point P132 in another space is shown. On the course 27c in the real space, go straight near the point P232, but on the course 26c in another space, turn right at the point P132. As the user travels, the display unit 12a changes from the image 24h of FIG. 12A turning right in another space to the image 24i of FIG. 12B. These images include

buildings

35a, 35b and

roads

42a, 42b taken in different spaces. In another space, it turns right there, but in real space, the direction of travel is always displayed as an image that can be seen in front. In other words, since the correspondence between the traveling directions is not fixed between the other space and the real space, the course shape of the real space with respect to the other space can be freely set. It should be noted that the fixed correspondence in the traveling direction means, for example, a state in which the correspondence is always shifted clockwise by 30 degrees.

FIG. 12C shows a display example reflecting the user's frontal orientation and posture (azimuth and elevation in the user's line-of-sight direction). FIG. 12B shows a display image 24j when a user twists his / her upper body to the left and turns his / her line of sight downward in order to see the image 35b of a building in another space. In the normal image display of a smartphone, even if the image has a wide viewing angle such as a 360 ° angle of view, only the image in the range of the angle of view in the direction of the smartphone that matches the traveling direction is displayed. Therefore, the orientation / posture detection unit 11a of the smartphone 40 can detect the movement of the user's upper body and display an image in the range of the angle of view in the line-of-sight direction. As a result, in the display image 24j, the image 35c of the building is displayed in front and the image 42c of the road is displayed in the lower part thereof.

According to the third embodiment, the present invention can be applied to a competition or the like that goes around the same course while looking at the scenery of another space. In particular, long-distance running with many laps makes it easy for runners to get bored, but since they can run while looking at different scenery from time to time, it is effective in eliminating the feeling of boredom.

FIG. 11 shows an example in which the smartphone 40 is attached to the body using a fixing device 41 so that it can be seen hands-free. Normally, the smartphone 40 is put in a pocket such as a zack and the smartphone is approached to each point. You may set the 40 to emit an alarm sound, and when you hear the alarm sound, you may pick up the smartphone 40 and look at the screen of the smartphone 40 because you are approaching the next point.

Further, in the third embodiment, not only a still image but also a moving image can be displayed on the display unit. In the above explanation, the beacon is used, but at a minimum, it is sufficient if the video can be played only in the range where the beacon is received. Alternatively, if it is desired to display even outside the reception range of the beacon, the running speed of the user between adjacent beacons may be calculated, and the playback speed of the recorded moving image may be changed accordingly.

If the actual space is a wide circuit course and the passing point can be determined only by the position detection unit without using a beacon, the moving image is aligned based on the position coordinates obtained from the position detection unit. You can play the video on the course. In that case, the position information may be added to the captured moving image at regular time intervals or when passing through points.

In each of the above embodiments, the case where the user moves by walking or running is shown, but it may be a bicycle, a car that the user does not drive, or a public means of transportation. In addition, the user may change the intermediate points and goals of the course during the movement. In addition, when displaying an image on the image display device while moving in each section, even if it is an image such as a view of a car window registered in a state in which another space is photographed in advance and can be accessed from the image display device. Good.

Although some examples have been described above, the present invention is not limited to the above-mentioned examples, and includes various modifications. For example, the above-described embodiment has been described in detail in order to explain the present invention in an easy-to-understand manner, and is not necessarily limited to the one including all the described configurations. It is also possible to add the configuration of another embodiment to the configuration of one embodiment. Further, it is possible to add / delete / replace other configurations for a part of each embodiment.

1: Image display system, 2: Image display device, 3: Image pickup device, 4: Web server, 5: Communication network, 6a, 6b: Image sensor, 7a, 7b: Position detection unit, 8a, 8b, 8c: Memory, 9a, 9b, 9c: Communication interface, 10a, 10b, 10c: Control unit, 11a, 11b: Orientation / orientation detection unit, 12a, 12b: Display unit, 13: Audio output unit, 19: Head-mounted display (HMD), 20: User, 40: Smartphone.

Claims

In an image display device that is worn by the user and displays an image in a different space that is different from the actual space in which the user is actually located.
A position detector that detects the user's position and
A communication interface that acquires course information including image data of another space and shooting position from the outside,
A memory for storing image data and course information acquired by the communication interface, and
A display unit that displays an image of another space stored in the memory, and
A control unit that controls the display of an image in another space with respect to the display unit is provided.
The control unit
Based on the course information of another space, the course consisting of multiple points in another space is associated with the course consisting of multiple points in the real space.
An image display device characterized in that when a user moves through a course in real space, an image of associated points in another space is displayed according to the position of the user.
In the image display device according to claim 1,
The control unit is characterized in that a plurality of points on the course in the other space and the real space are set so that the relationship between the distances of the sections of the points corresponding to each other is approximately constant scale. Display device.
In the image display device according to claim 2,
The control unit is characterized in that a plurality of points on the course in the other space and the real space are set so that the relationship between the traveling directions of the sections of the points corresponding to each other is approximately a constant directional difference. Image display device.
In the image display device according to claim 2,
When at least one of the other space and the real space is a course having an altitude difference, the control unit determines the distance of the section for setting a plurality of points on the course and the travel time of the user due to the altitude difference. An image display device characterized by using a distance corrected so as to cancel the difference.
In the image display device according to claim 3,
In addition, it is equipped with an audio output unit that outputs audio.
When the directional difference between the traveling direction of the course in the real space in which the user is moving and the traveling direction of the corresponding course in another space becomes a predetermined value or more, the control unit uses the voice output unit to speak. An image display device comprising notifying or displaying a message on the display unit.
In the image display device according to claim 3,
In addition, it is equipped with an audio output unit that outputs audio.
When the control unit detects that the movement position of the user deviates from the course set in the real space by a predetermined value or more, the voice output unit emits an alarm sound or displays a warning message on the display unit. An image display device characterized by.
In the image display device according to claim 2,
When the real space course that the user moves is a lap course that goes around a predetermined course multiple times
The control unit is an image display device characterized in that a plurality of points in a real space corresponding to a plurality of points in another space are set according to a distance traveled by the user on the circuit course.
In the image display device according to claim 7,
Beacons that emit predetermined signals are installed at a plurality of points set in the lap course.
An image display device characterized in that the distance traveled by a user is calculated by receiving a signal emitted by the beacon by the position detection unit.
In the image display device according to claim 1,
In addition, it is equipped with an orientation / attitude detection unit that detects the orientation and orientation of the user in the front direction.
The image display device is a transmissive head-mounted display that the user wears on the head and uses.
On the display unit, an image of another space is superimposed on the landscape of the real space in front and displayed.
The control unit displays an image when the direction of the user's line of sight in the real space detected by the orientation / posture detection unit matches the direction of the shooting direction in another space. apparatus.
In the image display device according to claim 1,
Equipped with an image sensor that captures an image at a point in another space
The position detection unit detects the shooting position in the other space,
Image data and course information taken by the image sensor are stored in the memory.
An image display device characterized in that when a user moves through a course in real space, the control unit reads image data and position information from the memory and displays the corresponding image on the display unit.
An image display device, an image pickup device, and a Web server are connected via a communication network, and an image in a different space different from the actual space in which the user is actually located is displayed on the image display device worn by the user. In the image display system
The image pickup device
An image sensor that captures images at multiple points on a course in another space,
It is equipped with a position detection unit that detects the position of each point where the image was taken.
The captured image data and shooting position information are transmitted to the Web server,
The Web server
The image data in another space received from the image pickup device and the shooting position are saved as course information.
The image display device is
A position detector that detects the user's position and
A memory for storing image data in another space acquired from the Web server and course information including a shooting position, and
A display unit that displays an image of another space stored in the memory, and
A control unit that controls the display of an image in another space with respect to the display unit is provided.
The control unit
Based on the course information of another space, the course consisting of multiple points in another space is associated with the course consisting of multiple points in the real space.
An image display system characterized in that when a user moves through a course in real space, an image of associated points in another space is displayed according to the position of the user.
In an image display method that displays an image in a different space whose position is different from the actual space in which the user is actually located.
Steps to take images at multiple points on a course in another space,
Steps to detect the position of each point where the image was taken, and
Steps to save image data and shooting position in another space as course information,
Based on the course information of another space, the step of associating a course consisting of multiple points in another space with a course consisting of multiple points in real space,
Steps to detect the position of the user moving through the course in real space,
A step to display an image of the associated point in another space according to the movement position of the user,
An image display method comprising.