JP6691589B2 - Information processing system and display method - Google Patents

Description

  The present invention relates to a technique for displaying an image corresponding to a route on a map.

  There is known a technique of displaying a part of a photograph (panoramic image) taken at a certain point on a map. Non-Patent Document 1 discloses a technique for displaying a panoramic image of a place when a predetermined object is dropped with a mouse at a place to be seen on the map in a map display service. The displayed image can be rotated or moved along the arrow displayed on the road according to the operation by the mouse.

"What you can do with Street View", [online], [November 20, 2012 search], Internet <URL: http://maps.google.co.jp/intl/ja/help/maps/streetview/learn /using-street-view.html〉

In the technique described in Non-Patent Document 1, in order to continuously view a panoramic image corresponding to a route from a certain starting point to an arriving point, the user must make a correct route from the starting point to the arriving point on the panoramic image. I had to keep giving instructions, and the operation was complicated.
On the other hand, the present invention provides a technique for continuously displaying panoramic images along a route from a departure point to an arrival point with a simpler operation.

  The present invention relates to an acquisition means for acquiring a plurality of images, each of which corresponds to a point on a map and which is cut out from a part of each of a plurality of panoramic images determined according to a route on the map, and the acquisition. There is provided an information processing system having display control means for sequentially displaying a plurality of images acquired by the means on a display means.

  The display control unit may sequentially switch and display one image displayed on the display unit among the plurality of images.

  The display control unit may automatically switch and display one image displayed on the display unit among the plurality of images, sequentially without input by the user.

  The information processing system further includes an image determination unit that determines a plurality of panoramic images according to the route, and the acquisition unit includes the image extracted from each of the plurality of panoramic images determined by the image determination unit. Multiple images may be acquired.

  The information processing system further includes a selection unit that selects a part of the points from a plurality of points included in the route, and the acquisition unit includes a plurality of points corresponding to the part of the points selected by the selection unit. The plurality of images cut out from each of the panoramic images may be acquired.

  The information processing system further includes a cutout unit that cuts out an image corresponding to the direction determined according to the route from each of the plurality of panoramic images, and the acquisition unit includes the plurality of images from the cutout unit. May be obtained.

  The cutout unit may cut out, from one panoramic image of the plurality of panoramic images, an image corresponding to an orientation corresponding to a positional relationship between the one panoramic image and another panoramic image corresponding to an adjacent point. Good.

  The clipping means positions, from one panoramic image of the plurality of panoramic images, the one panoramic image and another panoramic image corresponding to a point two or more points away from the point corresponding to the one panoramic image. You may cut out the image according to the direction according to the relationship.

  The cutout unit cuts out two or more images corresponding to different directions from one panoramic image of the plurality of panoramic images, and cuts out the two or more images from the one panoramic image. The image may be cut out from the point corresponding to the panoramic image of 1 and another panoramic image corresponding to the adjacent point.

  When the direction of the route changes by a predetermined value or more at a point corresponding to the one panoramic image, the cutout unit selects from the one panoramic image among the plurality of panoramic images two or more images corresponding to different directions. May be cut out.

  The display control unit may stop the process of displaying the plurality of images in order for one image, and change the one image according to the orientation of the display unit.

  The information processing system further includes a route determination unit that determines the route according to a user's instruction, and the acquisition unit includes a plurality of panoramic images determined according to the route determined by the route determination unit. A plurality of images cut out from a part of each may be acquired.

  The information processing system further includes a receiving unit that receives the route, and the acquiring unit is cut out from a part of each of the plurality of panoramic images determined according to the route received by the receiving unit. Multiple images may be acquired.

  Further, the present invention, each corresponding to a point on the map, an acquisition means for acquiring a plurality of images cut out from a part of each of the plurality of panoramic images determined according to the route on the map, There is provided an information processing device comprising: a display control unit that sequentially displays a plurality of images acquired by the acquisition unit on a display unit.

  Further, according to the present invention, a computer is used to obtain a plurality of images, each of which corresponds to a point on a map and which is cut out from a part of each of a plurality of panoramic images determined according to the route on the map. And a program for causing the display unit to display the plurality of images acquired by the acquisition unit in order.

  Further, the present invention includes a step of acquiring a plurality of images each of which corresponds to a point on a map and which is cut out from a part of each of a plurality of panoramic images determined according to the route on the map, And a step of displaying a plurality of images acquired by the acquiring step in order on a display means.

  According to the present invention, it is possible to continuously display the panoramic images along the route from the departure point to the arrival point with a simpler operation.

The figure which illustrates the outline of one embodiment. The figure which shows the structure of the information processing system 1 which concerns on one Embodiment. The figure which shows the specific example of the information processing system 1. The block diagram which shows the hardware constitutions of the terminal device 200. 3 is a block diagram showing the hardware configuration of the information processing device 300. FIG. FIG. 3 is a block diagram showing a hardware configuration of a server device 500. The flowchart which shows operation | movement of the information processing system 1. The figure which illustrates the map displayed by a map program. The figure which illustrates the point provided on the map. The figure which illustrates the structure of a panoramic image. FIG. 8 is a diagram exemplifying a specific sharing of processing when executing the flow of FIG. 7. The flowchart which illustrates the process which cuts out a partial image. The flowchart which illustrates the display process of a partial image. The figure which illustrates the partial image displayed according to a route.

1. Overview FIG. 1 is a diagram illustrating an overview of an embodiment. In the present embodiment, when a route on the map is given, a plurality of images cut out from each of the plurality of panoramic images along the given route (hereinafter, the image cut out from the panoramic image is referred to as a “partial image”). Are displayed in the order according to the route. That is, when a route is given, a plurality of images are sequentially displayed like a flipbook according to the route.

2. Configuration FIG. 2 is a diagram showing a configuration of the information processing system 1 according to the embodiment. The information processing system 1 includes a storage unit 11, a route acquisition unit 12, a panoramic image determination unit 13, a panoramic image selection unit 14, an image cutout unit 15, an image acquisition unit 16, a display control unit 17, and a display. It has means 21 and input means 22.

  The storage unit 11 stores a plurality of panoramic images. The panoramic image is an image obtained by photographing a wide range. A panoramic image is an image in which a predetermined range for at least one of pitch (elevation angle) and yaw (azimuth angle) is contained in one image when viewed from a certain shooting point. The plurality of panoramic images respectively correspond to different points on the map. The route acquisition means 12 acquires a route on the map. The panoramic image determination means 13 determines a plurality of panoramic images corresponding to the route from the panoramic images stored in the storage means 11. The panoramic image selection means 14 thins out the plurality of determined panoramic images according to a predetermined rule to select the plurality of panoramic images. The image cutout unit 15 cuts out a part of the plurality of panoramic images selected by the panoramic image selection unit 14 to be displayed on the display unit 21. The image acquisition unit 16 acquires the image (partial image) cut out by the image cutout unit 15. The display control unit 17 causes the display unit 21 to display the plurality of partial images acquired by the image acquisition unit 16 in the order according to the route.

  The display unit 21 displays the partial image. The input unit 22 inputs an instruction regarding display of an image into the information processing system 1. In this example, the display control unit 17 stops updating the partial images in the next order in the middle of the order in response to the instruction input via the input unit 22. Further, the display control means 17 moves the displayed portion in the panoramic image to which the stopped partial image belongs in accordance with the instruction input via the input means 22.

  FIG. 3 is a diagram showing a specific example of the information processing system 1. In this example, the information processing system 1 includes a display device 100, a terminal device 200, an information processing device 300, and a server device 500. The information processing device 300 is, for example, a stationary game machine. The information processing device 300 and the server device 500 are connected via a network 400. The network 400 is, for example, the Internet.

  The display device 100 is a stationary display device, for example, a television receiver. The terminal device 200 is a device that receives an instruction input from a user. The terminal device 200 is a device that further displays information. The terminal device 200 has a shape that can be held by one or both hands. The information processing device 300 is a computer device that performs information processing for displaying an image on the display device 100 and the terminal device 200. The information processing device 300 receives data required for displaying an image from the server device 500 and supplies the display data to the display device 100 and the terminal device 200 in response to a user operation. The display data is data for displaying an image on each of the display device 100 and the terminal device 200. The server device 500 is a computer device that provides information about a map.

  FIG. 4 is a block diagram showing the hardware configuration of the terminal device 200. The terminal device 200 includes a control unit 210, a storage unit 220, an interface unit 230, an input unit 240, a display unit 250, a touch screen unit 260, and a motion sensor unit 270.

  The control unit 210 controls the operation of each unit of the terminal device 200. The control unit 210 has a CPU (Central Processing Unit) and a memory. Various processes are executed by the CPU executing the program. The storage unit 220 is a device that stores data, such as a flash memory. The interface unit 230 is a device that transmits and receives data to and from the information processing device 300, and has, for example, an antenna and a modem. The interface unit 230 communicates with the information processing device 300 by a predetermined communication method. The input unit 240 includes operators that receive user operations, such as a keypad, buttons, and an analog stick. The input unit 240 supplies a signal according to a user operation to the control unit 210. The display unit 250 has a device that displays an image, such as a liquid crystal panel or an organic EL (electroluminescence) panel, and a drive circuit thereof. The display unit 250 displays an image according to the display data. The touch screen unit 260 is a device that receives a user instruction, and is provided on the display unit 250. The touch screen unit 260 supplies the control unit 210 with coordinate data indicating the position (coordinates) designated by the user.

  The motion sensor unit 270 detects the movement of the terminal device 200 and outputs information indicating the detected movement. The motion sensor unit 270 detects an acceleration sensor (three-axis acceleration sensor) for detecting an acceleration acting on the terminal device 200, a gyro sensor for detecting a change in an angle or an angular velocity of the terminal device 200, and a geomagnetism. It has a geomagnetic sensor. The motion sensor unit 270 supplies the signals output by these sensors to the control unit 210. When the terminal device 200 is tilted vertically and horizontally, it outputs a signal indicating the tilted movement.

  FIG. 5 is a block diagram showing the hardware configuration of the information processing device 300. The information processing device 300 includes a control unit 310, a storage unit 320, an interface unit 330, and a communication unit 340. The control unit 310 includes a device that controls the operation of each unit of the information processing device 300, for example, a CPU, a GPU (Graphics Processing Unit), and a memory. The storage unit 320 has a device that stores data, such as a flash memory or a hard disk drive. The interface unit 330 is a device that transmits and receives data to and from the display device 100 and the terminal device 200. The interface unit 330 wirelessly communicates with the terminal device 200 and wiredly communicates with the display device 100. The communication unit 340 communicates data with the server device 500 via the network 400.

  In this example, the storage unit 320 stores a program (hereinafter, referred to as a map program) for displaying a plurality of partial images cut out from each of the plurality of panoramic images in an order according to a route. When the control unit 310 executes the map program, some of the functions shown in FIG. 2 are installed in the information processing device 300.

  FIG. 6 is a block diagram showing the hardware configuration of the server device 500. The server device 500 includes a control unit 510, a storage unit 520, and a communication unit 540. The control unit 510 has a device that controls the operation of each unit of the server device 500, for example, a CPU and a memory. The storage unit 520 has a device for storing data, such as a flash memory or a hard disk drive. The communication unit 540 communicates data with the information processing device 300 via the network 400.

  In this example, the storage unit 520 stores a database in which a plurality of panoramic images are recorded. Further, the storage unit 520 stores a program (hereinafter referred to as a server program) for supplying data in response to a request from the information processing device 300 (terminal device). When control unit 510 executes the map program, a part of the functions shown in FIG. 2 is installed in server device 500.

  In the information processing system 1, the storage unit 520 of the server device 500 is an example of the storage unit 11. The control unit 510 executing the server program is an example of the panoramic image determination unit 13. The control unit 310 executing the map program is an example of the route acquisition unit 12, the panoramic image selection unit 14, the image cutout unit 15, the image acquisition unit 16, and the display control unit 17. The display unit 250 of the terminal device 200 is an example of the display unit 21. The input unit 240 and the motion sensor unit 270 are an example of the input unit 22.

3. Operation FIG. 7 is a flowchart showing the operation of the information processing system 1. The flow of FIG. 7 is started in response to an instruction to display a partial image along the route in a state where the server program is executed in the server device 500 and the map program is executed in the information processing device 300, for example. It The processing described below is provided by the map program and the server program. In step S100, the route on the map is acquired.

  FIG. 8 is a diagram illustrating a map displayed by the map program. The map has a portion corresponding to a road and a portion other than that. In FIG. 8, portions other than the road are hatched for convenience of explanation. In addition, an image representing a building or facility may exist in a portion other than the road. Further, the position on the map can be uniquely specified by the coordinates of a predetermined coordinate system such as latitude and longitude. Further, in this example, a plurality of points are discretely provided on the map. A unique identifier is assigned to each point, and each point can be uniquely specified by this identifier.

  FIG. 9 is a diagram illustrating points provided on the map. In this example, 20 points from point P1 to point P20 are set. Note that these points are virtually set points and may not be displayed as images on the map. With this map displayed, the user touches the departure point and drags the route from the arrival point to specify the route from the departure point to the arrival point. FIG. 9 illustrates the route designated by the user. In this example, a route starting from the vicinity of the point P12 and reaching the vicinity of the point P10 via the points P13, P3, and P9 is shown.

  Referring back to FIG. In step S110, a panoramic image corresponding to the route is determined (specified). In this example, the server device 500 determines a plurality of panoramic images corresponding to the route. One panoramic image is associated with each of the plurality of points described in FIG. When the route is determined, the points included in the route are determined, that is, the panoramic image corresponding to the points included in the route is determined. In addition, the fact that an identifier is attached to each point is synonymous with the fact that an identifier is attached to a panoramic image.

  FIG. 10 is a diagram illustrating the structure of a panoramic image. In this example, the panoramic image is a rectangle whose vertical axis (pitch) is 90 ° (directly above) to −90 ° (directly below) and whose horizontal axis (yaw) is 0 ° (north) to 360 ° (north). Is an image obtained by photographing all directions at corresponding points on the map. Note that this angle corresponds to the direction of the user's viewpoint. For example, the image of the origin on this coordinate axis corresponds to a landscape that can be visually recognized when the user faces directly in front of the north, assuming that the user is at the position where the panoramic image is captured.

  Referring back to FIG. In step S120, an image used for cutting out a partial image is selected from the plurality of panoramic images determined in step S110. That is, the plurality of panoramic images determined in step S110 are thinned out. The reason for this processing is as follows. If you try to display a partial image cut out from all panoramic images corresponding to all the points included in the route, if the number of points included in the route is large, the partial image from the start point to the arrival point will be displayed. Can take a very long time. In order to prevent the time required to display a plurality of partial images from becoming too long, panoramic images are thinned out under predetermined conditions. As the condition for thinning out the panoramic image, for example, a condition for thinning out the distance on the map to be as uniform as possible is used. For example, if there are areas where points are set every 100 m and areas where points are set every 10 m in the route, panoramic images corresponding to the areas where points are set every 10 m are thinned out. . Further, an upper limit may be set for the number of panoramic images, and the panoramic images may be thinned out so that the number of selected panoramic images is equal to or less than the upper limit value. The upper limit value may be a fixed value or a value that changes according to a distance on the map or a user's instruction.

In the thinning process, the following conditions are used, for example.
・ Preferably leave the points corresponding to the intersections.
・ The area near the intersection (before and after the intersection) should be preferentially left.
・ Pull out preferentially where straight lines continue.

  In step S130, a partial image is cut out from each of the plurality of panoramic images selected in step S120. The partial image is cut out from the panoramic image in the following procedure, for example. First, with respect to the panoramic image, a mapping is formed on a stereoscopic area corresponding to the range in which the panoramic image is captured. Further, in the panoramic image mapped in three dimensions, a part corresponding to the specified pitch (elevation depression angle) and yaw (azimuth angle) is mapped onto a plane corresponding to the displayed area. This mapping is a partial image, and forming a mapping from the panoramic image onto a plane of a predetermined size is called “cutting out a partial image from the panoramic image”. A well-known technique is used for the process of mapping the panoramic image on a solid and the process of mapping the stereoscopic image on a plane.

  In step S140, a plurality of partial images are acquired. In step S150, the plurality of partial images are sequentially displayed one by one in the order along the route.

  FIG. 11 is a sequence chart exemplifying a specific sharing of processing in the terminal device 200, the information processing device 300, and the server device 500 when executing the flow of FIG. 7. In this example, the storage unit 520 corresponds to the storage unit 11. The control unit 310 executing the map program corresponds to the route acquisition unit 12, the image acquisition unit 16, and the display control unit 17. The control unit 510 executing the server program corresponds to the panoramic image determination unit 13, the panoramic image selection unit 14, and the image cutout unit 15. The display unit 250 corresponds to the display unit 21. The input unit 240 and the touch screen 260 correspond to the input unit 22. Data for displaying the map illustrated in FIG. 8 (hereinafter referred to as “map data”) is stored in the storage unit 520 of the server device 500.

  In step S200, control unit 310 of information processing device 300 transmits a request for map data to server device 500. This request includes parameters that determine the map data used for display, such as the coordinates and scale of the center point of the displayed area. Upon receiving this request, control unit 510 of server device 500 extracts map data determined by the parameters included in the request from the data stored in storage unit 520 (step S201). Control unit 510 transmits the extracted map data to information processing device 300 (step S202). The control unit 310 of the information processing device 300 causes the display unit 250 to display an image (map) according to the received map data (step S203).

  The user touches the touch screen unit 260 while looking at the map displayed on the display unit 250 to specify a route (step S204). The control unit 310 acquires information indicating the route designated by the user from the terminal device 200 (step S100). The control unit 310 generates data for determining a route (hereinafter referred to as “route data”) from the information acquired from the terminal device 200 (step S205). The route data includes, for example, identifiers of points included in the route (points P12, P13, P3, P9, and P10 in the example of FIG. 9). When the route data is generated, the control unit 310 transmits a data request to the server device 500 (step S206). This request includes, for example, route data and parameters used for thinning out panoramic images (for example, the upper limit number of panoramic images).

  Upon receiving the request from the information processing device 300, the control unit 510 of the server device 500 selects, from the database stored in the storage unit 520, panoramic images corresponding to a plurality of points indicated by the route data included in the request. It is determined (step S110). Further, control unit 510 thins out the determined panoramic image using the parameters included in the received request (step S120). Further, control unit 510 cuts out partial images from each of the thinned panoramic images (step S130). Note that when cutting out a partial image from a panoramic image, it is necessary to specify the direction of the viewpoint. This process is performed as follows, for example. The processing described below includes (1) processing of cutting out a plurality of partial images having different viewpoint directions from one panoramic image at a point where a rotation angle in the viewpoint direction is large, such as (1) bending corner, and (2) bending corner, etc. Before the point where the rotation angle in the direction of the viewpoint is large, the process of tilting the viewpoint in the direction after the curve is included.

  FIG. 12 is a flowchart illustrating a process of cutting out a partial image. In the following description, the points on the route are assigned numbers according to the order. A point corresponding to the i-th panoramic image in the order after thinning is referred to as a point PP (i), and a panoramic image corresponding to the point PP (i) is referred to as I (i). In the example of FIG. 9, when thinning is not performed, PP (1) = P12, PP (2) = P13, PP (3) = P3, PP (4) = P9, PP (5) = P10. In addition, the direction of the viewpoint when cutting out the i-th partial image in the display order is represented by d (i). The i-th partial image in the display order is represented as IP (i). Further, a vector from the point PP (i-1) to the point PP (i) is represented as r (i).

  In step S1301, the control unit 510 updates the loop counter i (initializes the loop counter i when the loop is the first round). In step S1302, control unit 510 determines whether the angle α formed by d (i−1) and r (i + 1) is larger than threshold value th1. The threshold value th1 is an angle corresponding to an intersection or a turning angle, and is 45 °, for example. The case where the angle α is larger than the threshold value th1 means that the route is approaching an intersection (or a corner), and it means that the orientation of the viewpoint is greatly rotated at that point. When it is determined that the angle α is larger than the threshold value th1 (step S1302: YES), control unit 510 advances the process to step S1303. When it is determined that the angle α is equal to or less than the threshold value th1 (step S1302: NO), the control unit 510 shifts the processing to step S1308.

In step S1303, control unit 510 determines division number n of the rotation angle of the viewpoint. The number of divisions n is determined, for example, according to the size of the angle α, specifically, the number of divisions increases as the angle α increases. In step S1304, control unit 510 updates loop counter j (initializes loop counter j when the loop is the first round). In step S1305, control unit 510 calculates the direction d of the viewpoint using the following equation (1).
d (i) = d (i−1) + [{r (i + 1) −d (i−1)} × j] / n (1) In this case, one panoramic image has a plurality of different orientations. Since a plurality of partial images corresponding to is cut out, the total number of partial images is larger than the number of panoramic images. That is, when a plurality of partial images are cut out from one panoramic image, PP (i) and d (i) do not correspond to the same panoramic image in the subsequent panoramic images. For example, when three partial images are cut out from a certain panoramic image, the direction of the viewpoint corresponding to PP (i) is dd (i + 2), but in the following, PP (i) and d (i) will be referred to for convenience. Will correspond to the same panoramic image. In step S1305, control unit 510 cuts out partial image IP (i + j) from panoramic image I (i) using calculated viewpoint orientation d (i) (step S1306). Control unit 510 stores the data of the cut out partial image in storage unit 520. This data contains the identifier of the panoramic image that is the basis of this partial image.

  In step S1307, control unit 510 determines whether the loop termination condition is satisfied. The end condition of the loop is j = n here. When it is determined that the ending condition is satisfied (step S1307: YES), control unit 510 moves the process to step S1308. When it is determined that the ending condition is not satisfied (step S1307: NO), control unit 510 moves the process to step S1304.

  In step S1308, control unit 510 determines whether the angle β formed by vector r (i) and vector r (i + 1) is larger than threshold value th2. The threshold value th2 is an angle corresponding to an intersection or a turning angle, and is 45 °, for example. Although the threshold value th2 is equal to the threshold value th1 described above, the two may be different. The case where the angle β is larger than the threshold value th2 means that the route is approaching one point before the intersection (or the corner), and means that the direction of the viewpoint is greatly rotated at the next point. In this example, in order to prevent the viewpoint from significantly rotating at the point corresponding to the intersection, the viewpoint is rotated in advance at the point immediately before that point. When it is determined that the angle β is larger than the threshold value th2 (step S1308: YES), the control unit 510 shifts the processing to step S1309. When it is determined that the angle β is equal to or less than the threshold value th2 (step S1308: NO), control unit 510 advances the process to step S1310.

In step S1309, control unit 510 calculates viewpoint direction d by the following equation (2).
d (i) = r (i) + {rr (i) -r (i)} * k ... (2)
Here, the vector rr (i) is a vector from the point PP (i) to the point PP (i + 2). The coefficient k is a predetermined constant in the range of 0 <k ≦ 1.

In step S1310, control unit 510 calculates the viewpoint direction d by the following equation (3).
d (i) = r (i) (3)
The formula (4) may be used instead of the formula (3).
d (i) = d (i-1) (4)

  In step S1311, control unit 510 cuts out partial image IP (i + j) from panoramic image I (i) using calculated viewpoint direction d. Control unit 510 stores the data of the cut out partial image in storage unit 520. This data contains the identifier of the panoramic image that is the basis of this partial image.

  In step S1312, control unit 510 determines whether the loop termination condition is satisfied. The end condition of the loop is i = imax here. imax is the number of panoramic images selected in step S120. When it is determined that the ending condition is satisfied (step S1312: YES), control unit 510 ends the process of FIG. When it is determined that the termination condition is not satisfied (step S1312: NO), control unit 510 moves the process to step S1301.

  In the flow of FIG. 12, steps S1302 to S1307 correspond to the processing of cutting out a plurality of partial images having different viewpoint directions from one panoramic image, and steps S1308 to S1312 have a large rotation angle. This corresponds to the processing of tilting the viewpoint in the direction before and after the turn. In the above example, in the process of tilting the viewpoint in the direction after the bend, it is determined whether or not the viewpoint is tilted by referring to the one point ahead, but the reference point is limited to the one point ahead. Not done. Two or more points ahead may be referenced. Alternatively, the centroids at a plurality of points one or more ahead may be referred to.

  Referring back to FIG. When the partial images are cut out for all the selected panoramic images, control unit 510 transmits the data of the cut-out partial images to information processing device 300 that is the transmission source of the request (step S207).

  In step S140, the control unit 310 of the information processing device 300 acquires the data of the partial image group from the server device 500 (step S140). The control unit 310 controls the display unit 250 to display the partial images in the order along the route according to the acquired data. The display unit 250 sequentially displays the partial images under the control of the information processing device 300 (step S150). The process of displaying the partial images in order is performed as follows, for example.

  FIG. 13 is a flowchart illustrating a display process of a partial image. In step S1501, control unit 310 updates loop counter i. When the loop is the first round, the loop counter i is initialized. When the loop termination condition is satisfied, the flow of FIG. 13 is terminated.

  In step S1502, control unit 310 causes display unit 250 to display the i-th partial image. In step S1503, it is determined whether a trigger for displaying the next partial image has occurred. The trigger for shifting to the next partial image is, for example, a user instruction. The user's instruction is input, for example, via the input unit 240 (for example, pressing a button) or via the touch screen 260 (for example, touching a predetermined position). Alternatively, a lapse of a predetermined time after the i-th partial image is displayed on the display unit 250 may be treated as a trigger. When it is determined that the trigger for displaying the next partial image has occurred (step S1503: YES), the control unit 310 proceeds to step S1501. When it is determined that the trigger for displaying the next partial image has not occurred (step S1503: NO), control unit 310 advances the process to step S1504.

  In step S1504, control unit 310 determines whether a trigger for changing the direction of the viewpoint has occurred. The user's instruction is input, for example, via the motion sensor unit 270. For example, when the terminal device 200 is rotated 30 ° from north to east (clockwise), an instruction to rotate the viewpoint direction by 30 ° clockwise is input. When it is determined that the trigger for changing the direction of the viewpoint has occurred (step S1504: YES), control unit 310 advances the process to step S1505. When it is determined that the trigger for changing the direction of the viewpoint has not occurred (step S1504: NO), the control unit 310 shifts the processing to step S1503.

  In step S1505, control unit 310 acquires a partial image corresponding to the changed viewpoint orientation. Specifically, the control unit 310 transmits to the server device 500 a request for a partial image according to the changed viewpoint orientation. This request includes the identifier of the panoramic image corresponding to the currently displayed partial image and the parameter for determining the orientation of the viewpoint. The control unit 510 of the server device 500 cuts out a partial image from the panoramic image in response to a request. Control unit 510 transmits the cut-out partial image data to information processing device 300 that is the transmission source of the request. Control unit 310 causes display unit 250 to display the partial image according to the data received from the server (step S1506). That is, at this time, the transition of the displayed partial image to the partial image corresponding to the next point on the path is stopped, and in the meantime, the other direction of the viewpoint at the point corresponding to the currently displayed partial image is stopped. The partial image of is displayed. After that, if it is determined in step S1503 that a trigger has occurred, the transition to the partial image at the next point is restarted.

  FIG. 14 is a diagram illustrating a partial image displayed according to the route in the information processing system 1. Here, the partial images displayed in order along the route illustrated in FIG. 9 are shown. For the sake of explanation, a panoramic image is shown on the left side of each partial image together with an identifier. Further, in each panoramic image, a portion cut out as a partial image is shown by hatching. It should be noted that the mapping of the partial images in the panoramic image is not necessarily rectangular, but here, in order to simplify the drawing, it is shown as a rectangle. At points P12 and P13, since the route is oriented in the northeast direction, the part corresponding to the northeast direction in the panoramic image is cut out. Since the point P3 corresponds to an intersection, three partial images are cut out from one panoramic image. The three partial images correspond to the rotation of the viewpoint from northeast to south. At points P9 and P10, since the route is oriented in the northeast direction, the portion corresponding to the northeast direction in the panoramic image is cut out. Thus, according to the information processing system 1, the partial images corresponding to the direction of the viewpoint at the points set on the route are continuously displayed.

4. Modifications The present invention is not limited to the above-described embodiments, and various modifications can be made. Hereinafter, some modified examples will be described. Two or more of the following modifications may be used in combination.

  The division of processing in the information processing device 300, the terminal device 200, and the server device 500 is not limited to that illustrated in FIG. 11. The information processing apparatus 300 may perform part or all of the processing described as being performed by the server apparatus 500 in FIG. 11. Alternatively, the server device 500 may perform part of the processing described as being performed by the information processing device 300 in FIG. 11.

  As an example of the sharing of the processing in the information processing apparatus 300 and the server apparatus 500, the panoramic image thinning processing (step S120) may be performed by the information processing apparatus 300. In this case, the information processing apparatus 300 acquires the route and then thins out points on the route by the algorithm described in the embodiment. The information processing device 300 transmits a request including the identifiers of the thinned points to the server device 500. In another example, the information processing apparatus 300 uses a method other than transmitting a panoramic image and a point identifier corresponding to one-to-one correspondence (for example, a method of transmitting latitude and longitude of a curved point). The route may be sent to the device 500. In this case, the server device 500 identifies the identifier of the point included in the route from the route information acquired from the information processing device 300.

  Some of the processes illustrated in FIGS. 7, 11, 12, and 13 may be omitted. For example, the process of thinning out panoramic images (step S120) may be omitted. Alternatively, the order of some of the processes illustrated in FIGS. 7, 11, 12, and 13 may be exchanged with other processes.

  In the embodiment, an example has been described in which a plurality of partial images along the route are sequentially displayed one by one (images to be displayed are sequentially switched). The method of displaying a plurality of partial images along the route is not limited to this. For example, a plurality of partial images may be simultaneously displayed on one screen.

  The route on the map is not limited to that specified by the user. For example, the user may specify only the departure point and the arrival point, and the route on the way may be automatically determined by the server device 500 or the information processing device 300. As another example, the server apparatus 500 or the information processing apparatus 300 may provide the user with a predetermined route by the server apparatus 500. Alternatively, the route data created by one user may be provided to the information processing device 300 of another user. In this case, the route data may be transmitted from one information processing device 300 to another information processing device 300 via the server device 500.

  In another example, the route acquired from the server device 500 may be corrected by the user touching the touch screen unit 260 on the terminal device 200. In this case, the information processing device 300 transmits again the route corrected using the terminal device 200 to the server device 500. In addition, the route may be changed in the middle of displaying a plurality of partial images along the route. In this case, the information processing device 300 transmits information indicating the user's operation on the terminal device 200 to the server device 500. The server device 500 transmits the partial image corresponding to the changed route to the information processing device 300.

  When displaying a partial image, the trigger for displaying the next partial image (step S1503) is not limited to the instruction by the user. For example, after a partial image along the route is instructed to be displayed, the next image may be automatically and sequentially displayed like a so-called slide show without the user's instruction. In this case, the reproduction speed of the plurality of partial images may be automatically determined by the system or may be designated by the user. When the playback speed is specified by the user, the playback speed corresponding to the one selected from the plurality of options (eg, turtle, foot, bicycle, car, train, airplane, and UFO) may be used.

  If the panoramic image corresponding to the determined point on the route is not present on the server 500, that is, if (a part of) the partial image corresponding to the route cannot be acquired, the information processing apparatus 300 replaces the partial image. The movement of points on the map may be displayed on the display portion 250.

  After transmitting the partial images along the route to the information processing device 300, the server device 500 may acquire the panoramic image of points on the route from the database and store the panoramic image in the storage unit 520 as a cache. In this case, when the information processing device 300 requests a partial image of a route (return route) opposite to this route, the server device 500 may extract the partial image from the cached panoramic image.

  When displaying multiple partial images, first display a low-resolution image, and if it stops at a certain partial image (or if the display time of a certain partial image exceeds the threshold value), that partial image is displayed at high resolution. It may be displayed.

  When the user inputs a direction instruction during reproduction of the partial image along the route, a partial image corresponding to the instructed direction may be displayed at the point where the instruction is input. In this case, the information processing device 300 does not acquire a plurality of partial images along the route from the server device 500, but acquires a plurality of panoramic images along the route. That is, the information processing device 300 cuts out the partial image from the panoramic image, not the server device 500. The information processing apparatus 300 cuts out a plurality of partial images according to the route from the plurality of panoramic images acquired from the server apparatus 500. When the user inputs a direction instruction during the reproduction of the partial image along the route, the information processing apparatus 300 selects a partial image corresponding to the input instruction from the panoramic image corresponding to the point where the instruction is input. cut. The information processing device 300 causes the terminal device 200 to display the cut out partial image. The direction instruction is input, for example, by changing the attitude of the terminal device 200 (for example, tilting the terminal device 200).

  In the embodiment, the image displayed on the display device 100 is not particularly referred to, but a point corresponding to the map and the currently displayed partial image may be displayed, or the same image as the display unit 250 may be displayed. May be done. Alternatively, a screen unrelated to the map program such as a television program may be displayed.

  The map indicated by the map data does not necessarily have to be on the ground. For example, the map may show underwater or seabed topography. Alternatively, the map may represent the topography of celestial bodies other than the earth, such as the moon. In yet another example, the map may be a so-called virtual space, that is, a map of a non-existent fictitious space. In this case, an image created by three-dimensional computer graphics or the like may be used as the panoramic image.

  The hardware configuration of the information processing system 1 is not limited to the one described in the embodiment. For example, the function of the information processing device 300 and the function of the terminal device 200 may be realized by a single device, for example, a portable game machine, a tablet terminal, or a smartphone.

  Further, in the present invention, the functional functions of the main body device 300 shown in FIG. 3 may be realized by either hardware or software. When the functional functions of main device 300 are realized by software, the functions may be realized not by a single program but by cooperation of a plurality of programs. Further, these programs do not have to be executed in the same device, and may be executed in separate devices. It should be noted that such a program can be provided in a form recorded on a recording medium such as an optical disk or a semiconductor memory, or can be provided in a form downloaded to an information processing device via a network such as the Internet.

  The mathematical formulas and parameters described in the embodiments are merely examples, and the present invention is not limited to these.

DESCRIPTION OF SYMBOLS 1 ... Information processing system 11 ... Storage means 12 ... Route acquisition means 13 ... Panorama image determination means 14 ... Panorama image selection means 15 ... Image cut-out means 16 ... Image acquisition means 17 ... Display control means 21 ... Display means 22 ... Input means 100 ... display device 200 ... terminal device 210 ... control unit 220 ... storage unit 230 ... interface unit 240 ... input unit 250 ... display unit 260 ... touch screen unit 270 ... motion sensor unit 300 ... information processing device 310 ... control unit 320 ... storage unit 330 ... Interface part 340 ... Communication part 400 ... Network 500 ... Server device 510 ... Control part 520 ... Storage part 540 ... Communication part

Claims (17)

A server device,
And an information processing device,
The server device is
Storage means for storing a panoramic image group each corresponding to a point on the map,
Specifying means for specifying a plurality of panoramic images from the panoramic image group according to the route on the map;
Image cutting means for cutting out an image of a portion corresponding to the direction in which the path faces from each of the specified plurality of panoramic images,
The image cropping unit corresponds to a direction in which the route after turning at the intersection or the turn angle is tilted in a direction in which the route is turned, as compared with a direction in which the route is turned, with respect to the panoramic image before the intersection or the turn angle on the route. Cut out the image of the part,
The information processing device is
An acquisition means for acquiring a plurality of images cut out by the image cutout means,
An information processing system, comprising: a display control unit that sequentially displays a plurality of images acquired by the acquisition unit on a display unit. A server device,
And an information processing device,
The server device is
Storage means for storing a panoramic image group each corresponding to a point on the map,
Specifying means for specifying a plurality of panoramic images from the panoramic image group according to a given route on the map;
Control means for calculating the direction of the viewpoint at the point on the path,
Image cutting means for cutting out one or more images corresponding to the viewpoint from each of the specified plurality of panoramic images,
When the rotation angle of the viewpoint at the first point on the route exceeds a threshold value, the image cutout unit sets the viewpoint at the second point immediately before the first point on the route to the first point. Rotate according to the viewpoint in
The information processing device is
An acquisition means for acquiring a plurality of images cut out by the image cutout means,
An information processing system, comprising: a display control unit that sequentially displays a plurality of images acquired by the acquisition unit on a display unit. The server device is
Thinning means for thinning out at least one panoramic image from the plurality of panoramic images identified by the identifying means
Have
The image cutout unit cuts out an image from each of the plurality of panoramic images thinned by the thinning unit.
The information processing system according to claim 1. The information processing system according to any one of claims 1 to 3 , wherein the display control unit sequentially switches and displays one image displayed on the display unit among the plurality of images. The information processing system according to claim 4 , wherein the display control unit causes one image displayed on the display unit among the plurality of images to be automatically switched and displayed sequentially without input by a user. The server device further includes an image determination unit that determines a plurality of panoramic images according to the route,
It said acquisition means, the information processing system according to any one of claims 1 to 5 to obtain a plurality of images extracted from each of the plurality of panorama images determined by the image determining unit. The server device further includes selection means for selecting a part of points from a plurality of points included in the route,
The information processing system according to claim 6 , wherein the acquisition unit acquires the plurality of images cut out from each of the plurality of panoramic images corresponding to some of the points selected by the selection unit. The information processing system according to any one of claims 1 to 7 , wherein the image cutout unit cuts out, from each of the plurality of panoramic images, an image according to a direction determined according to the route. The image cutout unit cuts out, from one panoramic image of the plurality of panoramic images, an image corresponding to an orientation corresponding to a positional relationship between the one panoramic image and another panoramic image corresponding to an adjacent point. Item 8. The information processing system according to item 8 . The image cropping unit separates one panoramic image of the plurality of panoramic images from the one panoramic image and another panoramic image corresponding to a point two or more points away from the point corresponding to the one panoramic image. The information processing system according to claim 8 , wherein an image corresponding to the orientation according to the positional relationship is cut out. The image cutout unit cuts out two or more images corresponding to different directions from one panoramic image of the plurality of panoramic images,
After cutting out the image of the two or more from the one of the panoramic image, the other of the panoramic image corresponding to the point of the point and the adjacent corresponding to the one of a panoramic image, any one of claims 8 cuts out image 10 The information processing system according to one item. When the direction of the route changes by a predetermined value or more at a point corresponding to the one panoramic image, the image cutout unit selects from the one panoramic image among the plurality of panoramic images two or more corresponding to different directions. The information processing system according to claim 11 , wherein an image is cut out. The display control means stops the process of displaying the plurality of images in order for one image, and the direction of the viewpoint when cutting out the image from the panoramic image corresponding to the one image according to the direction of the display means. the information processing system according to any one of claims 1 to 12 to change the. The server device further includes a route determination unit that determines the route according to a user's instruction,
It said acquisition means, in any one of claims 1 to 13 to obtain a plurality of images extracted from a portion of each of the plurality of panorama images that are determined in accordance with the route determined by the routing means Information processing system described. The server device further includes a receiving unit that receives the route,
Said acquisition means, according to any one of claims 1 to 14 to obtain a plurality of images extracted from a portion of each of the plurality of panorama images that are determined in response to the received route by the receiving means Information processing system. Further having a terminal device,
The information processing system according to any one of claims 1 to 15 , wherein the terminal device includes the display unit. A method for displaying an image in an information processing system including a server device and an information processing device,
The server device storing panoramic image groups each corresponding to a point on the map;
The server device specifies a plurality of panoramic images from the panoramic image group according to a given route on the map;
The server device calculating a viewpoint orientation at a point on the route;
A step in which the server device cuts out one or more images corresponding to the viewpoint from each of the specified plurality of panoramic images;
When the rotation angle of the viewpoint at the first point on the route exceeds a threshold value , the server device sets the viewpoint at the second point immediately before the first point on the route at the first point. Rotating according to the viewpoint,
A step in which the information processing device acquires the plurality of cut out images;
The information processing device causes the display unit to sequentially display the plurality of images acquired in the acquiring step.

Images