JP6359870B2 - Information processing apparatus and moving image data reproducing method - Google Patents

Description

  The present invention relates to an information processing apparatus and moving image data reproduction method for displaying moving image data.

  Conventionally, a system capable of reproducing a plurality of moving image data in synchronization with each other is known. For example, in Patent Document 1 below, in a video browsing apparatus that performs multi-screen display, what is a video at the start of playback? There is a description that an independent playback time axis is provided, and synchronized playback of the video is realized by comparing the time stamp value given to each extracted video with the playback time axis.

International Publication No. 2008-044265

  However, the technique disclosed in Patent Document 1 has a problem in that an appropriate display process according to the positional relationship with the imaging target cannot be performed. For example, when moving image data obtained by shooting that the shooting target is moving is reproduced and displayed, there is a problem that the shooting target becomes smaller as the shooting target moves. For this reason, there is a demand for the user to select moving image data that can display a photographing target of a desired size. In addition, there is a demand for grasping the positional relationship with other moving image data.

  Accordingly, an object of the present invention is to provide an information processing apparatus and a moving image data reproduction method capable of reproducing moving image data to be photographed in an appropriate form as the photographing object moves.

  In order to solve the above-described problem, an information processing apparatus of the present invention includes a storage unit that stores management data describing a shooting position and a shooting direction for each scene in association with a plurality of moving image data, and a plurality of moving image data. Based on the positional relationship between the shooting position of one video data being played and the shooting target, and the positional relationship between the shooting position of other video data and the shooting target, the playback target is changed from one video data to another. Switching means for switching to moving image data.

  Further, the moving image data reproducing method of the present invention is a moving image data reproducing method in an information processing apparatus including a storage unit that stores management data describing a shooting position and a shooting direction for each scene in association with a plurality of moving image data. Based on the positional relationship between the shooting position of one video data being played and the shooting target, and the positional relationship between the shooting position of other video data and the shooting target among a plurality of video data, A switching step of switching from the moving image data to another moving image data.

  According to the present invention, from one moving image data to another based on the positional relationship between the shooting position of the one moving image data being reproduced and the shooting target, and the positional relationship between the shooting position of the other moving image data and the shooting target. Switch to the video data. Thereby, it is possible to switch to moving image data corresponding to the distance to the shooting target. For example, it is possible to switch to moving image data from a shooting position where the shooting target is close, and to view moving image data that better captures the shooting target. be able to.

  The information processing apparatus according to the present invention further includes a transmission unit that transmits one of the plurality of video data in a reproducible manner on the terminal side, and the transmission unit is another unit that is switched to the switching unit. The video data may be transmitted so as to be reproducible on the terminal side.

  In the information processing apparatus of the present invention, the switching unit generates a virtual line based on a shooting position of one moving image data being reproduced and a shooting position of another moving image data, and the virtual line and the one When the angle formed by the shooting direction of the moving image data and the angle formed by the shooting direction of the virtual line and the other moving image data transition with the reproduction of the one moving image data, When the angles match, the one moving image data is switched to the other moving image data.

  According to this invention, the virtual line based on the shooting position of the one moving image data being reproduced and the shooting position of the other moving image data is generated, and the angle formed by the virtual line and the shooting direction of the one moving image data And the angle formed by the imaginary line and the shooting direction of the other moving image data, and when the transition occurs along with the reproduction, the video data is switched from one moving image data to the other moving image data when the angles match. Accordingly, it is possible to determine that the distance between the shooting position and the shooting target is the same by a simple arithmetic process, and it is possible to view the moving image data in which the shooting target is better shot.

  The information processing apparatus according to the present invention further includes a determining unit that determines a reproduction start position of another moving image data based on a reproduction end position of the one moving image data at the time of switching when the switching unit performs switching. In addition, other moving image data is reproduced from the reproduction start position determined by the determining means.

  According to the present invention, the reproduction start position of other moving image data is determined based on the reproduction end position of one moving image data at the time of switching, and the reproduction of other moving image data is performed from the determined reproduction start position. As a result, since moving image data having overlapping shooting time zones is not reproduced, moving image data can be switched smoothly.

  Further, the information processing apparatus of the present invention is reproduced by using storage means for storing shooting positions for each scene in association with a plurality of moving image data, and information on shooting positions stored in the storage means. Calculating means for calculating a positional relationship between a shooting position for each scene in one moving image data and a shooting position for each scene in other moving image data, and based on the positional relationship for each scene calculated by the calculating means, Determining means for determining a display position of the other moving image data or image data indicating the other moving image data during reproduction of the one moving image data.

  Also, the moving image data reproducing method of the present invention is a moving image data reproducing method in an information processing apparatus including a storage unit that stores a shooting position for each scene in association with moving image data. A calculation step for calculating a positional relationship between each shooting position and a shooting position for each scene in other moving image data, and reproduction of the one moving image data based on the positional relationship for each scene calculated in the calculation step Determining the display position of the other moving image data or image data indicating the other moving image data.

  According to the present invention, the positional relationship between the shooting position for each scene in one moving image data being reproduced and the shooting position for each scene in other moving image data stored in advance is calculated, and each calculated scene is calculated. Based on the positional relationship, the display position of the other moving image data or the image data during the reproduction of one moving image data is determined. Thereby, other moving image data or its image data can be displayed at the display position based on the positional relationship between the shooting position of the other moving image data and the shooting target, and the viewer can select the other moving image data from the display position. The positional relationship between the shooting position and the shooting target can be grasped.

  The information processing apparatus according to the present invention includes a reproducing unit that reproduces one moving image data among a plurality of moving image data, and another moving image data or the other moving image data together with the one moving image data reproduced by the reproducing unit. Display means for displaying image data indicating the image data, and the display means displays other moving image data or image data indicating the other moving image data at the display position determined by the determining means.

  In addition, the information processing apparatus of the present invention transmits one moving image data to be reproduced so as to be reproducible, and displays the other moving image data determined by the determining unit or the display position of image data indicating the other moving image data Is transmitted to the terminal, and the terminal reproduces and displays one moving image data transmitted by the transmitting unit, and another moving image data or the other moving image is displayed at the transmitted display position. Image data indicating the data is displayed.

  In the information processing apparatus of the present invention, the storage unit further stores the shooting direction for each scene together, and the calculation unit includes the virtual line generated based on the shooting positions of a plurality of moving image data, and the virtual line. An angle formed by the shooting direction for each scene stored in the storage unit is calculated for each scene, and the determination unit determines the display position of the other moving image data or image data indicating the other moving image data, It is determined based on the angle calculated by the calculation means.

  According to the present invention, the angle formed by the virtual line generated based on the shooting positions of a plurality of moving image data and the shooting direction for each scene stored in advance is calculated for each scene. When playing and displaying images, the display position of other video data or its image data is determined based on the calculated angle, so that the intuitive relationship between the shooting position and the shooting target can be grasped by simple calculation. Can be made possible.

  According to the present invention, it is possible to switch to moving image data corresponding to the distance to the subject to be photographed. In addition, the viewer can grasp the positional relationship between the shooting position of the other moving image data and the shooting target from the display position of the other moving image data.

1 is a system diagram showing a system configuration of a communication system including a mobile terminal 100 which is an information processing apparatus of the present embodiment. 2 is a block diagram showing a functional configuration of a mobile terminal 100. FIG. 2 is a hardware configuration diagram of the mobile terminal 100. FIG. It is explanatory drawing which shows the specific example of management data. It is a conceptual diagram which shows the positional relationship of an imaging | photography object and a photographer. It is explanatory drawing which shows the imaging | photography direction and its angle when located in the initial position P of the imaging | photography object p. It is explanatory drawing which also shows the imaging | photography direction and its angle in each position AC when the imaging | photography object p moves from the position P to the position P1. It is explanatory drawing which showed the movement of the imaging | photography target p after that, and the imaging | photography direction from each position. 5 is a flowchart illustrating moving image data switching processing in the mobile terminal 100. 5 is a flowchart showing a reproduction process in the mobile terminal 100. It is a block diagram which shows the function structure of the portable terminal 100a in 2nd Embodiment. It is explanatory drawing which shows the specific example of management data. It is explanatory drawing which shows explanatory drawing which shows the display screen structure of the portable terminal. FIG. 14 is a conceptual diagram showing a schematic diagram of the display area 105a, a thumbnail map, and the positional relationship between each position and the shooting direction. It is explanatory drawing which shows the transition of the imaging | photography direction in each position with the movement of the imaging | photography object p, and the movement. It is explanatory drawing which showed the imaging | photography direction in each position AF, and the figure which shows the specific example of an imaging | photography direction management table. It is explanatory drawing for demonstrating matching with the angle described in the imaging | photography direction management table, and a moving image thumbnail area. It is explanatory drawing of the imaging | photography direction in each position when the imaging | photography target p moves, and the figure which shows the specific example of an imaging | photography direction management table. It is explanatory drawing for demonstrating matching with the angle described in the imaging | photography direction management table after a movement, and a moving image thumbnail area. It is explanatory drawing which shows the display screen structure by which the thumbnail image was displayed on the position based on the updated thumbnail map. It is a flowchart which shows the creation process of a thumbnail map. 4 is a flowchart showing moving image data reproduction processing and display screen control processing in the mobile terminal 100. It is explanatory drawing which shows the grouping process in a simple distribution. It is explanatory drawing which shows the grouping process based on a distribution pattern. It is explanatory drawing explaining the grouping method combined with direction information.

  Embodiments of the present invention will be described with reference to the accompanying drawings. Where possible, the same parts are denoted by the same reference numerals, and redundant description is omitted.

[First Embodiment]
The communication system according to the first embodiment is characterized in that the moving image data displayed on the mobile terminal can be automatically switched according to the relationship between the shooting target and the shooting position.

  FIG. 1 is a system diagram showing a system configuration of a communication system including a mobile terminal 100 which is an information processing apparatus of the present embodiment. As shown in FIG. 1, this communication system includes a Web server 200, a DB server 300, and a video processing server 400 in addition to the mobile terminal 100. The mobile terminal 100 can browse the moving image data stored in the video processing server 400 via the Web server 200. That is, the Web server 200 distributes a Web screen that enables selection of a plurality of moving image data in response to a request from the mobile terminal 100, and the mobile terminal 100 selects arbitrary moving image data via the Web screen. can do. The Web server 200 makes an acquisition request for the selected moving image data to the video processing server 400. The video processing server 400 distributes the moving image data specified in response to the acquisition request and distributes the moving image data to the mobile terminal 100 via the Web server 200. And the portable terminal 100 memorize | stores the distributed moving image data, The user can browse the content which is moving image data by reproducing | regenerating and displaying the moving image data memorize | stored by the user's operation.

  FIG. 2 is a block diagram illustrating a functional configuration of the mobile terminal 100. As illustrated in FIG. 2, the mobile terminal 100 includes a storage unit 11 (storage unit), a playback unit 12 (switching unit, determination unit, playback unit), a display unit 13, and an arithmetic processing unit 14. .

  The portable terminal 100 is configured by hardware such as a CPU, a ROM, and a RAM. FIG. 3 is a hardware configuration diagram of the mobile terminal 100. As shown in FIG. 3, the portable terminal 100 shown in FIG. 2 physically includes one or more CPUs 10a, a main memory RAM 10b and ROM 10c, input devices 10d such as a keyboard and a mouse, The computer system includes an output device 10e such as a display, a communication module 10f that is a data transmission / reception device such as a network card, and an auxiliary storage device 10g such as a hard disk or a semiconductor memory. Each function in FIG. 2 operates the input device 10d, the output device 10e, and the communication module 10f under the control of the CPU 10a by loading predetermined computer software on the hardware such as the CPU 10a and the RAM 10b shown in FIG. In addition, it is realized by reading and writing data in the RAM 10b and the auxiliary storage device 10g. Hereinafter, each functional block will be described based on the functional blocks shown in FIG.

  The storage unit 11 stores moving image data and management data related to the moving image data, and stores moving image data distributed from the video processing server 400 and its management data. FIG. 4 is an explanatory diagram showing a specific example of the management data. As shown in FIG. 4, the management data is described in association with a content ID, a scene ID, a position, and an azimuth angle, and a plurality of scene IDs, positions, and azimuth angles are described for each scene. Here, the content ID is an ID capable of uniquely specifying the moving image data, and the scene ID is an ID for specifying a frame (scene) determined arbitrarily or at predetermined intervals in the moving image data. The position and orientation indicate the shooting position and shooting direction. In this embodiment, the shooting position and the shooting direction are associated with each scene. However, the present invention is not limited to this, and the shooting position and the shooting direction may be associated with each frame. It may be associated with a scene determined every arbitrary time or every predetermined period.

  The playback unit 12 is a part that plays back the moving image data stored in the storage unit 11. The playback unit 12 can switch moving image data to be played back in response to an instruction from the arithmetic processing unit 14. In addition, the playback unit 12 plays back the video data after switching that is delayed based on the playback end position (playback time) of the video data before switching so that the switched video data has a temporal connection. The start position is determined, and the moving image data after switching is reproduced from that position.

  The display unit 13 is a part that displays moving image data reproduced by the reproduction unit 12.

  The arithmetic processing unit 14 is a part that performs arithmetic processing for specifying the positional relationship of the shooting positions of each moving image data based on the management data stored in the storage unit 11. That is, the arithmetic processing unit 14 determines the position of the shooting target and the moving direction thereof based on the shooting position and shooting direction described in the management data of each moving image data, and is determined to be closest to the shooting target. Switch to video data. The reproduction unit 12 reproduces the switched moving image data, and the display unit 13 displays the reproduced moving image data.

  Specific processing of the arithmetic processing unit 14 will be described with reference to the drawings. First, as a premise, grouping processing is performed according to the shooting position and shooting direction, and further, grouping processing including the same shooting target or shooting target person is performed by recognizing the shooting target by image recognition processing, and each moving image is processed. It is assumed that the same shooting target is included in the data.

  FIG. 5 is a conceptual diagram showing the positional relationship between the subject to be photographed and the photographer. In FIG. 5, the person to be photographed is a child who participates in an athletic meet, and moves in the moving direction S in the athletic field F that is the venue for the athletic meet. On the other hand, along the periphery of the athletic field F, the photographers a to c positioned at the shooting positions A to C are shooting the child that is the shooting target p. Each of the photographers a to c may photograph the same child or photograph another child within a range including the child.

  Then, as the subject p moves in the moving direction S, the photographer a located at the photographing position A changes the direction from the photographing direction Da1 to the photographing direction Da2. Similarly, the photographer b located at the photographing position B changes the direction from the photographing direction Db1 to the photographing direction Db2. The same applies to the photographer c.

  In the moving image data shot by the photographer a at the shooting position A, the shooting direction Da1 and the shooting direction Da2 are described in association with each scene as management data of the moving image data. The same applies to moving image data photographed by the photographer b and moving image data photographed by the photographer c.

  Next, as shown in FIG. 5, the shooting direction at each position according to the movement of the shooting target p and the angles with the respective positions A and B as vertices will be described. FIG. 6 is an explanatory diagram showing the shooting direction and its angle when it is positioned at the initial position P of the shooting target p. Note that the initial position P is a position calculated based on the position A, the position B, the shooting direction Da1, and the shooting direction Db1. Note that the calculation may be further performed using the position C and the shooting direction Dc1.

  As shown in FIG. 6A, the photographers a to c at the shooting positions A to C are directed in the direction of the shooting target p (shooting directions Da1, Db1, and Dc1), respectively. The angle Θa formed at the position P, the position A, and the position B, with the position A as the vertex, and the angle Θb formed at the position P, the position B, and the position A, with the position B as the vertex, This is expressed as shown in FIG. FIG. 6B shows the relationship of angle Θa> angle Θb, as is apparent from the figure. In this case, it can be determined that the imaging target p is close to the position A.

  Next, the shooting direction and the angle at each of the positions A to C when the shooting target p moves from the position P to the position P1 will be described. FIG. 7 is an explanatory view showing the state. Note that the position P1 is a position calculated based on the positions A and B and the imaging directions Da11 and Db11, as described above.

  As shown in FIG. 7A, the photographers a to c at the shooting positions A to C are directed to the shooting target directions (shooting directions Da11, Db11, and Dc11) at the position P1, respectively. The angle Θa1 formed in the order of the position P1, the position A, and the position B, and the angle Θb1 formed in the order of the position P1, the position B, and the position A are expressed as shown in FIG. 7B. The As shown in FIG. 7B, here, the angle Θa1 and the angle Θb1 are almost the same angle. Therefore, it can be determined that the isosceles triangle is formed by the line segment formed by connecting the position P1, the position A, and the position B, and the distance from the position A to the imaging target p (position P1); It can be determined that the distance from the position B to the imaging target p (position P1) matches.

  The imaging target p has moved from the position P to the position P1, and has moved from left to right in FIG. Then, it can be seen that the angle Θa changes to the angle Θa1, and the angle having the position A as the apex becomes small. Therefore, since it is determined that the distance from the position B to the shooting target p is shorter than the distance from the position A to the shooting target p, when these distances match, the moving picture data is switched to switch the shooting target p. Can be selected.

  FIG. 8 is an explanatory diagram showing the subsequent movement of the imaging target p and the imaging direction from each position. As shown in FIG. 8A, the imaging target p is further moved to the position P2. Here, as shown in FIG. 8B, the angle Θb2 formed by the position P2, the position B, and the position C having the position B as the apex, and the position P2, the position C, and the position having the position C as the apex It is determined that the angle Θc1 formed by B is the same. In that case, it is determined that the distance from the position P2 to the position B is the same as the distance from the position P2 to the position C, and the moving image data is changed from the position B to the position C. Can be switched.

  As described above, the arithmetic processing unit 14 calculates the position P of the shooting target p from the shooting position and shooting direction of each moving image data based on the management data of each moving image data, and further, based on the shooting direction, the shooting process is performed. The moving direction of the object p, the positions P1 and P2 that are the movement destinations, and the like can be calculated. Based on these positions P to P2 and positions A to C, the distance from each position to the object to be imaged p can be determined to determine from which position the moving image data is close. Can be selected automatically.

  In FIGS. 6 to 8, the position P is calculated when calculating the angle Θa. However, the present invention is not limited to this. For example, the line segment AB formed at the position A and the position B, The angle Θa can also be calculated from the shooting direction Da. The same applies to other angles Θb and the like.

  Next, processing of the mobile terminal 100 configured as described above will be described. FIG. 9 is a flowchart showing moving image data switching processing in the mobile terminal 100. Here, processing when focusing on position A and position B will be described. Note that, among the positions A to C, the position A and the position B that are closest to the photographing target p (position P9) are selected as processing targets.

  The arithmetic processing unit 14 identifies two positions close to the position P based on the management data stored in the storage unit 11, and extracts the shooting direction. Here, first, a position A and a position B are specified (S101), and shooting directions at the positions A and B are extracted (S102).

  Then, the arithmetic processing unit 14 calculates an angle having each of the two angles as a vertex, and calculates Δα that is a difference (subtract the other angle from one angle). Here, an angle Θa having the position A as the vertex and an angle Θb having the position B as the vertex are calculated (S103), and Δα = angle Θa−angle Θb, which is the difference between them, is calculated (S104). Here, the arithmetic processing unit 14 determines whether or not the first shooting position Δα> = 0 (S105).

  If it is determined that Δα> = 0, it is determined to display moving image data with the position A as the shooting position (S106). On the other hand, if it is determined that Δα> = 0, it is determined to display moving image data with the position B as the shooting position (S107). In the above-described method, Δα is calculated by subtracting the angle based on the position B from the angle based on the position A. However, the present invention is not limited to this, and may be reversed, and formed at any position. The calculation method is not limited to the above-described method as long as it can determine whether or not the angle is large.

  When playback is continued, if the playback unit 12 determines that the video data being played has reached the next scene, the process of specifying the two shooting positions at that time is performed to determine the display image. Repeat. This iterative process is performed for each scene assigned by the scene ID. Then, during the repeated processing, two positions that are close to each other again are extracted from positions A to C and set as processing targets. For example, when the position B and the position C are taken out, Δα is calculated according to the angle formed based on the position B and the position C, and the moving image data determined based on the Δα is reproduced.

  If it is determined that the same image as the currently displayed image is displayed at the time of determining the display image in S105, the content is continuously displayed without performing the switching process.

  In the process of FIG. 9, the process focuses on the position A and the position B, but it is assumed that the shooting position is actually 3 or more. In that case, the position to be the starting point is determined based on the shooting direction at each position. In the case of the present embodiment, since the shooting directions of the positions A to C each change clockwise (clockwise), the end point is the one in which the shooting direction is more leftward. In the case of the present embodiment, it can be determined that the shooting direction of the position C is directed leftward from the shooting direction of other positions, and the position A at the end opposite to the position C is the position A. It can be determined as the starting point. Therefore, the reproduction order of the moving image data can be determined based on these shooting positions and shooting directions.

  As another method, a position where the angle between the straight line connecting two adjacent points out of the three points and the shooting direction at each of the two points is closest to 90 degrees is defined as position A. Another position adjacent to the selected position may be selected as the position B.

  In this way, moving image data from the shooting position where the shooting distance is the closest can be automatically selected with the movement of the shooting target p.

Next, playback processing of the mobile terminal 100 will be described. FIG. 10 is a flowchart showing the reproduction process. The moving image data M _n is selected by an operation by the user of the mobile terminal 100 (S201), and the selected moving image data is reproduced by the reproducing unit 12 and displayed on the display unit 13 (S202).

Here, when the reproduction is started, in parallel with (or before the reproduction), the reproduction unit 12 acquires the shooting start times of the moving image data M _n before the switching and the moving image data M _{n + 1} after the switching. (S203). Then, the reproduction unit 12 calculates an offset value, which is a difference in shooting start time (S204).

When a switching instruction is received from the arithmetic processing unit 14 (S204), the playback unit 12 acquires the playback time T of the video data M _n before switching (S206), and the playback start position of the video data M _{n + 1} after switching. Is calculated (S207).

When the delay time T1 is calculated, the playback unit 12 plays back the moving image data M _{n + 1} after switching from the position of the delay time T1 (S208).

  In this way, it is possible to perform switching reproduction with a temporal connection between the moving image data before switching and the moving image data after switching.

  In addition, in this embodiment mentioned above, although the portable terminal 100 described as switching video data, it does not restrict to this. In order for the video processing server 400 to be able to reproduce one video data on the mobile terminal 100 to the mobile terminal 100, the video data is automatically switched according to the shooting position and streamed while streaming. May be performed.

  In addition, the mobile terminal 100 stores only management data of each moving image data, and appropriately requests distribution of moving image data to the video processing server 400 based on the shooting position and shooting direction of each moving image data. It may be.

  For example, when the mobile terminal 100 receives streaming distribution of moving image data before switching from the video processing server 400, the portable terminal 100 performs processing of switching to other moving image data based on the positional relationship. At that time, the video data after switching is designated, and a switching request designating the reproduction position is transmitted to the video processing server 400. When the video processing server 400 receives a switching request from the portable terminal 100, the video processing server 400 performs switching processing of other moving image data and performs streaming distribution of the other moving image data. With such a configuration, the mobile terminal 100 can perform switching control, and the video processing server 400 can perform moving image data distribution processing according to the control.

  Next, effects of the mobile terminal 100 used in the communication system according to the first embodiment will be described.

  According to the portable terminal 100, the storage unit 11 stores a plurality of pieces of moving image data including the same shooting target describing the shooting position and shooting direction for each scene. Then, the reproduction unit 12 reproduces one specified moving image data among the moving image data stored in the storage unit 11 and displays the display unit 13.

  On the other hand, the arithmetic processing unit 14 determines the positional relationship between the shooting position of one moving image data being played back by the playing unit 12 and the shooting target, and the positional relationship between the shooting position of another moving image data and the shooting target for each predetermined period. (For example, for each scene), the playback unit 12 performs a process of switching from one video data to another video data based on these positional relationships, and plays back the switched video data. The display unit 13 Display this. Thereby, it is possible to switch to moving image data from a shooting position where the shooting target is close, and it is possible to view moving image data obtained by shooting the shooting target better.

  In addition, according to the mobile terminal 100, the arithmetic processing unit 14 generates a virtual straight line based on the shooting position of one moving image data being played back by the playing unit 12 and the shooting position of other moving image data, The angle formed by the straight line and the shooting direction of one moving image data, the angle formed by the virtual straight line and the shooting direction of the other moving image data, and the angles coincide with each other when the moving image data is reproduced. Determine whether or not. When it is determined that they match, the playback unit 12 switches to other moving image data and performs playback processing. Accordingly, it is possible to determine that the distance between the shooting position and the shooting target is the same by a simple arithmetic process, and it is possible to view the moving image data in which the shooting target is better shot.

  Further, according to the portable terminal 100, the playback unit 12 determines the playback start position of other video data based on the playback end position of one video data at the time of switching, and plays back the determined other video data Playback from the start position. Thereby, since the moving image data in which the photographing time zones overlap are not reproduced, smooth reproduction can be performed.

  In the above-described effects, the description has been made on the assumption that the storage unit stores moving image data and management data in the mobile terminal 100. However, the present invention is not limited to this, and the mobile terminal 100 stores only management data. In addition, the moving image data switching process described in the present embodiment may be determined, and the moving image data may be distributed to, for example, the video processing server 400 as a switching instruction. At this time, an ID indicating moving image data may be sent to the video processing server 400 and the reproduction start position may be given.

It is also conceivable that the video processing server 400 is provided with the above switching function. In this case, the video processing server 400 includes a transmission unit (transmission unit) that performs streaming distribution instead of the display unit 13. In the video processing server 400, the transmission unit switches the moving image data to be reproduced based on the positional relationship determination processing by the arithmetic processing unit 14, and the moving image data is streamed. Thus, the present invention can be applied to the video processing server 400 side without being limited to the portable terminal 100.

[Second Embodiment]
Next, a communication system in the second embodiment will be described. The communication system according to the second embodiment displays one moving image data at a display position corresponding to the positional relationship between a shooting target and a shooting position in a mobile terminal that displays moving image data, and an image showing other moving image data Data is displayed around it.

  In the second embodiment, the same system configuration as that of the communication system shown in FIG. 1 is adopted, and the mobile terminal 100a receives the video data distributed from the video processing server 400, and reproduces and displays it. Is. In addition, the mobile terminal 100a handles a moving image data group that has been grouped based on the shooting position and the shooting direction, and appropriately sets the display position for the plurality of grouped moving image data. It is arranged and displayed at a position.

  FIG. 11 is a block diagram illustrating a functional configuration of the mobile terminal 100a according to the second embodiment. As shown in FIG. 11, the mobile terminal 100 includes a receiving unit 101, a storage unit 102 (storage unit), a playback unit 103, a display screen control unit 104 (calculation unit, determination unit), and a display unit 105 (display unit). It is configured to include. The portable terminal 100 is configured by hardware including a CPU, a ROM, a RAM, and the like, and is realized by, for example, the hardware illustrated in FIG. Hereinafter, each component in the block diagram shown in FIG. 11 will be described.

  The receiving unit 101 is a part that receives moving image data and its management data from the video processing server 400. In the video processing server 400, the moving image data is grouped according to a predetermined condition, and moving image data and management data of the same group are transmitted and received.

  The storage unit 102 stores moving image data and management data received by the receiving unit 101. FIG. 12 shows a specific example of management data. As shown in FIG. 12, the management data is described in association with the content ID, scene ID, position, and azimuth. This is the same as the management data shown in FIG. 4 of the first embodiment, where the position indicates the shooting position and the azimuth indicates the shooting direction. The position and azimuth are measured for each scene and described in the management data.

  The storage unit 102 stores a thumbnail map and a shooting direction management table, which will be described later.

  The playback unit 103 is a part that plays back the moving image data stored in the storage unit 102. When the user of the mobile terminal 100 selects one moving image data from among several moving image data, the reproducing unit 103 reproduces the one moving image data.

  The display screen control unit 104 is a part that controls the screen configuration when displaying moving image data reproduced by the reproduction unit 103.

  FIG. 13 is an explanatory diagram showing a display screen configuration. As shown in FIG. 13, in the display area 105a of the display unit 105, a moving image reproduction area 105b and a moving image thumbnail area 105c are formed. The moving image reproduction area 105 b is an area for displaying moving image data that has been reproduced by the reproducing unit 103. The moving image thumbnail area 105c is an area for displaying thumbnail images indicating other moving image data belonging to the same group as the moving image data being reproduced. This thumbnail image is composed of the first frame portion of the moving image data, but may be image data prepared for other thumbnail images.

  The moving image thumbnail area 105c is formed so as to surround the moving image reproduction area 105b. A predetermined number of thumbnail images are displayed in the moving image thumbnail area 105c. In FIG. 13, three thumbnail images are displayed per side of the moving image playback area 105b.

  In the moving image thumbnail area 105c, a thumbnail image of moving image data corresponding to the relationship with the shooting position of the moving image data displayed (reproduced) in the moving image reproduction area 105b is displayed. For example, in the example of FIG. 13, thumbnail images A to C are displayed in the moving image thumbnail area 105c arranged on the upper side of the moving image reproduction area 105b. Similarly, a thumbnail image F is displayed in the moving image thumbnail area 105c arranged on the right side of the moving image reproduction area 105b, and thumbnail images D and E are displayed in the lower moving image thumbnail area 105c. The display positions of the thumbnail images A to F are defined according to the positional relationship with the shooting position. Hereinafter, a specific method for defining will be described.

  FIG. 14A is a schematic diagram of the display area 105a. A moving image thumbnail area 105c is formed around the moving image reproduction area 105b, and the moving image thumbnail area 105c includes a unit area No. 1 to 12 are specified. This unit area No. In 1 to 12, thumbnail images of moving image data corresponding to the shooting position are displayed.

  FIG. 14B shows the unit region No. It is a management table for prescribing which moving image data is arranged in any of 1-12. In the present embodiment, this management table is referred to as a thumbnail map. This thumbnail map is stored in the storage unit 102, and is updated with reference to the management data as appropriate according to the elapsed time of the moving image data being reproduced. Specifically, as shown in FIG. 14B, a unit area corresponding to the shooting position of each moving image data is designated. In 1, moving image data having a position A as a photographing position is designated. Similarly, the unit area No. In 2, moving image data having the position B as the shooting position is designated. Similarly, moving image data is designated in other unit areas.

  On the other hand, “-” may be described in the thumbnail map. For example, the unit area No. 5 describes “-”. This indicates that there is no moving image data at the corresponding shooting position, and nothing is displayed in this unit area.

  FIG. 14C shows the positional relationship between the positions A to F and the photographing direction thereof. The example here shows the arrangement of photographers on both sides of the road in order to photograph the runner running on the road. The positions A to C are gathered to some extent and are located in the upper region. On the other hand, the positions D and E are located on the lower side. Further, the position F is located on the upper side, but is slightly away from the other positions. A thumbnail map is generated based on such a positional relationship. As shown in FIG. 14B, the positions A to C located on the upper side are unit area Nos. 1 to 3, respectively. The position F is a unit area No. 4 is specified. This allows a viewer of moving image data to intuitively grasp the shooting position of moving image data by displaying a thumbnail image at a display position corresponding to the actual positional relationship. For the same reason, the positions D and E are unit areas No. 8 and 9 are specified.

  Next, a more specific method for updating the thumbnail map will be described. FIG. 15 is an explanatory diagram illustrating the movement of the photographing target p and the transition of the photographing direction at each position accompanying the movement. As shown in FIG. 15A, the photographing target p is located at the position P. At that time, the photographers at the positions A to F are each directed in the direction of the photographing target p.

  Then, as shown in FIG. 15B, when the photographing target p moves to the position P1, the photographer at each of the positions A to F changes to the direction of the photographing target p after the movement, respectively. ing.

  As can be seen from FIG. 15, for example, the shooting direction at position A has shifted from substantially downward to diagonally downward to the right. On the other hand, the shooting direction at the position F has shifted from the diagonally lower left direction to the substantially lower direction. With such a transition, it is necessary to change the display position of the thumbnail image to be displayed in the moving image thumbnail area.

  The thumbnail map update process will be described below. FIG. 16A is an explanatory diagram showing the photographing directions at the positions A to F. FIG. Each moving image data having the shooting positions at positions A to F is grouped into one group and associated. Here, the moving image data at positions A to C and F are further grouped as the upper group, and the moving image data at positions D and E are grouped as the lower group.

  In each group, the shooting direction at each position is described in the shooting direction management table. FIG. 16B is a shooting direction management table table1 for managing the shooting directions of the positions A to F of the upper group. FIG. 16C is a shooting direction management table table2 for managing the shooting directions of the position D and the position E of the lower group. An angle is described for each shooting position in each shooting direction management table. Also, the arrangement order to be displayed in the moving image thumbnail area is described.

  This angle is calculated by the following process. That is, a virtual line is generated by connecting any two or all of the positions A to C and F belonging to the upper group, and based on the virtual line and the shooting direction at each position, the virtual line is used as a reference. The calculated angle is calculated. For example, in FIG. 16A, the angle ΘA at the position A is represented by an angle formed by a virtual line (vector in the CA direction) and the shooting direction from the position A, but is not limited thereto. It may be expressed as an angle formed by a virtual line (AC direction vector) and a shooting direction from the position A.

  In the present embodiment, as the angles at the positions A to C and F, the angle formed on the lower side with respect to the virtual line is calculated, and as shown in FIG. 16B, the position A and the virtual line (CA direction) Is formed as 95 degrees.

  On the other hand, as the angles at the positions D and E, the angle formed on the upper side with respect to the virtual line is calculated, and as shown in FIG. 16C, the angle formed between the position D and the virtual line is 100 degrees. Is calculated.

  Further, the arrangement order of the positions A to F in each shooting direction management table is defined based on the respective position information. Here, the thumbnail images are arranged in order from the left with reference to two virtual lines. It stipulates that It should be noted that which one of the virtual lines is to be the upper side is defined in advance, but is not limited to this, and may be changed as appropriate, such as raising the one with the larger number of shooting positions. Good.

  FIG. 17 is an explanatory diagram for explaining the association between the angle described in the shooting direction management table and the moving image thumbnail area. As shown in FIG. 1 to 3 are defined as areas for displaying thumbnail images of moving image data whose shooting direction is close to 90 degrees in the shooting direction management table table1 of the upper group. Note that the angle in the shooting direction is not limited to a value close to 90 degrees, and may be, for example, 45 degrees or more.

  In the present embodiment, thumbnail images of moving image data having positions A to C described in the shooting direction management table table1 of the upper group as shooting positions are displayed as unit area numbers. It is defined to display in the areas 1 to 3 (see FIG. 17B). On the other hand, in the shooting direction management table table1 of the upper group, the shooting direction at the position F is 30 degrees. Therefore, when the angle is close to 0 degrees or less than a predetermined angle (for example, 45 degrees or less), the thumbnail map is defined so as to be displayed in the thumbnail areas defined on the left and right sides of the moving image reproduction area 105b. (See FIG. 17B).

  In the moving image thumbnail area 105c, a thumbnail image is displayed at a position corresponding to the positional relationship with the shooting target. For example, in FIG. 17, in the moving image thumbnail area 105c on the lower side, thumbnail images from position D and those from position E are moved to the left side (corresponding to unit areas No. 9 and No. 8). Defined) to be displayed. The right side (unit area No. 7) is in a vacant state. In this case, the shooting positions from the position D and the position E are intuitively expressed as those shot from slightly behind the shooting target.

  In order to perform such processing, with respect to a predetermined angle (for example, 90 degrees or less) described in the shooting direction management table table2 in FIG. No. 8 or No. 9 is defined.

  In the example of FIG. 17, the definition process is performed so that the object close to 0 degree (for example, 0 degree to 45 degree or less) is displayed on the right side of the moving image reproduction area 105b, and the object close to 180 degree (for example, 135 The definition processing is performed so that the video is displayed on the left side of the moving image reproduction area 105b. For example, the unit area No. Definition is performed so that a thumbnail image of moving image data having the position F as the shooting position is displayed at the position 4.

  Similarly, the thumbnail image of the moving image data having the shooting positions at the positions D and E described in the shooting direction management table table2 of the lower group is a unit area number. Define to display in 7-9.

  Next, update of the shooting direction management table when the shooting direction at the positions A to F also changes when the shooting target p moves will be described.

  FIG. 18 is a diagram illustrating the shooting direction at each position when the shooting target p moves and the description content of the shooting direction management table at each position.

  FIG. 18A is an explanatory diagram showing the shooting direction at positions A to F after the movement of the shooting target p. FIG. 18B and FIG. 18C are explanatory diagrams showing the arrangement order and the angles based on the photographing direction.

  Similarly to the above, the angle based on the shooting direction at each of the positions A to F is described in the shooting direction management table table1 of the upper group. Since the object to be photographed p has moved, each angle has transitioned to an obtuse angle (in the direction in which the angle increases). Similarly, the lower group shooting direction management table table2 describes angles based on the shooting direction at the positions D and E, respectively.

  As shown in FIG. 18A, the position C has moved greatly, and the angle thereof is about the same as the position F and the position E.

  Then, according to the shooting direction management table table1 of the upper group and the shooting direction management table table2 of the lower group, the unit area No. shown in FIG. It is determined which of 1 to 12 is to be defined. As in FIG. 17, it is defined which unit region is associated with the angle based on the angle.

  As shown in FIG. 18A, since the position C has moved, as shown in FIGS. 19A and 19B, the position C has the unit region No. 2 and position F is unit region No. 3 is defined. Similarly, unit areas based on angles are defined for moving image data at other positions.

  FIG. 20 is an explanatory diagram showing a display screen configuration in which thumbnail images are displayed at positions based on the thumbnail map updated as shown in FIG. As shown in FIG. 20, a moving image thumbnail image is displayed at a position according to the thumbnail map of FIG. Compared with the display screen shown in FIG. 13, the display position of the thumbnail image displayed in the moving image thumbnail area 105 c is changed with the movement.

  In this way, the display screen control unit 104 controls the display screen configuration, so that the user can grasp the positional relationship between the moving image data reproduced in the reproduction moving image area and other moving image data.

  Next, processing of the mobile terminal 100 configured as described above will be described. FIG. 21 is a flowchart showing thumbnail map creation processing, which shows processing by the display screen control unit 104. Here, the processing is described as processing of the mobile terminal 100 that acquired the moving image data and the management data thereof from the video processing server 400, but the video processing server 400 may perform thumbnail map creation processing.

  First, upon receiving a thumbnail map creation instruction, the display screen control unit 104 extracts moving image data and management data in the same group from the storage unit 102 (S301). Then, the position and shooting direction in one scene are extracted for each moving image data from the management data of each moving image data (S302).

  Based on the position and shooting direction for each moving image data in the extracted one scene, the moving image data (position and shooting direction) is grouped into the upper group and the lower group (S303). Then, in each of the upper group and the lower group, the angle between the shooting direction of the moving image data at each position and the virtual line generated based on the position of each moving image data is calculated (S304). Then, an angle is described for each position in the photographing direction management table (S305), and a thumbnail map is created so as to be defined in a unit area determined based on the angle (S306).

  Then, it is determined whether or not there is a next scene. If there is a next scene (S307), the position and shooting direction for each corresponding moving image data are extracted from the management data (S308). Then, the moving image data in the scene is grouped, and a thumbnail map creation process is performed. If it is determined in S307 that there is no next scene, the thumbnail map creation process ends.

  In this way, when the thumbnail map is created and the moving image data is reproduced, display screen control according to the thumbnail map is performed.

  Next, display screen control using the thumbnail map created in this way will be described. FIG. 22 is a flowchart showing the moving image data reproduction processing and display screen control processing in the portable terminal 100.

  First, one moving image data is selected by a user operation of the mobile terminal 100 (S401), and reproduction processing by the reproduction unit 103 is performed (S402). Then, the display screen control unit 104 performs the display so that the thumbnail image is displayed at a position corresponding to the created thumbnail map (S403).

  Here, there is a transition of scene transition (S404). When a scene transitions, a thumbnail map for the next scene is read, and a thumbnail image display process corresponding to the thumbnail map is performed (S403).

  In addition, when the thumbnail image is selected in the middle and an instruction to switch the moving image data is given (S405), the moving image is reproduced. Then, until these processes are finished, an interruption check is performed to determine whether or not there is a scene transition and whether or not moving image data is switched (S404 to S406).

  As in the first embodiment, when switching the moving image data, the reproduction position corresponding to the reproduction time of the moving image data before switching is set so that the reproduction of the moving image data has a temporal connection. It is preferable to reproduce the moving image data for each switching. For this processing, the playback unit 103 first acquires the shooting start time of each grouped moving image data prior to playback, and calculates an offset value that is the difference in shooting start time between the moving image data. .

  Thereafter, when a thumbnail image is selected by the user's operation and the playback unit 103 plays back the moving image data associated with the thumbnail image, the playback time T between the moving image data before switching and the moving image data after switching is between The delay time T1 can be calculated by subtracting the offset value. Using the delay time T1, the reproducing unit 103 can reproduce the moving image data from the position where the reproduction start position is shifted by the delay time T1.

  In the above-described embodiment, the thumbnail image display processing is performed using the moving image data and management data stored in the mobile terminal 100. However, the present invention is not limited to this, and the video processing server 400 performs streaming distribution. A transmission means for performing streaming distribution, and streaming distribution of thumbnail images of other video data associated with the video data to be distributed by designating a display position on the mobile terminal 100 May be performed.

  Further, as described in the first embodiment, the mobile terminal 100 stores only management data, makes a switching determination based on the management data, and requests the video processing server 400 to distribute the moving image data after switching. The video processing server 400 may select and distribute the requested moving image data.

  Next, operational effects of the mobile terminal 100 in the communication system of the present embodiment will be described.

  According to the mobile terminal 100, the playback unit 103 plays back one video data among a plurality of video data stored in the storage unit 102, while the display screen control unit 104 plays back one video The positional relationship between the shooting position for each scene in the data and the shooting position for each scene in other moving image data stored in advance is calculated. Then, the display unit 105 displays a thumbnail image that is image data indicating other moving image data at a display position based on the calculated positional relationship for each scene. Thereby, the viewer can grasp the positional relationship between the shooting position of other moving image data and the shooting target from the display position.

  Further, in the portable terminal 100, the display unit 105 reproduces and displays one moving image data in the center, and displays image data indicating other moving image data around the reproduction display position of the one moving image data. Accordingly, the viewer can intuitively grasp the positional relationship between the shooting position of other moving image data and the shooting target.

  In addition, according to the mobile terminal 100, the display screen control unit 104 determines the angle formed by the virtual line generated based on the shooting positions of the plurality of moving image data and the shooting direction for each scene stored in advance. The calculation is performed for each scene, and the display unit 105 displays the display position of the thumbnail image at the display position based on the calculated angle when reproducing and displaying one moving image data. It is possible to intuitively grasp the positional relationship between the position and the photographing target.

In this embodiment, thumbnail images are displayed. However, the present invention is not limited to this, and the moving image data itself may be reproduced and displayed.

[Third Embodiment]
Next, an embodiment of the grouping process will be described. The mobile terminal 100 in the first and second embodiments described above can perform moving image data switching processing and thumbnail image display processing on moving image data that has been grouped in advance. This grouping process may be performed prior to each mobile terminal 100 playing a moving image, or a grouping processing device (not shown) that exists separately from the video processing server 400 and the mobile terminal 100 may be used. A grouping process may be performed to provide a group of moving image data groups (a plurality of moving image data and management data) to the video processing server 400 and the mobile terminal 100, respectively. Hereinafter, a grouping process applicable to the first embodiment and the second embodiment will be described.

  FIG. 23 is an explanatory diagram showing grouping processing in a simple distribution. As shown in FIG. 23, the position A to the position D are included in the grouping range a to the grouping range d centering on each position. Here, the position X is not included in any grouping range. In this case, a virtual line Q is generated based on the positions A to D, and when there is a position X on the virtual line Q, a grouping process is performed to treat the position X as the same group as the positions A to D. Note that there are various methods for generating the virtual line Q, and a position connecting the positions at the ends of the positions A to D (in this case, the position A and the position D) may be used as the virtual line Q. A virtual line may be obtained by averaging several lines obtained by connecting two positions among the positions A to D. Further, it is not necessary to be positioned directly above the virtual line Q, and a certain amount of width may be given, and the positions within the width may be grouped as the same group.

  FIG. 24 is an explanatory diagram illustrating grouping processing based on the distribution pattern. Since the positions A to D shown in FIG. 24 are in the same grouping range as shown in FIG. 23, the grouping process is performed assuming that they are the same group. Further, the position X is determined to be the same group based on the virtual line Q1 generated based on the positions A to D.

  However, the positions X1 to X4 cannot be determined to be the same group based on the virtual line Q1. Even in this case, if it can be determined that the positions X to X4 are in the same grouping range, the positions X1 to X4 can be the same group.

  In the present embodiment, the association processing unit searches for a group distributed in parallel with the virtual line Q1. That is, first, the distribution generation unit may generate a virtual line Q1 that is a distribution pattern based on the positions A to D, or the virtual line Q1 indicated by the distribution pattern information and distribution size information may be generated as a distribution pattern. You may select from a memory | storage part. Here, the virtual line Q1 which is a “straight line” is generated or selected as the distribution shape. On the other hand, the distribution generation unit performs grouping processing on the assumption that the positions X1 to X4 are the same group. Here again, the virtual line Q2, which is the distribution pattern, is generated or selected.

  When the relevance processing unit can determine that the virtual line Q1 and the virtual line Q2 are within a predetermined distance and are substantially parallel, the positions X1 to X1 used to generate the virtual line Q2 are used. The grouping process is performed assuming that the position X4 and the positions A to D are the same group.

  FIG. 25 is an explanatory diagram for explaining a grouping method combined with orientation information. As shown in FIG. 24, the grouping process is performed on the positions A to D and the positions X to X4 as being in the same group. In this example, a process for removing the grouping process is further performed based on the shooting direction. In FIG. 25, the content with the position X3 as the shooting position does not face between the virtual lines Q1 and Q2, but faces the outside. Therefore, it is not appropriate to group contents facing in such a shooting direction as the same group, and therefore processing for removing from the same group is performed. In other words, the relevance processing unit performs processing for excluding content having the position X3 as the shooting position from the group information storage unit.

  By performing further image recognition processing on the content having the positions grouped by such grouping processing as shooting positions, only moving image data including the same shooting target or the same shooting target is associated as the same group. Perform processing.

  In this way, moving image data that has been subjected to grouping processing is automatically switched based on the shooting position and shooting direction, or the arrangement of thumbnail images is determined. Can be smoothly switched, and the positional relationship can be intuitively determined when viewing a moving image.

DESCRIPTION OF SYMBOLS 11 ... Memory | storage part, 12 ... Playing part, 13 ... Display part, 14 ... Arithmetic processing part, 100 ... Portable terminal, 100a ... Portable terminal, 101 ... Receiving part, 102 ... Memory | storage part, 103 ... Reproducing part, 104 ... Display screen Control unit, 105 ... display unit, 200 ... Web server, 300 ... DB server, 400 ... video processing server.

Claims (4)

Storage means for storing shooting positions for each scene in association with a plurality of moving image data;
Calculation means for calculating the positional relationship between the shooting position for each scene in the reproduced moving image data and the shooting position for each scene in the other moving image data, using the information of the shooting position stored in the storage means When,
Determining means for determining a display position of the other moving image data or a thumbnail image indicating the other moving image data during reproduction of the one moving image data, based on the positional relationship for each scene calculated by the calculating means;
Playback means for playing back one of the plurality of video data;
Display means for displaying other moving image data or a thumbnail image indicating the other moving image data together with the one moving image data being reproduced by the reproducing means;
With
The determination means creates a thumbnail map that defines the display position of another moving image data or a thumbnail image indicating the other moving image data,
The display means displays other moving image data or a thumbnail image indicating the other moving image data at a position corresponding to the thumbnail map created by the determining means.
Information processing device. Transmitting means for reproducibly transmitting one moving image data to be reproduced, and transmitting the moving image data determined by the determining means or a display position of a thumbnail image indicating the other moving image data to the terminal ,
On the terminal, one moving image data transmitted by the transmitting unit is reproduced and displayed, and another moving image data or a thumbnail image indicating the other moving image data is displayed at the transmitted display position.
The information processing apparatus according to claim 1. The storage means
Furthermore, memorize the shooting direction for each scene together,
The computing means is
An angle formed by a virtual line generated based on a shooting position of a plurality of moving image data and a shooting direction for each scene stored in the storage unit is calculated for each scene,
The determining means includes
Determining the display position of the other moving image data or the thumbnail image indicating the other moving image data based on the angle calculated by the calculating means;
The information processing apparatus according to claim 1 or 2 . In a moving image data reproduction method in an information processing apparatus including a storage unit that stores a shooting position for each scene in association with moving image data,
A calculation step for calculating a positional relationship between a shooting position for each scene in the reproduced video data and a shooting position for each scene in the other video data;
A determination step of determining a display position of the other moving image data or a thumbnail image indicating the other moving image data during reproduction of the one moving image data, based on the positional relationship for each scene calculated in the calculating step;
With
A playback step of playing back one of the plurality of video data;
A display step of displaying another video data or a thumbnail image indicating the other video data together with the one video data being played in the playback step;
With
The determination step creates a thumbnail map that defines the display position of another moving image data or a thumbnail image indicating the other moving image data,
The display step displays other moving image data or a thumbnail image indicating the other moving image data at a position corresponding to the thumbnail map created in the determining step.
Video data playback method.

Images