JP5967126B2 - Terminal device and program - Google Patents

Description

  The present invention relates to a technical field such as an apparatus for creating virtual camera data representing the orientation of a virtual camera and the angle of view of a virtual camera in an image frame of moving image data composed of a plurality of image frames.

  Conventionally, there is a case where pseudo camera work is performed on predetermined moving image data by a user's operation of a virtual camera. In the pseudo camera work, a specific pixel region is displayed on the display screen by, for example, the user specifying the direction of the virtual camera, the angle of view of the virtual camera, or the like in the image frame constituting the moving image data. Such virtual camera data representing the orientation of the virtual camera and the angle of view of the virtual camera can be uploaded from the terminal device to the distribution server. Patent Document 1 discloses a communication system that can share pseudo camera work by other users without editing uploaded video data itself.

JP, 2014-011623, A

  However, for example, when an object (display target) in a moving image moves unexpectedly, the user may not be able to smoothly track a desired object by the camera work.

  The present invention provides a terminal device and a program capable of smoothly tracking and displaying an object in a moving image.

  In order to solve the above-mentioned problem, the invention according to claim 1 is an operation of a virtual camera in the image frame of moving image data including a plurality of the image frames, the image frame including a plurality of pixels. A first display control step of displaying a first pixel area specified from the plurality of pixels on a display screen, and a position of the virtual camera displaying the first pixel area on the display screen or the virtual Transmission of first virtual camera data representing at least one of camera orientations and a first reproduction time in which the first pixel area is displayed on the display screen in association with each other in time series and transmitted to the server device A second virtual camera representing at least one of a position of the virtual camera in the image frame of the moving image data and a direction of the virtual camera Camera work data in which data and a second reproduction time in which a second pixel area specified from the plurality of pixels is output by the second virtual camera data are associated in time series. Specific camera work including specific virtual camera data corresponding to the first reproduction time and representing at least a part of the first pixel area from among the plurality of camera work data stored in the server device An acquisition step of acquiring data, and a second display control step of displaying a pixel region according to the specific camera work data acquired in the acquisition step on the display screen along with the output of the moving image data. A program to be executed by a computer.

  The invention according to claim 2 is an image frame composed of a plurality of pixels, and in accordance with an operation of a virtual camera in the image frame along an output of moving image data composed of the plurality of image frames, A first display control step of displaying a first pixel area specified from a plurality of pixels on a display screen; a position of the virtual camera displaying the first pixel area on the display screen; or a direction of the virtual camera Acquiring the first virtual camera data representing at least one of the first virtual camera data and the first reproduction time when the first pixel area is displayed on the display screen in association with each other in time series, and the moving image data Second virtual camera data representing at least one of the position of the virtual camera in the image frame and the orientation of the virtual camera, and the second virtual camera. Camera work data in which a second reproduction time in which a second pixel area specified from the plurality of pixels is output in accordance with time data is correlated in time series, and the plurality of camera work data is stored. A first determination step for determining whether the storage means has the second reproduction time corresponding to the first reproduction time acquired by the acquisition step; and the determination by the first determination step that the storage means is present A second determination step of determining whether the second pixel area specified by the specific second virtual camera data associated with the second reproduction time includes at least a part of the first pixel area. When the second determination step determines that at least a part of the first pixel area is included, a specific second virtual camera is selected from the plurality of camera work data. A pixel area in accordance with a particular camera work data including data, a program for executing a second display control step of displaying on said display screen along with the output of the video data, to the computer.

According to a fourth aspect of the present invention, in the program according to any one of the first to third aspects, the first display control step displays the first pixel area on a main screen in the display screen. And the second display control step displays the second pixel region in the display screen by a sub-screen having a smaller area than the main screen, and the program is displayed on the sub-screen. A receiving step for accepting designation of a second pixel area, and a third display control step for displaying the second pixel area displayed on the sub-screen on the main screen when designation of the second pixel area is accepted Are further executed by the computer.

According to a fifth aspect of the present invention, in the program according to the fourth aspect , the second display control step sets a number of the sub-screens according to the number of specific second virtual camera data, The plurality of second pixel regions specified by each of the second virtual camera data are displayed on each of the plurality of sub-screens, and the accepting step includes a plurality of the second pixel regions respectively displayed on the plurality of sub-screens. The specification of the specific second pixel region is received from the two pixel regions, and the third display control step is performed when the specification of the specific second pixel region is received, A pixel region is displayed on the main screen.

The invention according to claim 3, in the program according to claim 2, wherein the second determining step, the center coordinates of the second pixel area specified by the specific second virtual camera data, the first pixel It is determined whether it is within a predetermined range from the center coordinates of the region.

According to a sixth aspect of the present invention, there is provided an image frame composed of a plurality of pixels, wherein the plurality of pixels are determined according to an operation of a virtual camera in the image frame of the moving image data composed of the plurality of image frames. First display control means for displaying the first pixel region specified from the display screen on the display screen, and at least one of the position of the virtual camera and the direction of the virtual camera that displayed the first pixel region on the display screen Transmitting means for associating the first virtual camera data representing the first reproduction time in which the first pixel area is displayed on the display screen in time series with the server device, and the transmission data of the moving image data Second virtual camera data representing at least one of the position of the virtual camera in the image frame and the orientation of the virtual camera; and the second virtual camera data The camera work data in which the second reproduction time in which the second pixel area specified from the plurality of pixels is output is associated in time series, and the server device stores the plurality of Acquisition means for acquiring specific camera work data including specific virtual camera data corresponding to the first reproduction time and representing at least a part of the first pixel area from the camera work data; and the acquisition And a second display control means for displaying a pixel area according to the specific camera work data acquired by the means on the display screen in accordance with the output of the moving image data.

  The invention according to claim 7 is an image frame composed of a plurality of pixels, and in accordance with an operation of a virtual camera in the image frame along an output of moving image data composed of the plurality of image frames, First display control means for displaying a first pixel area specified from a plurality of pixels on a display screen; a position of the virtual camera displaying the first pixel area on the display screen; or a direction of the virtual camera Acquisition means for acquiring the first virtual camera data representing at least one of the first virtual camera data and the first reproduction time when the first pixel area is displayed on the display screen in time series, and the moving image data Second virtual camera data representing at least one of the position of the virtual camera in the image frame and the orientation of the virtual camera, and the second virtual camera data Is a camera work data in which a second reproduction time in which a second pixel region specified from the plurality of pixels is output is associated in time series, and the plurality of camera work data are stored. It is determined by the first determination means that the storage means has the second reproduction time corresponding to the first reproduction time acquired by the acquisition means, and the storage means is determined by the first determination means. A second determination means for determining whether a second pixel area specified by the specific second virtual camera data associated with the second reproduction time includes at least a part of the first pixel area; When the second determination means determines that at least a part of the first pixel area is included, the specific camera camera including the specific second virtual camera data among the plurality of camera work data. A pixel area in accordance with Kudeta, characterized in that it comprises a second display control means for displaying on said display screen along with the output of the video data.

According to the first, second, third , sixth, and seventh aspects of the present invention, for example, even when an object in a moving image moves unexpectedly, the object in the moving image can be smoothly displayed and tracked.

According to the invention described in claim 4 , it is possible to make the user confirm on the sub screen the camera work close to the pseudo camera work provided on the main screen.

According to the fifth aspect of the present invention, it is possible to allow the user to efficiently confirm a plurality of camera works close to the pseudo camera work provided on the main screen at a time on the plurality of sub screens.

It is a figure showing an example of outline composition of communications system S of this embodiment. It is a flowchart which shows an example of the process performed by the control part 21 or CPU of the client terminal 2a. It is a figure which shows an example of the storage process of camera work data. It is a figure which shows an example of the display screen in the display of the client terminal 2a. It is a flowchart which shows an example of the process performed by the control part 11 or CPU of the delivery server 1. It is a conceptual diagram which shows an example of the relationship between viewing camera work data and model camera work data in a plurality of image frames. (A) is a figure which shows an example of the content of the viewing camera work data and model camera work data shown in FIG. (B) is a figure which shows an example of the determination result for every reproduction time for every model camera work data.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[1. Overview of communication system configuration and operation]
First, with reference to FIG. 1, the structure and operation | movement outline | summary of the communication system S of embodiment of this invention are demonstrated. FIG. 1 is a diagram illustrating a schematic configuration example of a communication system S of the present embodiment. As shown in FIG. 1, the communication system S includes a distribution server 1 and client terminals 2a and 2b. The client terminals 2a and 2b are examples of the terminal device of the present invention. The user of the client terminal 2a is a viewer of moving image data, for example. The user of the client terminal 2b is, for example, a video data manager. The number of client terminals shown in FIG. 1 is an example, and the number is not limited to this number. Distribution server 1 and client terminals 2a and 2b are each connected to a network NW. The network NW is configured by, for example, the Internet.

  The distribution server 1 transmits content to the client terminal 2a in response to the content request from the client terminal 2a. The content includes moving image data. Note that the content may include audio data. A content ID is assigned to the content. The content ID is identification information for identifying the content. The content is transmitted by streaming distribution via the network NW, for example. The client terminal 2a receives the content distributed by streaming from the distribution server 1. The client terminal 2a reproduces the moving image data included in the received content. The moving image data is composed of a plurality of image frames. Each image frame is composed of a plurality of pixels.

  Then, the client terminal 2a displays the pixel area specified from the plurality of pixels in the image frame of the reproduced moving image data on the display screen. This pixel area corresponds to a drawing area drawn on the display screen within one image frame. In other words, the pixel area is a display range cut out from the image frame. This pixel region can be specified by, for example, an operation of pseudo camera work (hereinafter referred to as “pseudo camera work”). The pseudo camera work operation is performed by operating the virtual camera in the image frame. That is, the client terminal 2a displays the pixel area specified from the plurality of pixels on the display screen according to the operation of the virtual camera in the image frame along with the output of the moving image data. Here, the virtual camera refers to a viewpoint virtually set for a moving image projected on a virtual screen in a two-dimensional plane or a three-dimensional virtual space. An example of a moving image is a panoramic moving image. A panoramic video is a video in which a subject is shot by a camera equipped with a lens capable of shooting a wide range, for example, with a high-resolution camera. Examples of lenses capable of photographing a wide range include a wide lens, a fisheye lens, and a 360 degree lens. A panoramic video may be created by combining a plurality of camera videos. The virtual screen includes a flat screen, a cylindrical screen, and a spherical screen. For example, when the lens of a camera that has captured a moving image is a fisheye lens or a 360-degree lens, the virtual screen is a spherical screen.

  The virtual camera operation includes, for example, a virtual camera pan (left / right swing) operation, a virtual camera tilt (up / down swing) operation, and a virtual camera zoom (enlargement or reduction) operation. The direction (angle) of the virtual camera is determined by the pan and tilt operations of the virtual camera. The panning and tilting operations of the virtual camera can be realized by, for example, a mouse dragging operation by the user or a flicking operation by the user with a finger or a pen. The flick operation is, for example, an operation in which the user quickly slides a contact object such as a finger or a pen on a touch panel display screen. Further, the angle of view of the virtual camera is determined by the zoom operation of the virtual camera. The zoom operation of the virtual camera can be realized, for example, when the user presses a button provided on an operation unit such as a mouse or a button displayed on a touch panel display screen. Alternatively, the zoom operation of the virtual camera can be realized by, for example, a pinch operation for pinching a display screen of a method with two user fingers. The pinch operation includes pinch-in that reduces the pixel area and pinch-out that enlarges the pixel area.

  The client terminal 2a generates camera work data during reproduction of moving image data. The camera work data includes virtual camera data representing at least one of the position of the virtual camera displaying the pixel area on the display screen, the orientation of the virtual camera, and the angle of view of the virtual camera, and the pixel area is the display screen. Are displayed in association with each other along the time series. The pixel area is specified by the position of the virtual camera or the direction of the virtual camera. On the other hand, the zoom magnification (display magnification) of the pixel area is specified by the angle of view of the virtual camera. The reproduction time indicates the reproduction time corresponding to the reproduction position from the reproduction start of the moving image. In other words, the reproduction time is a reproduction time during which the pixel area specified by the virtual camera data is output. The camera work data of the present embodiment includes, for example, parameters of pan, tilt, and zoom as virtual camera data. The pan and tilt parameters are an example of data representing the position of the virtual camera or the direction of the virtual camera. The zoom parameter is an example of data representing the zoom magnification (display magnification). When the virtual screen is a flat screen, the position of the virtual camera in the image frame is defined by data representing (x, y) coordinates. In this case, the (x, y) coordinates indicate the center coordinates of the virtual camera, for example.

  Here, the relationship between pan and tilt and the pixel region is expressed by, for example, the following equations (1) and (2).

pan = x '+ width / 2 (1)
tilt = y '+ height / 2 (2)

  Here, (x ′, y ′) indicates, for example, the upper left corner coordinates of the pixel area when the upper left corner coordinates of the image frame are set to the origin (0, 0). width indicates the width of the pixel region. height indicates the height of the pixel region. Moreover, width and height are respectively represented by the following formulas (3) and (4), for example.

width = W / α (3)
height = H / α (4)

  Here, W indicates the width of the image frame. H indicates the height of the image frame. α is a value that varies depending on zoom. For example, if the zoom magnification is 1, α = 1/4. If the zoom magnification is 2, α = ½. The relationship between α and the zoom magnification is defined on a program, for example.

  From the above formulas (1) to (4), the pixel region in the image frame can be specified from the pan, the tilt, and the zoom.

  According to the camera work data, it is possible to define the movement of a virtual camera that virtually captures all or part of a moving image projected onto a virtual screen in a two-dimensional plane or a three-dimensional virtual space. Thereby, pseudo camera work can be reproduced. For example, a plurality of viewing camera work data is generated every predetermined time in the reproduction time from the reproduction start to the reproduction end of the moving image data. The camera work data generated by the client terminal 2a is referred to as “viewing camera work data”. The virtual camera data included in the viewing camera work data is referred to as “viewing camera data”. The viewing camera data is an example of first virtual camera data. Further, the pixel area specified by the viewing camera data is referred to as “first pixel area”. The first pixel region is a viewing range. In addition, the reproduction time included in the viewing camera work data is referred to as “first reproduction time”. The pseudo camera work based on the viewing camera data is called viewing camera work.

  Then, the client terminal 2a associates the content ID with the generated viewing camera work data and transmits them to the distribution server 1 at predetermined time intervals during the reproduction of the content. This content ID is the content ID of the content being reproduced. That is, the client terminal 2a transmits the viewing camera data and the first reproduction time to the distribution server 1 in association with each other in time series. The distribution server 1 receives the content ID and viewing camera work data transmitted from the client terminal 2a. That is, the distribution server 1 acquires the viewing camera data and the first reproduction time in association with each other in time series. The viewing camera work data is given a camera work ID. The camera work ID is identification information for identifying camera work data.

  The distribution server 1 stores model camera work data. The model camera work data is model camera work data distributed to the client terminal 2a. For example, the model camera work data is generated by the client terminal 2b. The distribution server 1 stores the camera work data and the camera work ID generated by the client terminal 2b in the storage unit. That is, the distribution server 1 stores the virtual camera data and the reproduction time in the storage unit in association with each other in time series. The virtual camera data included in the model camera work data is referred to as “model camera data”. The model camera data is an example of second virtual camera data. In addition, the pixel area specified by the model camera data is referred to as a “second pixel area”. Further, the reproduction time included in the model camera work data is referred to as “second reproduction time”. The pseudo camera work represented in the model camera data is called model camera work. The model camera work data is given a camera work ID. The client terminal 2b may generate model camera work data for automatically tracking a predetermined object (display target) using, for example, a predetermined tracking program. Alternatively, the client terminal 2b may generate model camera work data for tracking a predetermined object by operating a virtual camera by an administrator, for example. By using the model camera work data, it is possible to reproduce a camera work that smoothly tracks a predetermined object in the moving image.

  Then, when receiving the viewing camera work data from the client terminal 2a, the distribution server 1 searches for the model camera work data related to the viewing camera work data. Specifically, the distribution server 1 stores a content ID and a plurality of model camera work data in association with each other. Further, the distribution server 1 stores the received viewing camera work data and the content ID for identifying the moving image data used for generating the viewing camera work data in association with each other. The distribution server 1 searches for a content ID that matches the content ID associated with the viewing camera work data. Then, the distribution server 1 identifies a plurality of model camera work data associated with the matched content ID. For a plurality of identified model camera work data, the distribution server 1 determines for each first playback time whether the second playback time corresponding to the first playback time included in the viewing camera work data is in the storage means. Here, the second reproduction time corresponding to the first reproduction time corresponds to, for example, a second reproduction time that matches the first reproduction time. However, the second reproduction time corresponding to the first reproduction time may have a certain width. In this case, the second reproduction time corresponding to the first reproduction time corresponds to the second reproduction time included in the range of plus or minus several tens of ms of the first reproduction time. The distribution server 1 determines whether or not the second pixel area specified by the specific model camera data includes at least a part of the first pixel area using the specific model camera data. Here, the specific model camera data is model camera data associated with the second reproduction time determined to be in the storage unit. The first pixel area is a pixel area specified by the viewing camera data associated with the first reproduction time. The determination as to whether or not the second pixel area includes at least a part of the first pixel area is, in other words, a determination as to whether or not there is a degree of coincidence between the second pixel area and the first pixel area. Then, the distribution server 1 includes a template camera that includes at least time-series template camera data that specifies a plurality of second pixel regions and a second reproduction time in which each of the plurality of second pixel regions is output in association in time series. Work data is transmitted to the client terminal 2a. Here, the plurality of second pixel regions are pixel regions subsequent to the second pixel region including at least a part of the first pixel region. “Subsequent to the second pixel region” means that the second pixel region is output after the second reproduction time during which the second pixel region is output.

  The client terminal 2a receives from the distribution server 1 model camera work data including the model camera data and the second reproduction time in association with each other in time series. That is, the client terminal 2a acquires specific model camera work data from a plurality of model camera work data. The specific model camera work data includes specific model camera data corresponding to the first reproduction time and representing at least a part of the first pixel region. And the client terminal 2a displays the pixel area according to the acquired specific model camera work data on the display screen along with the output of the moving image data. For example, the client terminal 2a displays, on the display screen, the second pixel area following the second pixel area including at least a part of the first pixel area according to the model camera data instead of the viewing camera data by the operation of the virtual camera. Display.

[2. Configuration of each device]
Next, the configuration of each device included in the communication system S of the present embodiment will be described with reference to FIG. As shown in FIG. 1, the distribution server 1 includes a control unit 11, a storage unit 12, an interface unit 13, and the like. These components are connected to the bus 14. The interface unit 13 is connected to the network NW. The control unit 11 includes a CPU (Center Processing Unit) as a computer, a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. The control unit 11 controls transmission / reception of contents and transmission / reception of camera work data. The storage unit 12 is configured by, for example, a hard disk drive. The storage unit 12 stores an OS, a server program, and the like. For example, the content received from the photographing apparatus is stored in the storage unit 12 in association with the content ID. The storage unit 12 stores a plurality of model camera work data received from the client terminal 2b or the like in association with the content ID. The storage unit 12 stores a camera work ID assigned to each of the plurality of model camera work data and feature information of the model camera work data. This feature information includes the name of the object tracked by the model camera work data, the object ID, and the like. When the object is a person, the name of the object is a person name. The object ID is identification information for identifying the object.

  Next, as shown in FIG. 1, the client terminals 2a and 2b include a control unit 21, a storage unit 22, a video RAM 23, a video control unit 24, an operation processing unit 25, an audio control unit 26, an interface unit 27, and the like. Configured. These components are connected to the bus 28. The control unit 21 includes a CPU (Center Processing Unit) as a computer, a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. The control unit 11 is an example of a first display control unit, a transmission unit, an acquisition unit, a first determination unit, a second determination unit, and a second display control unit of the present invention. The control unit 11 has a timer function. A display unit 24 a including a display is connected to the video control unit 24. An operation unit 25 a is connected to the operation processing unit 25. Examples of the operation unit 25a include a mouse, a keyboard, and a remote controller. A touch panel serving both as the display unit 24a and the operation unit 25a may be applied. The control unit 21 receives an operation instruction from the operation unit 25 a by the user via the operation processing unit 25. The user can operate the virtual camera described above using the operation unit 25a. A speaker 26 a is connected to the audio control unit 26. The interface unit 27 is connected to the network NW. The storage unit 22 is configured by, for example, a hard disk drive (HDD). The storage unit 22 stores an OS, player software, and the like. The player software is a program that causes the CPU to execute content reception and playback processing and camera work data transmission / reception processing.

  The control unit 21 functions as a player that reproduces the received content by executing the player software. Moreover, a display screen is displayed on the display in the display unit 24a by the function of the player. The display screen is also called a window. The display screen includes a main screen and a sub screen. On the main screen in the display screen, a moving image is displayed according to viewing camera data or model camera work data by operation of the virtual camera. This moving image is a moving image corresponding to a pixel region specified from a plurality of pixels in the image frame of the received moving image data. On the sub screen, moving images that are candidates to be displayed on the main screen are displayed according to the camera work data. The sub screen may be displayed superimposed on the main screen. The sub screen has a smaller area than the main screen. The control unit 21 sets the number of sub-screens corresponding to the number of specific model camera work data received from the distribution server 1. And the control part 21 displays the 2nd pixel area specified by the said specific model camera work data on a display screen on a subscreen, for example. When receiving the designation of the second pixel area displayed on the sub screen, the control unit 21 displays the second pixel area displayed on the sub screen on the main screen. Further, the RAM in the control unit 21 is provided with a buffer memory. The buffer memory temporarily holds moving image data included in the received content. The control unit 21 outputs moving image data from the buffer memory to the video RAM 23. The video RAM 23 is provided with a frame buffer into which drawing data is written. The video control unit 24 displays the drawing data written in the frame buffer by drawing on the display screen according to the control signal from the control unit 21. In some cases, audio data is included in the content held in the buffer memory. In this case, the control unit 21 reproduces the audio data from the buffer memory and outputs it to the audio control unit 26. The voice control unit 26 generates an analog voice signal from the voice data, and outputs the generated analog voice signal to the speaker 26a.

[3. Operation of communication system S]
Next, the operation of the communication system S of the present embodiment will be described. First, the operation of the client terminal 2a will be described with reference to FIG. FIG. 2 is a flowchart illustrating an example of processing executed by the control unit 21 or the CPU of the client terminal 2a. The process shown in FIG. 2 is started, for example, when the user of the client terminal 2a selects desired content after the player is activated in the client terminal 2a. When the process shown in FIG. 2 is started, the control unit 21 sets the main screen on the display in the display unit 24a (step S1). Next, the control unit 21 starts content reception and playback processing (step S2). The content reception and playback processing is performed in parallel with the processing shown in FIG. 2 by the multitask function of the OS, for example. In the content reception and reproduction process, the control unit 21 accesses the distribution server 1 and transmits a request for content. When the client terminal 2a accesses the distribution server 1, the distribution server 1 generates a session ID. This session ID is identification information for identifying a communication session between the distribution server 1 and the client terminal 2a. The session ID is transmitted from the distribution server 1 to the client terminal 2a. Then, in response to the content request, the control unit 21 receives the content distributed by streaming from the distribution server 1 and stores it in the buffer memory. And the control part 21 reproduces | regenerates the moving image data contained in the content memorize | stored in the buffer memory. Note that the content reception and playback processing continues until the end of the processing shown in FIG.

  Next, the control unit 21 determines whether there is model camera work data (step S3). For example, when model camera work data is stored in a RAM or the like, the control unit 21 determines that there is model camera work data. When it is determined that there is no model camera work data (step S3: NO), the control unit 21 proceeds to step S4. On the other hand, if it is determined that there is model camera work data (step S3: YES), the control unit 21 proceeds to step S5.

  In step S4, the control unit 21 sets the camera work input to the main screen to “manual”, and proceeds to step S6. Here, the setting “manual” means that the operation input of the virtual camera by the user is set to ON. On the other hand, in step S5, the control unit 21 sets the camera work input to the main screen to “model camera work data”, and proceeds to step S6. In step S <b> 6, the control unit 21 starts a camera work data storage process. And the control part 21 starts the moving image display process to a main screen (step S7). Note that the camera work data storage process and the moving image display process on the main screen are performed in parallel with the process shown in FIG. 2, for example, by the multitask function of the OS. In the moving image display process on the main screen, the control unit 21 displays a moving image on the main screen based on the reproduced moving image data. Here, it is assumed that the camera work input is set to “manual”. In this case, the control unit 21 causes the pixel area specified from the plurality of pixels in the image frame of the moving image data to be displayed on the main screen according to the operation of the virtual camera. That is, the control unit 21 displays the pixel area specified according to the viewing camera data by the operation of the virtual camera on the main screen. On the other hand, it is assumed that the camera work input is set to “model camera work data”. In this case, the control unit 21 displays the pixel region specified according to the model camera work data on the main screen. When “manual” is set in step S4, the control unit 21 temporarily stores information indicating that “manual” is set in the RAM. When the “model camera work data” is set in step S5, the control unit 21 temporarily stores information indicating that the “model camera work data” is set in the RAM.

  Here, with reference to FIG. 3, the details of the storage process of the camera work data will be described. FIG. 3 is a diagram illustrating an example of storage processing of camera work data. In the camera work data storage process, the control unit 21 determines whether or not the main screen is displayed in accordance with the operation of the virtual camera (step S61). For example, the control unit 21 determines whether or not the main screen is displayed in accordance with the operation of the virtual camera based on information temporarily stored in the RAM. When it is determined that the main screen is displayed according to the operation of the virtual camera (step S61: YES), the control unit 21 proceeds to step S62. In step S62, the control unit 21 stores the viewing camera data and the first reproduction time in time series in association with each other in accordance with the operation of the virtual camera, for example, in a predetermined area of the RAM. On the other hand, when it determines with the main screen not being displayed according to operation of a virtual camera (step S61: NO), the control part 21 progresses to step S63.

  In step S63, the control unit 21 determines whether or not the main screen is displayed according to the model camera work data. For example, the control unit 21 determines whether or not the main screen is displayed according to the model camera work data based on information temporarily stored in the RAM. When it is determined that the main screen is displayed according to the model camera work data (step S63: YES), the control unit 21 proceeds to step S64. In step S64, the model camera data and the second reproduction time are associated with each other in time series and stored in, for example, a predetermined area of the RAM. On the other hand, when it determines with the main screen not being displayed according to model camera work data (step S63: NO), the control part 21 progresses to step S65. In step S65, the control unit 21 determines whether or not the moving image data being reproduced has reached the end of reproduction. When it is determined that the moving image data has not reached the end of reproduction (step S65: NO), the control unit 21 returns to step S61. Thereby, the storage process of camera work data is continued. On the other hand, when it is determined that the moving image data has reached the end of reproduction (step S65: YES), the control unit 21 proceeds to step S66. In step S66, the viewing camera work data file in which the viewing camera data stored in the predetermined area of the RAM in step S62 or step S64 is associated with the reproduction time in time series is output, and the process shown in FIG. Exit. The output destination of the camera work data file is, for example, the storage unit 22 or the distribution server 1.

  In step S8, the control unit 21 determines whether or not the camera work input to the main screen is set to “manual”. For example, the control unit 21 determines whether or not the camera work input to the main screen is set to “manual” based on the information temporarily stored in the RAM. When it is determined that the camera work input to the main screen is not set to “manual” (step S8: NO), the control unit 21 proceeds to step S9. On the other hand, when it is determined that the camera work input to the main screen is set to “manual” (step S8: YES), the control unit 21 proceeds to step S10. In step S9, the control unit 21 determines whether or not the virtual camera has been operated. For example, when the user operates the mouse, it is determined that the virtual camera is operated. When it is determined that the virtual camera has been operated (step S9: YES), the control unit 21 returns to step S4. Returning to step S4, the control unit 21 sets the camera work input to the main screen to “manual”. On the other hand, when the control unit 21 determines that there is no operation of the virtual camera (step S9: NO), the control unit 21 proceeds to step S12.

  In step S10, the control unit 21 determines whether or not a predetermined time has elapsed since the previous transmission of the viewing camera work data. Specifically, every time the viewing camera work data is transmitted, the control unit 21 may temporarily store the transmitted date and time in the RAM. This predetermined time is set to, for example, several tens of seconds by the timer function of the control unit 21. When it is determined that the predetermined time has not elapsed since the previous transmission of the viewing camera work data (step S10: NO), the control unit 21 proceeds to step S12. On the other hand, when it is determined that the predetermined time has elapsed since the previous transmission of the viewing camera work data (step S10: YES), the control unit 21 proceeds to step S11. The passage of the predetermined time means that the time set by the timer has been counted up. In step S11, the control unit 21 transmits the viewing camera work data, the content ID, and the session ID in which the viewing camera data and the first reproduction time are associated in time series to the distribution server 1, and the process proceeds to step S12. . Here, the viewing camera data and the first reproduction time to be transmitted are the viewing camera data and the first reproduction time stored in a predetermined area of the RAM by the camera work data storage process. The transmitted content ID is the content ID of the content being reproduced. The control unit 21 deletes the transmitted viewing camera data and the first reproduction time from a predetermined area of the RAM. This is to prevent transmission of overlapping viewing camera data and first reproduction time to the distribution server 1. When viewing camera work data is transmitted, the count value of the timer is reset to zero. Then, the timer count restarts. Thereby, the control part 21 can transmit viewing camera work data at predetermined time intervals, for example. Thereby, since the viewing camera work data transmitted by the control unit 21 can be changed every transmission, the model camera work data returned from the distribution server 1 and the number thereof can be changed every time. For example, the model camera work data a and the model camera work data b may be returned in the first transmission, and the model camera work data c may be returned in the second transmission.

  In step S <b> 12, the control unit 21 determines whether the template camera work data transmitted from the distribution server 1 has been received. This model camera work data is model camera work data related to the viewing camera work data transmitted in step S11. When it determines with the control part 21 not having received model camera work data (step S12: NO), it progresses to step S15. On the other hand, when it determines with the control part 21 having received model camera work data (step S12: YES), it progresses to step S13.

  In step S13, the control unit 21 sets the number of sub-screens corresponding to the number of model camera work data received in step S12 on the display. For example, when the number of received model camera work data is two, two sub-screens are set. On the other hand, when the received number of model camera work data is five, five sub-screens are set. Note that the number of received model camera work data may exceed the upper limit for setting the sub screen. In this case, the control unit 21 sets the upper limit number of sub-screens. Next, the control unit 21 starts a moving image display process on the sub screen (step S14). The moving image display processing on the sub screen is performed in parallel with the processing shown in FIG. 2 by, for example, the multitask function of the OS. In the moving image display process on the sub screen, the control unit 21 displays the pixel region specified according to the template camera work data received in step S12 on the sub screen. It is possible to make the user confirm the model camera work close to the viewing camera work provided on the main screen on the sub screen. When a plurality of model camera work data is received, the control unit 21 displays a plurality of second pixel areas specified by the plurality of model camera work data on each of the plurality of sub-screens. In some cases, the distribution server 1 transmits the feature information of the model camera work data together with the model camera work data. In this case, the feature information of the received model camera work data is displayed on the sub-screen corresponding to the model camera work data. Thereby, the user can grasp the name of the object tracked by the model camera work data.

  FIG. 4 is a diagram illustrating an example of a display screen on the display of the client terminal 2a. In the example of FIG. 4A, a main screen MV and two sub screens SV1 and SV2 are displayed. Further, the names of persons to be tracked are displayed on the sub-screens SV1 and SV2, respectively. On the other hand, in the example of FIG. 4B, a main screen MV and five sub-screens SV1 to SV5 are displayed. As described above, the number of sub-screens increases or decreases according to the number of received template camera work data. Thereby, a plurality of model camera work close to the viewing camera work provided on the main screen can be efficiently confirmed at once by the user on the plurality of sub screens.

  In step S15, the control unit 21 determines whether or not designation of the second pixel area displayed on the sub screen has been received. For example, when the user designates on the sub screen with a mouse or a finger, it is determined that the designation of the second pixel area displayed on the sub screen is accepted. When the designation of the second pixel area displayed on the sub-screen is received (step S15: YES), the control unit 21 proceeds to step S5. For example, when the control unit 21 receives designation of a specific second pixel region from the plurality of second pixel regions respectively displayed on the plurality of sub-screens, the control unit 21 proceeds to step S5. If transfering it to step S5, the control part 21 will set the camera work input to a main screen to the model camera work data which identify the 2nd pixel area designated by step S15. Thereby, the control unit 21 displays the second pixel area displayed on the sub screen SV5 as shown in FIG. 4B on the main screen MV as shown in FIG. 4C in step S7. . In the example of FIG. 4C, the sub screen is erased by designating the second pixel region. By the way, before the designation of the second pixel area is accepted, the control unit 21 may display the pixel area specified according to the viewing camera data by the operation of the virtual camera on the main screen. In this case, when the control unit 21 receives the designation of the second pixel area in step S15, the control unit 21 sets the second pixel area according to the model camera data that identifies the specified second pixel area instead of the viewing camera data. It will be displayed on the sub screen. Note that the control unit 21 may receive designation of a specific second pixel area from among the plurality of second pixel areas respectively displayed on the plurality of sub-screens. In this case, the control unit 21 displays the specified specific second pixel region on the main screen. On the other hand, when it determines with the control part 21 not having received designation | designated of the 2nd pixel area currently displayed on the subscreen (step S15: NO), it progresses to step S16.

  In step S16, the control unit 21 determines whether there is model camera work data set in step S5. For example, when the model camera work data set in step S5 is stored in the RAM or the like, the control unit 21 determines that there is model camera work data. When it is determined that there is no model camera work data (step 16: NO), the control unit 21 returns to step S4. Thereby, the camera work input to the main screen is set to “manual”. On the other hand, when it determines with the control part 21 having the set model camera work data (step S16: YES), it progresses to step S17.

  In step S17, the control unit 21 determines whether or not the moving image data being reproduced has reached the end of reproduction. When it is determined that the moving image data has not reached the end of reproduction (step S17: NO), the control unit 21 returns to step S8. On the other hand, when it determines with the moving image data having reached | attained completion | finish of reproduction | regeneration (step S17: YES), the control part 21 complete | finishes the process shown in FIG. Although not shown, if the user stops playback during the process shown in FIG. 2, the process shown in FIG. 2 ends.

  Next, the operation of the distribution server 1 will be described with reference to FIG. FIG. 5 is a flowchart illustrating an example of processing executed by the control unit 11 or the CPU of the distribution server 1. The process shown in FIG. 5 is started when the distribution server 1 receives and acquires viewing camera work data, content ID, and session ID transmitted from the client terminal 2a. The acquired viewing camera work data, content ID, and session ID are stored in a predetermined area of the RAM.

  When the process shown in FIG. 5 is started, the control unit 11 acquires model camera work data associated with the acquired content ID from the storage unit 12 (step S21). The acquired model camera work data is stored in a predetermined area of the RAM. The control unit 11 searches for the previous determination result information based on the session ID acquired together with the viewing camera work data from the client terminal 2a. The previous determination result is information indicating the determination result of the degree of coincidence performed at the time of the previous reception of the viewing camera work data. The degree of coincidence is the degree of coincidence between the second pixel area and the first pixel area. When the previous determination result information associated with the session ID is stored in the storage unit 12, for example, the control unit 11 reads the determination result information into a predetermined area of the RAM.

  Next, the control unit 11 specifies model camera work data including a second playback time that matches the first playback time included in the viewing camera work data from the model camera work data acquired in step S21 (step S21). S22). That is, the control unit 11 determines, for each first reproduction time, whether or not the second reproduction time that matches the first reproduction time included in the viewing camera work data is in a predetermined area of the RAM. Here, it is assumed that the viewing camera work data defines the first reproduction time every several tens of milliseconds (milliseconds). In this case, the control unit 11 determines every several tens of milliseconds whether or not the second reproduction time that matches the first reproduction time is in the predetermined area of the RAM. That is, it is determined for each image frame. Alternatively, in this case, the control unit 11 may determine whether the second reproduction time that matches the first reproduction time is in a predetermined area of the RAM at intervals longer than several tens of ms. That is, the determination may be made for each of a plurality of image frames. If the second reproduction time that coincides in all the reproduction times or the first reproduction time equal to or greater than the predetermined number is not in the predetermined area of the RAM, the process shown in FIG. 5 ends. Then, the control unit 11 specifies one or a plurality of model camera work data including the second reproduction time determined to be in the predetermined area of the RAM. The second playback time and the first playback time determined to match as described above are simply referred to as “playback time” hereinafter.

  Next, the control unit 11 determines, for each reproduction time, whether the second pixel area specified by the model camera data includes at least a part of the first pixel area specified by the viewing camera data (step S23). That is, it is determined whether or not there is a degree of coincidence between the second pixel region and the first pixel region. Here, the model camera data is model camera data included in the model camera work data specified in step S22. The viewing camera data is viewing camera data included in the acquired viewing camera work data. For example, the control unit 11 determines whether the center coordinates of the second pixel area specified by the specific model camera data are within a predetermined range from the center coordinates of the first pixel area. When the center coordinates of the second pixel area are within a predetermined range from the center coordinates of the first pixel area, the second pixel area is determined as including at least a part of the first pixel area. That is, the predetermined range is composed of a plurality of pixels. The positions of the plurality of pixels within the predetermined range are each represented by coordinates having the origin at the upper left position of the image frame, for example. If there is a coordinate that matches the center coordinate of the second pixel region among the plurality of coordinates corresponding to the plurality of pixels within the predetermined range, the second pixel region is determined to include at least a part of the first pixel region. The In this case, the control unit 11 calculates center coordinates from the pan and tilt included in the viewing camera data before this determination. Then, the control unit 11 determines the predetermined range from the calculated center coordinates and zoom magnification. This predetermined range changes according to the zoom magnification. In addition, when all or a part of the first pixel region and the second pixel region overlap, the second pixel region may be determined as including at least a part of the first pixel region. For example, at least one coordinate among a plurality of coordinates corresponding to a plurality of pixels constituting the first pixel region and at least one coordinate among a plurality of coordinates corresponding to the plurality of pixels constituting the second pixel region Is determined to include at least a part of the first pixel region.

  FIG. 6 is a conceptual diagram showing an example of the relationship between viewing camera work data and model camera work data in a plurality of image frames. R1 to R5 shown in FIG. 6 indicate the first pixel region. H1 to H5 shown in FIG. 6 indicate a predetermined range. FIG. 7A is a diagram showing an example of the contents of the viewing camera work data and the model camera work data shown in FIG. The model camera work data ac shown in FIG. 7A is the model camera work data specified in step S22. The camera work IDs of the model camera work data ac are different from each other. 6 and 7A show an example in which pan and tilt indicated by virtual camera data are converted into center coordinates (x, y) of the pixel area. In the example of FIG. 7A, the zoom magnification included in the model camera work data ac is omitted. In the example of FIGS. 6 and 7A, for convenience of explanation, the reproduction time is 1 second between 1 s and 5 s, but it is actually several tens of milliseconds. In the example of FIG. 6, the model camera work data ac tracks different objects. However, the model camera work data ac may track the same object.

  FIG. 7B is a diagram illustrating an example of a determination result for each reproduction time for each model camera work data. Such a determination result is registered in a table stored in a predetermined area of the RAM, for example. When the previous determination result information is acquired in step S21, the previous determination result is added to the table. In the image frame at the reproduction time 1s shown in FIG. 6, the center coordinates (xa1, ya1), the center coordinates (xb1, yb1), and the center are within a predetermined range H1 from the center coordinates (x01, y01) of the first pixel region R1. There are coordinates (xc1, yc1). Therefore, the determination results of the model camera work data a to c at the reproduction time 1 s are all OK as shown in FIG. OK indicates that it is determined that the center coordinates of the second pixel region are within a predetermined range from the center coordinates of the first pixel region. That is, OK indicates that the degree of coincidence between the second pixel area and the first pixel area is determined to be present. In the image frame at the reproduction time 2s shown in FIG. 6, the center coordinates (xa2, ya2) and the center coordinates (xb2, yb2) are within the predetermined range H2 from the center coordinates (x02, y02) of the first pixel region R2. is there. Therefore, the determination results of the model camera work data a and b at the reproduction time 2s are OK as shown in FIG. 7B. On the other hand, the center coordinates (xc2, yc2) are not included in the predetermined range H2 from the center coordinates (x02, y02) of the first pixel region R2. Therefore, the determination result of the model camera work data c at the reproduction time 2s is NG as shown in FIG. 7B. NG indicates that it is determined that the center coordinates of the second pixel region are not included within a predetermined range from the center coordinates of the first pixel region. That is, NG indicates that the degree of coincidence between the second pixel area and the first pixel area is determined to be none. Note that the determination results of the model camera work data ac in the reproduction time 3 s shown in FIG. 6 are NG as shown in FIG. 7B, respectively.

  In the example of FIG. 6, the predetermined range is a circle within a predetermined distance from the center coordinates of the first pixel region. However, the predetermined range may be a rectangular range including the center coordinates of the first pixel region. Alternatively, the predetermined range may be the entire first pixel region. Note that the control unit 11 sets the distance from the center coordinate in the predetermined range so as to change according to the zoom magnification. For example, as shown in FIG. 7A, the zoom magnification during the reproduction time 1 s is 1. On the other hand, the zoom magnification in the reproduction time 2s is 2 times. Therefore, as shown in FIG. 6, the distance (radius) from the center coordinate when the zoom magnification is 2 times is set to ½ of the distance (radius) R from the center coordinate when the zoom magnification is 1 time. .

  Next, the control unit 11 determines whether there is model camera work data satisfying a predetermined transmission condition in the model camera work data specified in step S22 (step S24). This transmission condition is a condition that the ratio determined to have a coincidence within a predetermined reproduction period is equal to or higher than the reference ratio. Here, the reproduction period is set, for example, between 5 seconds and 10 seconds. A period retroactive to the reproduction period from the second reproduction time associated with the latest viewing camera data is determined in step S24. Further, the reference ratio is set between 60% and 80%, for example. FIG. 7B shows an example in which the playback period is set to 5 seconds. In this case, the model camera work data a has a ratio of 4/5 (80%) determined to have a degree of coincidence in step S23. In addition, the model camera work data b has a ratio of 3/5 (60%) determined to have a degree of coincidence in step S23. Therefore, if the reference ratio is 60%, the model camera work data a and the model camera work data b satisfy the transmission condition. On the other hand, if the reference ratio is 80%, only the model camera work data a satisfies the transmission condition. When it is determined that there is model camera work data that satisfies a predetermined transmission condition (step S24: YES), the control unit 11 proceeds to step S25. On the other hand, if the control unit 11 determines that there is no model camera work data that satisfies a predetermined transmission condition (step S24: NO), the control unit 11 transmits a NULL message to the client terminal 2a and ends the process shown in FIG. The NULL message is a message indicating that there is no model camera work data.

  In step S25, the control unit 11 acquires model camera work data determined to satisfy the predetermined transmission condition in step S24. Here, the acquired model camera work data includes a plurality of model camera data for specifying a plurality of second pixel areas following the second pixel area determined to include at least a part of the first pixel area. It is. Further, the acquired model camera work data includes a second reproduction time associated with a plurality of model camera data specifying a plurality of second pixel areas that follow. That is, at least the model camera data and the second playback time in the playback period to be played back by the client terminal 2a are included in the model camera work data. However, the acquired model camera work data includes, for example, a plurality of model camera data specifying the second pixel area output before the second pixel area determined to be partially included as described above. May be. That is, the acquired model camera work data may include the model camera data and the second playback time from the start of playback of the moving image data to the end of playback.

  Next, the control unit 11 transmits the model camera work data acquired in step S25 to the client terminal 2a (step S26). Accordingly, when the client terminal 2a receives the model camera work data in step S12, the client terminal 2a displays the second pixel area following the second pixel area including at least a part of the first pixel area on the sub screen. . And the control part 11 matches the determination result in step S24 with the session ID of the client terminal 2a, for example, memorize | stores in the memory | storage part 12, and complete | finishes the process shown in FIG.

  As described above, according to the present embodiment, the client terminal 2a corresponds to the first reproduction time among the plurality of model camera work data stored in the distribution server 1, and at least in the first pixel region. Specific model camera work data including specific model camera data representing a part is acquired. And the client terminal 2a displays the pixel area according to the acquired specific model camera work data on the display screen along with the output of the moving image data. Therefore, for example, even when an object in a moving image moves unexpectedly, the object in the moving image can be smoothly displayed for tracking.

  In the embodiment described above, the distribution server 1 is configured to determine whether or not there is a degree of coincidence between the second pixel area and the first pixel area when viewing camera work data is received from the client terminal 2a. However, the control unit 21 of the client terminal 2a may be configured to determine whether or not there is a degree of coincidence between the second pixel area and the first pixel area. In this case, the control unit 21 of the client terminal 2 a acquires a plurality of model camera work data from the distribution server 1 in advance and stores the data in the storage unit 22. Then, the control unit 21 of the client terminal 2a has at least a part of the second pixel area specified by the specific model camera data associated with the second reproduction time corresponding to the first reproduction time. Determine whether to include. When it is determined that the control unit 21 of the client terminal 2a includes at least a part of the first pixel area, the control unit 21 according to the specific model camera work data including the specific model camera data among the plurality of model camera work data. The pixel area is displayed on the display screen along with the output of the moving image data. For example, the control unit 21 of the client terminal 2a uses the plurality of model camera work data stored in the storage unit 22 to perform the processes of steps S21 to S25 illustrated in FIG. In addition, the control unit 21 of the client terminal 2a refers to the storage unit 12 of the distribution server 1 that stores a plurality of model camera work data, and uses the plurality of model camera work data to perform steps S21 to S21 shown in FIG. You may perform the process of S25.

1 Distribution server 2a, 2b Client terminal S Communication system

Claims (7)

A first pixel region identified from among the plurality of pixels according to an operation of a virtual camera in the image frame of the moving image data composed of the plurality of image frames, the image frame including a plurality of pixels; A first display control step to be displayed on the display screen;
First virtual camera data representing at least one of the position of the virtual camera or the direction of the virtual camera that causes the first pixel area to be displayed on the display screen, and the first pixel area are displayed on the display screen. A transmission step of associating the first reproduction time with a time series and transmitting to the server device;
Second virtual camera data representing at least one of a position of a virtual camera or a direction of the virtual camera in the image frame of the moving image data, and a second specified from the plurality of pixels by the second virtual camera data. Camera work data in which a second reproduction time at which a two-pixel region is output is associated in time series, and the first reproduction time is selected from a plurality of camera work data stored in the server device. And acquiring specific camera work data including specific virtual camera data representing at least a part of the first pixel region;
A second display control step of displaying a pixel area according to the specific camera work data acquired in the acquisition step on the display screen along with the output of the moving image data;
A program that causes a computer to execute. An image frame composed of a plurality of pixels, identified from among the plurality of pixels according to an operation of a virtual camera in the image frame along with output of moving image data composed of the plurality of image frames A first display control step for displaying the first pixel region on the display screen;
First virtual camera data representing at least one of the position of the virtual camera or the direction of the virtual camera that causes the first pixel area to be displayed on the display screen, and the first pixel area are displayed on the display screen. An acquisition step of acquiring the first reproduction time in association with each other in time series;
Second virtual camera data representing at least one of a position of a virtual camera or a direction of the virtual camera in the image frame of the moving image data, and a second specified from the plurality of pixels by the second virtual camera data. The camera work data in which the second reproduction time for outputting the two-pixel region is associated with each other in time series, and a plurality of the camera work data stored in the storage unit is acquired by the acquisition step. A first determination step of determining whether there is the second reproduction time corresponding to the first reproduction time;
When it is determined in the storage means by the first determination step, the second pixel area specified by the specific second virtual camera data associated with the second reproduction time is the first pixel area. A second determination step for determining whether at least a part of
When it is determined by the second determination step that at least a part of the first pixel area is included, the specific camera work data including the specific second virtual camera data is obeyed among the plurality of camera work data. A second display control step of displaying a pixel region on the display screen along with the output of the moving image data;
A program that causes a computer to execute. In the second determination step, it is determined whether the center coordinates of the second pixel area specified by the specific second virtual camera data are within a predetermined range from the center coordinates of the first pixel area. The program according to claim 2 . In the first display control step, the first pixel region is displayed on a main screen in the display screen,
In the second display control step, the second pixel region is displayed in the display screen by a sub-screen having a smaller area than the main screen,
The program is
An accepting step of accepting designation of the second pixel area displayed on the sub-screen;
A third display control step of displaying the second pixel area displayed on the sub-screen on the main screen when designation of the second pixel area is accepted;
The program according to any one of claims 1 to 3 , further causing the computer to execute. The second display control step sets a number of the sub-screens according to the number of specific second virtual camera data, and a plurality of the second pixel regions specified by each specific second virtual camera data Is displayed on each of the plurality of sub-screens,
The accepting step accepts designation of a specific second pixel region from among the plurality of second pixel regions respectively displayed on the plurality of sub-screens,
The third display control step, when the specification of a particular said second pixel region is accepted, wherein the specified particular the second pixel regions in claim 4, characterized in that to be displayed on the main screen Program. A first pixel region identified from among the plurality of pixels according to an operation of a virtual camera in the image frame of the moving image data composed of the plurality of image frames, the image frame including a plurality of pixels; First display control means for displaying on the display screen;
First virtual camera data representing at least one of the position of the virtual camera or the direction of the virtual camera that causes the first pixel area to be displayed on the display screen, and the first pixel area are displayed on the display screen. Transmitting means for associating the first reproduction time with a time series and transmitting the first reproduction time to the server device;
Second virtual camera data representing at least one of a position of a virtual camera or a direction of the virtual camera in the image frame of the moving image data, and a second specified from the plurality of pixels by the second virtual camera data. Camera work data in which a second reproduction time at which a two-pixel region is output is associated in time series, and the first reproduction time is selected from a plurality of camera work data stored in the server device. And obtaining means for obtaining specific camera work data including specific virtual camera data representing at least a part of the first pixel region;
Second display control means for displaying a pixel region according to the specific camera work data acquired by the acquisition means on the display screen along with the output of the moving image data;
A terminal device comprising: An image frame composed of a plurality of pixels, identified from among the plurality of pixels according to an operation of a virtual camera in the image frame along with output of moving image data composed of the plurality of image frames First display control means for displaying the first pixel region on the display screen;
First virtual camera data representing at least one of the position of the virtual camera or the direction of the virtual camera that causes the first pixel area to be displayed on the display screen, and the first pixel area are displayed on the display screen. Acquisition means for acquiring the first reproduction time in association with each other in time series;
Second virtual camera data representing at least one of a position of a virtual camera or a direction of the virtual camera in the image frame of the moving image data, and a second specified from the plurality of pixels by the second virtual camera data. The camera work data in which the second reproduction time in which the two-pixel region is output is associated in time series, the storage means storing a plurality of the camera work data acquired by the acquisition means First determination means for determining whether there is the second reproduction time corresponding to the first reproduction time;
When it is determined by the first determination unit that the data is stored in the storage unit, the second pixel region specified by the specific second virtual camera data associated with the second reproduction time is the first pixel region. Second determination means for determining whether at least a part of
When it is determined by the second determination means that at least a part of the first pixel area is included, the specific camera work data including the specific second virtual camera data is obeyed among the plurality of camera work data. Second display control means for displaying a pixel region on the display screen in accordance with the output of the moving image data;
A terminal device comprising:

Images