JP2004104556A - Method, device and program for reproducing video, and recording medium for video reproduction program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enable a user to view video at his/her own desired angle with respect to a client for reproducing distributed video. <P>SOLUTION: A video distribution server 11 distributes video to each of clients 14. The clients 14 receive and reproduce the distributed video. When the user designates a portion of an area of the received video as a reproduction area, the part of the area is extracted from the received video, the extracted video area is deformed into a reproduction area to be presented to the user, and the deformed video is reproduced. In addition, the information of the reproduction-designated area designated by the user is made to be transmittable and receivable to/from a second device to enable the second device to extract, deform and reproduce a selected area part. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a technology for reproducing video distributed over a network, and in particular, distributes a high-definition video, a wide-angle fisheye video, a 360-degree video, a multi-angle video, or the like from a video distribution server. The present invention relates to a video playback method, a video playback device, a video playback program, and a recording medium for the video playback program.
[0002]
[Prior art]
2. Description of the Related Art In recent years, broadband is being used in IP networks such as the Internet and FLET'S network. Along with this, a video distribution service has become popular as a service.
[0003]
In the video distribution service, the image quality has been improved and the bandwidth has been increased to distribute high-definition video such as high-definition video. Services for distributing multi-angle video shot at angles are also being provided.
[0004]
In particular, services for shooting live images with a camera and distributing the images on the Internet have been actively performed. As a service for displaying an image of an angle desired by a user in a live image, a service of controlling a camera from a user terminal to change a viewing angle has also been realized (for example, see Non-Patent Document 1).
[0005]
[Non-patent document 1]
Hideyuki Tamura, Yuko Wakatsuki, "Evolution of Computer Image Frontier III Cyber Cyber Video Space 10 Internet Broadcasting and Live Camera", "O plus E" (February 1998), New Technology Communications, p. 220-235
[0006]
[Problems to be solved by the invention]
However, there are some problems with the conventional technology for changing the angle of the distribution video by controlling the camera.
[0007]
First, when the same distribution video is distributed to a plurality of clients (video reproduction devices), when the camera is moved under the control of one client, all clients receiving the video of the camera are transmitted to all clients. The problem is that videos of the same angle are all distributed. That is, all users watch the same distribution video, and each user cannot view the video at their favorite angle.
[0008]
Second, since the angle of the distribution video cannot be changed unless the user controls the camera in real time, the target is limited to live distribution. In other words, the angle that can be changed by moving the camera can be changed only for the video currently being shot from the camera. For example, this technology cannot be applied to video distribution for distributing file video.
[0009]
The third problem is a problem from a user's point of view. Even if a non-expert user tries to move a camera that can freely switch angles, there is a case where the user does not know which angle to view. For example, even if it is easy to switch angles to follow a ball in a soccer video, it is difficult for an expert to change angles so that tactics can be understood. The same applies to a case where a multi-angle video is distributed.
[0010]
The present invention solves the above problems, and allows each user to view an image at his / her favorite angle, and can change the angle not only for a live image but also for a file image. It is an object of the present invention to provide a video reproduction technology that enables a user to view a video whose angle is changed by a specialist.
[0011]
[Means for Solving the Problems]
In order to achieve the above object, the present invention cuts out a partial area of a video distributed from a video distribution server as a reproduction area according to a user's specification, and transforms the area into a video shape to be presented to a user on a video reproduction apparatus. The main feature is that by reproducing the deformed video, the user can trim and view the video. Also, the present invention is characterized in that it is possible to pass trimming information to another person, and that another person can view a video using the trimming information.
[0012]
Specifically, the present invention relates to a video playback method in a client (video playback device) that is connected to a video delivery server that delivers video via a predetermined communication network and plays back video delivered from the video delivery server. Then, the client receives the video to be distributed, allows the user to designate a partial area of the received video as a playback area, extracts the area specified by the user, and transforms the area into a playback area for presentation to the user. In addition, the transformed video is reproduced.
[0013]
In the client, a part of the video delivered from the video delivery server is cut out and transformed into the shape of the playback area presented to the user on the client before playback. For example, when a high-definition video such as a high-definition video is distributed from a video distribution server, if the client cuts out and reproduces only a part of the area, the distribution video is changed by changing the cut-out area on the client side. An operation can be performed as if the angle was changed by a video playback device without changing the camera angle at which the image was taken. Therefore, since the camera is not controlled, it is possible to view images at different angles by many clients. Also, since the camera is not actually moved, the present invention can be applied to the distribution of archived video.
[0014]
Further, the present invention is characterized in that, in the above-mentioned video reproducing method, information on an area to be taken out by a client is stored, and the information is transmitted to an information distribution server or another client. Further, in the above-mentioned video reproducing method, the client receives information on an area to be taken out from the information distribution server or another client, and selects a reproducing area.
[0015]
For example, a client transmits area designation information on a reproduction area to another client. The client that has received the information cuts out the video using the area indicated by this information as the cutout area and plays back the video. For this reason, it is possible to view an image at an angle selected by another person belonging to the group without operating the device. For example, if the video is viewed by a client in the same group as a person who has specialized knowledge about switching camera angles, it is possible to view the video at an expert angle.
[0016]
Further, the client (video reproduction device) according to the present invention is connected to a video distribution server that distributes a video via a predetermined communication network, and reproduces the video distributed from the video distribution server. Means for receiving a video to be reproduced, means for allowing the user to designate a partial area of the received video as a playback area, means for extracting the area designated by the user, and transformation of the area into a playback area to be presented to the user. And a means for reproducing the transformed image.
[0017]
By using this client, the above-described video reproduction method can be executed.
[0018]
Further, the client is provided with means for storing information on an area to be extracted and means for transmitting the information to the information distribution server or another client. Further, it is characterized by comprising means for receiving information on an area to be taken out and means for selecting a reproduction area.
[0019]
Thus, in the above-described video reproducing method, it is possible to view a video of an angle selected by another person or an expert belonging to the same group.
[0020]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a diagram illustrating a configuration example of a video distribution system according to the present embodiment. In the video distribution system shown in FIG. 1, 11 is a video distribution server, 12 is an information distribution server, 13 is a network, and 14-1 to 14-n are clients.
[0021]
The network 13 connects the video distribution server 11, the information distribution server 12, and each client 14. The network 13 includes, for example, a communication network such as an Internet Protocol (IP) network such as the Internet, and a network combining radio wave broadcasting and a telephone network.
[0022]
The video distribution server 11 distributes a live broadcast video captured by a camera or a filed archive video. The delivered video is delivered to each client 14 via the network 13. The delivered image may be a high-definition image or an image captured by a fish-eye camera. In the case of a low-resolution video, there is no problem with the mechanism, but the reproduction image quality on the client 14 is deteriorated.
[0023]
The video distribution format may be one encoded by a compression encoding method such as MPEG-2 or Motion JPEG, or one encoded by an uncompressed encoding method such as DV. Good.
[0024]
FIG. 2 is a diagram illustrating an example of a device configuration of a client according to the present embodiment. The client 14 in the present embodiment is a video reproduction device for reproducing a video distributed from the video distribution server 11. In the client 14, 21 is a video receiver, 22 is a video selector, 23 is a video transformer, 24 is a video reproducer, 25 is an information receiver, 26 is a controller, and 27 is an information transmitter.
[0025]
The operation of the client 14 shown in FIG. 2 will be described by taking as an example a case where the user A who watches the video on the client 14-1 shown in FIG.
[0026]
The client 14-1 receives a video from the video distribution server 11 with the video receiver 21. The received video is sequentially sent to the video selector 22 after decoding if necessary. At this point, for example, in the case of a rectangular image, in an X × Y image with the number of pixels, the RGB values of each pixel are calculated for each frame by the coordinates (0, 0), (1, 0),. 1,0), (0,1), (1,1), ..., (X-1,1), ..., (0, Y-1), (1, Y-1), ..., (X- 1, Y-1).
[0027]
The user A uses the controller 26 to inform the video selector 22 of the area on the video representing the angle he / she wants to view. The region in which the controller 26 instructs the cutout may be specified by, for example, a rectangle, or may be any other shape such as a polygon or a circle.
[0028]
FIG. 3 is a diagram illustrating an example of a rectangular cutout area according to the present embodiment. As a method for the user A to instruct the controller 26, for example, as shown in FIG. 3, when the region to be cut out is a rectangle, the coordinates (x ₁ , Y ₁ ), (X ₂ , Y ₁ ), (X ₁ , Y ₂ ), (X ₂ , Y ₂ ) Can be directly input with a keyboard or specified using a device such as a mouse.
[0029]
In addition, the user A operates an input device such as a joystick while watching the video reproduced by the video reproducer 24 to change the vertical and horizontal changes based on the clipped video currently being viewed, zoom up, zoom down, and the like. May be instructed to the controller 26. In this case, the controller 26 calculates the coordinate values of the new cutout area from the coordinate values of the area cut out one step before and the amount of change input from the input device, and transmits the coordinate values to the video selector 22. I do.
[0030]
The vertex coordinates (x ₁ , Y ₁ ), (X ₂ , Y ₁ ), (X ₁ , Y ₂ ), (X ₂ , Y ₂ ) Is instructed, the video selector 22 selects, for example, the coordinates (x ₁ , Y ₁ ), (X ₁ +1, y ₁ ), ..., (x ₂ , Y ₁ ), (X ₁ , Y ₁ +1), (x ₁ +1, y ₁ +1), ..., (x ₂ , Y ₁ +1), ..., (x ₁ , Y ₂ ), (X ₁ +1, y ₂ ), ..., (x ₂ , Y ₂ ), The data in which the RGB values of each pixel are listed in this order are extracted and sent to the video deformer 23.
[0031]
The video deformer 23 shapes (deforms) the video data sent from the video selector 22 so that the video data is reproduced by the video reproducer 24. For example, when the size of the screen reproduced by the video reproducer 24 is a rectangle of M × N, the video deformer 23 cuts out the image (x ₂ -X ₁ +1) × (y ₂ -Y ₁ The image of +1) is enlarged or reduced to create an image of M × N size.
[0032]
As the processing at this time, it suffices to obtain the number of the RGB value of the RGB value of the pixel at the coordinates (m, n) of the reproduced video corresponding to the RGB value of the video data. For example, if the pixel corresponding to the coordinates (m, n) of the reproduced video is the L-th data among the above-listed video data,
L = [(x ₂ -X ₁ +1) × m ÷ M] +
[(Y ₂ -Y ₁ +1) × n ÷ N] × (x ₂ -X ₁ +1) +1 (Equation 1)
It becomes. However, in Equation 1, [] represents a Gaussian symbol and represents the largest integer not exceeding the numerical value in [].
[0033]
The video deformer 23 sends the shaped (deformed) video data to the video reproducer 24. When the video reproducer 24 can input only a video in a special format such as an MPEG-2 player, the video deformer 23 performs encoding according to a special format such as MPEG-2 encoding. , And sends the video data to the video reproducer 24. The video reproducer 24 reproduces the video of the angle specified by the user by sequentially reproducing the video data.
[0034]
Next, as a second example, an example will be described in which the distribution video is a video in which a plurality of reproduction areas are specified in advance as a plurality of viewing angles, or a multi-angle video shot at a plurality of angles.
[0035]
The client 14-1 receives a video from the video distribution server 11 with the video receiver 21. The received video is sequentially sent to the video selector 22 after decoding if necessary. An angle ID for identifying each of a plurality of viewing angle and multi-angle videos specified in advance is assigned.
[0036]
The user A informs the video selector 22 of the angle he or she wants to view using the controller 26. An angle can be specified to the controller 26 by, for example, an angle ID indicating one of the multi-angle images.
[0037]
When an angle ID to be reproduced is instructed from the controller 26, the video selector 22 selects a video area of the angle ID or the video itself from the video transmitted from the video receiver 21, and sends the video to the video deformer 23. send.
[0038]
The video deformer 23 shapes (deforms) the video data sent from the video selector 22 so that the video data is reproduced by the video reproducer 24. For example, the video area of the designated angle ID is transformed into a reproduction area, or encoded in accordance with a video format that can be reproduced by the video reproducer 24. The video deformer 23 sends the shaped (deformed) video data to the video reproducer 24. The video reproducer 24 reproduces the video of the angle specified by the user by sequentially reproducing the video data.
[0039]
Next, as a third example, the client 14-1 is operated to transmit the angle of the image a viewed by the user A to the other client 14-2 via the information distribution server 12, and the client 14-2 transmits the angle. , User B views video a having the same angle as user A.
[0040]
The video reproduction method in the client 14-1 is as described in the above-described example. Here, the controller 26 has means for storing the area coordinates or the angle ID designated by the user A, and sends to the information transmitter 27 the same area coordinates or the angle ID sent from the controller 26 to the video selector 22. From the information transmitter 27, the area coordinates or the angle ID are transmitted to the information distribution server 12 via the network 13 as information on the area to be reproduced in the video a (reproduction designated area information).
[0041]
FIG. 4 is a diagram illustrating a configuration example of the information distribution server according to the present embodiment. In the information distribution server 12 shown in FIG. 4, 31 is an information receiver, 32 is a transmission client selector, 33 is an information transmitter, and 34 is a client database (DB).
[0042]
The information distribution server 12 receives the area coordinates or the angle ID received from the client 14-1 by the information receiver 31 and sends it to the transmission client selector 32.
[0043]
Here, the information distribution server 12 manages each of the clients 14-1 to 14-n as a group. This management is, for example, a group in which the client 14-1 and the client 14-2 look at the video a, and the clients 14-3 and 14-4 look at the video b for each viewing video. Further, the plurality of clients 14-i to 14-j that view one video may be further divided into a plurality of groups.
[0044]
Information on these groups is managed by the client DB 34. The method of registering a group in the client DB 34 is not particularly defined, but, for example, a group can be determined when a user who uses the system is registered in advance. Also, a group may be dynamically divided for each viewing video.
[0045]
The transmission client selector 32 selects, from the group, a client that transmits the area information and the angle ID corresponding to the viewing angle of the video a sent from the client 14-1 with reference to the client DB 34. In this example, since the client 14-1 and the client 14-2 are in the same group, it is determined that the area information and the angle ID corresponding to the viewing angle of the image a are transmitted to the clients 14-1 and 14-2.
[0046]
The information transmitter 33 transmits area information and an angle ID corresponding to the viewing angle of the video a to the clients 14-1 and 14-2 via the network 13. The area information or angle ID to be transmitted is area information or angle IDs of all clients belonging to the same group. At this time, one or a plurality of area information and angle IDs are transmitted to all clients belonging to the same group.
[0047]
Further, the area information or the angle ID sent from the information transmitter 33 can be limited to only information sent from a specific client. For example, when the client 14-1 is a client operated by an information provider such as a broadcasting station, that is, an expert, and the client 14-2 is a client operated by a general viewer, the client 14-1 sends the client 14-1. Information may be sent to all clients in the same group as client 14-1, but information sent from client 14-2 may not be sent to other clients. In this case, only information indicating the viewing angle of the expert is sent to each client.
[0048]
In the client 14-2, the information receiver 25 receives the area information and the angle ID. The area information and the angle ID received by the client 14-2 are sent to the controller 26. Which area information or angle ID is used is determined by the controller 26. For example, the user B informs the controller 26 to select an angle using the received area information or angle ID of the user A. Here, the controller 26 may set in advance that the user B views at the same angle as the user A.
[0049]
The area information or the angle ID of the selected user A is sent to the video selector 22. The above-described processing of clipping or angle selection, transformation, and reproduction is performed using this information, so that the video reproducer 24 of the client 14-2 reproduces the video at the same angle as the client 14-1.
[0050]
Further, as a fourth example, there is a case where the information distribution server 12 does not exist in the system of FIG. 1 and the clients 14-1 to 14-n directly exchange information via the network 13. Hereinafter, the client 14-1 is operated to directly transmit the angle of the image a being viewed by the user A to the client 14-2 who is listening to the same image. An example in the case of viewing videos of the same angle will be described.
[0051]
The video reproduction method in the client 14-1 is as shown in the above-described example. Here, the same area coordinates or angle ID sent from the controller 26 to the video selector 22 are also sent to the information transmitter 27. From the information transmitter 27, the area coordinates or the angle ID are sent directly to the client 14-2 via the network 13 as information on the area to be cut out of the image a.
[0052]
In the client 14-2, the information receiver 25 receives the area information and the angle ID sent from the client 14-1.
[0053]
The area information and the angle ID received by the client 14-2 are sent to the controller 26. If the user B informs the controller 26 to select an angle using the area information or the angle ID of the user A received in advance or on the spot, the area information or the angle ID of the user A is used as the designation of the reproduction area. It is sent to the selector 22. The processing of the above-described cutout or angle selection, deformation, and reproduction is performed using this information, so that the video reproducer 24 reproduces the video at the same angle as that of the client 14-1.
[0054]
The present invention can be implemented by combining the first to fourth examples described above.
[0055]
FIG. 5 is a flowchart of a video reproduction process in the client 14-1 according to the present embodiment. It is assumed that the same video is distributed by streaming from the video distribution server 11 to each of the clients 14-1 to 14-N.
[0056]
In the client 14-1, when the reproduction of the distribution video is started by turning on the power or the like, first, in step S10, the partial area reproduction flag indicating whether the mode is the mode of trimming and reproducing the video is initially set to OFF. Become The partial area reproducing flag indicates that only a partial area is trimmed and reproduced when ON, and that normal full area reproduction is performed when OFF.
[0057]
In step S11, the video distributed by the video distribution server 11 is received and reproduced. In step S12, it is determined whether or not the user has input reproduction designated area information for designating a partial area of the video as a reproduction area, or whether or not reproduction designated area information has been received from the information distribution server 12 or another client. . If there has been input or reception, in step S13, the reproduction designation area information is stored in the internal memory. If there is no input or reception, the process proceeds to step S14.
[0058]
In step S14, it is determined whether or not the user of the client 14-1 has instructed to reproduce the designated reproduction area. If there is an instruction to reproduce the designated reproduction area, a partial area reproduction flag is set to ON in step S15. If there is no instruction to reproduce the designated reproduction area, the process proceeds to step S16.
[0059]
In step S16, it is determined whether the partial area reproduction flag is ON or OFF. If it is OFF, the process returns to step S11, and normal reception and reproduction of the distribution video are continued. If it is ON, the process proceeds to step S17.
[0060]
In step S17, the distribution video is received, and a partial area of the received video is extracted according to the reproduction designated area information stored in step S13. If there is no input or reception of the designated reproduction area information in advance, the processing may be performed as an entire area as a default, or the user may be asked about the designated reproduction area.
[0061]
Subsequently, in step S18, the extracted area is transformed into a reproduction area to be presented to the user by the video deformer 23. In step S19, the transformed video is output to the video reproducer 24 and reproduced.
[0062]
Next, in step S20, it is determined whether or not the user has instructed the controller 26 to transmit reproduction designated area information. If there is no transmission instruction, the process proceeds to step S22. If there is an instruction to transmit the designated reproduction area information, the designated reproduction area information is transmitted to the information distribution server 12 or another client in the same group in step S21.
[0063]
In step S22, it is determined whether or not the user has instructed to cancel the reproduction instruction in the reproduction designated area. If the reproduction instruction has been released, the process returns to step S10, turns off the partial area reproduction flag, and returns to the normal distribution video reception and reproduction mode. If the reproduction instruction of the reproduction designated area has not been released, the process returns to step S12, and the processing of trimming, transforming, and reproducing the video is similarly repeated. If the reproduction designation area information is re-input or re-received during the reproduction of the partial area, the reproduction designation area is changed in steps S12 and S13.
[0064]
The above processing can be realized by a computer and a software program, and the program can be stored in an appropriate recording medium such as a computer-readable portable medium memory, a semiconductor memory, and a hard disk.
[0065]
【The invention's effect】
As described above, according to the present invention, even if the same video is distributed from the video distribution server to all clients (video reproduction devices), the user operating each client cuts out or selects a video at a different angle. As a result, it is possible to play back, so that it is possible to view the video at an angle of your choice.
[0066]
In addition, since the angle change at the client can be performed by the operation of the client without depending on the video distributed from the video distribution server, even if the archived video filed from the video distribution server is distributed, the user can change his / her favorite angle. It is possible to see the video at an angle.
[0067]
Furthermore, since the region from which the video is cut out can be shared between the grouped clients, even if the user is not an expert, if he or she is in the same group as the expert, he or she can receive the distribution of the area information or the angle ID, so that the expert can receive the information. It is also possible to watch the same angle of the video as the house.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating a configuration example of a video distribution system according to an embodiment.
FIG. 2 is a diagram illustrating an example of a device configuration of a client according to the present embodiment.
FIG. 3 is a diagram illustrating an example of a rectangular cutout area according to the present embodiment.
FIG. 4 is a diagram illustrating a configuration example of an information distribution server according to the present embodiment.
FIG. 5 is a flowchart of a video reproduction process in a client according to the present embodiment.
[Explanation of symbols]
11 Video distribution server
12 Information distribution server
13 Network
14-1 to 14-n client
21 Video receiver
22 Video selector
23 Video Transformer
24 Video player
25 Information receiver
26 Controller
27 Information transmitter
31 Information receiver
32 Transmission client selector
33 Information transmitter
34 Client DB

Claims (10)

A video playback method in a video playback device connected to a video delivery server for delivering a video via a predetermined communication network and for playing back a video delivered from the video delivery server,
Receiving a video distributed from the video distribution server;
Inputting area designation information for designating a partial area of the received video as a playback area;
Extracting a region designated as a reproduction region from the received video according to the region designation information;
Transforming the extracted video area into a playback area to be presented to the user;
Reproducing the deformed video. The video reproducing method according to claim 1,
Storing area designation information for designating a partial area of the video as a reproduction area;
Transmitting the stored area designation information to another device. The video reproducing method according to claim 1,
Receiving, from another device, region designation information for designating a partial region of the video as a reproduction region,
A video reproducing method, wherein an area specified as a reproduction area is extracted from a received video according to the received area specifying information. A video reproducing apparatus connected to a video distribution server for distributing video via a predetermined communication network and reproducing a video distributed from the video distribution server,
Means for receiving a video distributed from the video distribution server;
Means for inputting area specification information for specifying a partial area of the received video as a reproduction area;
Means for extracting an area designated as a reproduction area from the received video according to the area designation information;
Means for transforming the extracted video area into a playback area to be presented to the user;
Means for reproducing the deformed video. The video playback device according to claim 4,
Means for storing area specification information for specifying a partial area of the video as a reproduction area;
Means for transmitting the stored area designation information to another device. The video playback device according to claim 4,
Means for receiving, from another device, area designation information for designating a partial area of the video as a reproduction area,
A video reproducing apparatus, wherein the means for extracting the area specified as the reproduction area extracts an area specified as a reproduction area from a received video in accordance with the received area specification information. A video reproduction program for controlling a video reproduction device connected to a video distribution server that distributes a video via a predetermined communication network and reproducing a video distributed from the video distribution server,
A process of receiving a video distributed from the video distribution server;
Processing for inputting area designation information for designating a partial area of the received video as a playback area;
Processing for extracting an area specified as a reproduction area from the received video according to the area specification information;
Processing to transform the extracted video area into a playback area to be presented to the user;
The process of playing back the transformed video
A video playback program to be executed by a computer. The video playback program according to claim 7,
Processing for storing area designation information for designating a partial area of the video as a playback area;
Transmitting the stored area designation information to another device;
A video playback program to be executed by a computer. The video playback program according to claim 7,
A process of receiving, from another device, area designation information for designating a partial area of the video as a playback area;
Processing for extracting an area specified as a reproduction area from the received video according to the received area specification information;
A video playback program to be executed by a computer. A recording medium for a video reproduction program, wherein the video reproduction program according to any one of claims 7 to 9 is recorded.

Images