JP5594842B2 - Video distribution device - Google Patents

Description

  The present invention relates to a video distribution apparatus that distributes video content in a streaming manner, and in particular, a video distribution apparatus that automatically sets a zoom range including an important area of an original video, and cuts out the zoom range from the original video and distributes it as a zoom video. About.

  A so-called multi-screen service that enables viewing of content for a large screen such as a cable television on not only a TV but also a device such as a PC, a mobile phone, and a stylus portable terminal is becoming popular. Non-Patent Document 1 discloses a technology for distributing content for cable television to a PC. In such a multi-screen service, it is necessary to convert high-resolution content created for a television with a large display to a low resolution for a mobile phone with a small display.

  In a general multi-screen service, the entire content for a large screen is converted (transcoded) as it is and streamed to a mobile phone or the like. However, mobile phones and the like have a problem that the attractiveness of content cannot be fully exhibited because the display is small. Therefore, there has been a demand for a mechanism that allows the user to enlarge and view an arbitrary area that the user wants to view.

  As a technique for enlarging video content, Patent Document 1 discloses a digital zoom for trimming (cutting out) and presenting an arbitrary part in a video. However, such a digital zoom function is performed as a process at the time of video shooting, and does not take into consideration the bit rate limitation in video transmission and the low delay required for streaming in real time, and has a high load. Since processing is required, application to video streaming is difficult.

  As a method for solving such a technical problem, a method using real-time encoding in which an enlarged image desired by the user is encoded and transmitted in real time on the server side can be easily imagined. However, if the user always wants to encode and transmit the enlarged video, the load on the server increases, and the load increases in proportion to the number of users receiving the service. At the same time, it is not practical to provide services.

  Furthermore, as another solution to the above technical problem, Non-Patent Document 2 divides and encodes video content into tiles, and transmits a part of the entire tile from the server side, while receiving it at the client side. A method is disclosed in which an enlarged image is obtained by decoding an image for each tile and displaying the decoding result at the same time. However, in this method, it is necessary to individually decode the video divided into tiles on the client side and display it in synchronization. Therefore, it is not practical to install it on a terminal with low processing capability such as a portable terminal. No.

  In response to such a technical problem, in Patent Document 2 by the present inventors, a plurality of encoded files for zoom having different resolutions are prepared in advance in the video distribution server according to the stage of the zoom magnification that can be provided. In addition, when a zoom request is detected, a zoom position image is cut out from a zoom encoding file corresponding to the requested zoom magnification into a rectangular image, thereby reducing the load on the zoom request. A technique for distributing zoom video is disclosed.

JP2008-316739 Japanese Patent Application No. 2010-219728

Fancast (http://www.fancast.com/) A Study on Multi-resolution Distribution Method for Interactive Panorama Video Distribution System (Information Processing Society of Japan Vol.2010-AVM-70 No.8)

  In Patent Document 2, since the rectangular area cut out from the original image is fixed, it is difficult to obtain an ideal zoom image from an image in which important areas are dispersed at a plurality of locations in the screen. there were.

  FIG. 29 is a diagram for explaining such a technical problem. The original image [FIG. 29 (a)] is divided into nine 3 × 3 slices P11 to P33. This is realized by cutting out 4 slices of 2 × 2 from 9 slices of 3 × 3.

  In the example shown in the figure, the zoom operation is performed by the user on the non-zoom image in FIG. 5B, and generally, the display area of the person is an important area, so all five persons are included. The figure (c) is an ideal zoom image. However, in the illustrated example, five persons are dispersed in the screen. Therefore, if the slice is fixed as in the prior art, even if many persons are included in the zoom image, the figure (d). It is necessary to select 4 slices of P12, P13, P22, and P23 as shown in Fig. 4 or 4 slices of P22, P23, P32, and P33 as shown in Fig. 3 (e). Could not get.

  An object of the present invention is to solve the above-described problems of the prior art and to provide a video distribution apparatus in which a zoom range is automatically set so as to include an important region and an ideal zoom video can be easily obtained.

  In order to solve the above technical problem, the present invention is characterized in that the following means is taken in a video distribution apparatus for streaming distribution of a zoom video of a video content.

  (1) Means for storing a plurality of zoom videos having different resolutions according to the zoom rate for each video content, means for extracting an important area from a frame of the video contents, and a rectangle including the important area is the minimum distribution aspect ratio Means for expanding to the size and setting the zoom range; means for determining the zoom rate according to the size of the zoom range; and means for setting zoom conditions including a zoom video and a cutout range of the zoom video according to the zoom rate And means for cutting out the zoom image based on the zoom condition and means for distributing the cut out zoom image.

  (2) The video content further includes means for dividing the shot content and means for obtaining a representative frame from each shot, and the means for extracting the important area extracts the important area from each representative frame.

  (3) The means for setting the zoom range is a means for setting the rectangle including the important region for each representative frame as an important rectangle, and the position of the important rectangle in each representative frame is compared in time series, A means for integrating the important rectangles of each representative frame into one for each range where the distance is below a predetermined threshold is provided, and each important rectangle is expanded to the minimum size of the distribution aspect ratio and set to the zoom range. .

  (4) The means for cutting out the zoom image slices the zoom image according to the zoom ratio into n × m, and zooms the p × q slice including the zoom range from the zoom image sliced into the n × m. In addition to cutting out as an image, the zoom image is sliced more finely than the n × m when the zoom range cannot be included in the p × q slice.

  (5) A means for transmitting the coordinate position of the zoom image together with the zoom image is further provided.

  (6) comprising means for receiving an operation signal for requesting delivery of a zoom video from a video playback terminal; and means for analyzing the operation signal to obtain a zoom condition, wherein the means for cutting out the zoom video comprises the zoom condition The zoom condition obtained by analyzing the operation signal is prioritized over the zoom condition set by the means for setting.

  According to the present invention, the following effects are achieved.

  (1) The zoom range is automatically set based on the important area from the video content, and the video in the zoom range is automatically cut out from the original video and distributed as a zoom video. An optimal zoom video can be streamed according to the content of the video content without requiring a complicated zoom operation.

  (2) Since the important area is extracted for each frame, the zoom condition can be switched for each frame.

  (3) Since the positions of important rectangles in the same frame are compared in time series, and important rectangles that are close to each other are integrated into one important rectangle, the zoom condition can be switched frequently with only a slight difference in feature amount. Can be prevented.

  (4) When the zoom range cannot be included in the p × q slice, the zoom image is further sliced, so that the minimum range including the zoom range can be distributed as a zoom image. The important area can be displayed larger in the zoom image.

  (5) Since the coordinate position of the zoom video is transmitted to the playback terminal side together with the zoom video, the playback terminal side can easily recognize the zoom position in the original video.

1 is a block diagram of a video distribution system according to an embodiment of the present invention. It is the figure which showed the operation | movement outline | summary of this invention. It is the figure which showed an example of the encoding file of the image | video for zooms. It is the figure which showed the method of cutting out a zoom image from an original image. 5 is a flowchart illustrating a method for automatically setting a zoom range. It is a figure for demonstrating shot division | segmentation. It is the figure which showed the example of extraction of an important area | region. It is the figure which showed the identification method of telop with low importance. It is the figure which showed the setting method of the important rectangle. It is the figure which showed the method (the 1) which compares an important rectangle in a time series. It is the figure which showed the integration method (the 1) of the important rectangle. It is the figure (the 1) which expressed integration of the important rectangle in the tree form. It is the figure which showed the method (the 2) which compares an important rectangle in a time series. It is the figure which showed the integration method (the 2) of an important rectangle. It is the figure (the 2) which expressed integration of the important rectangle in the tree form. It is the figure which showed the method (the 3) which compares an important rectangle in a time series. It is the figure which showed the integration method (the 3) of an important rectangle. It is the figure (the 3) which expressed integration of the important rectangle in the tree form. It is the figure which showed the method (the 4) which compares an important rectangle in a time series. It is the figure which showed the integration method (the 4) of an important rectangle. It is the figure (the 4) which expressed integration of the important rectangle in the tree form. It is the figure which showed the method of extending an important rectangle to the minimum size of a delivery aspect ratio, and setting a zoom range. It is the flowchart which showed the method of cutting out a zoom range from the image | video for zooms, and delivering a zoom image | video. It is the figure which showed the relationship between the resolution of the image | video for zoom, the magnification with respect to a delivery size, and the value which converted the display range of the zoom image | video into the resolution of the original image | video. It is the figure which showed an example of the zoom range. It is the figure which showed the setting method of the extraction range which can include a zoom range. It is the figure which showed an example of zoom conditions. It is the figure which showed the example of reproduction | regeneration of the zoom image | video in a reproduction | regeneration terminal. It is a figure for demonstrating a technical subject.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing the configuration of a video distribution system according to an embodiment of the present invention, in which a video distribution device 1 that distributes video content in a streaming format and video reproduction that receives and reproduces the video content The terminal 2 is connected via a wide area network such as the Internet. The video distribution device 1 detects a scene suitable for zoom playback from the video content and distributes the zoom video of the video content automatically or in response to a zoom request from the playback terminal 2.

  FIG. 2 is a diagram showing an outline of the operation of the present invention. The video distribution apparatus 1 is a video content of high resolution (for example, 1920 × 1056 pixels) captured and edited for a large-sized TV [the same figure (a )] Is converted into distribution resolution, that is, the display resolution of the playback terminal 2 (for example, 640 × 352 pixels) for distribution [FIG. (B)], and further switched to zoom video for distribution [FIG. (C)]. To do.

  In the present invention, in order to realize such distribution of zoom video without adversely affecting the bit rate and transmission delay and without requiring high processing capability for the playback terminal 2, the zoom rate is determined for each video content. Encoding files for a plurality of zoom videos having different resolutions are prepared in advance.

  In the present embodiment, as shown in FIG. 3, four zoom image (large zoom, medium zoom, small zoom, and no zoom) files are prepared in advance according to the zoom magnification. The resolution of the image for large zoom [FIG. (A)] is 1920 × 1056, which is the same as the original image. The resolution of the medium zoom image [(b) in the same figure] is 1280 × 704. The resolution of the image for small zoom [FIG. (C)] is 960 × 528. The resolution of the non-zoom video [(d)] is 640 × 352, which is the same as the distribution video.

  The grid lines shown in each zoom image represent slice boundaries. For example, when middle zoom is requested, as shown in FIG. 4, for example, a middle zoom image file sliced 3 × 3. From this, only the 2 × 2 video range is cut out according to the zoom position and distributed as a zoom video. The position, shape, and size of each slice are defined so that a zoom video having a distribution resolution can be configured by a plurality of sets according to the zoom position.

  In the example shown in the figure, slices 1-1, 1-2, 2-1, and 2-2 are cut out when the zoom range is at the upper left of the screen, and slices 1-2, 1 and 1 are cut out when the zoom range is at the upper right of the screen. 3, 2-2 and 2-3 are cut out. Similarly, slices 2-1, 2-2, 3-1, 3-2 are cut out if the zoom range is in the lower left of the screen, and slices 2-2, 2-3, if the zoom range is in the lower right of the screen. 3-2 and 3-3 are cut out.

  Next, a method of automatically setting the zoom range based on the feature region of the original video will be described with reference to the flowchart of FIG.

  In step S1, as shown in FIG. 6 as an example, the original video is divided into shots at shot boundaries. A shot boundary is a point at which the viewpoint of a camera is switched by video editing (a point at which a video is cut). For example, as disclosed in Japanese Patent Application Laid-Open No. 2007-134986, a difference amount between consecutive frames. Can be obtained. Since the optimal zoom condition (zoom position and zoom plate ratio) is likely to change if the camera viewpoint changes, in this embodiment, the shot boundary is set as the basic timing for zoom switching.

  In step S2, the current shot of interest is selected in time series from a number of shots obtained by the shot division. In step S3, a representative frame is selected from the target shot. In step S4, as shown in an example in FIG. 7, important regions (indicated by circles or ellipses) are extracted from the representative frames f1 to f5.

  Here, the important region is a kind of ROI (Region of interest) obtained by analyzing the characteristics of the original video, and appears in an arbitrary number and an arbitrary area in an arbitrary place in the frame. In the present embodiment, the following areas are extracted as important areas.

  (1) Person area

  A person is an important area in many images, and is extracted by applying, for example, face recognition disclosed in Japanese Patent Laid-Open No. 2006-508461.

  (2) Telop area

  The telop often includes important character information in the video. Therefore, as disclosed in JP-A-12-23062, (a) the telop is displayed in a predetermined area at the top or bottom of the screen, (b) a luminance change occurs at the time of appearance and termination of the telop, The telop area is extracted using the feature.

  In the present embodiment, as illustrated in FIG. 8, among the extracted telops, telops that are displayed for a relatively long time across a number of representative frames are less important. It is judged and excluded.

  (3) Highly remarkable area

  In addition to a person or a telop, a highly significant area is extracted as an important area. In the present embodiment, “feature points”, “frequency components”, and “motion gaze models” are referred to as lower-level feature amounts that indicate high saliency.

  (1) Features

  The screen is divided by a fixed grid unit, and a grid area in which the number of feature points per unit area is equal to or larger than a fixed value is extracted as a remarkable area. It is assumed that there is a lot of significant information in the region with many feature points, and for the extraction, for example, the technique of Harris or Stephens can be adopted as a technique for detecting the corner of the image like the corner detection technique. .

  (2) Frequency component

  In the image, since there is more significant information in the region with more change, a remarkable region is extracted based on the total energy amount of the AC high-frequency component of the DCT coefficient. In general, the visual characteristics of the human eye are sensitive to low frequency components and insensitive to high frequency components. DCT is a technology that converts a pixel region of a video frame into a frequency region using discrete cosine transform.In this embodiment, a salient region is extracted based on the sum of high-frequency components (dAC) of DCT per block. Is done.

  (3) Motion gaze model

  Depending on the amount of movement of the local area on the screen, the coordinates of the area to be watched are calculated by a technique such as that described in Non-Patent Document 3. The important object acquired in each representative frame is weighted according to its spatial position and feature amount.

  (4) Other

  Not only the important area detection based on the feature analysis in the video, but also the important object in the video specified by the video producer can be set as the remarkable area. For example, in a certain video content, when the appearance time and the coordinates of the main performer are specified by the video producer and given as metadata, a remarkable area is extracted using the given information instead of the above analysis result To do.

  Returning to FIG. 5, in step S5, as shown in FIG. 9, for example, a rectangular range including all the important areas is set to the important rectangle C (C1, C2,... C5) for each representative frame f. In step S6, the second important rectangle in the time series in the shot of interest is selected as the current rectangle of interest. Here, the important rectangle C2 of the second representative frame f2 is selected as the target rectangle.

  In step S7, a logical sum (OR) area between the target rectangle C2 and two important rectangles adjacent to the target rectangle C2 is calculated. Here, as shown in FIG. 10, the OR region C1 / 2 between the target rectangle C2 and the important rectangle C1 of the representative frame f1, and the OR region C2 / 3 between the target rectangle C2 and the important rectangle C3 of the representative frame f3 Calculated.

  In step S8, the areas of the OR regions C1 / 2 and C2 / 3 are compared. In step S9, the important rectangle pair having the smaller area of the OR region is integrated into one important region. Here, since the area of the OR region C2 / 3 is smaller, the target rectangle C2 and the important rectangle C3 are integrated into one important rectangle C2-3 as shown in FIG.

  In step S10, as shown in FIG. 12, in the virtual node tree with the important rectangles C1 to C5 as leaves, the branch of the integrated rectangle of interest C2 and the branch of the important rectangle C3 are connected, and the branch length L1 (vertical axis) is set to the absolute value of the area difference between the OR regions C1 / 2 and C2 / 3. In step S11, it is determined whether or not all the important rectangles have been tree-formed. Here, it is determined that the tree-forming has not been completed, and the process returns to step S6.

  In step S6, the integrated important rectangle C2-3 is selected as the current attention rectangle as the second important rectangle in time series. In step S7, an OR region between the target rectangle C2-3 and two important rectangles adjacent to the target rectangle C2-3 is calculated. Here, as shown in FIG. 13, the OR region C1 / 2-3 between the target rectangle C2-3 and the important rectangle C1 of the representative frame f1, and the important rectangle C4 of the target rectangle C2-3 and the representative frame f4 OR region C2-3 / 4 is calculated.

  In step S8, the areas of the OR regions are compared, and in step S9, the smaller pair is integrated. Here, since the area of the OR region C1 / 2-3 is smaller, the important rectangle C1 and the target rectangle C2-3 are integrated into the important rectangle C1-3 as shown in FIG. In step S10, as shown in FIG. 15, the branch of the important rectangle C1 and the branch of the target rectangle C2-3 are connected, and the branch length L2 is the area of the OR regions C1 / 2-3 and C2-3 / 4. Set to the absolute value of the difference. In step S11, since it is determined that the tree formation of the important rectangle is not completed, the process returns to step S6 again.

  In step S6, the second important rectangle C4 in time series is selected as the current attention rectangle. In step S7, OR regions of the noticed rectangle C4 and two important rectangles adjacent to it are calculated. Here, as shown in FIG. 16, the OR region C1-3 / 4 of the target rectangle C4 and the important rectangle C1-3, and the OR region C4 / 5 of the target rectangle C4 and the important rectangle C5 of the representative frame f5 Calculated.

  In step S8, the areas of the OR regions are compared. In step S9, the smaller important region pairs are integrated. Here, since the area of the OR region C4 / 5 is smaller, the important rectangles C4 and C5 are integrated into one important rectangle C4-5 as shown in FIG. In step S10, as shown in FIG. 18, the branch of the rectangular region C4 and the branch of the important region C5 are connected, and the branch length L3 is the absolute value of the area difference between the OR regions C1-3 / 4 and C4 / 5. Set to In step S11, since it is determined that the tree formation of the important rectangle is not completed, the process returns to step S6 again.

  In step S6, the second important rectangle C4-5 in time series is selected as the current attention rectangle. In step S7, an OR area between the target rectangle C4-5 and two important rectangles adjacent to the rectangle is calculated. Here, since there is no adjacent important region later, only the OR region C1-3 / 4-5 of the target rectangle C4-5 and the important rectangle C1-3 is calculated as shown in FIG.

  In step S8, the areas of the OR regions are compared, and in step S9, the important rectangular pairs with smaller areas are integrated. Here, since only the OR region C1-3 / 4-5 has been calculated, as shown in FIG. 20, the important rectangles C1-3 and C4-5 are integrated into one important rectangle C1-5. In step S10, as shown in FIG. 21, the branch of the important rectangle C4 and the branch of the important area C5 are connected, and the branch length L4 is the absolute difference in area between the important rectangle C1-3 and the target rectangle C4-5. Set to a value. In step S11, since it is determined that the tree formation of the important rectangle is completed, the process proceeds to step S12.

  In step S12, a zoom switching point is detected based on the node tree of FIG. In this embodiment, each branch length L is compared with a threshold value, and here, since the branch length L4 exceeds the threshold value, a zoom switching point is set between the important rectangles C1-3 and C4-5. Therefore, zoom switching occurs at one location in the shot.

  In step S13, as shown in an example in FIG. 22, the integrated important rectangles Cx are expanded to the minimum size of the distribution aspect ratio and set to the zoom range Z (Cx). In step S14, it is determined whether or not the detection of the zoom switching point and the setting of the zoom range have been completed for all shots. If not completed, the process returns to step S2, and the above processes are repeated while changing the shot of interest.

  As described above, when the zoom range Z (Cx) including the important rectangle Cx is determined, the zoom range Z (Cx) is cut out from the zoom video and distributed as a zoom video.

  Next, with reference to the flowchart of FIG. 23, a method of cutting out the zoom range Z (Cx) from the zoom video and distributing the zoom video will be described.

  Also here, the resolution of the original video is 1920 × 1056, the resolution of the video delivered to the mobile phone is 640 × 352, and the four zoom videos described with reference to FIG. Large zoom, medium zoom, small zoom, and no zoom) are prepared in advance.

  FIG. 24 shows the relationship between the resolution of each zoom video, the magnification with respect to the distribution size (640 × 352), and the value obtained by converting the display range of the zoom video to the resolution of the original video. For example, a medium zoom image means that the resolution is cut out from a medium zoom image [FIG. 3 (b)] of 1280 × 704, and its display range corresponds to a pixel range of 960 × 528 on the original image. Yes.

  In step S21, a zoom rate corresponding to the minimum cutout size that can include the zoom range Z (Cx) is selected. For example, as shown in FIG. 25, if the size of the zoom range Z (Cx) on the original video is 1160 × 652, the cut-out size to be distributed as the zoom video is 1160 × 652 on the original video. It is desirable to set it to the smallest size that can contain 652 sizes.

  Here, since only a size of 960 × 528 can be cut out with the middle zoom, a part of the zoom range Z (Cx) is lost. Further, since it can be cut out with a size of 1920 × 1056 without zooming, it can include the entire zoom range Z (Cx), but also includes many areas other than the zoom range Z (Cx). On the other hand, since the small zoom can be cut out with a size of 1280 × 704, the entire zoom range Z (Cx) can be included, and the area other than the zoom range Z (Cx) can be minimized. Therefore, the small zoom is selected here.

  In step S22, it is determined whether or not the zoom range Z (Cx) can be accommodated within the cut-out size with a standard slice for small zoom. In this embodiment, as shown in FIG. 26A, the standard slice of the small zoom video is 3 × 3, but the cut slice is 2 × 2, so how to set the position of the cut slice Even if it is determined that the zoom range Z (Cx) cannot be completely included in the cut slice, the process proceeds to step S23.

  In step S23, as shown in FIG. 26B, the number of slices is set to 6 × 6, which is double. In step S24, it is determined whether or not the zoom range Z (Cx) can be included in 4 × 4 of the cut slice in the 6 × 6 slice. Here, since it is determined that it can be included, the process proceeds to step S25. In step S25, a 4 × 4 slice that can include the zoom range Z (Cx) is cut out. In step S26, the cut slice is distributed as a zoom video.

  Referring back to FIG. 1, in the video distribution apparatus 1, the zoom video generation unit 101 generates a plurality of zoom video files having different resolutions according to the zoom rate for each original video, and uses the H.264 encoding method. It is encoded and stored in the zoom video DB 102. In the present embodiment, the encoded files of the large zoom image, middle zoom image, small zoom image, and non-zoom image are stored.

  As described with reference to FIG. 6, the shot division unit 103 divides the original video into shots at shot boundaries. The zoom range setting unit 104 sets the zoom range Z (Cx) that is cut out from the original video and distributed as the zoom video based on the important area included in the original video. In the zoom range setting unit 104, the representative frame acquisition unit 104a acquires a representative frame from the original video. The important area extraction unit 104b extracts an important area from each representative frame. The important rectangle setting unit 104c sets, as the important rectangle Cx, a rectangular range that can include all the important areas for each representative frame.

  The zoom switching point detection unit 104d detects a zoom switching point for switching the zoom position and the zoom rate in the same shot based on the time series change in the same shot in the rectangular range. The zoom range expansion unit 104e expands the important rectangle Cx to the minimum range of the distribution aspect ratio and sets it to the zoom range Z (Cx). In the present embodiment, since the important rectangle changes in time series, the zoom range also changes in time series.

  The zoom rate determination unit 105 determines the zoom rate according to the size of the zoom range Z (Cx). The zoom condition determination unit 106 uses a zoom video file to be cut out and a slice of the zoom video file as a zoom condition for appropriately cutting out a zoom video including the zoom range from the zoom video file according to the zoom rate. The method and coordinate data to be cut out as a zoom image are determined and stored in the zoom condition DB 107.

  FIG. 27 is a diagram showing an example of the zoom condition. In this embodiment, the zoom condition is expressed by four numerical values, and “zoom switching frame number” and “display designated coordinate 1 ( “Upper left coordinates of zoom range)”, “designated display coordinates 2 (lower right coordinates of zoom range)”, and “enlargement ratio (four steps)”. Therefore, the zoom condition [275,320,240,2] in the figure means that “from the 275th frame, the zoom range in which the display designation coordinate 1 is 320 and the display designation coordinate 1 is 240 is zoomed at the enlargement ratio“ 2 ””. ing.

  Based on the zoom condition, the zoom control unit 108 cuts out a zoom video that includes the zoom range from the zoom video file, and distributes it from the distribution unit 109 to the playback terminal 2. At this time, the video data for zoom distribution is multiplexed with the audio data and transmitted in a transmission format such as RTP, for example. At this time, coordinate information regarding the zoom position may be transmitted to the playback terminal side at regular intervals (for example, every second). The coordinate information may be transmitted separately from the video data and audio data using, for example, RTCP, or may be described in the RTP header and the like in the same packet as the video data and audio data. It may be transmitted.

  In the playback terminal 2, the video reception unit 201 receives the zoom video from the video distribution system, transfers it to the video playback unit 202, and plays it back. The user operation reception unit 203 receives a user operation such as a key input requesting zoom reproduction and converts it into an operation signal. This operation signal is transmitted from the operation signal transmission unit 204 to the video distribution apparatus 1.

  FIG. 28 is a diagram showing an example of playback on the playback terminal side in the present embodiment. On the playback screen, along with the zoom video 251, a guidance display 252 indicating the range of the zoom video on the original video, and automatic A switch 253 for instructing start / stop of enlargement is provided. The content of the guidance display 252 is generated based on the coordinate information of the zoom position transmitted from the video distribution device 1, and the illustrated example shows that the current zoom position is at the upper center.

  In this embodiment, when the terminal user requests zoom distribution on the screen by tapping or the like, this operation signal is transmitted to the video distribution apparatus 1, received by the operation signal receiving unit 110, and notified to the user operation analysis unit 111. The The user operation analysis unit 111 analyzes the user operation and identifies the zoom condition requested by the terminal user. The zoom control unit 108 prioritizes the zoom condition specified by the terminal user over the zoom condition automatically set by the zoom condition setting unit, and cuts out the zoom image.

DESCRIPTION OF SYMBOLS 1 ... Video distribution apparatus, 2 ... Playback terminal, 101 ... Zoom video production | generation part, 102 ... Zoom video DB, 103 ... Shot division | segmentation part, 104 ... Zoom range setting part, 105 ... Zoom rate determination part, 106 ... Zoom conditions Determination unit 107 107 Zoom condition DB 108 Zoom control unit 109 Distribution unit 110 Operation signal reception unit 111 User operation analysis unit 201 Video reception unit 202 Video reproduction unit 203 User operation Reception unit, 204 ... operation signal transmission unit

Claims (6)

In a video distribution device for streaming distribution of zoom video of video content,
Means for storing a plurality of zoom images having different resolutions according to a zoom rate for each video content;
Means for extracting important regions from the frame of video content;
Means for expanding a rectangle including the important area into a minimum size rectangle among rectangles including the rectangle and having the same aspect ratio as the distribution aspect ratio, and setting the zoom range;
Means for determining a zoom rate according to the size of the zoom range;
Means for setting zoom conditions including a zoom image and a cutout range of the zoom image according to the zoom rate;
Means for cutting out a zoom image based on the zoom condition;
A video distribution apparatus comprising: means for distributing the clipped zoom video. Means for dividing the video content into shots;
Means for obtaining a representative frame from each shot,
2. The video distribution apparatus according to claim 1, wherein the means for extracting the important area extracts an important area from each representative frame. The means for setting the zoom range includes:
Means (104c) for setting a rectangle including the important region for each representative frame as an important rectangle;
Means for comparing the positions of the important rectangles of each representative frame in time series, and integrating the important rectangles of each representative frame into one for each range in which the distance between the important rectangles falls below a predetermined threshold;
The each important rectangle is expanded and set to a zoom range so as to be a rectangle having a minimum size among rectangles that include the rectangle and have the same aspect ratio as the distribution aspect ratio. 2. The video distribution device according to 2. The means for cutting out the zoom video slices the zoom video according to the zoom ratio into n × m, and uses the zoom video sliced into the n × m as a zoom video with a p × q slice including the zoom range. Cut out,
The means for cutting out the zoom image slices the zoom image finer than the n × m when the zoom range cannot be included in the p × q slice. 4. The video distribution device according to any one of items 3 to 3. 5. The video distribution apparatus according to claim 1, further comprising means for transmitting a coordinate position of the zoom video together with the zoom video. Means for receiving an operation signal for requesting distribution of zoom video from a video playback terminal, and means for analyzing the operation signal to obtain a zoom condition;
6. The zoom condition acquired by analyzing the operation signal is prioritized over the zoom condition set by the zoom condition setting means. The video distribution device according to any one of the above.