JP2013090102A - Distribution system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique facilitating distribution of video data.SOLUTION: A distribution system performs: discriminating individual images constituting image data; generating sequence information in accordance with the sequence of individual images; comparing each image constituting the video data with another image; defining one image in which at least a part matches as a referred image and the other as a reference image; receiving input of storing-destination information indicating the storing-destination in storing an image without a match and a referred image as a result of the comparison in the storage parts of plural distribution servers; generating a meta file containing the sequence information, the storage destination information, and display instruction information that indicates a correlation between the reference image and a referred image of which at least a part matches the reference image; and performing distribution in accordance with an image file obtainment request based on the meta file.

Description

  This case relates to a technique for distributing moving image data.

  With the development of digital technology, a technology for creating a moving image with digital data, storing it in a server, and distributing it via a network has become widespread.

  Digital video data (hereinafter also simply referred to as video data) generally has a large amount of data, and when it is distributed over a network, the data must be compressed using an appropriate compression technique so as not to waste network bandwidth. Is required.

  In addition, when the video data to be played back is selected and received from the server to the terminal, and the received video data is played back on the terminal, it takes time until the video data is received, so the user requests the video data. I have to wait a long time before I can play it.

  Therefore, a technique for reducing the waiting time of the user by simultaneously reproducing the moving image data while receiving it from the server, so-called streaming, is also widely used.

JP 2009-290552 A JP 2003-102006 A JP 2000-090249 A

  Conventionally, when a streaming server connects to a user terminal and starts streaming, the connection with the user terminal must be maintained until the streaming ends, so even if the streaming request is congested, the connection during streaming Cannot be switched dynamically and the load cannot be adjusted.

  In addition, conventional streaming playback distributes video data in order from the beginning, so it takes a very long time when the user terminal wants to perform playback from the middle of the video, or fast forward / rewind. Alternatively, there is a problem that only operations on buffered data can be performed.

Further, when streaming is performed, a streaming server corresponding to a streaming protocol such as RTSP (Real Time Streaming Protocol) or RTMP (Real Time Messaging Protocol) is required, and the hurdle for introducing the system is high. In addition, when streaming is performed in an area where the network environment is insufficient, the reception of moving image data may not be in time for reproduction and may not be reproduced smoothly.

  Therefore, the disclosed system aims to provide a technique that facilitates the distribution of moving image data.

In order to solve the above problems, the distribution system of the present invention provides:
It has multiple distribution servers and meta servers,
A metafile providing unit for transmitting to the user terminal a metafile including storage location information indicating a storage location of each image constituting the video data in response to a video data distribution request received from the user terminal; Prepared,
The plurality of servers are
A storage unit for storing the image;
A distribution unit that reads an image corresponding to the acquisition request from the storage unit and distributes the image to the user terminal for each image when the image acquisition request based on the metafile is received from the user terminal. .

The distribution system includes
A request transmission unit that transmits a distribution request for moving image data to the meta server;
A metafile receiver that receives a metafile from the metaserver;
Based on the received storage location information of the metafile, an acquisition request unit that transmits an acquisition request for each image constituting the moving image data to the distribution server that is the storage location thereof,
You may further provide the user terminal which has the reproducing part which reproduces | regenerates the said image received by the said acquisition request.

The distribution system includes:
When each image constituting the moving image data is compared with other images, and when one of the images at least partially matched is a referenced image and the other is a reference image,
The storage location information of the metafile is information indicating the storage location of the unmatched image and the referenced image,
The metafile includes display instruction information indicating a correspondence relationship between the reference image and a referenced image at least partially matching the reference image, and the mismatched image, the referenced image, and the reference image. Further comprising order information indicating the playback order of
The acquisition request unit of the user terminal receives the image and the referenced image that did not match based on the storage location information from the distribution server,
When the playback unit of the user terminal displays the image and the reference image that do not match based on the order information and displays the reference image based on the order information, the playback unit displays the reference image based on the display instruction information. Instead of the reference image, the referenced image corresponding to the reference image may be displayed.

The distribution system includes:
When the referenced image and the reference image overlap in some areas, the data of the non-overlapping area of the reference image is added to the referenced image, and the original referenced image and the reference image are A reference image indicating a region corresponding to the reference image in the combined reference image, and a reference image indicating a region corresponding to the original reference image in the combined reference image; The display instruction information includes image information,
When the reproduction unit of the user terminal displays the referenced image, the original referenced image is extracted from the referenced image based on the original image information and displayed, and the reference image is displayed. The reference image may be extracted from the referenced image based on the reference image information and displayed.

The distribution system includes
The display instruction information includes display effect information related to the reference image,
When the playback unit of the user terminal displays the referenced image instead of the reference image, the playback unit may display the referenced image by performing image processing specified by the display effect information.

In addition, the distribution method of the present invention includes:
A distribution method using a distribution system including a plurality of distribution servers and a meta server,
The meta server executes a step of transmitting to the user terminal a meta file including storage destination information indicating a storage destination of each image constituting the moving image data in response to the distribution request of the moving image data received from the user terminal.
The plurality of distribution servers including a storage unit for storing the image,
When the image acquisition request based on the metafile is received from the user terminal, an image corresponding to the acquisition request is read from the storage unit and delivered to the user terminal for each image.

In addition, the metafile generation device of the present invention includes:
An order generator that identifies each image constituting the video data and generates order information according to the order of the images;
Each image constituting the moving image data is compared with other images, and at least a part of the matching image is a reference image, and the comparison unit uses the other as a reference image;
An input receiving unit that receives an input of storage destination information indicating a storage destination when an image that is not matched by the result of the comparison and a reference image is stored in a storage unit of a plurality of distribution servers;
A metafile generating unit that generates a metafile including display instruction information indicating a correspondence relationship between the reference image and a referenced image at least partially matching the reference image, the order information, and the storage destination information; Is provided.

The metafile generation method of the present invention includes:
Identifying each image constituting the video data and generating order information according to the order of each image;
Each image constituting the moving image data is compared with other images, and at least one of the matched images is a referenced image and the other is a reference image;
Receiving an input of storage location information indicating a storage location when an image that is not matched by the result of the comparison and a reference image is stored in a storage unit of a plurality of distribution servers;
Generating a metafile including display instruction information indicating a correspondence relationship between the reference image and a referenced image at least partially matching the reference image, the order information, and the storage destination information;
Is executed by the computer.

  The disclosed system can provide a technique that enables easy distribution of moving image data.

Schematic diagram of distribution system according to the present invention Schematic configuration diagram of a metafile generation device according to Embodiment 1 Functional block diagram of the metafile generation device according to the first embodiment Schematic configuration diagram of a grid server in the first embodiment Functional block diagram of the grid server according to the first embodiment The figure which shows the example of the web page provided by the web page provision part Schematic configuration diagram of a user terminal in the first embodiment Functional block diagram of a user terminal according to the first embodiment Diagram showing overall flow of image file and metafile generation method The figure which shows the flow of the process which calculates | requires the metadata of each image Explanatory drawing of the process which determines the change of a scene Explanatory drawing of the process which determines the change of a scene Explanatory diagram of processing to classify images Explanatory drawing of the process which determines the same image Diagram showing the flow of processing to synthesize partially matched images Explanatory drawing of processing to synthesize partially matched images Diagram showing the flow of image composition processing Illustration of video delivery method Diagram showing an example of a metafile The figure which shows an example of the user interface at the time of reproduction | regeneration Diagram showing metafile structure Diagram showing the flow of video playback processing using dedicated software

  Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. The configuration of the following embodiment is an exemplification, and the present invention is not limited to the configuration of the embodiment.

(1) System Configuration FIG. 1 is a configuration diagram of a distribution system according to the present invention. The distribution system 10 of this example includes a web server 5 that provides a web page, a metafile generation device 1 that creates meta information of moving image data (content), and a grid server (meta server) that provides meta information to the user terminal 3. 4, a plurality of distribution servers 2 that distribute moving image data, and a user terminal 3 that acquires moving image data from the distribution server 2 and reproduces it in real time.

  When the user terminal 3 requests the grid server 4 to distribute moving image data, the grid server 4 transmits a metafile to the user terminal 3. The user terminal 3 receives each image constituting the moving image data from the distribution server 2 based on the metafile and sequentially reproduces the images.

  In the present embodiment, each image constituting the moving image data can be acquired for each image as one image file, and the moving image data is stored in a plurality of distribution servers. For this reason, even when the moving image data is being reproduced, if the communication speed with the distribution server 2 cannot be sufficiently secured due to the congestion of the distribution request or the like, it is possible to switch to another distribution server and acquire subsequent images. it can.

(2) Hardware Configuration of Metafile Generation Device 1 FIG. 2 is a diagram showing a hardware configuration of the metafile generation device 1. The metafile generation device 1 includes a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, a host bus 104, a bridge 105, an external bus 106, and an interface 107. And an input device 108, an output device 110, a storage device (HDD) 111, and a communication device 115.

  The CPU 101 functions as an arithmetic processing device and a control device, and controls the overall operation in the metafile generation device 1 according to various programs. Further, the CPU 101 may be a microprocessor. The ROM 102 stores programs and calculation parameters used by the CPU 101. The RAM 103 is a so-called main memory that primarily stores programs used in the execution of the CPU 101, parameters that change as appropriate during the execution, and the like. These are connected to each other by a host bus 104 including a CPU bus.

The host bus 104 is connected to an external bus 106 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 105. Note that the host bus 104, the bridge 105, and the external bus 106 are not necessarily configured separately, and these functions may be mounted on one bus.

  The input device 108 is, for example, an input means for a user to input information, such as a mouse, keyboard, touch panel, button, microphone, switch, and lever, and an input that generates an input signal based on the input by the user and outputs it to the CPU 101 It consists of a control circuit. The operator of the metafile generation device 1 can input various data and instruct processing operations to the metafile generation device 1 by operating the input device 108. The input device 108 may be a storage medium reading device such as a CD-ROM drive or a memory card reader. The storage medium reading device reads information recorded on a removable storage medium such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and inputs the information to the CPU 101.

  The output device 110 includes, for example, a display device such as a CRT (Cathode Ray Tube) display device, a liquid crystal display (LCD) device and a lamp, and an audio output device such as a speaker and headphones. The output device 110 outputs the processing result of the CPU 101, for example. Specifically, various information such as input data and the operating status of the system is displayed as text or images. On the other hand, the audio output device outputs messages, warning sounds, audio data of contents, and the like as sounds.

  The output device 110 may be a storage medium writing device such as a CD-R drive or a memory card writer. The storage medium writing device outputs (writes) information such as the processing result of the CPU 101 to a removable storage medium such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

  Note that the input device 108 and the output device 110 may be provided integrally, such as a storage medium read / write device (such as a memory card reader / writer) or a display having a touch panel.

The storage device 111 is a data storage device configured as an example of a storage unit of the metafile generation device 1 according to the present embodiment. The storage device 111 is a storage medium, a recording device that records data on the storage medium, and data from the storage medium. A reading device that reads data and a deleting device that deletes data recorded in the storage medium can be included. The storage device 111 is configured with, for example, an HDD (Hard Disk Drive). The storage device 111 drives a hard disk and stores programs executed by the CPU 101 and various data. For example, the storage apparatus 111 stores input storage location information, a created metafile, and the like.

  The communication device 115 is a communication interface configured with, for example, a communication device for connecting to the communication network 12. The communication device 115 may be a wireless communication device such as a wireless local area network (LAN) or a wireless USB, or a wire communication device that performs wired communication.

(3) Function of Metafile Generation Device 1 FIG. 3 is a functional block diagram of the metafile generation device 1 according to the present embodiment. The metafile generation device 1 includes an order generation unit 131, a comparison unit 132, an input reception unit 133, and a metafile generation unit 134. The functions of the order generation unit 131, the comparison unit 132, the input reception unit 133, and the metafile generation unit 134 are mainly executed by the CPU 101 according to a program to control each unit such as the communication device 115 and the storage device 111. Realized.

  The order generation unit 131 identifies each image constituting the moving image data, and generates order information according to the order of each image.

  The comparison unit 132 compares each image constituting the moving image data with another image, and sets at least one of the matched images as a reference image and the other as a reference image.

  The input receiving unit 133 receives storage destination information indicating storage destinations when images that do not match as a result of the comparison and the referenced images are stored in storage units of a plurality of distribution servers.

  The metafile generation unit 134 generates a metafile including display instruction information indicating a correspondence relationship between the reference image and a referenced image at least partially matching the reference image, the order information, and the storage destination information. To do.

(4) Hardware Configuration of Grid Server FIG. 4 is a diagram illustrating a hardware configuration of the grid server 4. The grid server 4 includes a CPU (Central Processing Unit) 401, a ROM (Read Only Memory) 402, a RAM (Random Access Memory) 403, a host bus 404, a bridge 405, an external bus 406, an interface 407, The computer includes an input device 408, an output device 410, a storage device (HDD) 411, and a communication device 415.

  The CPU 401 functions as an arithmetic processing device and a control device, and controls the overall operation in the grid server 4 according to various programs. Further, the CPU 401 may be a microprocessor. The ROM 402 stores programs used by the CPU 401, calculation parameters, and the like. The RAM 403 is a so-called main memory that primarily stores programs used in the execution of the CPU 401, parameters that change as appropriate during the execution, and the like. These are connected to each other by a host bus 404 including a CPU bus.

The host bus 404 is connected to an external bus 406 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 405. Note that the host bus 404, the bridge 405, and the external bus 406 are not necessarily configured separately, and these functions may be mounted on one bus.

  The input device 408 generates an input signal based on an input by the user, such as a mouse, a keyboard, a touch panel, a button, a microphone, a switch, and a lever, and inputs input by the user and outputs the input signal to the CPU 401 It consists of a control circuit. The operator of the grid server 4 can input various data and instruct processing operations to the grid server 4 by operating the input device 408. The input device 408 may be a storage medium reading device such as a CD-ROM drive or a memory card reader. The storage medium reading device reads information recorded on a removable storage medium such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and inputs the information to the CPU 401.

  The output device 410 includes a display device such as a CRT (Cathode Ray Tube) display device, a liquid crystal display (LCD) device and a lamp, and an audio output device such as a speaker and headphones. The output device 410 outputs the processing result of the CPU 401, for example. Specifically, various information such as input data and the operating status of the system is displayed as text or images. On the other hand, the audio output device outputs messages, warning sounds, audio data of contents, and the like as sounds.

  The output device 410 may be a storage medium writing device such as a CD-R drive or a memory card writer. The storage medium writing device outputs (writes) information such as the processing result of the CPU 401 to a removable storage medium such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

The storage device 411 is a data storage device configured as an example of a storage unit of the grid server 4 according to the present embodiment, and is a storage medium, a recording device that records data in the storage medium, and a read that reads data from the storage medium A deletion device for deleting data recorded on the device and the storage medium can be included. The storage device 411 is configured by an HDD (Hard Disk Drive), for example. The storage device 411 drives a hard disk, and C
Stores programs executed by the PU 401 and various data. For example, the storage device 411 stores a metafile received from the metafile generating device 1 and each image constituting the moving image data.

  The communication device 415 is a communication interface configured with, for example, a communication device for connecting to the communication network 12. The communication device 415 may be a wireless communication device such as a wireless local area network (LAN) or a wireless USB, or a wire communication device that performs wired communication.

(5) Function of Grid Server 4 FIG. 5 is a functional block diagram of the grid server 4 according to the present embodiment. The grid server 4 includes a metafile providing unit 432 and a load control unit 434. The functions of the request receiving unit 431, the metafile providing unit 432, and the load control unit 434 are realized mainly by the CPU 401 executing processing according to a program to control each unit such as the communication device 415 and the storage device 411. .

  In response to the moving image data distribution request received from the user terminal, the meta file providing unit 432 transmits a meta file including storage location information indicating the storage location of each image constituting the moving image data to the user terminal 3.

  The load control unit 434 obtains the distribution status based on the number of moving image data being distributed, the transfer speed of the moving image data, the number of retransmission requests, and the like, and notifies the user terminal 3 of the distribution status so that the user with poor distribution status The terminal 3 is urged to switch the grid server 4 or the user terminal 3 having a poor distribution status is notified of another storage location extracted from the storage location information, and the other grid server 4 is switched based on the storage location. Prompt.

  In the present embodiment, the metafile providing unit 432 and the load control unit 434 are provided in the grid server 4. However, the present invention is not limited to this, and if each functional unit exists in the distribution system 10, another device is provided. The structure provided for may be sufficient. For example, the distribution server 2 or the metafile generation device 1 may include the metafile providing unit 432 and provide a metafile to the user terminal 3.

(6) Distribution server 2
Further, the distribution server 2 is similar to the grid server 4 in that the CPU (Central Processing Unit)
), ROM (Read Only Memory), RAM (Random Access Memory), host bus, bridge, external bus, interface, input device, output device, storage device (HDD), and communication device It is a computer provided with. Since these hardware configurations are substantially the same as those of the grid server 4, the illustration is omitted. According to the program of the distribution server 2, for example, functions as a distribution unit, and when an image acquisition request based on a metafile is received from the user terminal 3, an image corresponding to the acquisition request is read from the storage unit for each image. It also functions as a distribution unit that distributes to the user terminal.

(7) Web server 5
As with the grid server 4, the Web server 5 has a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a host bus, a bridge, an external bus, an interface, and an input. A computer includes a device, an output device, a storage device (HDD), and a communication device. Since these hardware configurations are substantially the same as those of the grid server 4, the illustration is omitted. The Web server 5 has a function of providing a Web page according to a program. For example, when a browsing request for the Web page specified by the URL is received from the user terminal 3, the Web server 5 reads the Web page from the storage device 111 and transmits it to the user terminal 3.
FIG. 6 is a diagram illustrating an example of a web page provided by the web page providing unit 431. The Web page of FIG. 6 is described by a markup language such as HTML, and includes hyperlinks 91-94 to moving image data.

(8) Hardware Configuration of User Terminal FIG. 7 is a schematic configuration diagram of the user terminal 3 in the present embodiment. The user terminal 3 includes a CPU (Central Processing Unit) 301, a ROM (Read Only Memory) 302, an RA
M (Random Access Memory) 303, host bus 304, bridge 305, external bus 306, interface 307, input device 308, output device 310, storage device (HDD) 311, and communication device 315 Computer. The user terminal 3 is, for example, a personal computer, a tablet computer, a smartphone, a mobile phone, semiconductor audio, a television, a TV tuner, an STB (Set Top Box), a digital signage, or the like.

  The CPU 301 functions as an arithmetic processing device and a control device, and controls the overall operation in the user terminal 3 according to various programs. Further, the CPU 301 may be a microprocessor. The ROM 302 stores programs used by the CPU 301, calculation parameters, and the like. The RAM 303 is a so-called main memory that primarily stores programs used in the execution of the CPU 301, parameters that change as appropriate during the execution, and the like. These are connected to each other by a host bus 304 including a CPU bus.

The host bus 304 is connected to an external bus 306 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 305. The host bus 304, the bridge 305, and the external bus 306 are not necessarily configured separately, and these functions may be mounted on one bus.

The input device 308 generates input signals based on the input by the user and input means for the user to input information such as a mouse, keyboard, touch panel, button, microphone, switch, remote controller and lever, for example. And an input control circuit for outputting to the CPU 301. The operator of the user terminal 3 can input various data and instruct processing operations to the user terminal 3 by operating the input device 308. The input device 308 may be a storage medium reading device such as a CD-ROM drive or a memory card reader. A storage medium reading device reads information recorded on a removable storage medium such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and inputs the information to the CPU 301.

  The output device 310 includes a display device such as a CRT (Cathode Ray Tube) display device, a liquid crystal display (LCD) device and a lamp, and an audio output device such as a speaker and headphones. The output device 310 outputs the processing result of the CPU 301, for example. Specifically, various information such as input data and the operating status of the system is displayed as text or images. On the other hand, the audio output device outputs messages, warning sounds, audio data of contents, and the like as sounds.

  The output device 310 may be a storage medium writing device such as a CD-R drive or a memory card writer. The storage medium writing device outputs (writes) information such as the processing result of the CPU 301 to a removable storage medium such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

The storage device 311 is a data storage device configured as an example of a storage unit of the user terminal 3 according to the present embodiment. The storage device 311 is a storage medium, a recording device that records data on the storage medium, and a read that reads data from the storage medium. A deletion device for deleting data recorded on the device and the storage medium can be included. The storage device 311 is composed of, for example, an HDD (Hard Disk Drive). The storage device 311 drives a hard disk and has a CPU.
A program executed by the server 301 and various data are stored.

  The communication device 315 is a communication interface configured with, for example, a communication device for connecting to the communication network 12. The communication device 315 may be a wireless communication device such as a wireless local area network (LAN) or a wireless USB, or a wire communication device that performs wired communication.

  Note that these hardware configurations are not limited to this, and can be appropriately changed as long as the function of the user terminal 3 such as playing a moving image based on a metafile can be realized. For example, if the digital signage is a device that does not require audio output, the speaker may be omitted. Further, a user terminal function may be realized in combination with other devices such as a television (monitor) and an STB. That is, it is not necessary to have all the elements in one apparatus.

(9) Function of User Terminal 3 FIG. 8 is a functional block diagram of the user terminal 3 according to the present embodiment. The user terminal 3 includes a request transmission unit 331, a metafile reception unit 333, an acquisition request unit 334, and a moving image reproduction unit 335. The functions of the request transmission unit 331, the metafile reception unit 333, the acquisition request unit 334, and the video playback unit 335 are mainly executed by the CPU 301 according to a program to control each unit such as the communication device 315 and the storage device 311. Realized.

  The request transmission unit 331 transmits a web page browsing request and a moving image data distribution request to the distribution server 2. For example, when the hyperlink 91-94 in FIG. 6 is selected, a video data distribution request is made by transmitting the URL of the video data that is the link destination to the distribution server.

  The metafile receiving unit 333 receives the metafile from the distribution server 2 and stores it in the RAM 303 or the storage device 311.

  Based on the received storage location information of the metafile, the acquisition request unit 334 transmits an acquisition request for each image constituting the moving image data to the distribution server 2 that is the storage location. In addition, the acquisition request unit 334 receives the distribution status from the load control unit 434 of the grid server 4, and if the distribution status of the distribution server 2 making the distribution request is bad, the distribution request destination is sent to the other distribution server 2. Switch. Further, the acquisition request unit 334 may determine the distribution status based on the transfer speed of the image file, the number of retransmission requests, the error rate, and the like, and switch the distribution server 2 that requests the distribution based on the distribution status. A plurality of distribution servers 2 that request distribution are recorded in a metafile described later, and are selected from the plurality of distribution servers 2.

  The moving image reproduction unit 335 sequentially reproduces the received images according to the metafile (display instruction information).

The program for executing the functions of the request transmission unit 331, the metafile reception unit 333, the acquisition request unit 334, and the moving image reproduction unit 335 may be a so-called web browser.

(10) Method for Generating Image File and Metafile A method for generating an image file and a metafile by the metafile generating apparatus 1 will be described with reference to FIGS. 9 is a diagram showing the overall flow of the image file and metafile generation method, FIG. 10 is a diagram showing the flow of processing for obtaining metadata of each image, and FIGS. 11 and 12 are diagrams for explaining processing for determining scene switching. FIG. 13 is an explanatory diagram of processing for classifying images, FIG. 14 is an explanatory diagram of processing for determining the same image, FIG. 15 is a diagram showing a flow of processing for synthesizing partially matched images, and FIG. 16 is partially matched. FIG. 17 is a diagram illustrating the structure of a metafile.

  When the metafile generation device 1 receives the information for specifying the moving image data to be processed, the storage location information of the generated image file, and the metafile generation instruction, as shown in FIG. 9, the metafile generation processing is performed. Start. First, the metafile generation device 1 reads the designated moving image data to be processed from the storage device 111 or a storage medium reading device (input device 108), and reads each image on the RAM 103 (step S10). If the moving image data is compressed, it is decoded and each image is used as bitmap data.

  Next, the metafile generating device 1 sets each file to one file (image file) for each image, and temporarily stores the file in the storage device 111 or the RAM 103 with a file name (step S20).

  Then, the metafile generation device 1 repeats the process of comparing each image with another image to obtain metadata for each image (step S40) for all images (steps S30 to S50).

In the process of comparing each image to obtain metadata (step S40), as shown in FIG. 10, the comparison unit 132 first compares each image with the previous image (step S410), and the degree of coincidence is high. It is determined whether or not (S420). For this comparison and determination, for example, the feature points of the image and the immediately preceding (previous) image are respectively extracted, the feature points of the image and the immediately preceding image feature points are correlated, and the corresponding feature points are obtained. The ratio between the number of feature points extracted in the image and the number of corresponding feature points is calculated as the degree of coincidence.

  FIG. 11 shows a case where the image 82 is compared with the immediately preceding image 81. A point 82 </ b> A is displayed as a point for convenience at the location extracted as the feature point of the image 82. Similarly, 81A is a point where points are displayed for convenience in the points extracted as feature points of the image 81. Information around the feature point in the image 82A is used as a template, a matching part is searched for by template matching on the image 81A, and the feature point of the matching part is set as a corresponding feature point. Thus, the ratio of the feature points that do not correspond to the corresponding feature points is calculated, and if the ratio of the corresponding feature points, that is, the degree of coincidence is equal to or higher than a threshold, for example, 60% or higher, the scenes are determined to be the same. Note that even within the same scene, the characters move and the background moves, and a certain change in the image is assumed. Therefore, the threshold for determining the same scene is set low. Also, the higher the frame rate, the smaller the difference between images, so this threshold and others may be set higher. Also, the lower the image resolution, the smaller the difference between the images, so this threshold value may be set higher.

  As shown in FIG. 11, when similar images are compared, feature points are obtained in substantially the same manner, and since there are many corresponding feature points, it is determined that the degree of coincidence is high.

FIG. 12 shows a case where the image 84 is compared with the immediately preceding image 83. A point 84A is displayed as a point for convenience at the location extracted as the feature point of the image 84. Similarly, 83A is a point where points are displayed for convenience in the extracted points of the image 83. Information on pixels around the feature point in the image 84A is used as a template, a matching portion is searched for by template matching on the image 83A, and the feature point of the matching portion is set as a corresponding feature point. Thus, the ratio of the feature points that do not correspond to the corresponding feature points is calculated, and if the ratio of the corresponding feature points, that is, the degree of coincidence is equal to or higher than a threshold, for example, 60% or more, the scene is determined to be the same.

  As shown in FIG. 12, when dissimilar images are compared, naturally the extracted feature points are not similar, and the corresponding feature points are few, so the degree of coincidence is determined to be less than the threshold value.

  If it is determined in step S420 that the degree of coincidence is high (S420, Yes), the image is recognized as the same scene as the immediately preceding image, and the comparison unit 132 has the same scene ID as the scene ID of the immediately preceding image. Images are listed from the metafile (step S430), sequentially read onto the RAM 103, and the images are compared with other images in the same scene (step S440). Based on this comparison, it is determined whether or not the image is a gradually shifting image (referred to as a scroll image in the present embodiment) as if the image was taken by panning or tilting the camera (step S450). . Note that the scroll image is not limited to an image photographed by a camera, and may be an animation or computer graphics created so as to shift in one direction, like pan and tilt.

FIG. 13 shows an example in which the image 86 is compared with another image 85. The feature points of these images 85 and 86 are extracted, divided into a plurality of areas and compared as in the images 85A and 86A, the movement of the feature points is detected, and the movement in the same direction is detected in all of the plurality of areas. If it is determined, the image 86 is determined as a scroll image (step S450, Yes). Then, the difference from the previous image is combined with the previous image as a referenced image (step S470), and the combined referenced image is registered in a predetermined storage area.
Here, the predetermined storage area may be a data area of the storage device 111 or a storage area on the distribution server 2 designated as a storage destination. Moreover, the file name at the time of storage is good also as what combined the information (work ID) which specifies moving image data, and the information (image ID) which specifies an image, for example. Then, the metafile generation unit 134 includes information for identifying the scroll image (reference image), information for identifying the reference image after synthesis, and the scroll image among the reference images after synthesis. The area information indicating the corresponding area is associated and recorded as metadata in the metafile (step S490). Thus, even if the delivery of the scroll image 86 is omitted, the moving image reproduction unit 335 of the user terminal 3 reads the corresponding area from the combined reference image based on the metafile, and displays it in place of the scroll image 86 to display the moving image. Can be played and the amount of data to be distributed can be reduced.
On the other hand, if it is not a scroll image in step S450 (No in step S450), the image is compared with other images in the same scene to determine whether there is the same image (step S460).

  FIG. 14 shows a specific example in the case of determining that the images are the same. In FIG. 14, images 71 to 75 are each image created from the moving image data in step S <b> 20, and processing for comparing the image 73 with other images will be described.

When comparing the image 73 with the immediately preceding image 72 in the previous step S420, the comparing unit 132 determines that the image as a whole is similar and the degree of coincidence is higher than the threshold, and the image 73 is different from the immediately preceding image 73. It is determined that they are the same scene, and other images (images 71 and 72 in this example) that are similarly determined to be the same scene are extracted and compared to determine whether they are the same (step S450). In FIG. 14, images obtained by extracting feature points of the image 73 and the other images 71 and 72 are 73A and 71A and 72A. Here, since the images 71-75 are scenes in which the characters are talking, comparing the image 73A and the image 72A with a plurality of regions, the region 73B including the mouth and the region 72B are different and are not the same image. Is determined. On the other hand, when the image 73A and the image 71A are compared, they are also images at the moment when the mouth is closed. Therefore, even if a plurality of areas are compared, the degree of coincidence between the areas 73B and 71B including the mouth and all other areas. Are high and are determined to be the same image. In FIG. 14, for the sake of convenience, only the difference between the areas 71B to 73B is shown. However, since it is not known in which area the difference actually appears, the entire image is compared.

  As described above, if the image 86 is the same as the other image 85, the moving image can be reproduced by redisplaying the other image 85 instead of displaying the image 86. Therefore, in this embodiment, when the comparison unit 132 compares in step S450 and classifies the image 86 as the same reference image as the image 85, the metafile generation unit 134 includes the metadata indicating the reference image 86, Metadata indicating the reference image 85 is recorded in the metafile (step S490). As a result, even if the distribution of the same reference image 86 is omitted, the moving image reproduction unit 335 of the user terminal 3 refers to the referenced image 85 based on the metafile and reproduces the moving image by displaying it instead of the reference image 86. And the amount of data to be distributed can be reduced.

  Further, as a result of the comparison by the comparison unit 132 in step S460, if the same image is not in the same scene, this image (general image) is registered as a distribution target image file in a predetermined storage area (step S480), The metafile generation unit 134 records metadata (image ID, storage destination, etc.) for identifying this image in the metafile (step S490).

  Similarly, when the comparison unit 132 recognizes in step S420 that an image having a low degree of coincidence with the immediately preceding image is the first image of the new scene as a result of comparison (step S425), the image is registered in a predetermined storage area. In step S480, the metafile generation unit 134 records the metadata of the image in the metafile (step S490). As the metadata of the image, for example, the scene ID of the new scene recognized in step S425 may be registered in the metafile in association with the information of the image. The scene ID is identification information for specifying a scene, and for example, serial numbers are given in the order of recognition.

  Note that, when the steps S30 to S50 in FIG. 9 are repeated for the subsequent image when the general image matches the subsequent image, it is set as a referenced image and the metafile is updated.

  The metafile generation device 1 transmits the metafile created as described above to the grid server 4.

(11) Details of Processing for Combining Reference Images of Scroll Images The composition processing in step S470 will be described in detail with reference to FIGS. In FIG. 15, images 61 to 64 are images created from the moving image data in step S <b> 20, and a process for first classifying the image 62 will be described.

When the comparison unit 132 compares the image 62 with the immediately preceding image 61 in step S420, the comparison unit 132 determines that the entire image is similar and the matching degree is higher than the threshold, and extracts the image 61 as another image of the same scene. Are compared (steps S430 and S440). 62A and 61A are images obtained by quantifying the image 62 and the other image 61 and extracting feature points. Here, since the image 61-64 is a scene taken by tilting from the top of the mountain toward the top of the mountain, the plurality of regions A1-A9 of the image 62A match each other in the region A1-A3 in the mountain top direction when compared with the image 61A. The degree of coincidence is low, and the degree of coincidence is high in the other regions A4-A9. In addition, the feature points in the region A4-A9 are moved in one direction. For example, the feature point P11 of the image 62A corresponding to the feature point P11 in the region A1 of the image 61A has moved in the direction of the arrow 60. Further, the feature points P12 to P16 of the area A5-A9 are also moved (scrolled) in the same direction as the arrow 60.

  That is, the image 62 is an image scrolled in the direction of the arrow 60 as compared with the image 61, and a region 62 </ b> C newly entered in the image 62 by scrolling is different from the image 61, and the other region 62 </ b> D matches the image 61. To do. Therefore, the difference image 62C is added to the image 61 to synthesize the reference image 61B, and region information for specifying the region 62E that matches the reference image 62 in the combined reference image 61B is generated.

  Similarly, for the subsequent scroll images, the image quantification, comparison, and synthesis processes are repeated, and as shown in FIG. 16, the difference area 63C of the image 63 from the image 62 is added to the referenced image 61B, and the referenced image 61C. Are combined, and region information for specifying a region 63E that matches the reference image 63 in the combined reference image 61C is generated.

  Further, a difference area 64C between the image 64 and the image 63 is added to the referenced image 61C to synthesize the referenced image 61D, and area information for specifying the area 64E that matches the reference image 64 in the combined referenced image 61D. Is generated.

  A detailed flow of the process of combining these referenced images, that is, the process of step S470 shown in FIG. 10 will be described with reference to FIG. If the image 92 is determined to be a scroll image in step S450, the comparison unit 132 extracts an image with a high degree of coincidence, usually the immediately preceding image 61 from a predetermined storage area (step S510). Then, the area near the center of the scroll image 62 and the previous image 61 is compared (step S520), and the movement amount and direction are obtained (step S530).

  Based on the movement amount and direction, the image composition unit 135 cuts out the difference area in the scroll image (step S540), adds the difference area to the immediately preceding image 61 (step S550), and composes the referenced image 61B ( Step S560). In step S560, the metafile generation unit 134 generates area information indicating an area corresponding to the scroll image 62 in the combined reference image 61.

  In addition, the metafile generation unit 134 deletes the reference image 61 before composition from the distribution data storage location (step S570).

  After the completion of step S570, the process returns to the process of step S480 shown in FIG. 10, and the reference image after synthesis is registered in a predetermined storage area.

(12) Moving Image Distribution Method and Playback Method FIG. 18 is an explanatory diagram of a moving image distribution method. When the user selects a link in the Web page as shown in FIG. 6, the user terminal 3 accesses the grid server 4 according to the link destination information (URL, etc.), and sends the URL and parameters included in the URL to the distribution server. 2 is notified (step S100).

  The grid server 4 that has received the distribution request from the user terminal 3 checks the load status of each distribution server 2 (step S103), and performs distribution to the user terminal 3 so that the load is evenly distributed, for example. The server 2 is determined (step S106). Then, the grid server 4 adds information indicating the distribution server to the metafile of the moving image corresponding to the identification information such as the URL and the parameter notified from the user terminal 3 as the distribution request, and sends it to the requesting user terminal 3. Transmit (step S110).

The user terminal that has received the metafile requests the metafile to acquire each image from the storage server 2 based on the image list (distribution list) (step S120). Here, the distribution server 2 that requests acquisition of an image is not limited to the server that requested the metafile, and may be any of a plurality of distribution servers 2 indicated as storage locations in the metafile. The distribution server 2 may request acquisition from a plurality of distribution servers 2 in parallel and receive image files from each of the distribution servers 2. The distribution server 2 that has received the acquisition request transmits the corresponding image file to the requesting user terminal 3 ( Step S130).

The user terminal 3 that has received the image file stores the image file in a storage unit serving as a buffer (step S140). Here, the storage unit for buffering the image file may be the storage device 311, but is preferably the RAM 303 in which the file does not remain after the distribution ends.
The user terminal 3 reproduces the moving image by displaying the received images sequentially according to the metafile (step S150).

  And the user terminal 3 repeats the process after step S120 until it receives all the images of a delivery list (step S160).

FIG. 19 is a diagram illustrating an example of a metafile. In this embodiment, the meta information is described in a markup language, and the moving image is reproduced by displaying the image according to the description by the software of the user terminal 3 (so-called browser).
In FIG. 19, fps = 60 in the description 51 is an attribute indicating the frame rate at the time of displaying an image, and indicates that 60 images are displayed per second.
Further, href = of the description 52 is information indicating that an image is acquired from the distribution server 2, and “http: //xxx.23.122.33/server1/1932108392089.png” of the description 54 following this is the storage destination. File name (identification information). The order of this description becomes order information indicating the reproduction order.
Also, “reload” scope = “scene” href = “/ 12903840298340929.html” in the description 53 is a description for redisplaying the referenced image.
As described above, by describing information necessary for reproduction in the markup language, the user terminal 3 can realize the reproduction of the moving image according to the present embodiment with a general browser.

Further, the present invention is not limited thereto, and dedicated software (player) that determines and executes redisplay, scroll display, and the like on the user terminal side based on the meta information may be used.
FIG. 22 is an explanatory diagram of a moving image playback method when a dedicated player is used. When the number of images buffered in step S140 in FIG. 18 exceeds a predetermined value, the moving image playback unit 335 of the user terminal 3 starts playback processing of each image according to the playback order of the metafile (step S610).

  The moving image reproduction unit 335 first determines whether or not the type of image to be reproduced is the same image (step S620). If the image is the same image, the referenced image recorded as the referenced image of the same image based on the metafile. Is read from the buffer and displayed on the display device (step S630).

  If the image types are not the same in step S620, it is determined whether the images are shifted images (step S640). Here, if it is a shift image, the reference image recorded as the reference image of the shift image is read from the buffer based on the metafile, and an area corresponding to the shift image is cut out based on the area information (step S650). ), The cut-out area is displayed instead of the shift image (step S660).

If the image type is not a shift image in step S640 (No in step S640), the image type is a general image or a referenced image, and the image is read from the buffer and displayed (step S670).

  Then, the moving image reproduction unit 335 determines whether there is an image to be reproduced next based on the order information of the metafile. If there is a next image, the process returns to step S610 to repeat the reproduction process, and the next image is determined. If not, the process in FIG. 19 ends.

  In this way, the image is distributed from the distribution server 2 to the user terminal 3 by the processing of FIG. 18 and the image received by the user terminal 3 and stored in the buffer in parallel is reproduced according to the metafile as shown in FIG. By doing so, it is possible to reproduce the moving image data.

  FIG. 20 shows an example of a user interface during playback. In FIG. 20, each image constituting the moving image is displayed in the moving image display field 51. The operation unit 50 arranged below the moving image display column 51 is a group of buttons for performing operations such as reproduction, fast forward, and rewind of moving images.

  When the play / stop button 52 is selected during the reproduction of the moving image, the moving image reproduction unit 335 stops the reproduction of the moving image, and continues to display the image displayed at this time to be in a stationary state. When the play / stop button 52 is selected while the moving image is stopped, the moving image is started to be played. When the fast-forward button 53 is selected, the moving image playback unit 335 skips the order and selects and plays back images when selecting an image to be played back based on the order information in step S610. For example, every other image is reproduced when reproducing at double speed, and every third image is reproduced when reproducing at quadruple speed. In parallel with this, the acquisition request unit 334 requests images to be acquired out of order and stores them in the buffer.

  When the scene advance button 54 is selected, the moving image reproduction unit 335 selects and reproduces the first image of the next scene when selecting an image to be reproduced based on the order information in step S610. In parallel with this, the acquisition request unit 334 requests an image to be acquired by skipping a scene and stores it in a buffer.

  When the rewind button 55 is selected, the moving image reproducing unit 335 selects the images by returning the order when the images to be reproduced are selected based on the order information in step S610, and performs reproduction. For example, when rewinding at double speed, every other image is returned, and when reproducing at quadruple speed, every third image is returned. In addition, when the images whose order has been returned exist in the buffer, they are read from the buffer, and when they do not exist in the buffer, the acquisition request unit 334 requests acquisition of these images in parallel with the reproduction. Store it in the buffer.

  When the scene return button 56 is selected, the moving image reproducing unit 335 selects and reproduces the first image of the previous scene when selecting an image to be reproduced based on the order information in step S610. In parallel with this, the acquisition request unit 334 receives the first image of the previous scene and stores it in the buffer.

  In the present embodiment, since delivery is performed for each image, fast-forwarding and rewinding can be handled even during playback.

(13) Configuration of Metafile FIG. 21 is a diagram showing the configuration of the metafile. As shown in FIG. 21, in this embodiment, the work ID, scene ID, image ID (frame ID), file name, image type (frame type), server ID (storage location information), and number of frames used are included in the metafile. Information indicating display effects (display effect information) is recorded.

The work ID is identification information assigned to each moving image data, and the scene ID is identification information assigned to each scene.

  The image type is information indicating the type of the same image, a shift image, a referenced image, and a general image. Accordingly, among the images recorded in the metafile, those that are the image types of the referenced image and the general image are the distribution targets, and this information corresponds to the distribution list.

  The image ID is a serial number assigned in the order of each image in the moving image data, that is, the reproduction order, and is information for uniquely identifying the image in the moving image data and also order information.

  The server ID (storage destination information) is information indicating the distribution server 2 storing each image, and a plurality of servers are recorded. The server ID is an address (IP address or the like) for accessing via the network, identification information associated with this address, or the like.

(14) Image Display Control The moving image playback unit 335 of the user terminal 3 may display and display the referenced image based on the display effect information of the metafile.

  In a moving image, a predetermined video effect such as fade-in or fade-out may be used. For example, a fade-out in which an image gradually changes to white or black due to a scene change or the like may be used. In this case, it is possible to reproduce the image by changing the color tone of the referenced image so as to approach white or black. In the case of fade-in, display is performed while changing from a white or black image to a reference image.

Therefore, the comparison unit 132 of the metafile generation device 1 detects the presence or absence of a predetermined change such as a change in color tone or a change in focus between images when comparing the images in step S450 or step S460. For example, if a change in color tone in which the image gradually changes to white or black is detected in step S460, the metafile generating device 1 recognizes that the same image fades out in step S490, and displays the display effect information of the same image. As a fade-out (for example, fade-out to white) is recorded in the metafile. If a change in focus in which the image gradually blurs is detected in step S450, the metafile generation device 1 recognizes that the scroll image is blurred (focus change) in step S490, and the display effect information of the same image. Record the change in focus in the metafile.

Then, the moving image reproduction unit 335 of the user terminal 3 performs image processing on the referenced image based on the metafile and displays the image at step S630.
For example, display control is performed according to the following metafile description.
<source type = "image" href = "http: //xxx.23.122.33/server1/123456.png"
style = "focus: to-defocus 3.5px 30"/>
The Focus: indicates that the attribute is a focus change, and in the case of to-defocus, the referenced image (123456.png in the above example) is changed from a focused state to a blurred state. On the other hand, in the case of to-focus, the state is changed from a blurred state to a focused state.
The Focus attribute 3.5px indicates the amount of blur and is specified in units of pixels. The above example shows blurring in the range of 3.5 pixels.
Further, the Focus attribute 30 is a blurring time, which indicates the number of seconds or frames (images) from the start of the blurring process until the blurring amount is reached.

<source type = "image" href = "http: //xxx.23.122.33/server1/123456.png"
style = "fade: out white 100% 15 />
The fade: indicates a fade attribute, and if it is out, the referenced image (123456.png in the above example) is gradually changed to the designated color (white in the above example) and faded out. The color designation may be described in RGB such as #ffffff. On the other hand, in the case of in, the designated color is gradually changed to the reference image and faded in.
Further, 100% of the fade attribute indicates a fade amount and indicates how close the designated color is to the designated image.
Furthermore, 15 of the fade attribute is a fade time, and indicates the number of seconds or the number of frames (images) from the start of the fade process until the fade amount is reached.

  Note that this video effect is not limited to fade-in / fade-out, and any video effect can be used as long as it can display the referenced image by image processing such as solarization or monochromeization.

  As described above, according to the present embodiment, it is possible to reproduce a moving image by displaying a referenced image instead of a reference image, and to reduce the amount of data distributed when reproducing the moving image.

  In addition, the distribution server can be switched during the reproduction of the moving image, so that it is possible to flexibly cope with the load concentration.

<Others>
The present invention is not limited to the illustrated examples described above, and various modifications can be made without departing from the scope of the present invention.

DESCRIPTION OF SYMBOLS 1 Metafile production | generation apparatus 2 Distribution server 3 User terminal 10 Distribution system

Claims (8)

A distribution system including a plurality of distribution servers and a meta server,
A metafile providing unit for transmitting to the user terminal a metafile including storage location information indicating a storage location of each image constituting the video data in response to a video data distribution request received from the user terminal; Prepared,
The plurality of distribution servers are
A storage unit for storing the image;
A distribution unit that, when receiving an acquisition request for the image based on the metafile from the user terminal, reads an image corresponding to the acquisition request from the storage unit and distributes the image to the user terminal for each image;
Having a distribution system. A request transmission unit that transmits a distribution request for moving image data to the meta server;
A metafile receiver that receives a metafile from the metaserver;
Based on the received storage location information of the metafile, an acquisition request unit that transmits an acquisition request for each image constituting the moving image data to the distribution server that is the storage location thereof,
A distribution system further comprising: a user terminal having a reproduction unit that reproduces the image received in response to the acquisition request. When each image constituting the moving image data is compared with other images, and when one of the images at least partially matched is a referenced image and the other is a reference image,
The storage location information of the metafile is information indicating the storage location of the unmatched image and the referenced image,
The metafile includes display instruction information indicating a correspondence relationship between the reference image and a referenced image at least partially matching the reference image, and the mismatched image, the referenced image, and the reference image. Further comprising order information indicating the playback order of
The acquisition request unit of the user terminal receives the image and the referenced image that did not match based on the storage location information from the distribution server,
When the playback unit of the user terminal displays the image and the reference image that do not match based on the order information and displays the reference image based on the order information, the playback unit displays the reference image based on the display instruction information. The distribution system according to claim 1, wherein the reference image corresponding to the reference image is displayed instead of the reference image. When the referenced image and the reference image overlap in some areas, the data of the non-overlapping area of the reference image is added to the referenced image, and the original referenced image and the reference image are A reference image indicating a region corresponding to the reference image in the combined reference image, and a reference image indicating a region corresponding to the original reference image in the combined reference image; The display instruction information includes image information,
When the reproduction unit of the user terminal displays the referenced image, the original referenced image is extracted from the referenced image based on the original image information and displayed, and the reference image is displayed. The distribution system according to claim 3, wherein the reference image is extracted from the referenced image and displayed based on the reference image information. The display instruction information includes display effect information related to the reference image,
5. The display unit according to claim 3, wherein when the playback unit of the user terminal displays the referenced image instead of the reference image, the playback unit displays the referenced image by performing image processing specified by the display effect information. Delivery system. A distribution method using a distribution system including a plurality of distribution servers and a meta server,
The meta server executes a step of transmitting to the user terminal a meta file including storage destination information indicating a storage destination of each image constituting the moving image data in response to the distribution request of the moving image data received from the user terminal.
The plurality of distribution servers including a storage unit for storing the image,
A distribution method of executing a step of reading an image corresponding to the acquisition request from the storage unit and distributing the image to the user terminal for each image when the image acquisition request based on the metafile is received from the user terminal . An order generator that identifies each image constituting the video data and generates order information according to the order of the images;
Each image constituting the moving image data is compared with other images, and at least a part of the matching image is a reference image, and the comparison unit uses the other as a reference image;
An input receiving unit that receives an input of storage destination information indicating a storage destination when an image that is not matched by the result of the comparison and a reference image is stored in a storage unit of a plurality of distribution servers;
A metafile generating unit that generates a metafile including display instruction information indicating a correspondence relationship between the reference image and a referenced image at least partially matching the reference image, the order information, and the storage destination information;
A metafile generating device comprising: Identifying each image constituting the video data and generating order information according to the order of each image;
Each image constituting the moving image data is compared with other images, and at least one of the matched images is a referenced image and the other is a reference image;
Receiving an input of storage location information indicating a storage location when an image that is not matched by the result of the comparison and a reference image is stored in a storage unit of a plurality of distribution servers;
Generating a metafile including display instruction information indicating a correspondence relationship between the reference image and a referenced image at least partially matching the reference image, the order information, and the storage destination information;
To generate a metafile that the computer executes.

Images