JP6024002B2 - Video distribution system - Google Patents

Description

  The present invention relates to a moving image distribution system for reproducing a plurality of moving image contents distributed via a network.

  In recent years, in addition to the widening of communication lines and improvements in communication technology, along with the diversification of users' use of the Internet (communication devices), an environment that facilitates transmission and reception of large volumes of data has been established. Distribution and viewing of moving image contents via the Internet are actively performed. In many cases, the viewable moving image content is arranged and displayed in a tile shape or an array shape on the display of the communication device in a state where each content is reduced. The user can start viewing by selecting one moving image content to be viewed with reference to this array display.

  In the distribution of moving image content in such a format, an image processing apparatus whose main problem is to suppress an increase in processing load on the distribution server and an image distribution system using the same have been disclosed (for example, Patent Documents). 1).

  In this system, the state (enlargement, reduction, movement, etc.) of the display area in the client used by the user is analyzed one by one, and accordingly (1) a plurality of moving image contents to be arrayed are temporarily decoded, (2) Based on these, one moving image for array display is generated, and (3) the one moving image is encoded into one bit stream and distributed to the client. Also, when any one of a plurality of moving image contents included in the distributed moving image for array display is enlarged to a size equal to or larger than a predetermined threshold (for example, 80% of the display area) by a user operation. The server analyzes the state of the display area transmitted from the client again, determines that it is clear that the video content is noticed and selected, and the bit stream of the selected video content alone is used as it is in the client. Deliver to.

JP2015-192278A

According to the above-described prior art, the processing of (1) to (3), which is necessary for displaying an array while any moving image content is selected, can be omitted. It is thought that the processing load in the can be reduced.
However, in this prior art configuration, after the user selects any one of the plurality of moving image contents displayed in an array, the moving image content is displayed in the display area of the client at the same magnification. Since a time lag occurs, there may be a problem in that a delay is caused in distribution and video reproduction is interrupted.

  That is, whenever a change occurs in the state of the display area due to a user operation, information regarding the state of the display area is transmitted from the client to the server. When the server analyzes the received information and determines that any one of the moving image contents is selected, the server switches the bit stream to be distributed to the client from the moving image for array display to the selected moving image content. When the bit stream of the selected moving image content alone is delivered from the server, the client receives it and displays it in the display area at the same magnification. The discontinuity of the moving image reproduction described above is caused by a time lag from when the user operates in this way until the selected moving image content is displayed at the same magnification. When the reproduction of the moving image is interrupted, the user who is viewing the content is less focused on the content, and as a result, the user's work efficiency may be reduced.

  Therefore, an object of the present invention is to provide a technique capable of reproducing a moving image without reducing the user's concentration.

In order to solve the above problems, the present invention employs the following means.
In other words, the moving image distribution system of the present invention distributes a moving image bitstream multiplexed with one synthesized image obtained by combining a plurality of images in advance in an array and a plurality of sounds related to the plurality of images. Via a distribution means, a UI providing means for providing composition information relating to the composite video and a user interface linked to the configuration information, accompanying the distribution of the bit stream by the moving picture distribution means, and a user interface provided by the UI provision means And a reproduction means for realizing the reproduction of the moving image by outputting a predetermined region selected from one composite video to the video output unit and downmixing a plurality of sounds to the audio output unit. ing.

  In the moving image distribution system, the moving image is distributed from the server side to the client side and is reproduced on the client side. The moving image distributed from the server side is multiplexed with a plurality of videos and a plurality of sounds. A plurality of videos constituting a moving image are synthesized in a state where they are arranged in advance (for example, tiles, rows, etc.), and are delivered as one synthesized video at the stage of delivery. In addition, a plurality of sounds constituting the moving image are distributed as one sound data having a plurality of channels.

  When the distributed video is played back on the client side (video and audio are output), the audio is extracted from the audio data and output in a mixed state. As for the video, only the region selected via the user interface is extracted from the synthesized video and output. When the user wants to switch the video to be displayed, the video to be switched to is delivered to the client side as part of the composite video in the same way as the video before switching (during playback). There is no need to request the server separately and wait for the distribution, and the processing is completed only by changing the area extracted from the synthesized video on the client side. For this reason, communication between the server and the client due to switching of the video to be displayed does not occur, and there is no possibility of delay or interruption in moving image reproduction due to a time lag associated with communication.

  Therefore, according to the above-described aspect, even when the video to be displayed is switched during the reproduction of the multiplexed moving image, the switching can be realized smoothly without interrupting the video and audio. Therefore, the user can continue watching the video without losing concentration.

  In addition, the moving image distribution means collects a plurality of videos obtained by photographing the same target object or a plurality of mutually related objects and a sound emitted from the target object, or adds a plurality of videos in association with the target object. The bit stream multiplexed with the audio of

  In the moving image distributed in this aspect, a plurality of videos and sounds capturing the same object or a plurality of objects related to each other are synchronized and multiplexed.

  For example, the video to be distributed is a video that was shot and collected simultaneously by cameras and microphones installed at multiple locations in the venue at events that accommodate a large number of people, such as live music, sports competitions, entrance ceremonies, etc. And audio. The moving image distribution means distributes the multi-view moving images multiplexed in this way. In such a venue, the appearance varies depending on the position of the audience seat. Also, there are ten colors for the scene you want to see, that is, from which position you want to see. According to this aspect, the user can select and switch the video to be displayed from his / her plurality of delivered videos according to his / her preference. Further, as described above, video and audio are not interrupted due to this switching. Therefore, the user can enjoy watching the moving image without losing the excitement as if he / she is traveling around the venue that is the target of the moving image.

  Moreover, this aspect can be applied to various fields such as education, medical care, and industry other than the above-described event examples. For example, the video that is distributed is learning content used in the field of education, and the teacher gives a lecture, a plurality of videos that capture the board written content, lecture text, etc. at the same time as the lecture progresses. And a plurality of sounds such as sound effects and BGM added along with the progress of the lecture. Or, it is applied to foreign language teaching materials, and video and audio prepared for a plurality of languages are multiplexed in a synchronized state in accordance with the progress timing of the lecture. Also, in the industrial field, for example, a plurality of video and audio captured simultaneously at a plurality of positions (entrance, exit, etc.) on the production line are multiplexed. Thus, according to this aspect, the user can efficiently confirm the target object to be confirmed by switching the display in real time without interruption of video and audio. Therefore, the user's work efficiency can be improved in a scene where quickness is required.

  Preferably, the moving image distribution system further includes an output adjustment unit that adjusts an output mode of the video output by the video output unit and an output mode of the audio output by the audio output unit, and the UI providing unit includes: A user interface is provided for causing the output adjustment means to adjust the video and audio output modes based on the operation.

  In this mode, an operation menu for adjusting the video and audio output mode is provided by the UI providing means. Examples of the video output mode include switching of output video (video to be displayed), enlargement / reduction of video, change of display position, insertion of effects, and the like. Further, examples of the sound output mode include the volume of each sound to be mixed, the localization and silence of each sound, and the like. When the user performs an operation related to the adjustment of the output mode using the operation menu, the output adjustment unit adjusts the video and audio output mode to be actually output based on the operation.

  Therefore, according to this mode, it is possible to finely adjust the output mode of the multiplexed video and audio via the user interface. Therefore, the user can further concentrate on viewing the moving image by outputting video and audio in a manner that the user likes.

  More preferably, the moving image distribution system includes an adjustment recording unit that records an adjustment history that is a transition of a video and audio output mode by the output adjustment unit that is made through a user interface based on a user operation during reproduction of the moving image, and An adjustment reproduction means for reproducing the adjustment of the video and audio output mode based on the user's operation upon the reproduction of the moving image on the output adjustment means according to the adjustment history recorded by the adjustment recording means. I have.

  In this mode, the adjustment recording unit records the transition of the output mode (adjustment history) based on the user's operation performed on a certain moving image on the server side. The adjustment history recorded on the server side can be used when the same user or another user views this moving image. In that case, the adjustment history is distributed to the client side together with the moving image, and the adjustment reproduction unit reproduces the adjustment of the output mode of the video and audio recorded along the distributed adjustment history.

  Therefore, according to this aspect, the moving image can be viewed in a state where the adjustment of the output aspect performed by itself or another user is reproduced. Thus, when a user views a video, the user reproduces the output adjustment that the user made in the past according to his / her preference as it is, or reproduces the output adjustment made according to another user's preference. It becomes possible to concentrate more by viewing.

  The moving image distribution system further includes notification means for notifying a user different from the user of identification information that uniquely defines a combination of the moving image and the adjustment history recorded by the adjustment recording means, and the identification information notified by the notification means When a distribution request is made, the moving image distribution means distributes the bit stream of the moving image identified by the identification information and the adjustment history for the moving image, and the adjustment reproduction means regenerates the moving image by the user along the adjustment history. The adjustment of the video and audio output mode made based on the above operation is reproduced.

  In this aspect, there is a notification means for guiding information (for example, a combination of a moving image ID and a user ID) that can uniquely identify a combination of a certain moving image and an adjustment history for the moving image to other users. When a video distribution request is made together with this identification information, the distribution means distributes the video identified by the identification information and the adjustment history, and output of video and audio recorded along the adjustment history distributed by the adjustment reproduction means Reproduce the adjustment of the aspect.

  Therefore, according to this aspect, it is possible to easily guide information (for example, URL) that enables viewing of a moving image in a state in which the adjustment of the output aspect performed by itself is reproduced. In addition, other users can easily view a specified moving image using the guided information in exactly the same state as the reproduction by the guiding user. Therefore, other users can easily view a moving image without performing an operation related to adjustment of the output mode.

  The present invention can reproduce a moving image without reducing the user's concentration.

It is a block diagram of the environment where a moving image delivery system operates. It is a figure which shows the example of the user interface provided by a moving image delivery system. It is a schematic diagram which shows the aspect with which several video and an audio | voice are multiplexed. It is a functional block diagram (1/2) of a moving image delivery system. It is a functional block diagram (2/2) of a moving image delivery system. It is a figure explaining the structure information of a moving image. It is a figure (1/3) which shows the output adjustment example of the image | video via the operation menu provided with a moving image delivery system. It is a figure (2/3) which shows the output adjustment example of the image | video via the operation menu provided with a moving image delivery system. It is a figure (3/3) which shows the output adjustment example of the image | video via the operation menu provided with a moving image delivery system. It is a flowchart which shows the example of a procedure of the process performed by the server side. It is a flowchart (1/2) which shows the example of a procedure of the process performed by the client side. It is a flowchart (2/2) which shows the example of a procedure of the process performed by the client side. It is a figure which shows the specific example of adjustment log | history data. It is a sequence diagram which shows the operation example at the time of reproducing | regenerating a moving image using an animation delivery system, and recording adjustment history. It is a sequence diagram which shows the operation example at the time of reproducing a moving image by reproducing the output adjustment made by the other user using the moving image distribution system. It is a block diagram of the environment where the moving image delivery system of 2nd Embodiment operate | moves. It is a block diagram of the environment where the moving image delivery system of 3rd Embodiment operate | moves.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. The following embodiment is a preferred example, and the present invention is not limited to this example.

  FIG. 1 is a configuration diagram of an environment in which the moving image distribution system 100 according to an embodiment operates. The moving image distribution system 100 can be applied to a computer environment connected to a network such as the Internet, for example.

  As shown in this figure, the video as the video material distributed in the video distribution system 100 is shot by n cameras VC1 to VCn, and the voice as the video material is collected by m microphones MC1 to MCm. Sounded. The n images captured by the cameras VC1 to VCn and the m sounds collected by the microphones MC1 to MCm are input to the combining and multiplexing apparatus 10. The synthesizer / multiplexer 10 synthesizes n videos into a multi-screen video to generate one video signal and generates one audio signal having m channels with m voices as prime elements.

  The encoding device 20 converts the video signal and the audio signal generated by the combining and multiplexing device 10 into a band and a format that can be distributed to the network, respectively, and collects them into one bit stream. The bit stream generated by the encoding device 20 is stored in the moving image database 32 in the database server 30.

  In addition, the process in which each material which comprises a moving image is put together into one bit stream is later mentioned using another figure. Further, the multiplexing / multiplexing device 10, the encoding device 20, and the database server 30 do not need to be connected to each other via a network, and can select a form in which signals and data generated by each device can be exchanged with each other. it can. For example, video / audio signals are transmitted from the multiplexing / multiplexing apparatus 10 to the encoding apparatus 20 via a cable, and data is input from the encoding apparatus 20 to the database server 30 using a recording medium or the like. May be.

  The database server 30 arranges / manages various databases used in the moving image distribution system 100 and includes a history database 34 and a user master 36 in addition to the moving image database 32 described above. In addition to the bit stream of the moving image, the moving image database 32 stores the configuration information of the moving image (for example, the number of videos, the number of sounds, the compression format, the resolution, the coordinate information of each video, etc.). The history database 34 stores history data that records changes in video and audio output adjustments made by user operations during playback of distributed video. The user master 36 stores registration information (for example, user ID, user name, authentication password, etc.) of users who use the moving image distribution system 100. Each database 32, 34, and 36 is not necessarily configured as a database system. For example, a storage area recognized by the database server 30 may be used as a file storage.

  The distribution server 40 distributes a moving image in response to a request from the user (more precisely, the user terminal 50). The distribution server 40 can be connected to the database server 30 via a network or the like, and at the same time is connected to the Internet 70. A user terminal 50 used when a user uses the moving image distribution system 100 is also connected to the Internet 70. That is, the distribution server 40 and the user terminal 50 can be connected to each other via the Internet 70.

  The user terminal 50 performs a moving image distribution request to the distribution server 40, reproduction of a moving image through an operation menu provided by the distribution server 40, and the like. Examples of the form of the user terminal 50 include a mobile device such as a computer, a smartphone, and a tablet PC, an Internet-compatible TV, an Internet-compatible game machine, and the like. As described above, the moving image distribution system 100 can be used by any client that can be connected to the Internet and has a standard web browsing function and video / audio output function.

  The functions of the distribution server 40 and the user terminal 50 and the processing executed in the distribution server 40 and the user terminal 50 will be described in detail later.

  FIG. 2 is a diagram illustrating an example of an operation menu 140 (user interface) provided by the moving image distribution system 100 according to an embodiment. In order to facilitate the understanding of the invention, in the following description, a case where the moving image to be distributed is composed of four videos and sounds each will be exemplified. In addition, although the operation input to the operation component provided in the operation menu 140 varies depending on the form of the user terminal 50 (for example, click, touch, tap, press, etc.), these are collectively expressed as “selection” here. I will do it.

  The operation menu 140 is a part of a user interface provided from the distribution server 40 at the time of moving image distribution. The operation menu 140 includes a menu for adjusting the video output mode (video selection, enlargement / reduction, position change, effect, etc.), and a menu for adjusting the audio output mode (volume, localization, mute, etc.) For example, it is expanded to a web browser or a dedicated application.

  In addition to the operation menu 140, the user interface provided from the distribution server 40 includes a control program. The control program renders the operation menu 140 according to the moving image based on the configuration information distributed along with the moving image, and executes processing associated with each operation performed via the operation menu 140.

  For example, as shown in FIG. 2, the operation menu 140 is provided with four main video switching buttons 160 (160a to 160d) and four audio adjustment menus 161 (161a to 161d). Such an arrangement of the operation menu 140 is determined by the control program based on the configuration data indicating that the moving image to be distributed this time is composed of four videos and four sounds, Rendered.

  In addition, on each of the main video switching buttons 160a to 160d, four moving images with a reduced display size are displayed as thumbnails as sub-videos. For example, as shown in FIG. 2, the main image switching button 160a for the image 1 displays a reduced image of the pup with the head tilted, and the main puppy switching button 160b for the image 2 displays the same puppy. Is reduced, the main video switching button 160c for video 3 is displayed in a reduced size, and the same puppy is displayed on the left side in the main video switching button 160d for video 4. The video captured from is reduced. These sub-videos are displayed on the buttons 160a to 160d to which switching to each video 1 to 4 is assigned based on coordinate (position) information indicating each area of the video 1 to 4 in the composite video described in the configuration data. In addition, the control program displays reduced images of the respective images 1 to 4.

At this time, when the main video switching button 160a for video 1 is selected, the video that is reduced and displayed on the main video switching button 160a is displayed in the main video display area 150. This display is also made by the control program. Based on the selection of the main video switching button 160a, the control program extracts only the area from the synthesized video based on the coordinate information indicating the area of the video 1 described in the configuration data, and displays it in the main video display area 150.
As described above, the main video switching buttons 160 (160a to 160d) provided by the number of videos extract the selected video from the synthesized video and display it in the main video display area 150.

  Each audio adjustment menu 161 (161a to 161d) provided with the number of sounds is provided with a volume slider 162 (162a to 162d), a localization knob 164 (164a to 164d), and a mute button 166 (166a to 166d). . The volume slider 162 adjusts the volume of the designated voice. The localization knob 164 adjusts the localization of the designated sound. The mute button 166 switches whether to mute the designated sound.

  By using these menus, the user can freely adjust the volume and balance of individual voices, and can mix and output a plurality of multiplexed voices according to preferences. For example, as shown in FIG. 2, when the volume slider 162a for the voice 1 is slid to the upper end, the voice 1 is output at the maximum volume. When the localization adjustment knob 164b for the sound 2 is selected and brought to “R”, the sound 2 is output only from the right side. When the mute button 166c for the sound 3 is selected, the display of the mute button 166c changes to a cross, and the sound 3 is muted and is not output at all.

  The character input display area 158 accepts and displays characters input by the user during the reproduction of the moving image. The enlarge button 152 enlarges the video displayed in the main video display area 150. The reduction button 154 reduces the video displayed in the main video display area 150. The move button 156 moves an area to be displayed in the video displayed in the main video display area 150. The enlargement / reduction of the video and the change (movement) of the display position in the video will be described in detail later.

  The effect selection buttons 168 to 178 insert effects associated with the respective buttons during video reproduction. For example, when the cross-fade button 168 is selected, a cross-fade is inserted into the video, that is, the video after the button selection is faded in while the video before the button selection is faded out, and the video before and after is smoothly switched. When the auto effect button 176 is selected, any one of a plurality of types of prepared effects is automatically inserted.

  The adjustment record button 180 records the transition (adjustment history) of the video and audio output mode adjusted via the operation menu 140. Once the adjustment recording button 180 is selected, recording in the local temporary database is started. Thereafter, when the user performs an operation related to video or audio output adjustment with the operation menu 140, the process associated with each operation component is executed and the video and audio output mode is changed. When the output adjustment is performed, it is added to the temporary database one by one. When the reproduction of the moving image ends, a message prompting the user to save the data is displayed on the operation menu 140. Here, when the adjustment recording button 180 is selected again, the data recorded in the temporary database during the reproduction of the moving image, that is, the adjustment history of the video and audio output mode is transmitted to the distribution server 40 and recorded in the history database 34. .

  The adjustment reproduction button 182 reproduces the adjustment of the video and audio output mode based on the adjustment history developed in the temporary database. When another user requested the adjustment history when viewing the movie at the time of the distribution request of the movie, or there is an adjustment history recorded on the server side when the user viewed the same movie in the past Is distributed together with the adjustment history when the moving image is distributed, and is expanded in a local temporary database. When the adjustment reproduction button 182 is selected, the adjustment history is read from the temporary database, and the recorded output adjustments are executed one by one along the reproduction time axis. It is reproduced. That is, the user can automatically reproduce the output adjustment according to the adjustment history without operating the operation menu 140, and can view the moving image in the same state as when viewing the past (when recording the adjustment history).

  Note that if there is no adjustment history in the temporary database, the output adjustment cannot be reproduced, so the adjustment reproduction button 182 is displayed in a disabled state.

  Notification tool activation buttons 190 to 196 are URLs for activating notification tools such as e-mail and SNS so that output adjustments made by the user to this video can be reproduced when other users view this video. Is set as the text. For example, when the e-mail activation button 190 is selected, a mail client installed in the user terminal 50 is activated and a message creation screen is displayed. At this time, a URL in which information (for example, a moving image ID and a user ID) that can identify the moving image and the user who has adjusted the output is specified as a query is set as the body of the message. Other users who have received this message simply access the URL described in the text, and in the same state as when the transmitting user viewed, that is, in a state where the operation performed by the transmitting user is reproduced, Can watch.

  When the user's adjustment history is not recorded on the server side, the notification tool activation buttons 190 to 196 are displayed in a disabled state because the adjustment history cannot be referred to by other users.

  FIG. 3 is a schematic diagram showing an aspect in which a plurality of video and audio are multiplexed.

First, four videos V1 to V4 photographed by the cameras VC1 to VC4 are each input to the synthesis and multiplexing apparatus 10 as video signals. The four videos V1 to V4 are combined into one multi-screen video MV. For example, assuming that each of the videos V1 to V4 is a video having a resolution of 960 × 540, the multi-screen video MV is a resolution of 1920 × 1080 (HDTV) in which the videos V1 to V4 are arranged in two rows and columns. Are combined into a video image having a multi-screen shape. Through such synthesis processing, the four video signals are combined into one.
Note that the arrangement of the multi-screen video MV is not limited to this mode. For example, the videos V1 to V4 may be arranged in a line in either the vertical or horizontal direction. When arranged in a vertical row, the multi-screen video MV is combined with a video having a resolution of 960 × 2160.

  The four voices A1 to A4 collected by the microphones MC1 to MC4 are each input to the synthesis and multiplexing apparatus 10 as voice signals. The four voices A1 to A4 are multiplexed into a multi-track voice MA having a plurality of channels each configured as a separate and independent channel. Through these processes, the four audio signals are combined into one.

  Further, the video signal and the audio signal are synchronized on the time axis. In this way, in the multiplexing / multiplexing apparatus 10, one multi-screen video MV obtained by synthesizing a plurality of video materials and a multi-track audio MA multiplexed by a plurality of audio materials are multiplexed into one. A video signal and one audio signal are generated.

  The video signal of the multi-screen video MV and the audio signal of the multi-track audio MA multiplexed by the synthesizer / multiplexer 10 are input to the encoder 20. The encoder 20 encodes the input video signal and audio signal as data into a band and format that can be distributed to the Internet 70, and generates one bit stream. As the encoding (compression) format, any one of popular formats is adopted. For example, the video signal is H.264. H.264 format video data is encoded, and the audio signal is encoded into AAC format audio data. The encoded video data and audio data are combined to generate one bit stream BS. The bit stream BS is a form that can be delivered to the network as it is.

  The bit stream BS generated by the encoder 20 is stored in the moving image database 32. The video database 32 stores a bitstream BS corresponding to each video distributed by the video distribution system 100. When the distribution server 40 receives the moving image distribution request from the user terminal 50, the distribution server 40 refers to the moving image database 32 to acquire the corresponding bit stream BS and distributes it to the user terminal 50.

  4 and 5 are functional block diagrams of the moving image distribution system 100. FIG. The video distribution system 100 is configured across the server side (database server 30 and distribution server 40) and the client side (user terminal 50), and the server side is mainly equipped with functions related to video distribution, On the client side, functions related to the playback of the distributed video are implemented.

  FIG. 4 is a functional block diagram showing functions that the moving image distribution system 100 has on the server side. As shown in this figure, the moving image distribution system 100 includes a moving image storage unit 110, an adjustment history storage unit 112, a user information storage unit 114, a request response unit 120, a moving image distribution unit 122, a UI providing unit 124, an adjustment history input / output. Unit 126 and user authentication unit 128 on the server side.

The database server 30 is equipped with a function for storing data used in the moving image distribution system 100.
The moving image storage unit 110 corresponds to the moving image database 32 and stores a moving image bit stream and configuration information distributed on the moving image distribution system 100. The adjustment history storage unit 112 corresponds to the history database 34, and stores history data that records transitions in video and audio output adjustments made based on user operations during playback of distributed video. The user information storage unit 114 corresponds to the user master 36 and stores registration information and authentication information of users who use the moving image distribution system.

  The distribution server 40 is provided with a function for performing processing in response to a request from the user terminal 50, specifically, a function from the request response unit 120 to the user authentication unit 128.

  The request response unit 120 operates as a window for the distribution server 40. The request response unit 120 receives the request transmitted from the user terminal 50, confirms the content, and distributes the request to an appropriate function unit. The request response unit 120 also relays data, notifications, and the like acquired when the request is processed by each functional unit, and transmits these to the user terminal 50 that is the transmission source of the request.

  In response to the distribution by the request response unit 120, the video distribution unit 122 acquires a bit stream of the video requested to be distributed from the user terminal 50 from the video storage unit 110, and sends this to the user terminal 50 via the request response unit 120. To deliver.

  The UI providing unit 124 displays the configuration information of the moving image requested to be distributed from the user terminal 50 according to the distribution by the request response unit 120, and a user interface such as an operation menu and a control program necessary for reproducing the moving image. 50 via the request response unit 120. The moving image configuration information provided by the UI providing unit 124 is generated in accordance with the screen resolution based on the terminal information transmitted together with the distribution request from the user terminal 50.

  The adjustment history input / output unit 126 executes processing according to the request for recording / referencing the adjustment history transmitted from the user terminal 50 in accordance with the distribution by the request response unit 120. Specifically, the adjustment history input / output unit 126 first transmits the adjustment history (history data in which adjustment details of the output mode are recorded) in response to the adjustment history recording request transmitted from the user terminal 50. Are recorded in the adjustment history storage unit 112 in a state where the user and the moving image can be identified. Further, the adjustment history input / output unit 126, as a response to the adjustment history reference request transmitted from the user terminal 50, an adjustment history storage unit stores an adjustment history for a specific video by a specific user based on the transmitted identification information. 112, and this is distributed to the user terminal 50 via the request response unit 120.

  The user authentication unit 128 executes an authentication process for a user who uses the moving image distribution system 100 according to the distribution by the request response unit 120. When the authentication information transmitted from the user terminal 50 is passed through the request response unit 120, the user authentication unit 128 inquires the user information storage unit 114 about the validity of the authentication information and performs user authentication processing. The result is notified to the user terminal 50 via the request response unit 120.

  FIG. 5 is a functional block diagram showing functions that the moving image distribution system 100 has on the client side. As shown in this figure, the moving image distribution system 100 includes a distribution request unit 130, a reception unit 132, a reproduction unit 136, an operation reception unit 140, an adjustment reproduction unit 142, an output adjustment unit 144, a temporary recording unit 145, and a recording request unit. 146, an authentication request unit 147 and a notification unit 148 are provided on the client side (user terminal 50). These functions are implemented by a control program provided by the UI providing unit 124.

The distribution request unit 130 requests the distribution server 40 to distribute a specific moving image and to refer to the adjustment history corresponding thereto. When transmitting the request, the distribution request unit 130 also transmits the terminal information of the user terminal 50 (for example, the screen resolution).
The receiving unit 132 receives the bit stream of the moving image distributed from the distribution server 40, the configuration information, the user interface, and the adjustment history as a response to the moving image distribution request transmitted by the distribution request unit 130.
The decoding unit 134 corresponds to decoding means provided as standard on the user terminal, decodes the moving image bit stream received by the receiving unit 132, and makes video and audio reproducible. The decoded video and audio are output to the video output unit 138 and the audio output unit 139 through the playback unit 136, respectively.

  The reproduction unit 136 reproduces the moving image by outputting the video and audio decoded by the decoding unit 134 in a predetermined output mode. More specifically, the reproduction unit 136 reproduces a moving image by causing the video output unit 138 and the audio output unit 139 to output video and audio, respectively, in accordance with output modes specified by the output adjustment unit 144 described later. Is realized.

The video output unit 138 corresponds to video output means (for example, a display) provided as standard on the user terminal 50, and a predetermined area designated by the playback unit 136 among videos (multi-screen video) constituting a moving image. Output.
The audio output unit 139 corresponds to audio output means (for example, a speaker) provided as standard on the user terminal 50, and outputs audio that is downmixed from a plurality of channels by the reproduction unit 136.

The operation reception unit 140 receives a user operation performed during the reproduction of a moving image. The operation reception unit 140 forms a part of the user interface received by the reception unit 132 along with the moving image distribution. In other words, the operation reception unit 140 is the operation menu 140 itself provided from the UI providing unit 124 and generated on the user terminal 50.
User operation related to video output adjustment (output video switching, enlargement / reduction, display area movement, effect insertion, etc.) or audio output adjustment (volume, localization, mute, etc. of each audio to be mixed) by the operation menu 140 Then, the operation reception unit 140 instructs the output adjustment unit 144 to perform output adjustment assigned to the operation.

  The adjustment reproduction unit 142 expands the adjustment history transmitted from the distribution server 40 in response to the moving image distribution request transmitted by the distribution request unit 130 in a local temporary database, and adjusts the output adjustment recorded in the adjustment history to the time axis. Is instructed to the output adjustment unit 144. As the adjustment reproduction unit 142 operates in this way, the moving image distribution system 100 can reproduce the output adjustment performed by the user via the operation menu 140 during past viewing.

The output adjustment unit 144 reflects the output adjustment instructed by the operation reception unit 140 or the adjustment reproduction unit 142 in the current output mode, determines a new output mode as a whole, and outputs video and audio in accordance with this. To the playback unit 136. More specifically, the output adjustment unit 144 includes a new video output mode (a display area for the main video display area 150 determined after reflecting the current output adjustment) and a new audio output mode ( Each volume of LR after mixing determined after reflecting the current output adjustment) is designated to the reproduction unit 136.
The output mode at the start of playback is determined based on the moving image configuration information received by the receiving unit 132 along with moving image distribution. The output adjustment unit 144 instructs the playback unit 136 to output the video specified as the main video at the start of playback in the configuration information to the main video display area 150.

The temporary recording unit 145 appends the output adjustment based on the user operation reflected by the output adjustment unit 144 to the local temporary database. When the user selects the adjustment record button 180 on the operation menu 140 after the state in which the moving image can be played back is prepared, the temporary recording unit 145 creates a temporary database and starts recording output adjustment. Add output adjustments to the temporary database one by one. As a result, a history of output adjustments made during moving image reproduction is accumulated on the temporary database.
The temporary database is created as a work area in the local disk 53 or the RAM 52 of the user terminal 50, and is cleared at a predetermined timing.

  The recording request unit 146 transmits the output adjustment history stored in the temporary database to the distribution server 40 and requests recording of the adjustment history. As described above, when the adjustment recording button 180 is selected once after the state in which the moving image can be reproduced is prepared, the temporary recording unit 145 starts recording the adjustment history locally. Thereafter, when the user selects the adjustment recording button 180 again after the reproduction of the moving image is completed, the recording request unit 146 transmits the adjustment history stored in the temporary database to the distribution server 40.

The authentication request unit 147 transmits authentication information input by the user to the distribution server 40 and requests execution of authentication processing.
The notification unit 148 passes information that can uniquely identify the moving image and the adjustment history for the moving image to a notification tool such as an e-mail or an SNS. More specifically, the process assigned to the notification tool activation buttons 190 to 196 provided in the operation menu 140 is executed.

  FIG. 6 is a diagram for explaining the configuration information of a moving image. 6A shows a specific example of the moving image configuration information, and FIG. 6B shows a specific example of the size and coordinate information of the multi-screen video MV interpreted from the configuration information. .

The configuration information is distributed along with the moving image from the distribution server 40 to the user terminal 50, and is used by a control program included in the user interface.
The configuration information includes information related to the configuration of the video (for example, the number of videos, the number of audio, resolution and coordinate information of each video, initial main video, compression format, frame rate, total playback time, language, video title, etc.) It is described together with attribute information related to information (for example, configuration information ID, creation date, etc.). Among these, a part of the configuration information (for example, resolution and coordinate information of each video) is generated in accordance with the screen resolution of the user terminal 50.

  For example, the configuration information shown in FIG. 6A includes the number of configuration videos (4), the width (960 pixels) and the height (540 pixels) of the video identified by the ID “5618”. ) And upper left coordinates of each video area in the multi-screen video, main video at the start of playback (video 01), video compression format (H.264), video frame rate (24 fps), number of audio channels (4 channels) , Audio compression format (AAC), total playback time (1 hour 3 minutes 22 seconds), language (Japanese), video title (Indian people are surprised! Ukiuki Magic Show), creation date and time of composition information (2016/5 / 21) etc.

  From this configuration information, four images V1 to V4 having a width w of 960 pixels and a height h of 540 pixels, that is, a resolution of 960 × 540, are arranged two by two vertically and horizontally, and have a resolution of 1920 × 1080. It can be seen that the multi-screen video MV is synthesized. Therefore, as shown in FIG. 6B, when the horizontal position in the multi-screen video MV is represented as x and the vertical position is represented as y (coordinate (x, y)), the coordinate (0, 0). A region of width w and height h starting from the top left corresponds to the video V1, a region of width w and height h starting from the upper left corner of the coordinates (960, 0) corresponds to the video V2, and coordinates ( An area having a width w and height h starting from 0,540) corresponds to the video V3, and an area having a width w and height h starting from coordinates (960,540) corresponds to the video V4. You can see that

  Further, it can be seen from the relationship between the resolution of each video and the coordinate information that this configuration information is generated for a horizontally long screen. As described above, the user terminal 50 transmits its own terminal information together with the distribution request for the moving image to the distribution server 40, and the distribution server 40 that has received the distribution distributes the configuration information according to the screen resolution of the request source. To do. For example, when the user terminal 50 has a horizontally long screen such as a computer or an Internet-compatible TV, coordinate information suitable for display on the horizontally long screen is specified in the configuration information of the moving image distributed from the distribution server 40. Is done. If the user terminal 50 has a vertically long screen such as a smartphone or tablet PC, the distribution server 40 distributes configuration information in which coordinate information suitable for display on the vertically long screen is designated. Therefore, in that case, coordinate information is specified when the multi-screen video MV is rotated 90 degrees to the left in the diagram shown in FIG. 6B and the lower left vertex is (0, 0).

  The control program interprets the configuration information in this way, and performs playback control of the distributed moving image. For example, since the moving image is designated to have the video 01, that is, the video V1 as the main video at the start of reproduction, the control program extracts only the area of the video V1 from the multi-screen video MV at the start of reproduction. To display. When the user performs an operation related to the output of video or audio from the operation menu 140 after the reproduction of the moving image starts, the control program (more specifically, each functional unit mounted on the user terminal 50) detects this, and the user By adjusting the output assigned to the operation, the moving image is reproduced in the output mode designated by the user.

  For convenience of explanation, all the moving images exemplified below are moving images having the configuration information shown in FIG. However, this configuration is merely an example, and the moving image distributed by the moving image distribution system 100 is not limited to this configuration. For example, it is possible to increase or decrease the number of videos constituting the multi-screen video MV, or to adopt a configuration video having a smaller (or larger) resolution according to the screen resolution of the user terminal 50.

[Specific example of video output adjustment]
FIGS. 7 to 9 are diagrams illustrating examples of video output adjustment via the operation menu 140 provided in the moving image distribution system 100. FIG. Of these, FIG. 7 shows an example in which the video displayed in the main video display area 150 is switched, and FIG. 8 shows an example in which the video displayed in the main video display area 150 is expanded and the display area is moved. FIG. 9 shows an example of moving the display area of the video displayed in the main video display area 150 and further reducing the video. In these drawings, a part of the operation menu 140 is simplified.

  FIG. 7 shows an example of switching the video to be displayed in the main video display area 150. The moving image shown in FIG. 7 includes four videos V1 to V4 taken by a plurality of cameras set at different positions of the magic show held in the hall, and four collected by microphones arranged in the vicinity of each camera. It is a multi-view video composed of audio. The multi-screen video MV consists of a video V1 that captures a part of the audience seats, a video V2 that captures the interior of the venue from behind the audience seats, a video V3 that captures the person holding the key to Magic, and a stage where Magic is being demonstrated. The captured video V4 is synthesized in a state where two images are arranged vertically and horizontally.

At the time when reproduction of this moving image is started, video output based on the configuration information is performed. In the illustrated configuration information, video 1 (video V1) is designated as the main video at the start of playback. Therefore, at the time of starting playback, as shown in FIG. 7, only the area of the video V1 that captures a part of the passenger seat is extracted from the multi-screen video MV and displayed in the main video display area 150. The video output according to this aspect is continued until the user performs some operation related to the video or audio output adjustment on the operation menu 140 thereafter.
Here, when the user selects the main video switching button 160d for video 4, the video to be displayed as the main video is switched to video 4 (V4), and only the area of video V4 capturing the stage is displayed in the main video display area 150. It is extracted from the screen image MV and displayed.

FIG. 8 shows an example of enlarging the video displayed in the main video display area 150 and moving the display area.
8A: When the user selects the enlargement button 152 several times while the video V4 is displayed as the main video, the target area in the video V4 displayed in the main video display area 150 is narrowed. As a result, in the main video display area 150, only the central area narrowed with the same aspect ratio in the video V4 is extracted and displayed. Accordingly, a part of the magician's body, a bunny girl, a head of a pigeon appearing from a hat, and the like that are stored outside this area are out of the target area and are not displayed in the main video display area 150.

  (B) in FIG. 8: In the state in which video is output according to the mode shown in FIG. 8A, the user moves the right / bottom part of the movement button 156 (the part indicated by the black triangle in the figure). ) Is selected, the position of the target area in the video V4 displayed in the main video display area 150 moves in the lower right direction. As a result, the display target area shown in FIG. 8A changes its position to the lower right while maintaining the size, and only the lower right area of the video V4 is extracted and displayed in the main video display area 150. The Therefore, in the main video display area 150, only the hat and bunny girl's legs of the magician within the display target area are displayed.

  FIG. 9 shows an example in which the video display area displayed in the main video display area 150 is moved and reduced.

  9A: In the state where the video is output in the mode shown in FIG. 8B, the user selects the enlargement button 152 again, and the left / upper part of the move button 156 (see FIG. 9). When the part indicated by a black triangle is selected, the target area of the video V4 displayed in the main video display area 150 is further narrowed, and then the position of the displayed target area moves in the upper left direction. As a result, the display target area is slightly narrower than the display target area shown in FIG. 8B, and the position changes to the upper left. Only the upper left area of the video V4 is extracted from the main video display area 150. Displayed. Therefore, in the main video display area 150, the magician's face that is within the display target area is displayed in an up position.

  9B: When the user selects the reduction button 154 several times in a state where the video is output in the mode shown in FIG. 9A, the video V4 displayed in the main video display area 150 is displayed. The target area is expanded. As a result, in the main video display area 150, only the upper left area whose range is wider than the target area in FIG. 9A with the same aspect ratio is extracted from the video V4 and displayed. Accordingly, the main image display area 150 displays the upper body of the magician and the magic tool within the display target area, and the user can view the magic demonstration scene moderately.

  As described above, the user can switch the video to be displayed in the main video display area 150 via the operation menu 140, or can display the main video display area 140 by enlarging or reducing the video and moving the display area. The output mode of the video to be adjusted is adjusted.

  In the main video display area 150 and each of the main video switching buttons 160a to 160d, a part of the multi-screen video MV is extracted and displayed. In the present embodiment, these operation components 150, Each of 160a to 160d was formed in a fixed size and arranged at a fixed position. In order to enhance operability and video visibility, the main video display area 150 and the main video switching buttons 160a to 160d may be mounted so that the user can freely change the size and position thereof.

[Processing on the server side]
FIG. 10 is a flowchart illustrating a procedure example of request response processing executed on the server side. Each step shown in this flowchart is executed by each function unit 110 to 128 that the moving image distribution system 100 has on the server side. The main body that operates each function unit is the CPU of each server on which these function units are mounted, that is, distribution. These are the CPU 41 of the server 40 and the CPU 31 of the database server 30. That is, the CPU 31 and 41 cause each function unit to execute each step, so that the moving image distribution system 100 functions on the server side.

Step S100: The CPU 31 causes the request response unit 120 to receive a request transmitted from the user terminal 50.
Step S102: The CPU 31 causes the request response unit 120 to check whether or not the request received in the previous step S100 is a request for moving image distribution. When it is a request for moving image distribution (Yes), the CPU 31 next executes step S104. If it is not a request for moving image distribution (No), the CPU 31 next executes step S114.

Step S104: The CPU 31 causes the request response unit 120 to confirm specific designation contents of the request (moving image to be delivered, delivery designation of adjustment history, terminal information, etc.).
Step S106: The CPU 31 causes the moving image distribution unit 122 to acquire the configuration information from the moving image storage unit 110 in addition to the bit stream of the moving image specified by the request.

Step S108: The CPU 31 causes the request response unit 120 to confirm whether or not the distribution of the adjustment history is specified together with the moving image distribution request. When the adjustment history is designated (No), the CPU 31 next executes step S110. When the adjustment history is not designated (Yes), the CPU 31 next executes step S112.
Step S <b> 110: The CPU 31 causes the adjustment history input / output unit 126 to acquire the adjustment history designated for distribution together with the moving image distribution request from the adjustment history storage unit 112.

  Step S112: The CPU 31 provides the request response unit 120 with the data acquired in the previous steps, that is, the moving image bitstream and its configuration information, the adjustment history (if specified), and the UI providing unit 124. A user interface for reproduction control is transmitted to the user terminal 50 that is the request transmission source.

  Step S114: The CPU 31 causes the request response unit 120 to confirm whether or not the request received in step S100 is a request for recording the adjustment history. If it is a request to record the adjustment history (Yes), the CPU 31 next executes step S116. If it is not a request for recording the adjustment history (No), the CPU 31 next executes step S120.

  Step S116: The CPU 31 causes the user authentication unit 128 to confirm whether or not the user using the user terminal 50 as the request transmission source has already been authenticated in the current session. If the user has been authenticated (Yes), the CPU 31 next executes step S118. If the user has not been authenticated (No), the CPU 31 next executes step S120.

  Step S118: The CPU 31 causes the adjustment history input / output unit 126 to execute an adjustment history recording process. In the adjustment history recording process, the adjustment history transmitted from the user terminal 50 is stored in the adjustment history storage unit 112.

  Step S120: The CPU 31 causes the user authentication unit 128 to execute user authentication processing. In the user authentication process, the authentication information transmitted from the user terminal 50 is inquired of the user information storage unit 114, and its validity is checked.

  When the above processing is completed, the request response processing executed on the server side (distribution server 40 and database server 30) with respect to one request transmitted from the client side (user terminal 50) ends. The request response process is started when the service of the moving image distribution system 100 is started, and is continued until the service is stopped.

[Processing on the client side]
FIG. 11 and FIG. 12 are flowcharts illustrating an example of a procedure of processing executed on the client side. Each step shown in these flowcharts is executed by each function unit 130 to 148 provided on the client side of the moving image distribution system 100. The main body that operates each function unit is the CPU 51 of the user terminal 50 on which these functions are mounted. is there. That is, when the CPU 51 causes each function unit to execute each step, the moving image distribution system 100 functions on the client side.

  (A) in FIG. 11 is a flowchart showing an example of a procedure for moving image reception / playback processing. The moving image receiving / reproducing process is a process executed by the CPU 51 for data distributed from the distribution server 40 in response to a moving image distribution request made by the user terminal 50.

Step S200: The CPU 51 causes the receiving unit 132 to receive data distributed from the distribution server 40.
Step S202: The CPU 51 causes the receiving unit 132 to check whether the adjustment history is included in the data received in step S200. When the adjustment history is not included (Yes), the CPU 51 next executes step S206. When the adjustment history is included (No), the CPU 51 next executes step S204.

Step S206: The CPU 51 causes the decoding unit 134 to decode the bit stream of the moving image out of the data received in the previous step S200.
Step S208: The CPU 51 causes the reproduction unit 136 to reproduce the moving image decoded in step S202, and causes the video output unit 138 and the audio output unit 139 to output the decoded video and audio, respectively.

  Step S204: The CPU 51 causes the adjustment reproduction unit 142 to create a temporary database as advance preparation for using the adjustment history, and develops the adjustment history received in step S200 on the temporary database.

  (B) in FIG. 11 is a flowchart showing a procedure example of the moving image reproduction process. The moving image reproduction process is a process that is continuously executed by the CPU 51 from the start to the end of reproduction of a moving image.

  Step S220: The CPU 51 causes the output adjustment unit 144 to execute output adjustment processing. In this process, when video or audio output adjustment is made, a new output mode reflecting this output adjustment is determined.

  Step S222: The CPU 51 causes the reproduction unit 136 to output video and audio to the video output unit 138 and audio output unit 139, respectively, and reproduces a moving image. At this time, if the newly determined output mode is returned in the previous step S220, the CPU 51 causes the playback unit 136 to output video and audio along this output mode. On the other hand, if a new output mode is not returned in the previous step S220, the output is not adjusted (the output mode is not changed), so the CPU 51 causes the playback unit 136 to output the video in the same output mode as before. And output audio.

  Step S224: The CPU 51 confirms with the playback unit 136 whether or not the playback of the moving image has ended. When the reproduction is finished (Yes), the CPU 51 finishes the moving image reproduction process. If the reproduction has not ended (No), the CPU 51 returns to the first step S220 and repeats the subsequent processing.

  FIG. 12 is a flowchart illustrating an exemplary procedure of output adjustment processing. The output adjustment process is a process executed by the CPU 51 in order to adjust the output of video and audio as the moving image is reproduced.

  Step S240: The CPU 51 causes the operation reception unit 140 to check whether an operation related to video or audio output adjustment has been detected (received). When the operation is detected (Yes), the CPU 51 next executes step S242. When the operation is not detected (No), the CPU 51 next executes step S248.

Step S242: The CPU 51 causes the operation receiving unit 140 to instruct the output adjusting unit 144 to perform output adjustment according to the operation received in the previous step S210.
Step S244: The CPU 51 causes the temporary recording unit 145 to check whether a temporary database has been created. When the temporary database has been created (Yes), the CPU 51 next executes step S246. When the temporary database has not been created (No), the CPU 51 next executes step S252.
Step S246: The CPU 51 causes the temporary recording unit 145 to add the contents of the output adjustment reflected in step S242 to the temporary database.

Step S248: The CPU 51 causes the adjustment reproduction unit 142 to check whether there is an adjustment history corresponding to the current reproduction time. If there is an adjustment history (Yes), the CPU 51 next executes step S250. When there is no adjustment history (No), nothing is executed and the process returns to the moving image reproduction process ((B) in FIG. 11).
Step S250: The CPU 51 causes the adjustment reproduction unit 142 to instruct the output adjustment unit 144 to perform output adjustment recorded in the adjustment history confirmed in the previous step S248.

  Step S252: The CPU 51 causes the output adjustment unit 144 to reflect the output adjustment instructed in Step S242 or Step S250 in the current output mode and determine a new output mode. When the above processing is completed, the CPU 51 returns the new output mode determined in step S252 to step S220 in FIG. 11B, which is the caller of the output adjustment processing, and returns to the moving image playback processing.

  On the other hand, when an operation relating to output adjustment is not performed (step S240: No) and there is no adjustment history to be reproduced (step S244: No), the CPU 51 returns to the moving image reproduction process without returning anything to the caller.

  In this embodiment, when the operation receiving unit 140 receives a user operation related to output adjustment while reproducing a moving image while reproducing the output adjustment recorded in the adjustment history, the output adjusting unit 144 preferentially processes output adjustment based on the user's operation.

  For example, it is assumed that the user operates any one of the operation components for output adjustment provided in the operation menu 140 (for example, the enlarge button 152) for the first time after 10 minutes from the start of the reproduction of the moving image. At this time, the output adjustment unit 144 ignores the instruction based on the adjustment history made by the adjustment reproduction unit 142 and gives priority to the instruction based on the user's operation received by the operation receiving unit 140, and is associated with the user's operation. The output adjustment is reflected (for example, the video displayed in the main video display area is enlarged and displayed). Therefore, in this case, the moving image is played back in the state where the output mode is reproduced as recorded in the adjustment history until 10 minutes from the start of playback, and after 10 minutes from the start of playback, the operation menu 140 is operated by the user's operation. Thus, the reproduction is performed in a state in which the output adjustment made via this is reflected.

  In addition, when an interruption due to a user operation occurs in this way, the effective period of the action caused by the interruption, that is, the period in which output adjustment based on the adjustment history is not performed is temporary or until the end of playback. This can be appropriately selected according to the requirements of the system. For example, when the user's operation is interrupted and the user's operation is not performed at all for a certain period (for example, 3 minutes), the output adjustment based on the adjustment history is performed again to reproduce the moving image. May be.

  FIG. 13 is a diagram illustrating a specific example of the adjustment history data. (A) in FIG. 13 shows an example of the header information table, and (B) in FIG. 13 shows an example of the adjustment history table.

  All of these tables are created when the user selects the adjustment record button 180. Among them, the header information table is a table for recording management information related to the recording of the adjustment history, and the session ID in the moving image distribution system 100, the user ID for identifying the current user, and the moving image (the current reproduction) The recording date of the adjustment history and the start date / time thereof are recorded together with the moving image ID for identifying the moving image).

  The adjustment history table is a table for recording details of adjustment contents regarding the output of video or audio during playback of a moving image, and is internally assigned to each operation component for output adjustment provided in the operation menu 140. Along with the component ID and the parameter of the operation component, the time on the reproduction timeline where the output adjustment has been made is recorded in milliseconds. The parameter here is an operation amount corresponding to a numerical value predetermined for each operation component.

  For example, in the first record in FIG. 13B, the component ID “005” and the parameter “3” are recorded for the adjustment time “00: 10: 04.236”. The component ID “005” is assigned to the enlargement button 152, and the parameter “3” is a numerical value indicating that the enlargement button 152 has been selected three times in succession (or continuously for a time corresponding to this). . Therefore, when the output adjustment is reproduced based on the adjustment history recorded in the first record, the reproduction of the moving image identified by the moving image ID “5618” is started at the timing of 00: 10: 04.236. The video displayed in the main video display area 150 is displayed in a state where it is enlarged by three stages.

  In the second record, the component ID “011” and the parameter “15” are recorded for the adjustment time “00: 11: 35.001”. The component ID “011” is assigned to the volume slider 162a, and the parameter “16” is a numerical value indicating that the volume slider 162a has been slid to the 15th memory. Therefore, when the output adjustment is reproduced based on the adjustment history recorded in the second record, the reproduction of the moving image identified by the moving image ID “5618” is started at the timing of 00: 11: 35.001. The sound 1 is mixed with the other sounds 2 to 4 and output at a volume corresponding to the 15th memory.

  FIG. 14 is a sequence diagram illustrating an operation example when a moving image is viewed using the moving image distribution system 100 and an adjustment history is recorded. Steps corresponding to protocol negotiation and session establishment that are performed in advance when packets are transmitted and received between computers are general matters and are not described here. Further, for convenience of explanation, it is assumed here that the user terminal 50 requests delivery of a free video. Hereinafter, it demonstrates along an operation example.

  When the user A starts a web browser on the user terminal 50 to access the distribution menu URL of the moving image distribution system 100 (S11) and selects a moving image to be viewed from the displayed distributed moving image list, the user terminal 50 As a trigger, a distribution request for the selected moving image is transmitted to the distribution server 40 (S12). The distribution server 40 receives this and confirms the content of the request (S13). Then, referring to the moving image database 32 (S14), the requested moving image bit stream and its configuration information are acquired (S15), and delivered to the requesting user terminal 50 together with the user interface for reproduction control (S16).

  The user terminal 50 decodes the distributed moving image bit stream, and outputs the decoded video and audio via a user interface for reproduction control to reproduce the moving image (S17). Here, when the user A selects the adjustment record button 180 provided in the operation menu 140, the user terminal 50 detects this operation (S18), creates a temporary database on the local disk, and starts recording output adjustments. (S18a). When the user A performs an operation relating to the output adjustment of the video or audio using the operation menu 140 during the reproduction of the moving image, the user terminal 50 detects this operation (S19), and simultaneously performs the output adjustment associated with the operation ( S20), the contents of the output adjustment are added to the temporary database as an adjustment history (S21). Thereafter, whenever the user A performs some operation, the user terminal 50 repeats the processes of S19 to S21 described above.

  When the user A selects the adjustment recording button 180 again after the reproduction of the moving image, the user terminal 50 transmits an adjustment history recording request to the distribution server 40 in response to this (S22). The distribution server 40 receives this and confirms the content of the request (S23). Since the user identification is required to record the adjustment history, the distribution server 40 confirms whether or not the user who transmitted this request has been authenticated from the current session state (S24). As a result, if it is found that the user is in an unauthenticated (anonymous access) state, authentication information is requested from the user terminal 50 (S25). In response to this request, when the user A inputs authentication information and transmits it (S26), the distribution server 40 receives it (S27) and inquires of the user master 36 to confirm the validity of the authentication information (S28). If the authentication information is valid, the authentication process is normally completed (S29), and the distribution server 40 notifies the user terminal 50 to that effect (S30). In response to this, when the user terminal 50 transmits the adjustment history recorded in the temporary data to the distribution server 40 (S31), the distribution server 40 receives this (S32) and stores it in the history database 34 (S33). When the adjustment history is stored normally (S34), the distribution server 40 notifies the user terminal 50 to that effect (S35).

  FIG. 15 is a sequence diagram illustrating an operation example when the user B uses the moving image distribution system 100 to view the moving image while reproducing the output adjustment made by another user, for example, the friend user A. For convenience of explanation, it is assumed here that the moving image guided by another user is a free moving image. Hereinafter, it demonstrates along an operation example.

  User B receives an e-mail from user A at user terminal 50. In the body of this e-mail, a URL for viewing a moving image whose output has been adjusted according to user A's preference is provided. When the user B selects this URL, the user terminal 50 activates the web browser to access the URL, and at the same time transmits a distribution request for the moving image specified by the URL to the distribution server 40 (S51). The distribution server 40 receives this, and confirms that distribution of the adjustment history recorded by the user A together with the moving image is requested (S52). Therefore, the user first refers to the moving image database 32 (S53) to acquire the requested moving image bit stream and its configuration information (S54), and further refers to the history database 34 (S55) to correspond to the requested moving image. The adjustment history data of A is acquired (S56). The distribution server 40 distributes the acquired data to the user terminal 50 together with the user interface for reproduction control (S57).

  The user terminal 50 first develops the distributed adjustment history in a local temporary database (S58). Then, the user terminal 50 decodes the distributed video bit stream, and outputs video and audio via the user interface for playback control in the output mode recorded in the adjustment history while referring to the temporary database. The moving image is reproduced (S59). As a result, the same output adjustment as that performed when the user A viewed this moving image is reproduced. That is, the user B can view the moving image in exactly the same state as the user A without operating the operation menu 140 at all.

  When the user B performs an operation related to the video or audio output adjustment with the operation menu 140 during the reproduction of the moving image, the user terminal 50 detects this operation (S60), and simultaneously performs the output adjustment associated with the operation. (S61) The contents of the output adjustment are added to the temporary database as an adjustment history (S62). Thereafter, the output adjustment recorded in the adjustment history of the user A is not reproduced, and the output adjustment is performed based on the operation performed by the user B using the operation menu 140.

  When the user B selects the adjustment recording button 180 after the reproduction of the moving image, the user terminal transmits an adjustment history recording request to the distribution server in response to this (S63). The distribution server 40 receives this and confirms the content of the request (S64), and confirms whether or not the user who transmitted this request from the current session state has been authenticated (S65). Thereafter, similar to the flow of the sequence diagram shown in FIG. 12, the distribution server 40 identifies the user who performs user authentication and records the adjustment history, receives the adjustment history data, and stores it in the history database 34. It will be.

[Second Embodiment: Live Distribution]
FIG. 16 is a configuration diagram of an environment in which the moving image distribution system 200 according to another embodiment operates. The moving image distribution system 200 as the second embodiment is an environment assuming live distribution of moving images, and differs from the first embodiment described above in that the database server 30 is not interposed.

  The video and audio constituting the moving image to be distributed in the moving image distribution system 200 are input to the synthesis and multiplexing apparatus 10 simultaneously with shooting and sound collection. When the synthesis and multiplexing apparatus 10 synthesizes video and multiplexes audio, each generated video / audio signal is input to the encoding apparatus 20 and is encoded in real time. The bit stream encoded by the encoding device 20 is directly transmitted to the distribution server 40. The bit stream transmitted to the distribution server 40 is sequentially distributed to the user terminal 50.

  As described above, in the moving image distribution system 200, the bit stream of the moving image encoded by the encoding device 20 is not accumulated in the database server 30. In addition, the distribution server 40 distributes the bit stream of the moving image synthesized and multiplexed in real time to the user terminal 50.

[Third Embodiment: Hybrid Cast]
FIG. 17 is a configuration diagram of an environment in which the moving image distribution system 300 according to a different embodiment operates. The moving image distribution system 300 as the third embodiment is an environment assuming a hybrid cast. Unlike the first embodiment, the moving image distribution system 300 includes a plurality of means (broadcasting device 42 and data distribution server 44) for distributing moving images, data associated therewith, and the like to user terminals.

The moving image distributed in the moving image distribution system 300 is output from the broadcasting device 42 installed in the broadcasting station, and is transmitted as a broadcast wave to the Internet-compatible television 55 (user terminal as a TV signal receiver) via the radio tower 80. Is done. In addition, data associated with the moving image transmitted from the broadcasting device 42 (for example, moving image configuration information, user interface, etc.) is transmitted from the data distribution server 44 installed in the broadcasting station or service provider via the Internet 70. And transmitted to the Internet-compatible television 55 (a user terminal as a communication device).
Thus, in the moving image distribution system 300, distribution is performed using broadcasting and communication. Further, the Internet-compatible television 55 (hereinafter, abbreviated as “TV 55”) receives a moving image and data accompanying it from these two routes, and via a user interface developed on the screen of the television 55. To achieve video playback.

  Further, when the device 55a connected to the television 55 via a home network or the like is set to be paired with the television 55 and an application linked to the movie transmitted to the television 55 is installed in advance, The distribution system 300 can make the device 55a function auxiliary. At this time, the device 55a can receive the data distributed from the data distribution server 44 from the television 55 as necessary, and can operate the television 55 by developing a user interface on its own display, for example.

  As shown in FIG. 17, in this embodiment, both the broadcasting device 42 and the data distribution server 44 are connected to the database server 30, and the moving image distributed to the television 55 and the accompanying data are acquired from the same server. The moving image database 32 and the other databases 34 and 36 may be arranged on different servers. That is, the broadcasting device 42 and the data distribution server 44 do not need to be connected to the same server, and can be appropriately changed according to the use environment. Also, in the distribution by hybrid cast, it is also possible to distribute a moving image live in real time without using the database server 30 as in the environment of the second embodiment.

  The present invention can be implemented in various modifications without being limited to the above-described embodiments.

  For example, in the above-described embodiment, the databases 32, 34, and 26 used in the video distribution system 100 are provided in the database server 30 and managed on a server different from the distribution server 40. However, the present invention is limited to this form. It is possible to adopt an optimum form according to the operating environment. For example, all the databases 32, 34, and 26 may coexist in the distribution server 40, or some databases may be appropriately arranged on different servers. By doing so, it is possible to reduce the operating cost of the server.

  Further, in the operation menu 140, six types of effect selection buttons 168 to 178 and four types of notification tool activation buttons 190 to 196 are provided. However, these are merely examples, and different types of buttons may be added. Good.

  Regarding the recording of the adjustment history, in order to avoid the influence on the video playback due to the communication delay or the like, the server is triggered once the adjustment recording button 180 is selected after the video playback is completed after being temporarily stored in the local temporary database. The form is recorded on the side. In consideration of the network bandwidth and the like, if there is no serious influence on the reproduction of the moving image, it is possible to mount the recording directly on the server side each time without going through the local recording.

  Further, the adjustment history can be recorded only by one pattern for each moving image for each user, and when the adjustment history for this moving image is recorded on the server side a plurality of times, the latest adjustment history is left. It is also possible to adopt a form in which the adjustment history of each time can be left. In such a form, the operation menu 140 is appropriately changed according to this, and information that can identify each is assigned to each adjustment history recorded on the server side. When the notification unit 148 generates a URL that allows viewing of a moving image reflecting the output adjustment by the user, in addition to the moving image and the user who performed the output adjustment, information that can identify which output adjustment is to be used ( For example, a URL that specifies a moving image ID, user ID, and number of times ID) as a query may be set in the body of the notification tool.

DESCRIPTION OF SYMBOLS 10 Synthesis | combination multiplexing apparatus 20 Encoding apparatus 30 Database server 40 Distribution server (Video distribution means, UI provision means, adjustment recording means)
50 user terminals (reproduction means, output adjustment means, adjustment reproduction means, notification means)
100 video distribution system 140 operation menu (user interface)
150 Main video display area 160 Main video switching button 161 Audio adjustment menu 180 Adjustment record button 182 Adjustment reproduction button

Claims (5)

A moving image distribution means for distributing a bit stream of a moving image multiplexed by one synthesized image obtained by combining a plurality of images in advance in an array and a plurality of sounds related to the plurality of images;
UI providing means for providing configuration information related to the synthesized video and a user interface linked to the configuration information accompanying the delivery of the bitstream by the moving image delivery means;
Via the user interface provided by the UI providing means, the plurality of audios are downmixed and output to the audio output unit while outputting a predetermined area selected from the one synthesized video to the video output unit. A moving image distribution system comprising: a reproducing unit that realizes reproduction of the moving image. The video distribution system according to claim 1,
The video delivery means
Multiple images obtained by photographing the same object or a plurality of mutually related objects and a plurality of sounds collected from or added to the object in association with the object are multiplexed. A video distribution system that distributes a bitstream that has been processed. In the moving image distribution system according to claim 1 or 2,
An output adjusting means for adjusting an output mode of the video output by the video output unit and an output mode of the audio output by the audio output unit;
The UI providing means includes:
A video distribution system, characterized in that a user interface is provided for causing the output adjustment means to adjust an output mode of the video and audio based on a user operation. In the moving image distribution system according to claim 3,
An adjustment recording means for recording an adjustment history which is a transition of the video and audio output modes by the output adjustment means made through the user interface based on a user operation during reproduction of the moving image;
Upon reproduction of the moving image, an adjustment that causes the output adjustment unit to reproduce the adjustment of the video and audio output mode made based on the operation of the user according to the request along the adjustment history recorded by the adjustment recording unit. A video distribution system further comprising reproduction means. The video distribution system according to claim 4,
A notification unit for notifying a user different from the user of identification information that uniquely defines a combination of the moving image and the adjustment history recorded by the adjustment recording unit;
When a distribution request with the identification information notified by the notification means is made,
The video delivery means
Deliver the video bitstream identified by the identification information and the adjustment history for the video,
The adjustment reproduction means includes
When reproducing the moving image, the moving image distribution system reproduces the adjustment of the video and audio output mode made based on the user's operation along the adjustment history.

Images