JP2022132378A - Information processing apparatus, moving image distribution method, and moving image distribution program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information processing apparatus, a moving image distribution method and a moving image distribution program capable of giving a boost to a viewer and a moving image itself during distribution of the moving image of a user who performs singing or the like.

SOLUTION: A server receives a tip input as a display instruction of a tip effect image 60 for decorating a song moving image from a portable terminal of a viewing user, superimposes the tip effect image 60 on a song moving image at a superimposition timing T after receiving the tip input, and distributes the song moving image on which the tip effect image 60 is superimposed, to the portable terminal.

SELECTED DRAWING: Figure 7

COPYRIGHT: (C)2022,JPO&INPIT

Description

The present invention relates to an information processing device, a moving image distribution method, and a moving image distribution program.

In recent years, online services such as websites (web pages, web services), online games, and application software (hereinafter referred to as "apps") delivered to information processing devices such as mobile terminals via computer networks have become widespread. there is

As one of the online services, there is a video posting site where a user himself/herself can post a video shot and other users can watch the video. Performance videos such as singing videos are also posted by users themselves to this video posting site. A video posting site may have a function for a viewing user who views a posted video to evaluate (determine) the video in some way.

Here, Patent Literature 1 discloses a karaoke performance apparatus that changes a singer's image as a result of determination based on the singer's singing ability. In this karaoke performance device, a viewer who listens to a singer's singing inputs the judgment result of the singer's singing ability, and according to the judgment result, a specific image is added to the singer's video. Then, in Patent Document 1, as an addition of an image to the image of the singer according to the determination result, in the case of the determination result that the singing ability is good, the image of the singer, that is, the face of the singer, is covered with a crown on the head. , In the case of the judgment result that the singing ability is bad, the image signal from the camera is changed so that the image of the singer's face is displayed with a peckmark, or the image of the singer's face explodes or is severely deformed. Modulating is disclosed.

JP-A-6-250683

The karaoke device disclosed in Patent Document 1 counts the determination signal indicating the determination result with a counter at the end of singing by the singer, and changes the image of the singer according to the counting result. It is not intended to excite the audience while singing by the singer.

The present invention has been made in view of such circumstances, and an information processing device, a moving image distribution method, and a moving image distribution that can excite viewers and the moving image itself during distribution of a moving image of a user who sings or the like. The purpose is to provide a program.

In order to solve the above problems, the information processing device, moving image distribution method, and moving image distribution program of the present invention employ the following means.

In order to solve the above problems, an ``information processing device'' according to one aspect of the present invention is an information processing device that distributes a moving image to a terminal device so that a viewing user can view the video, wherein the terminal device of the viewing user receiving means for receiving a display instruction of a decoration image that decorates the moving image being distributed from the receiving means; superimposing means for superimposing the decoration image on the moving image after the receiving means receives the display instruction; and superimposing the decoration image on the moving image and a distribution means for distributing the video thus obtained to the terminal device.

In order to solve the above problems, a ``moving image distribution method'' according to one aspect of the present invention includes: a first step of distributing a moving image to a terminal device so that a viewing user can view the moving image; a second step of receiving a display instruction of a decoration image that decorates the moving image inside; a third step of superimposing the decoration image on the moving image after receiving the display instruction; and the moving image on which the decoration image is superimposed. and a fourth step of delivering to the terminal device.

In order to solve the above problems, a ``moving image distribution program'' according to one aspect of the present invention provides a computer provided in an information processing device that distributes a moving image to a terminal device so that a viewing user can view it. receiving means for receiving, from a device, an instruction to display a decoration image that decorates the moving image being distributed; superimposing means for superimposing the decoration image on the moving image after the receiving means receives the display instruction; and It functions as distribution means for distributing the superimposed moving image to the terminal device.

Various technical limitations may be added to the above "information processing apparatus" as exemplified below. In addition, technical limitations to the same effect may be added to the processing steps executed by the "moving image distribution method" and the functions of the "moving image distribution program".

The superimposing means superimposes the decoration image on the moving image at a predetermined timing of the moving image.

A plurality of timings are provided in the moving image, and the superimposing means superimposes the decoration image corresponding to the display instruction on the moving image at the timing after the next time the display instruction is received.

The moving image is a moving image that is recorded while music is being played back, and a plurality of timings are provided according to the music.

A maximum number of the decoration images that can be superimposed on the moving image at the same timing is set in advance, and the superimposing means superimposes the decoration images on the moving image in the order in which the display instructions are received so as not to exceed the maximum number. do.

The superimposing means superimposes all of the decoration images on the moving image at the predetermined timing when there is a decoration image which is not superimposed on the moving image even though the display instruction is received at a predetermined timing immediately before the end of the moving image. do.

Points are set for the decorative images, and the design differs according to the points.

An acquisition unit is provided for acquiring, from the moving image, an image of a scene in which there are relatively many decorative images in the moving image.

ADVANTAGE OF THE INVENTION According to this invention, it has the effect that a viewer and a moving image itself can be excited during distribution of the moving image of the user who sings.

1 is a configuration diagram of a karaoke system according to a first embodiment of the present invention; FIG. 2 is a block diagram showing the electrical configuration of the server according to the first embodiment of the present invention; FIG. 1 is a block diagram showing an electrical configuration of a mobile terminal according to a first embodiment of the invention; FIG. FIG. 4 is a diagram showing a screen display of a mobile terminal when shooting a singing video according to the first embodiment of the present invention; FIG. 4 is a diagram showing a screen display of a mobile terminal when viewing a singing video according to the first embodiment of the present invention; FIG. 7 is a diagram showing a screen display when selecting a coin-throwing effect image according to the first embodiment of the present invention; FIG. 4 is a diagram showing a singing video on which a coin-throwing effect image is superimposed according to the first embodiment of the present invention; FIG. 4 is a schematic diagram showing the superimposition timing of the coin-throwing effect image according to the first embodiment of the present invention; FIG. 4 is a functional block diagram relating to a coin-throwing effect function according to the first embodiment of the present invention; 4 is a flow chart showing the flow of coin-throwing effect processing according to the first embodiment of the present invention. FIG. 10 is a schematic diagram showing the superimposition timing of the coin-throwing effect image according to the second embodiment of the present invention; FIG. 11 is a flow chart showing the flow of coin-throwing effect processing according to the second embodiment of the present invention; FIG. FIG. 10 is a diagram showing a coin-throwing effect image according to another embodiment of the present invention;

An embodiment of an information processing apparatus, a moving image distribution method, and a moving image distribution program according to the present invention will be described below with reference to the drawings.

[1. First Embodiment]
A first embodiment of the present invention will be described below.

In this embodiment, the performer records his or her own performance as a moving image using a portable terminal and transmits the moving image to the server, thereby uploading it to the moving image posting site. A moving image (posted moving image) uploaded to the moving image posting site can be viewed via a mobile terminal. In this embodiment, the performance is singing, the performer is the singing user, and the video uploaded to the video posting site is the singing video. A user who views a singing video uploaded to a video posting site is called a viewing user.

[1-1. Configuration of karaoke system]
FIG. 1 is a schematic configuration diagram of a karaoke system 1 according to this embodiment. The karaoke system 1 includes a communication line 2 , a plurality of mobile terminals 3 (mobile terminals 3A and 3B), and a server 4 .

The communication line 2 forms a computer network, and is, for example, a wide area communication line provided by an electric power company.

The mobile terminal 3 is, for example, an information processing terminal such as a smart phone, a tablet terminal, or a notebook computer, and is used by a user to use online services. The mobile terminal 3 includes a touch panel display 3a for displaying images, a speaker 3b for outputting sound, a microphone 3c for inputting sound, a camera 3d for photographing an object, an earphone terminal 3e for connecting an earphone (not shown), and the like. Prepare. It should be noted that the photographing here means a state in which the camera 3d is functioning and the subject is being displayed on the touch panel display 3a regardless of whether or not recording is being performed. The touch panel display 3a includes, for example, an LCD (Liquid Crystal Display) and a touch sensor. The LCD displays various images, and the touch sensor receives various input operations performed using an indicator such as a finger, stylus, or pen. In the following description, the touch panel display 3a is also called screen 3a.

The server 4 is an information processing device that provides online services to the mobile terminal 3 via the communication line 2 . In the example of FIG. 1, the singing user uploads the singing video to the video posting site by transmitting his/her own singing video (singing video data) from the portable terminal 3A to the server 4 . Then, the viewing user accesses the video posting site using the portable terminal 3B and views the singing video. In addition, the singing user can also view the singing video uploaded by himself/herself by accessing the video posting site using the mobile terminal 3A. It is also possible for the user of the mobile terminal 3B to become a singing user and upload a singing video to a video posting site.

[1-2. Server configuration]
FIG. 2 is a block diagram showing the electrical configuration of the server 4 according to this embodiment.

The server 4 according to the present embodiment includes a CPU (Central Processing Unit) 20 which is a main control unit that controls the operation of the entire server 4, a ROM in which various programs and various data are stored in advance.
(Read Only Memory) 22, RAM (Random Access Memory) 24 used as a work area or the like when various programs are executed by the CPU 20, and HDD (Hard Disk Drive) 26 as storage means for storing various programs and various data. ing.

The HDD 26 stores singing video data transmitted from the mobile terminal 3A, that is, singing video data uploaded to a video posting site, song data indicating songs that can be sung by the singing user, and the like. Note that the storage means is not limited to the HDD 26, and may be other storage media such as semiconductor memory such as flash memory.

Further, the server 4 includes an operation input unit 28 configured with a keyboard, a mouse, etc., for receiving input of various operations, a monitor 30 such as a liquid crystal display device for displaying various images, and a mobile terminal 3 or the like via the communication line 2. It is provided with an external interface 32 that is connected to other information processing devices or the like and performs transmission/reception of various data with the other information processing devices or the like.

The CPU 20 , ROM 22 , RAM 24 , HDD 26 , operation input section 28 , monitor 30 and external interface 32 are electrically connected to each other via a system bus 34 . Therefore, the CPU 20 accesses the ROM 22, the RAM 24, and the HDD 26, grasps the operation state of the operation input unit 28, displays images on the monitor 30, and exchanges various data with other information processing devices or the like via the external interface 32. Transmission and reception can be performed.

[1-3. Electrical Configuration of Mobile Terminal]
FIG. 3 is a functional block diagram showing the electrical configuration of the mobile terminal 3. As shown in FIG.

The mobile terminal 3 includes a main control section 40, a main storage section 42, an auxiliary storage section 44, a communication section 46, and operation buttons 48 in addition to the configuration shown in FIG.

The main control unit 40 is, for example, a CPU, a microprocessor, a DSP (Digital Signal Processor), etc., and controls the overall operation of the mobile terminal 3 .

The main storage unit 42 is composed of, for example, a RAM or a DRAM (Dynamic Random Access Memory), and is used as a work area or the like when the main control unit 40 executes processing based on various programs.

The auxiliary storage unit 44 is, for example, a non-volatile memory such as a flash memory, and stores various data such as images, programs used for processing by the main control unit 40, and the like. The programs stored in the auxiliary storage unit 44 include, for example, an OS (Operating System) for realizing the basic functions of the mobile terminal 3, drivers for controlling various hardware, e-mail and web browsing, and various other programs. It is a program or the like for realizing a function. In addition, the auxiliary storage unit 44 stores in advance an application (hereinafter referred to as a “video posting viewing application”) for shooting and posting singing videos and viewing video posting sites.

The communication unit 46 is, for example, a NIC (Network Interface Controller) and has a function of connecting to the communication line 2 . Note that the communication unit 46 has a function of connecting to a wireless LAN (Local Area Network), a function of connecting to a wireless WAN (Wide Area Network), a function of connecting to a wireless LAN (Local Area Network), a function of connecting to a wireless WAN (Wide Area Network), or a short-range communication such as Bluetooth (registered trademark), instead of or together with the NIC. It may have a function of enabling wireless communication, infrared communication, and the like.

The operation button 48 is provided on the side surface of the mobile terminal 3, and includes a power button for starting or stopping the mobile terminal 3, a volume adjustment button for sound output by the speaker 3b, and the like.

The main control unit 40, main storage unit 42, auxiliary storage unit 44, communication unit 46, operation buttons 48, touch panel display 3a, speaker 3b, microphone 3c, camera 3d, and earphone terminal 3e are interconnected via a system bus 49. is electrically connected to Therefore, the main control unit 40 accesses the main storage unit 42 and the auxiliary storage unit 44, displays an image on the touch panel display 3a, grasps the operating state of the touch panel display 3a and the operation buttons 48 by the user, and transmits sound to the microphone 3c. Input, sound output from the speaker 3b or the earphone connected to the earphone terminal 3e, control of the camera 3d, access to various communication networks and other information processing devices via the communication unit 46, and the like can be performed.

[1-4. Recording of singing videos by singing users]
A case where a singing user shoots a singing video using the mobile terminal 3A will be described.

A singing user activates a video posting viewing application on the mobile terminal 3A when shooting a singing video. When the moving image posting viewing application is activated, the mobile terminal 3A accesses the server 4 storing a plurality of music data. Then, the singing user arbitrarily selects a song for singing by himself/herself from the moving image posting/listening application, and downloads the song data from the server 4 to the portable terminal 3A. Then, the singing user sings by reproducing the music at an arbitrary timing using the video posting viewing application. The video posting viewing application starts playing the music and also starts shooting a video with the camera 3d. That is, the singing video is a video captured by the mobile terminal 3A while the music is played back from the mobile terminal 3A.

The song data is also associated with lyric data, and when the song data is downloaded from the server 4 to the mobile terminal 3A, the associated lyric data is also downloaded to the mobile terminal 3A. In the following description, it is assumed that song data also includes lyrics data.

FIG. 4 is an example of a display state (hereinafter referred to as "screen display") on the screen 3a of the mobile terminal 3A when shooting a singing video.

As shown in FIG. 4, the screen 3a is divided into a lyrics display area 50A and a photographed image display area 50B. The lyric display area 50A includes a lyric image 52 indicating the lyrics of the song sung by the user, a pitch image 54 indicating the pitch of the song, and a progress bar 56 indicating the progress of shooting.

The lyric image 52 and the pitch image 54 are updated as the music progresses. In this embodiment, as an example, the lyrics image 52 and the pitch image 54 are updated by several phrases and displayed in the lyrics display area 50A. Note that the update timing of the lyrics image 52 and the pitch image 54 may be the same or different.

As an example, the lyrics image 52 displays the lyrics in multiple stages (two stages in the example of FIG. 4), and the colors of the lyrics in the upper stage match the progress of the music so that the singing user can grasp the lyrics to be sung at present. Together, it changes from the left end to the right end. When the color change of the lyrics in the upper row reaches the right end, the lyrics in the lower row rise and are displayed in the upper row, and new lyrics are displayed in the lower row. change to the right end.

In the pitch image 54, for example, a plurality of pitch bars 54A are displayed in steps in the horizontal direction according to the intensity of the pitch. Then, the color of the pitch bar 54A changes from the left end to the right end and the pointer 54B moves from the left end to the right end as the music progresses so that the singing user can grasp the pitch of the lyrics to be sung at present. When the color of the pitch bar 54A changes and the pointer 54B reaches the right end, the pitch image 54 showing the next pitch is updated and displayed.

As an example, the progression bar 56 indicates the length of the entire song from the left end to the right end. When the music starts to be reproduced, the pointer 56A indicating the music reproduction position moves from the left end to the right end, and when the pointer 56A reaches the right end, the music ends. The progress bar 56 passed by the pointer 56A is displayed thicker than the position before progress.

Recording of the singing video is started after a predetermined time (for example, 10 seconds) after the singing user selects a song and clicks a recording start button (not shown) displayed on the screen 3a. Also, the start and end of video recording may coincide with the start and end of the music, but this is not limiting, and the video recording may start a predetermined time (for example, 5 seconds) before the start of the music. However, the video recording may end after a predetermined time (for example, 5 seconds) after the end of the music.

A singing user connects an earphone to the earphone terminal 3e, listens to the reproduced music using the earphone, and sings along with the music. The mobile terminal 3A shoots the singing user with the camera 3d and records the singing user's singing with the microphone 3c. That is, the microphone 3c does not acquire the sound of the music being reproduced. Then, the portable terminal 3A records the singing voice of the singing user acquired by the microphone 3c and uses it as singing data.

The singing data may be obtained by extracting the frequency band of human voice by filtering. This filtering process removes noise caused by the surrounding environment of the singing user from the singing data, so that the recorded singing voice of the singing user becomes clearer.

Then, the moving image posting viewing application combines the song data and the singing data with the recorded data to create singing moving image data that can be transmitted to the server 4 . As an example, the user can select one of the following two types of timing for transmitting the song video data to the server 4, that is, timing for uploading to the video posting site.

One is live distribution in which a singing user sings and uploads singing video data to a video posting site in real time. In the live distribution, the viewing user will view the singing by the singing user in real time. The other is non-live distribution in which the singing user uploads the singing video data to the video posting site at any timing after the singing of the song is completed.

When live distribution is to be performed, the singing user performs settings for live distribution before recording the singing video, so that the singing video data is uploaded to the video posting site at the same time as the recording of the video is started. In the case of live distribution, the singing video data may be uploaded to the video posting site without being stored in the mobile terminal 3A.

In addition, as settings for live distribution, the first live distribution setting that allows the viewing user to view the singing video in the live distribution, and the second live distribution that allows the viewing user to view the singing video even after the live distribution Any of the settings can be set by the singing user. That is, in the first live distribution setting, when the live distribution ends, the singing video data is deleted from the server 4, and the viewing user cannot view the live-distributed singing video after the live distribution ends. On the other hand, in the second live distribution, the server 4 continues to store the singing video data even after the live distribution ends, so that the viewing user can view the singing video as a non-live distribution even after the live distribution ends.

In the case of non-live distribution, the singing video data is temporarily stored in the mobile terminal 3A, and the singing user operates the video posting viewing application to upload the singing video to the video posting site at any timing.

[1-5. Viewing of singing video by viewing user]
A case where the viewing user uses the mobile terminal 3B to view the singing video will be described.

When viewing a singing video, the viewing user activates the video posting viewing application on the mobile terminal 3B. When the video posting viewing application is activated, the mobile terminal 3B accesses the server 4 storing a plurality of singing video data, that is, the video posting site. Then, the viewing user selects a singing video to be viewed via the video posting viewing application and displays it on the screen 3a. An example of the method of distributing the singing video to the mobile terminal 3B by the server 4 is streaming distribution.

FIG. 5 is an example of the screen display of the portable terminal 3B when viewing a singing video, and shows the screen display when live distribution is being performed.

A singing video is displayed on the screen 3a, and a singing user display area 50C, a lyrics display area 50D, and a comment input display area 50E are provided. The singing user display area 50C, the lyrics display area 50D, and the comment input display area 50E may be superimposed and displayed on the singing video.

The singing user display area 50C displays the user name of the singing user who posted the singing video, whether or not it is a live distribution, and the name of the song being sung.

The lyrics of the singing video are displayed in the lyrics display area 50D. The displayed lyrics are, for example, a plurality of phrases each, and the color of the lyrics changes from the left end to the right end as the music progresses. As an example, the lyrics display area 50D may display the lyrics in a plurality of stages. In this case, when the change in the color of the lyrics in the upper row reaches the right end, the lyrics in the lower row rise and are displayed in the upper row, and new lyrics are displayed in the lower row. changes from the left end to the right end.

In the comment input display area 50E, a comment input field is displayed, and the comments of the viewing user who is viewing the singing video are displayed together with the user name. As an example, each time a comment is input by the viewing user, the comment is additionally displayed at the top of the comment input display area 50E, and the comments that have been displayed up to that point are moved downward. When the comment cannot be displayed in the comment input display area 50E, a scroll bar (not shown) is displayed on the right side of the comment display area. Comments that were not displayed in 3a are displayed.

Further, the screen 3a displays operation icons 58A to 58D for the viewing user to perform various operations.

The operation icon 58A is an icon that is clicked when the viewing user sympathizes with the singing video that the viewing user is viewing, and the total number of clicks of the operation icon 58A for the singing video is displayed above the operation icon 58A. be.

The operation icon 58B is an icon that is clicked when the viewing user requests a battle (hereinafter referred to as "competition singing") against the singing user who is live-distributing the singing video displayed on the screen 3a. In competition singing, a plurality of singing videos (a first singing video and a second singing video) by different singing users are displayed simultaneously on the screen 3a of the mobile terminal 3B used by the viewing user, and the singing video alternately sings the same song. is. That is, the viewing user who clicked the operation icon 58B becomes the singing user who performs the competition singing.

The operation icon 58C is an icon that is clicked when the viewing user makes various settings for the video posting viewing application.

The operation icon 58D is an icon that is clicked by the viewing user when superimposing a decoration image on the singing video displayed on the screen 3a. Note that the decoration image according to the present embodiment is priced according to its type, and can be purchased by the viewing user by charging. Then, the viewing user clicks the operation icon 58D to superimpose the decoration image on the singing video that the viewing user is viewing. The singing user of the singing video on which the decoration image is superimposed receives money corresponding to the superimposed decoration image from the operator of the video posting site. That is, the superimposition (display instruction) of the decoration image on the singing video by the viewing user corresponds to a so-called tipping for the singing user.

[1-6. Coin effect function]
The video posting and viewing application according to the present embodiment receives an instruction to display a decorative image (hereinafter referred to as a "money effect image") that decorates the singing video being distributed from the viewing user's mobile terminal 3B, and then displays the money effect image. It has a coin-throwing effect function that is superimposed on the singing video. Note that the coin-throwing effect function performs processing for superimposing a coin-throwing effect image on the singing video displayed on the screen 3a after the operation icon 58D displayed on the screen 3a of the portable terminal 3B used by the viewing user is clicked. . That is, since the viewing user decorates the singing video by superimposing a coin-throwing effect image on the singing video that he/she sympathizes with, the coin-throwing effect function is used to liven up the viewers and the singing video itself during the distribution of the singing video. can be done. Note that the coin-throwing effect function is a function executed by the server 4 in response to a display instruction from the viewing user via the video posting viewing application.

[1-6-1. Throwing effect image]
FIG. 6 shows a screen display of the mobile terminal 3B that transitions when the viewing user clicks the operation icon 58D, and a plurality of different coin effect images 60 selectable by the viewing user are displayed. A point is set for each of the coin-throwing effect images 60, and the design differs according to the point. The viewing user selects one of the plurality of coin-throwing effect images 60 displayed on the screen, which can be acquired with the points owned. Note that different designs refer to, for example, different shapes, colors, directions, or the like.

As an example, a charge amount is determined for each point, and the viewing user purchases the points, and the types of the coin effect images 60 that can be selected increase or decrease according to the points owned. When the viewing user selects the coin-throwing effect image 60, the points owned by the viewing user decrease according to the points of the selected coin-throwing effect image 60. - 特許庁It should be noted that the points may be increased by using other online services, etc., in addition to the acquisition by billing.

The coin-throwing effect image 60 illustrated in FIG. 6 is four types of spotlight images that simulate spotlights and are divergent in different directions, and four types of merged spotlight images each having a different color by combining a plurality of spotlight images. By setting the coin-throwing effect image 60 as an image simulating a spotlight, the viewing user can feel as if he/she is participating in a concert or the like. Note that the coin-throwing effect image 60 is not limited to the example shown in FIG. 6, and may be another image.

The time for which the coin effect image 60 is superimposed on the singing video, in other words, the time for which it is displayed on the screen 3a (hereinafter referred to as "display time") is predetermined. good too. For example, the display time of the coin-throwing effect image 60 of 5p shown in FIG. 6 is 0.3 seconds, and the display time of the coin-throwing effect image 60 of 20p is 0.5 seconds. Note that the coin-throwing effect image 60 shown in FIG. 6 imitates a spotlight, so as an example, it dynamically changes within the display time so that the light extends downward from the top of the screen 3a. may

Further, the selection of the coin effect image 60 itself corresponds to an instruction to display the coin effect image 60 on the singing video. In the following description, the display instruction (selection) of the coin-throwing effect image 60 by the viewing user via the mobile terminal 3B is referred to as the coin-throwing input as described above.

Inputting coins is possible using a video posting and viewing application regardless of live distribution or non-live distribution of singing videos. In addition, since the input of the coin is performed while the singing video is being distributed, the viewing user can input the coin when he/she sympathizes with the user, and the viewing user can reproduce the effect-superimposed singing video in real time by the input of the coin. can be viewed at

FIG. 7 shows a singing video superimposed with the coin throwing effect image 60 (hereinafter referred to as "effect superimposed singing video") when the coin throwing input is performed by the viewing user selecting the united spotlight image as the coin throwing effect image 60. ) is an example. The effect-superimposed singing video is stored in the server 4 and distributed through the video posting site so that not only the viewing user who has input the coin but also other viewing users can view it.

As an example, the effect-superimposed singing video has a layer structure, and the coin-throwing effect image 60 is a layer (hereinafter referred to as an "effect layer") different from the singing video. The effect layer is synchronized with the singing video, and a new coin-throwing effect image 60 is added according to the coin-throwing input by the viewing user. In this way, the effect-superimposed singing video has a layer structure, so that the viewing user can change the settings of the video posting viewing application to cancel the superimposition of the coin effect image 60 and sing even if it is an effect-superimposed singing video. You can watch videos.

[1-6-2. Superimposition timing of coin effect image]
The coin-throwing effect function according to the present embodiment, after receiving a coin-throwing input from the viewing user's mobile terminal 3B, inserts the coin-throwing effect image 60 into the singing video at a predetermined timing (hereinafter referred to as "superimposition timing") of the singing video. superimpose. A singing moving image is a moving image captured while a song is played as described above, and a plurality of superimposition timings are provided discretely according to the song. That is, the singing video is associated with music data for which the superimposition timing is set, and the coin-throwing effect function superimposes the coin-throwing effect image 60 on the singing video based on the music data for which the superimposition timing is set.

FIG. 8 is a schematic diagram showing the superimposition timing of the coin-throwing effect image 60. As shown in FIG. In FIG. 8, T indicates the superimposition timing, and P indicates the period during the superimposition timing T (hereinafter referred to as "non-superimposition period").

The superimposition timing T is a timing that is considered appropriate for livening up the singing video, and differs for each song, and is determined by, for example, the tempo, pitch, beat sound, guide melody, measure, melody, tune, rhythm, etc. of the song. . Determination of the superimposition timing T may be determined, for example, manually, or may be determined automatically by analyzing the music using a predetermined algorithm. Moreover, regardless of the tempo of the music, the superimposition timing T may be set near the end of the music. In the example of FIG. 8, the superimposing timing T is determined based on the pitch of the music, and the superimposing timing T is the position where the pitch is relatively high.

The non-superimposing period P is a period during which the coin-throwing effect image 60 is not superimposed on the singing video, and is determined when the superimposing timing T is determined. In the example of FIG. 8, the non-superimposed period P1 is before the superimposed timing T1, the non-superimposed period P2 is between the superimposed timings T1 and T2, and the non-superimposed period P3 is after the superimposed timing T2.

Then, the coin-throwing effect function according to the present embodiment superimposes the coin-throwing effect image 60 corresponding to the coin-throwing input on the singing video at the superimposition timing T after the next time the coin-throwing input is accepted. Specifically, referring to FIG. 8, the coin-throwing effect image 60 corresponding to the coin-throwing input performed during the non-superimposed period P1 is superimposed on the singing video at the superimposition timing T1, and the coin-throwing input performed during the non-superimposed period P2 is superimposed on the singing video. is superimposed on the singing video at the superimposing timing T2, and the coin casting effect image 60 corresponding to the coin input performed during the non-superimposing period P3 is superimposed on the singing video at the next superimposing timing T.

More specifically, the server 4 stores the coin-throwing input information indicating the content of the coin-throwing input received during the non-overlapping period P in the coin-throwing input storage area provided in the RAM 24 or the HDD 26, and displays the coin-throwing effect image indicated by the coin-throwing input information. 60 is set so as to be superimposed on the singing video at the next superimposing timing T. That is, the larger the number of coin-throwing inputs received in the non-superimposing period P, the brighter the coin-throwing effect image 60 to be superimposed at the next superimposing timing T. The coin-throwing input information that has been superimposed on the singing video is deleted from the coin-throwing input storage area. Note that the coin-throwing input information includes the name of the user who performed the coin-throwing input, the coin-throwing effect image 60, the points, the timing of the coin-throwing input (the time from the start of the singing video), and the like.

When there are a plurality of coin-throwing inputs received in one non-superimposing period P, the plurality of coin-throwing effect images 60 corresponding to the received coin-throwing inputs are all superimposed and displayed at the next superimposing timing T. Therefore, when there are a plurality of the same coin-throwing effect images 60, it becomes difficult for the viewing user to recognize each if they are superimposed. Therefore, when there are a plurality of the same coin effect images 60 among the coin effect images 60 superimposed on the singing video at the superimposition timing T, the coin coin effect image 60 is displayed with the display angle shifted. good too. For example, when two coin effect images 60 with a single spotlight directed downward are input as a coin throw input, the coin throw effect image 60 is displayed diagonally to the right from above or to the left from above. is displayed on the screen 3a as a coin-throwing effect image 60 directed toward.

In addition, not limited to this, when there are a plurality of coin effect images 60 to be superimposed at the superimposition timing T, the timing of superimposing the coin effect images 60 on the singing video is shifted by a small amount of time (for example, by 0.1 seconds) and displayed. You may In this case, the superimposition timing T has a certain amount of time width (for example, 5 seconds).

Further, the maximum number of coin effect images 60 that can be superimposed at the superimposing timing T is predetermined, and if the number of coin effect images 60 to be superimposed at the next superimposing timing T exceeds the maximum number, the superimposing timing T of the next and subsequent times is determined. , the coin-throwing effect image 60 may be superimposed on the singing video.

Note that the coin-throwing effect function according to the present embodiment does not limit the number of coin-throwing inputs that a single viewing user can make, and as long as the viewing user has points, it is possible to make multiple coin-throwing inputs for one singing video. , and it is possible to input coins a plurality of times even in one non-superimposed period P.

In this way, the coin-throwing effect function according to the present embodiment superimposes the coin-throwing effect image 60 on the singing video at the superimposition timing T that is appropriate for livening up the singing video. You can enhance the video itself.

[1-6-3. Image cropping function]
The video posting viewing application has a function of obtaining an image of the scene by cutting (copying) a scene in the singing video in which there are relatively many coin-throwing effect images 60 from the singing video (hereinafter referred to as "image cutting function"). may have As a result, the viewing user can easily post exciting scenes in the singing video to other services or the like.

For example, the image cutting function cuts out a singing video of a predetermined period including the superimposition timing T at which the number of superimposed coin effect images 60 is the largest, or cuts out an image corresponding to the superimposition timing T as a still image. An image (moving image or still image) cut out by the image cutting function is transmitted to the mobile terminal 3B and stored in the auxiliary storage unit 44, as an example, but is not limited to this, and may be sent to an SNS registered by the viewing user ( Social Networking Service), etc.

[1-7. Function block of the coin-throwing effect function]
FIG. 9 is a functional block diagram relating to the coin-throwing effect function according to this embodiment. The CPU 20 included in the server 4 includes a coin-throwing input reception unit 70 , a coin-throwing effect superimposing unit 72 , a video distribution unit 74 , and an image clipping unit 76 . Processing executed by each function of the server 4 is implemented by a program stored in the HDD 26 .

The coin-throwing input receiving unit 70 receives a display instruction (throwing input) of the coin-throwing effect image 60 decorating the singing video from the portable terminal 3B of the viewing user to whom the singing video is distributed.

The coin-throwing effect superimposing unit 72 superimposes the coin-throwing effect image 60 on the singing video after the coin-throwing input receiving unit 70 receives the coin-throwing input. Note that the coin-throwing effect superimposing unit 72 according to the present embodiment superimposes the coin-throwing effect image 60 corresponding to the coin-throwing input on the singing video at the superimposition timing T after the next time the coin-throwing input is received.

The video distribution unit 74 distributes the singing video uploaded to the video posting site to the mobile terminal 3B used by the viewing user. In addition, when the coin-throwing effect image 60 is superimposed on the singing video, the video distribution unit 74 distributes the effect-superimposed singing video to the mobile terminal 3B.

The image cutting unit 76 acquires a scene in which the coin-throwing effect image 60 is relatively many in the singing video from the singing video, and acquires a still image of the scene or a moving image including the scene.

[1-8. Flowchart of video distribution processing]
FIG. 10 is a flow chart showing the flow of coin-throwing effect processing according to the present embodiment executed by the CPU 20 provided in the server 4 . A program for executing the coin-throwing effect process according to this embodiment is stored in advance in a predetermined area of the HDD 26 .

First, in step S100, the video distribution unit 74 distributes the singing video to the portable terminal 3B used by the viewing user.

In the next step S102, the coin input receiving unit 70 determines whether or not a coin input from the portable terminal 3B has been received.

In step S104, the coin input reception unit 70 stores coin input information indicating the content of the received coin input in the coin input storage area of the HDD 26 (or RAM 24).

In step S106, the coin-throwing effect superimposing unit 72 determines whether or not the progress of the singing video has reached the next superimposing timing T. If the determination is affirmative, the process proceeds to step S108, and if the determination is negative, the process proceeds to step S114. do.

In step S108, the coin effect superimposing unit 72 determines whether or not the coin input information is stored (memorized) in the coin input storage area. Move to

In step S110, the coin effect superimposing section 72 superimposes the coin effect image 60 indicated by the coin input information stored in the coin input storage area on the singing video.

In the next step S112, the coin effect superimposing unit 72 deletes the superimposed coin entry input information from the coin entry input storage area.

In the next step S114, the video distribution unit 74 determines whether or not the distribution of the singing video has ended. If the determination is negative, the process returns to step S102.

As described above, the server 4 receives a coin-throwing input, which is an instruction to display the coin-throwing effect image 60 that decorates the singing video being distributed, from the mobile terminal 3B of the viewing user, and at the superimposition timing T after receiving the coin-throwing input. A coin-throwing effect image 60 is superimposed on a singing video, and the singing video on which the coin-throwing effect image 60 is superimposed is distributed to a mobile terminal 3B.

As a result, the singing video that the viewing user has sympathized with is decorated with the coin-throwing effect image 60, so that the viewing user and the singing video itself are excited during the distribution of the singing video.

[2. Second Embodiment]
A second embodiment of the present invention will be described below.

The configurations of the karaoke system 1, the mobile terminal 3, and the server 4 according to the present embodiment are the same as the configurations of the karaoke system 1, the mobile terminal 3, and the server 4 according to the first embodiment, so the description thereof is omitted.

[2-1. Superimposition timing of coin effect image]
The coin-throwing effect function according to the present embodiment superimposes the coin-throwing effect image 60 corresponding to the coin-throwing input on the singing video each time the coin-throwing input is received, in other words, without a significant time delay from the reception of the coin-throwing input. . In the coin-throwing effect function according to the present embodiment, the maximum number of coin-throwing effect images 60 that can be superimposed on the singing video at the same timing (hereinafter referred to as "maximum superimposition number") is set in advance. Then, the coin-throwing effect function (throwing-effect superimposing unit 72) superimposes the coin-throwing effect images 60 on the singing video in the order in which the coin-throwing inputs are received so as not to exceed the maximum number of superimpositions.

FIG. 11 is a schematic diagram showing the superimposition timing of the coin-throwing effect image 60 according to this embodiment. FIG. 11(A) is a schematic diagram of the coin input storage area 80 according to the present embodiment, and the coin input storage area 80 stores the coin input information in chronological order when the coin input is received. It should be noted that the numbers at the end of the "input input information" in FIG. 11(A) indicate the order in which input input is accepted.

In FIG. 11(B), the horizontal axis indicates the passage of time, and the vertical axis schematically shows the superimposed coin-throwing effect image 60. The numbers in FIG. 11(B) are shown in FIG. Corresponds to the order of receipt of coin input. As an example, in FIG. 11, the maximum superimposition number is 4, and the display time of coin input information 3, 4, 5, and 6 is approximately half the time of coin input information 1, 2, and 7. FIG.

As exemplified in FIG. 11B, the coin effect images 60 are superimposed on the singing video in the order in which the coin input is received up to the maximum number of superpositions (the coin effect images 60-1 to 60-4). Then, when a given coin effect image 60 ends and the number of superimposed coin effect images 60 on the singing video becomes less than the maximum number, the next coin effect image 60 (the coin effect images 60-5 to 60- The superimposition of 7) is performed in the order in which the coin input is received. That is, the coin-throwing effect function according to the present embodiment superimposes the coin-throwing effect image 60 on the singing video in a first-in first-out (FIFO) manner so as not to exceed the maximum superimposition number, like so-called cueing.

In the example of FIG. 11B, since the display time of the coin effect image 60-3 is shorter than the display time of the coin effect image 60-2, the coin effect image 60- is displayed before the coin effect image 60-2. The superposition of 3 ends. Therefore, the superimposition of the coin effect image 60-5 is started before the superimposition of the coin effect image 60-2 ends.

Thus, the coin-throwing effect function according to the present embodiment superimposes the coin-throwing effect image 60 on the singing video so as not to exceed the maximum superimposition number. As a result, the coin-throwing effect function prevents the singing video from being filled with a large number of coin-throwing effect images 60 and makes it impossible for the viewing user to visually recognize the singing video itself, while the singing video itself is livened up during distribution of the singing video. be able to.

In addition, the coin-throwing effect superimposing unit 72 according to the present embodiment generates a coin-throwing effect image 60 (hereinafter referred to as "non-superimposed coin-throwing effect image") that has received a coin-throwing input but is not superimposed on the singing video at a predetermined timing immediately before the end of the singing video. ) exists, all non-superimposed coin effect images are superimposed on the singing video at a predetermined timing. Note that the predetermined timing is, for example, 10 seconds before the end of the singing video.

Thereby, the coin-throwing effect function can superimpose all the coin-throwing effect images 60 corresponding to the received coin-throwing input on the singing video, and can liven up the singing video at the end of the singing video.

It should be noted that, not limited to this, even if there is a non-superimposed coin effect image immediately before the end of the singing video, the singing video is terminated as it is without superimposing the non-superimposed coin effect image on the singing video at a predetermined timing as described above. You may

[2-2. Flowchart of video distribution processing]
FIG. 12 is a flow chart showing the flow of coin-throwing effect processing according to the present embodiment executed by the CPU 20 provided in the server 4 . A program for executing the coin-throwing effect process according to this embodiment is stored in advance in a predetermined area of the HDD 26 .

First, in step S200, the video distribution unit 74 distributes the singing video to the portable terminal 3B used by the viewing user.

In the next step, step S202, the coin input reception unit 70 determines whether or not the coin input from the mobile terminal 3B is received. .

In step S204, the coin input reception unit 70 stores coin input information indicating the content of the received coin input in the coin input storage area 80 of the HDD 26 (or RAM 24).

In step S206, the coin-throwing effect superimposing unit 72 determines whether or not the coin-throwing input information is stored (memorized) in the coin-throwing input storage area 80. If the determination is affirmative, the process proceeds to step S208. Move to S214.

In step S208, the coin effect superimposing unit 72 determines whether or not the number of coin effect images 60 superimposed on the singing video at the current timing is less than the maximum superimposed number. In the case of determination, the process returns to step S202.

In step S210, the coin effect superimposing unit 72 superimposes the coin effect image 60 on the singing video in the order in which the coin input is received.

In the next step S212, the coin effect superimposing section 72 deletes the superimposed coin entry input information from the coin entry input storage area 80. FIG.

In the next step S214, the video distribution unit 74 determines whether or not the distribution of the singing video has ended. If the determination is negative, the process returns to step S202.

[3. Other embodiments]
As described above, the present invention has been described using each of the above embodiments, but the technical scope of the present invention is not limited to the scope described in each of the above embodiments. Various modifications or improvements can be made to each of the embodiments described above without departing from the gist of the invention, and forms with such modifications or improvements are also included in the technical scope of the present invention. Also, the above embodiments may be combined as appropriate.

For example, in the first embodiment, the mode in which the viewing user selects the coin-throwing effect image 60 to be superimposed on the singing video has been described, but the present invention is not limited to this. For example, when a viewing user performs a coin-throwing input, he/she inputs points without selecting the coin-throwing effect image 60 by himself/herself, and the coin-throwing effect function generates a coin-throwing effect image according to the total points received by the superimposition timing T. 60 may be superimposed on the singing video. Then, for example, according to the total points input in the non-superimposed period P1, the coin effect image 60 to be superimposed at the next superimposing timing T1 changes, and according to the total points input in the non-superimposed period P2, the following The superimposed coin effect image 60 changes at the superimposition timing T2. It should be noted that the points to be totaled are not limited to points input by one viewing user, but points input by a plurality of viewing users.

With reference to FIG. 13, the change of the coin-throwing effect image 60 according to the points will be described. For example, if 5 points are input in a certain non-superimposed period P, at the next superimposition timing T, the coin effect image 60 shown in FIG. 13A is superimposed on the singing video. Also, if the number of points input in a certain non-superimposing period P is 10 points, at the next superimposing timing T, a coin effect image 60 shown in FIG. 13(B) is superimposed on the singing video. Similarly, when the points are 15 points, the coin effect image 60 shown in FIG. 13C is superimposed, and when the points are 20 points, the coin effect image 60 shown in FIG. 13D is superimposed. , when the points are 25 points, a coin-throwing effect image 60 shown in FIG. 13(E) is superimposed. In the example of FIG. 13, the shape of the coin-throwing effect image 60 differs according to the points so that the larger the total number of points, the more gorgeous it becomes. You may

Further, in each of the above-described embodiments, the performance has been described as singing, and the video uploaded to the video posting site has been described as a singing video, but the present invention is not limited to this. For example, the performance may be other than singing, such that the performance is dance and the video uploaded to the video posting site is a dance video.

In addition, the flow of the coin-throwing effect processing described in each of the above embodiments is also an example. You may

3B mobile terminal (terminal device)
4 Server (information processing device)
60 Coin effect image (decorative image)
70 coin input reception unit (reception means)
72 Coin effect superimposing unit (superimposing means)
74 Video distribution unit (distribution means)
76 image cutting unit (acquisition means)
T superimposition timing (predetermined timing)

Claims (10)

An information processing device that distributes a video to a terminal device so that a viewing user can view it,
receiving means for receiving display instructions from the terminal device of the viewing user to display a decoration image that decorates the moving image being distributed;
superimposing means for superimposing the decoration image on the moving image after the receiving means receives the display instruction;
and a distribution means for distributing the moving image on which the decoration image is superimposed to the terminal device. 2. The information processing apparatus according to claim 1, wherein said superimposing means superimposes said decoration image on said moving image at a predetermined timing of said moving image. A plurality of the timings are provided in the moving image,
3. The information processing apparatus according to claim 2, wherein said superimposing means superimposes said decoration image corresponding to said display instruction on said moving image at said timing after the next time said display instruction is received. The moving image is a moving image recorded while a song is being played,
4. The information processing apparatus according to claim 2, wherein a plurality of said timings are provided according to said music. setting in advance a maximum number of the decoration images that can be superimposed on the moving image at the same timing;
5. The information processing apparatus according to any one of claims 1 to 4, wherein said superimposing means superimposes said decoration images on said moving image in the order in which said display instructions are received so as not to exceed said maximum number. The superimposing means superimposes all of the decoration images on the moving image at the predetermined timing when the display instruction is accepted but the decoration image is not superimposed on the moving image at a predetermined timing immediately before the end of the moving image. 6. The information processing apparatus according to claim 5. 7. The information processing apparatus according to any one of claims 1 to 6, wherein a point is set for the decoration image, and a design differs according to the point. 8. The information processing apparatus according to any one of claims 1 to 7, further comprising acquisition means for acquiring, from the moving image, an image of a scene in which there are relatively many decorative images in the moving image. a first step of distributing a moving image to a terminal device so that a viewing user can view it;
a second step of receiving, from the terminal device of the viewing user, an instruction to display a decoration image that decorates the moving image being distributed;
a third step of superimposing the decoration image on the moving image after receiving the display instruction;
and a fourth step of distributing the moving image on which the decoration image is superimposed to the terminal device. A computer provided in an information processing device that distributes video to a terminal device so that a viewing user can view it,
receiving means for receiving display instructions from the terminal device of the viewing user to display a decoration image that decorates the moving image being distributed;
superimposing means for superimposing the decoration image on the moving image after the receiving means receives the display instruction;
A video distribution program for functioning as distribution means for distributing the video on which the decoration image is superimposed to the terminal device.

Images