JP2024056974A - Information processing device, video distribution method, and video distribution program - Google Patents

Abstract

[Problem] The objective is to provide an information processing device, a video distribution method, and a video distribution program that can liven up viewers and the video itself while a user who is singing or the like is distributing the video. [Solution] A server receives a tip input from a mobile device of a viewing user, which is an instruction to display a tip effect image 60 that decorates the singing video, superimposes the tip effect image 60 on the singing video at a superimposition timing T after receiving the tip input, and distributes the singing video with the tip effect image 60 superimposed to the mobile device. [Selected Figure] Figure 7

Description

The present invention relates to an information processing device, a video distribution method, and a video distribution program.

In recent years, online services such as websites (web pages, web services), online games, and application software (hereinafter referred to as "apps") that are distributed to information processing devices such as mobile terminals via computer networks have become widespread.

One type of online service is a video posting site that allows users to post videos that they have filmed themselves and that other users can view. Users also post performance videos, such as singing videos, on this video posting site. Some video posting sites also have a function that allows users who have viewed a posted video to make some kind of evaluation (judgment) of the video.

Patent Document 1 discloses a karaoke performance device that changes the image of a singer as a result of the singer's singing ability. In this karaoke performance device, a viewer who watches the singer sing inputs the result of the judgment of the singer's singing ability, and a specific image is added to the image of the singer depending on the judgment result. Patent Document 1 also discloses that, as an image to be added to the image of the singer depending on the judgment result, when the singing ability is judged to be good, a crown is placed on the image of the singer, i.e., the singer's face, and when the singing ability is judged to be poor, a check mark is displayed on the image of the singer's face, or the image of the singer's face explodes or is severely distorted, etc., by modulating the image signal from the camera.

Japanese Patent Application Laid-Open No. 6-250683

The karaoke device disclosed in Patent Document 1 uses a counter to count a judgment signal indicating the judgment result when the singer finishes singing, and changes the image of the singer according to the counting result, so it does not excite the audience while the singer is singing.

The present invention has been made in consideration of the above circumstances, and aims to provide an information processing device, a video distribution method, and a video distribution program that can liven up viewers and the video itself while a user is streaming a video of themselves singing, etc.

In order to solve the above problems, the information processing device, video distribution method, and video distribution program of the present invention employ the following means.

To solve the above problem, an "information processing device" according to one aspect of the present invention is an information processing device that distributes a video to a terminal device so that the video can be viewed by a viewing user, and includes a reception means for receiving a display instruction from the terminal device of the viewing user for a decorative image that decorates the video being distributed, a superimposition means for superimposing the decorative image on the video after the reception means receives the display instruction, and a distribution means for distributing the video with the decorative image superimposed to the terminal device.

To solve the above problem, a "video distribution method" according to one aspect of the present invention includes a first step of distributing a video to a terminal device so that the video can be viewed by a viewing user, a second step of receiving a display instruction from the terminal device of the viewing user to display a decorative image that decorates the video being distributed, a third step of superimposing the decorative image on the video after receiving the display instruction, and a fourth step of distributing the video with the decorative image superimposed to the terminal device.

In order to solve the above problem, the "video distribution program" of one aspect of the present invention causes a computer provided with an information processing device that distributes videos to a terminal device so that the viewing user can view them to function as a reception means for receiving a display instruction from the terminal device of the viewing user for a decorative image that decorates the video being distributed, a superimposition means for superimposing the decorative image on the video after the reception means receives the display instruction, and a distribution means for distributing the video with the decorative image superimposed to the terminal device.

The above-mentioned "information processing device" may be subject to various technical limitations, as exemplified below. In addition, technical limitations of the same nature may be applied to the processing steps executed by the "video distribution method" or the functions of the "video distribution program."

The superimposing means superimposes the decorative image onto the video at a predetermined timing of the video.

The timing is set multiple times in the video, and the superimposing means superimposes the decorative image on the video in response to the display instruction at the timing from the next time the display instruction is received onward.

The video is recorded while a song is being played, and multiple timings are provided according to the song.

The maximum number of decorative images that can be superimposed on the video at the same time is set in advance, and the superimposing means superimposes the decorative images on the video in the order in which the display instructions are received so as not to exceed the maximum number.

If, at a predetermined timing immediately before the end of the video, there are decorative images for which the display instruction has been received but which have not been superimposed on the video, the superimposing means superimposes all of the decorative images on the video at the predetermined timing.

The decorative image has a set point, and the design varies depending on the point.

The apparatus is provided with an acquisition means for acquiring images of scenes in which the decorative images are relatively numerous from the video.

The present invention has the effect of being able to liven up viewers and the video itself while a user is streaming a video of themselves singing, etc.

1 is a configuration diagram of a karaoke system according to a first embodiment of the present invention. FIG. 2 is a block diagram showing the electrical configuration of the server according to the first embodiment of the present invention. 1 is a block diagram showing an electrical configuration of a mobile terminal according to a first embodiment of the present invention; FIG. 2 is a diagram showing a screen display of a mobile terminal when shooting a singing video according to the first embodiment of the present invention. FIG. 2 is a diagram showing a screen display of a mobile terminal when viewing a singing video according to the first embodiment of the present invention. FIG. 13 is a diagram showing a screen display when selecting a coin throwing effect image according to the first embodiment of the present invention. FIG. 13 is a diagram showing a singing video on which a coin tip effect image is superimposed according to the first embodiment of the present invention. FIG. 13 is a schematic diagram showing the superimposition timing of a coin throwing effect image according to the first embodiment of the present invention. FIG. 11 is a functional block diagram relating to a coin throwing effect function according to the first embodiment of the present invention. 11 is a flowchart showing the flow of a coin throwing effect process according to the first embodiment of the present invention. FIG. 11 is a schematic diagram showing the superimposition timing of a coin throwing effect image according to a second embodiment of the present invention. 13 is a flowchart showing the flow of a coin throwing effect process according to a second embodiment of the present invention. FIG. 13 is a diagram showing a coin throwing effect image according to another embodiment of the present invention.

Below, an embodiment of an information processing device, a video distribution method, and a video distribution program according to the present invention will be described with reference to the drawings.

[1. First embodiment]
A first embodiment of the present invention will now be described.

In this embodiment, a performer records their own performance as a video using a mobile device and transmits it to a server, thereby uploading it to a video sharing site. The video uploaded to the video sharing site (posted video) can be viewed via a mobile device. Note that in this embodiment, the performance is referred to as singing, the performer is referred to as a singing user, and the video uploaded to the video sharing site is referred to as a singing video. Also, a user who views a singing video uploaded to the video sharing site is referred to as a viewing user.

[1-1. Karaoke system configuration]
1 is a schematic diagram of a karaoke system 1 according to the present embodiment. The karaoke system 1 includes a communication line 2, a plurality of mobile terminals 3 (mobile terminals 3A and 3B), and a server 4.

Communication line 2 forms a computer network and is, for example, a wide area communication line provided by an electric power company.

The mobile terminal 3 is, for example, an information processing terminal such as a smartphone, a tablet terminal, or a notebook computer, and is used by users to utilize online services. The mobile terminal 3 includes a touch panel display 3a for displaying images, a speaker 3b for outputting sound, a microphone 3c for inputting sound, a camera 3d for photographing a subject, and an earphone terminal 3e for connecting earphones (not shown). Note that photographing here refers to a state in which the camera 3d is functioning and the subject is displayed on the touch panel display 3a regardless of whether recording is occurring. The touch panel display 3a includes, for example, an LCD (Liquid Crystal Display) and a touch sensor. The LCD displays various images, and the touch sensor accepts various input operations performed using a pointing object such as a finger, a stylus, or a pen. In the following description, the touch panel display 3a is also referred to as a screen 3a.

The server 4 is an information processing device that provides online services to the mobile terminal 3 via the communication line 2. In the example of FIG. 1, the singing user uploads his/her singing video (singing video data) to a video posting site by transmitting the singing video from the mobile terminal 3A to the server 4. The viewing user then accesses the video posting site using the mobile terminal 3B and views the singing video. The singing user can also view the singing video that he/she has uploaded by accessing the video posting site using the mobile terminal 3A. Also, the user of the mobile terminal 3B can become a singing user and upload a singing video to the video posting site.

[1-2. Server configuration]
FIG. 2 is a block diagram showing the electrical configuration of the server 4 according to the present embodiment.

The server 4 according to this embodiment includes a CPU (Central Processing Unit) 20 which is a main control unit that controls the overall operation of the server 4, a ROM in which various programs and various data are stored in advance,
(Read Only Memory) 22, a RAM (Random Access Memory) 24 used as a work area when the CPU 20 executes various programs, and a HDD (Hard Disk Drive) 26 as storage means for storing various programs and various data.

The HDD 26 stores singing video data transmitted from the mobile terminal 3A, i.e., singing video data uploaded to a video posting site, and song data indicating songs that the singing user can sing. Note that the storage means is not limited to the HDD 26, and may be, for example, another storage medium such as a semiconductor memory such as a flash memory.

The server 4 further includes an operation input unit 28, which is composed of a keyboard, a mouse, etc. and accepts input of various operations, a monitor 30, such as a liquid crystal display device, which displays various images, and an external interface 32, which is connected to other information processing devices, such as a mobile terminal 3, via the communication line 2 and transmits and receives various data to and from the other information processing devices.

The CPU 20, ROM 22, RAM 24, HDD 26, operation input unit 28, monitor 30, and external interface 32 are electrically connected to each other via a system bus 34. Therefore, the CPU 20 can access the ROM 22, RAM 24, and HDD 26, grasp the operation status of the operation input unit 28, display images on the monitor 30, and send and receive various data to and from other information processing devices, etc. via the external interface 32.

[1-3. Electrical configuration of mobile terminal]
FIG. 3 is a functional block diagram showing the electrical configuration of the mobile terminal 3. As shown in FIG.

In addition to the components shown in FIG. 1, the mobile terminal 3 includes a main control unit 40, a main memory unit 42, an auxiliary memory unit 44, a communication unit 46, and an operation button 48.

The main control unit 40 is, for example, a CPU, a microprocessor, a DSP (Digital Signal Processor), etc., and controls the overall operation of the mobile terminal 3.

The main memory unit 42 is composed of, for example, RAM or DRAM (Dynamic Random Access Memory), and is used as a work area when the main control unit 40 executes processes based on various programs.

The auxiliary storage unit 44 is, for example, a non-volatile memory such as a flash memory, and stores various data such as images, and programs used in the processing of the main control unit 40. The programs stored in the auxiliary storage unit 44 are, for example, an operating system (OS) for implementing basic functions of the mobile terminal 3, drivers for controlling various hardware, and programs for implementing e-mail, web browsing, and various other functions. In addition, the auxiliary storage unit 44 prestores an application for shooting and posting singing videos and viewing video posting sites (hereinafter referred to as a "video posting and viewing application").

The communication unit 46 is, for example, a network interface controller (NIC), and has a function of connecting to the communication line 2. Note that the communication unit 46 may have a function of connecting to a wireless local area network (LAN), a function of connecting to a wireless wide area network (WAN), and a function of enabling short-range wireless communication such as Bluetooth (registered trademark), infrared communication, etc., instead of or together with the NIC.

The operation buttons 48 are provided on the side of the mobile terminal 3 and include a power button for starting and stopping the mobile terminal 3 and a volume adjustment button for the sound output by the speaker 3b.

The main control unit 40, main memory unit 42, auxiliary memory unit 44, communication unit 46, operation buttons 48, touch panel display 3a, speaker 3b, microphone 3c, camera 3d, and earphone terminal 3e are electrically connected to each other via a system bus 49. Therefore, the main control unit 40 can access the main memory unit 42 and auxiliary memory unit 44, display images on the touch panel display 3a, grasp the operation state of the touch panel display 3a and the operation buttons 48 by the user, input sound to the microphone 3c, output sound from the speaker 3b or an earphone connected to the earphone terminal 3e, control the camera 3d, and access various communication networks and other information processing devices via the communication unit 46.

[1-4. Filming of singing videos by singing users]
A case where a singing user shoots a singing video using the mobile terminal 3A will be described.

When a singing user shoots a singing video, the singing user launches a video posting and viewing app on the mobile terminal 3A. When the video posting and viewing app launches, the mobile terminal 3A accesses the server 4 that stores multiple pieces of music data. The singing user then arbitrarily selects a piece of music to sing from the video posting and viewing app, and downloads the music data from the server 4 to the mobile terminal 3A. The singing user then plays the music at a timing of their choice using the video posting and viewing app, and sings. The video posting and viewing app starts playing the music, and starts shooting the video with the camera 3d. In other words, the singing video is a video shot by the mobile terminal 3A while the music is being played from the mobile terminal 3A.

Note that song data is also associated with lyric data, and when song data is downloaded from server 4 to mobile terminal 3A, the associated lyric data is also downloaded to mobile terminal 3A. Note that in the following explanation, song data is considered to include lyric data.

Figure 4 shows an example of the display state (hereinafter referred to as "screen display") on the screen 3a of the mobile terminal 3A when shooting a singing video.

As shown in FIG. 4, the screen 3a is divided into a lyrics display area 50A and a captured image display area 50B. The lyrics display area 50A includes a lyrics image 52 that shows the lyrics of the song being sung by the user, a pitch image 54 that shows the pitch of the song, and a progress bar 56 that shows the progress of the capture.

The lyric image 52 and the pitch image 54 are updated according to the progress of the song. In the present embodiment, as an example, the lyric image 52 and the pitch image 54 are updated every few phrases and displayed in the lyrics display area 50A. Note that the update timing of the lyric image 52 and the pitch image 54 may be the same or different.

As an example, the lyrics image 52 displays lyrics in multiple rows (two rows in the example of FIG. 4), and the color of the lyrics in the upper row changes from the left end to the right end as the song progresses so that the singing user can understand the lyrics that should currently be sung. When the color change of the lyrics in the upper row reaches the right end, the lyrics in the lower row rises and are displayed in the upper row, and new lyrics are displayed in the lower row, and the color of the lyrics in the upper row changes again from the left end to the right end as the song progresses.

As an example, the pitch image 54 is displayed with multiple pitch bars 54A in a staircase pattern in the left-right direction according to the strength of the pitch. Then, in order to enable the singing user to grasp the pitch of the lyrics that should currently be sung, the color of the pitch bars 54A changes from the left end to the right end as the song progresses, and the pointer 54B moves from the left end to the right end. When the color of the pitch bars 54A changes and the pointer 54B reaches the right end, the pitch image 54 indicating the next pitch is updated and displayed.

As an example, the length of the progress bar 56 from the left end to the right end indicates the length of the entire song. When playback of the song begins, a pointer 56A indicating the playback position of the song moves from the left end to the right end, and when the pointer 56A reaches the right end, the song ends. Note that the part of the progress bar 56 that the pointer 56A has passed through is displayed thicker than its position before the progress.

Recording of a singing video starts a predetermined time (e.g., 10 seconds) after the singing user selects a song and clicks a recording start button (not shown) displayed on screen 3a. The start and end of video recording may coincide with the start and end of the song, but this is not limited thereto. Video recording may start a predetermined time before the start of the song (e.g., 5 seconds before), or end a predetermined time after the end of the song (e.g., 5 seconds after).

The singing user connects earphones to the earphone terminal 3e, listens to the played music using the earphones, and sings along with the music. The mobile terminal 3A captures an image of the singing user with the camera 3d, and records the singing of the singing user with the microphone 3c. In other words, the microphone 3c does not capture the sound of the played music. The mobile terminal 3A then records the singing user's singing voice captured by the microphone 3c, and uses this as singing data.

The singing data may be obtained by extracting the frequency band of the human voice through a filtering process. This filtering process removes noise caused by the singing user's surrounding environment from the singing data, making the recorded singing voice of the singing user clearer.

The video posting and viewing app then combines the music data and singing data with the recording data to generate singing video data that can be sent to server 4. As an example, the user can select one of the following two types of timing for sending the singing video data to server 4, i.e., the timing for uploading the data to the video posting site.

One is live streaming, in which the singing user uploads singing video data to a video sharing site in real time while singing. In live streaming, viewing users watch the singing user sing in real time. The other is non-live streaming, in which the singing user uploads singing video data to a video sharing site at a timing of their choice after finishing singing the song.

When performing live streaming, the singing user sets up the live streaming before recording the singing video, and uploads the singing video data to the video posting site when video recording starts. Note that in the case of live streaming, the singing video data may be uploaded to the video posting site without being stored in the mobile terminal 3A.

When live streaming is performed, the singing user can set either a first live streaming setting that allows viewing users to view the singing video during live streaming, or a second live streaming setting that allows viewing users to view the singing video even after the live streaming. That is, with the first live streaming setting, the singing video data is deleted from the server 4 when the live streaming ends, and the viewing user cannot view the live-streamed singing video after the live streaming ends. On the other hand, with the second live streaming setting, the server 4 continues to store the singing video data even after the live streaming ends, so the viewing user can view the singing video as a non-live stream even after the live streaming ends.

When non-live streaming is performed, the singing video data is temporarily stored in the mobile terminal 3A, and the singing user operates the video posting and viewing app to upload the singing video to the video posting site at any time.

[1-5. Viewing of singing videos by viewers]
A case where a viewing user views a singing video using the mobile terminal 3B will be described.

When the viewing user wishes to view a singing video, the viewing user launches a video posting and viewing application on the mobile terminal 3B. When the video posting and viewing application is launched, the mobile terminal 3B accesses a server 4 that stores multiple singing video data, i.e., a video posting site. The viewing user then selects the singing video they wish to view via the video posting and viewing application and displays it on the screen 3a. Note that one example of the method by which the server 4 distributes the singing video to the mobile terminal 3B is streaming distribution.

Figure 5 shows an example of the screen display of mobile terminal 3B when watching a singing video, showing the screen display when live streaming is taking place.

The screen 3a displays a singing video and also has a singing user display area 50C, a lyrics display area 50D, and a comment input display area 50E. The singing user display area 50C, the lyrics display area 50D, and the comment input display area 50E may be displayed superimposed on the singing video.

The singing user display area 50C displays the username of the singing user who posted the singing video, whether it is a live broadcast or not, and the name of the song being sung.

The lyrics of the singing video are displayed in the lyrics display area 50D. As an example, the lyrics displayed are multiple phrases at a time, and the color of the lyrics changes from the left end to the right end as the music progresses. As an example, the lyrics display area 50D may display lyrics in multiple rows. In this case, when the color change of the lyrics in the upper row reaches the right end, the lyrics in the lower row rises and is displayed in the upper row, and new lyrics are displayed in the lower row, and the color of the lyrics in the upper row changes again from the left end to the right end as the music progresses.

In the comment input display area 50E, a comment input field is displayed, and comments from viewing users who are watching the singing video are displayed together with their user names. As an example, each time a comment is input from a viewing user, that comment is added and displayed at the top of the comment input display area 50E, and comments that were previously displayed are moved down. If there are more comments than can be displayed in the comment input display area 50E, a scroll bar (not shown) is displayed on the right side of the comment display area, and comments that were not previously displayed on the screen 3a can be displayed by the viewing user operating the scroll bar.

In addition, operation icons 58A to 58D are displayed on screen 3a to allow the viewing user to perform various operations.

Operation icon 58A is an icon that is clicked by the viewing user when the viewing user empathizes with the singing video that the viewing user is viewing, and the total number of clicks of operation icon 58A for that singing video is displayed above operation icon 58A.

Operation icon 58B is an icon that is clicked when a viewing user challenges a singing user who is live streaming the singing video displayed on screen 3a to a battle (hereinafter referred to as a "singing battle"). A singing battle involves simultaneously displaying multiple singing videos (first singing video, second singing video) by different singing users on screen 3a of the mobile terminal 3B used by the viewing user, and the singing videos take turns singing the same song. In other words, the viewing user who clicks operation icon 58B becomes a singing user who will engage in a singing battle.

Operation icon 58C is an icon that is clicked by the viewing user to configure various settings for the video posting and viewing app.

Operation icon 58D is an icon that is clicked by the viewing user when a decorative image is to be superimposed on the singing video displayed on screen 3a. Note that the decorative images in this embodiment have a price determined according to their type, and can be purchased by the viewing user for a fee. Then, the viewing user superimposes a decorative image on the singing video that he or she is viewing by clicking operation icon 58D. The singing user of the singing video on which the decorative image is superimposed receives money according to the superimposed decorative image from the operator of the video posting site. In other words, the superimposition (display instruction) of a decorative image on a singing video by the viewing user corresponds to a so-called tipping to the singing user.

[1-6. Coin throwing effect function]
The video posting and viewing application according to the present embodiment has a coin-throwing effect function that superimposes a coin-throwing effect image on a singing video after receiving a display instruction from the viewing user's mobile terminal 3B of a decorative image (hereinafter referred to as a "coin-throwing effect image") that decorates the singing video being distributed. The coin-throwing effect function performs a process of superimposing a coin-throwing effect image on a singing video displayed on the screen 3a after an operation icon 58D displayed on the screen 3a of the mobile terminal 3B used by the viewing user is clicked. That is, the viewing user decorates a singing video by superimposing a coin-throwing effect image on a singing video that the viewing user sympathizes with, so that the coin-throwing effect function can liven up the viewers and the singing video itself during the distribution of the singing video. The coin-throwing effect function is a function executed by the server 4 in response to a display instruction from the viewing user via the video posting and viewing application.

[1-6-1. Coin throwing effect image]
6 shows a screen display of the mobile terminal 3B to which the viewing user transitions when the viewing user clicks the operation icon 58D, and displays a plurality of different tip effect images 60 that the viewing user can select. Points are set for each tip effect image 60, and the design differs according to the points. The viewing user selects one of the plurality of tip effect images 60 displayed on the screen that can be acquired with the points owned. Note that different designs refer to, for example, different shapes, colors, orientations, etc.

As an example, a charge is set for each point, and viewing users purchase points, and the number of selectable tip effect images 60 increases or decreases depending on the number of points they own. When a tip effect image 60 is selected by a viewing user, the number of points owned by the viewing user decreases according to the number of points of the selected tip effect image 60. Note that points can be obtained not only by charging a fee, but also by increasing the number of points by using other online services, etc.

The coin tip effect image 60 illustrated in FIG. 6 is four types of spotlight images that mimic spotlights and have different directions, and four types of combined spotlight images that combine multiple spotlight images and have different colors. By using an image that mimics a spotlight as the coin tip effect image 60, the viewing user can get the feeling that they are participating in a concert or the like. Note that the coin tip effect image 60 is not limited to the example in FIG. 6, and may be other images.

The time for which the coin tip effect image 60 is superimposed on the singing video, in other words the time for which it is displayed on the screen 3a (hereinafter referred to as the "display time"), is predetermined, and the display time may vary depending on the coin tip effect image 60. For example, the display time of the 5p coin tip effect image 60 shown in FIG. 6 is 0.3 seconds, and the display time of the 20p coin tip effect image 60 is 0.5 seconds. Note that since the coin tip effect image 60 shown in FIG. 6 imitates a spotlight, it may change dynamically within the display time, for example so that the light extends from the top to the bottom of the screen 3a.

The selection of the coin tip effect image 60 itself corresponds to an instruction to display the coin tip effect image 60 on the singing video. In the following description, the instruction (selection) by the viewing user to display the coin tip effect image 60 via the mobile terminal 3B is referred to as a coin tip input, as described above.

Tipping is possible using a video posting and viewing app regardless of whether the singing video is live or not. Tipping is also done while the singing video is being streamed, so viewers can tip when they feel like it, and can watch the effects-overlaid singing video in real time based on their tipping input.

Figure 7 shows an example of a singing video on which a tip effect image 60 is superimposed (hereinafter referred to as an "effect-superimposed singing video") when a viewing user inputs a tip by selecting a combined spotlight image as the tip effect image 60. The effect-superimposed singing video is stored on the server 4 and distributed so that it can be viewed not only by the viewing user who inputs a tip via the video posting site, but also by other viewing users.

As an example, the effect-superimposed singing video has a layer structure, and the tip effect image 60 is a layer (hereinafter referred to as the "effect layer") that is different from the singing video. The effect layer is synchronized with the singing video, and a new tip effect image 60 is added in response to a tip input by the viewing user. Because the effect-superimposed singing video has a layer structure in this way, the viewing user can change the settings of the video posting and viewing app to remove the superimposition of the tip effect image 60 and watch the singing video, even if it is an effect-superimposed singing video.

[1-6-2. Timing of coin throwing effect image overlay]
The tipping effect function according to this embodiment superimposes a tipping effect image 60 on a singing video at a predetermined timing (hereinafter referred to as "superimposition timing") of the singing video after accepting a tipping input from the viewing user's mobile terminal 3B. The singing video is a video filmed while a song is being played as described above, and multiple superimposition timings are provided discretely according to the song. That is, the singing video is associated with song data in which the superimposition timing is set, and the tipping effect function superimposes the tipping effect image 60 on the singing video based on the song data in which the superimposition timing is set.

Figure 8 is a schematic diagram showing the superimposition timing of the coin throwing effect image 60. In Figure 8, T indicates the superimposition timing, and P indicates the period between the superimposition timings T (hereinafter referred to as the "non-superimposition period").

The superimposition timing T is a timing that is considered appropriate for livening up a singing video and differs for each song, and is determined, for example, by the tempo, pitch, beat sound, guide melody, bars, melody, tune, rhythm, etc. of the song. The superimposition timing T may be determined, for example, manually, or automatically by analyzing the song using a predetermined algorithm. Also, regardless of the tempo, etc. of the song, the superimposition timing T may be set near the end of the song. In the example of Figure 8, the superimposition timing T is determined based on the pitch of the song, and the position where the pitch is relatively high is set as the superimposition timing T.

The non-overlapping period P is a period during which the coin tip effect image 60 is not superimposed on the singing video, and is determined when the superimposing timing T is determined. In the example of FIG. 8, the non-overlapping period P1 is before the superimposing timing T1, the non-overlapping period P2 is between the superimposing timings T1 and T2, and the non-overlapping period P3 is after the superimposing timing T2.

Then, the tip effect function according to this embodiment superimposes a tip effect image 60 corresponding to a tip input on the singing video at the superimposition timing T from the next time onward after the tip input is accepted. Explaining specifically with reference to FIG. 8, the tip effect image 60 corresponding to a tip input made in the non-overlapping period P1 is superimposed on the singing video at the superimposition timing T1, the tip effect image 60 corresponding to a tip input made in the non-overlapping period P2 is superimposed on the singing video at the superimposition timing T2, and the tip effect image 60 corresponding to a tip input made in the non-overlapping period P3 is superimposed on the singing video at the next superimposition timing T.

More specifically, the server 4 stores the tip input information indicating the content of the tip input received during the non-overlapping period P in a tip input storage area provided in the RAM 24 or HDD 26, and sets the tip input effect image 60 indicated by the tip input information to be superimposed on the singing video at the next superimposition timing T. In other words, the more tip inputs received during the non-overlapping period P, the more colorful the tip effect image 60 superimposed at the next superimposition timing T will be. The tip input information that has been superimposed on the singing video is deleted from the tip input storage area. The tip input information includes the name of the user who made the tip input, the tip effect image 60, points, the timing of the tip input (time from the start of the singing video), etc.

When multiple coin-throwing inputs are received during one non-overlapping period P, the multiple coin-throwing effect images 60 corresponding to the received coin-throwing inputs are all displayed overlapping at the next overlay timing T. For this reason, when multiple coin-throwing effect images 60 of the same kind exist, it becomes difficult for the viewing user to recognize each of them when they are overlaid. Therefore, when multiple coin-throwing effect images 60 of the same kind exist among the coin-throwing effect images 60 superimposed on the singing video at the overlay timing T, the coin-throwing effect function may shift the display angle of one of the coin-throwing effect images 60. For example, when two coin-throwing effect images 60 of a single spotlight pointing directly downward from above are input as coin-throwing inputs, the coin-throwing effect function displays one of the coin-throwing effect images 60 on the screen 3a as a coin-throwing effect image 60 pointing diagonally right from above or diagonally left from above.

Furthermore, without being limited to this, when there are a plurality of coin-throwing effect images 60 to be superimposed at the superimposition timing T, the timings at which the coin-throwing effect images 60 are superimposed on the singing video may be shifted by a small time (for example, 0.1 seconds). In this case, the superimposition timing T has a certain time span (for example, 5 seconds).

In addition, the maximum number of coin-throwing effect images 60 that can be superimposed at the superimposition timing T may be determined in advance, and if the number of coin-throwing effect images 60 superimposed at the next superimposition timing T exceeds the maximum number, the coin-throwing effect images 60 may be superimposed on the singing video at the superimposition timing T after the next one.

The tip effect function according to this embodiment does not place any restrictions on the number of tip inputs that a single viewing user can make, and as long as the viewing user has points, multiple tip inputs are possible for one singing video. Also, multiple tip inputs are possible even during one non-overlapping period P.

In this way, the coin tip effect function according to this embodiment superimposes the coin tip effect image 60 onto the singing video at the superimposition timing T that is deemed appropriate for livening up the singing video, thereby livening up viewers and the singing video itself during the distribution of the singing video.

[1-6-3. Image cropping function]
The video posting and viewing app may have a function of acquiring an image of a scene in a singing video in which the coin tip effect image 60 is relatively large by cutting (copying) the scene from the singing video (hereinafter referred to as an "image cutting function"). This allows a viewing user to easily post an exciting scene from the singing video to another service or the like.

The image cropping function, for example, crops out a singing video of a predetermined period including the superimposition timing T at which the number of superimposed coin tip effect images 60 is the greatest, or crops out an image corresponding to the superimposition timing T as a still image. The image (video or still image) cropped by the image cropping function is, for example, transmitted to the mobile terminal 3B and stored in the auxiliary storage unit 44, but is not limited to this and may also be posted to a social networking service (SNS) to which the viewing user is registered.

[1-7. Functional block of coin tip effect function]
9 is a functional block diagram of the tip effect function according to this embodiment. The CPU 20 of the server 4 includes a tip input receiving unit 70, a tip effect superimposing unit 72, a video delivery unit 74, and an image cropping unit 76. The processes executed by each function of the server 4 are realized by a program stored in the HDD 26.

The tip input receiving unit 70 receives a display instruction (tip input) for a tip effect image 60 that decorates the singing video from the mobile terminal 3B of the viewing user to whom the singing video is distributed.

The tip effect superimposition unit 72 superimposes the tip effect image 60 onto the singing video after the tip input acceptance unit 70 accepts the tip input. Note that the tip effect superimposition unit 72 according to this embodiment superimposes the tip effect image 60 corresponding to the tip input onto the singing video at the superimposition timing T from the next time onward after the tip input is accepted.

The video delivery unit 74 delivers the singing video uploaded to the video posting site to the mobile terminal 3B used by the viewing user. If a tip effect image 60 is superimposed on the singing video, the video delivery unit 74 delivers the effect-superimposed singing video to the mobile terminal 3B.

The image cropping unit 76 acquires scenes from the singing video in which there are relatively many coin tip effect images 60, and acquires still images of those scenes or video images including those scenes.

[1-8. Flowchart of video distribution processing]
10 is a flowchart showing the flow of the throwing effect process according to this embodiment, which is executed by the CPU 20 of the server 4. A program for executing the throwing effect process according to this embodiment is stored in a predetermined area of the HDD 26 in advance.

First, in step S100, the video distribution unit 74 distributes the singing video to the mobile terminal 3B used by the viewing user.

In the next step S102, the coin input acceptance unit 70 determines whether or not a coin input from the mobile terminal 3B has been accepted, and if the judgment is affirmative, the process proceeds to step S104, and if the judgment is negative, the process proceeds to step S106.

In step S104, the tip input reception unit 70 stores tip input information indicating the content of the received tip input in a tip input storage area of the HDD 26 (or RAM 24).

In step S106, the coin tip effect superimposition unit 72 determines whether the progression of the singing video has reached the next superimposition timing T, and if the determination is positive, the process proceeds to step S108, and if the determination is negative, the process proceeds to step S114.

In step S108, the tip effect superimposition unit 72 determines whether or not tip input information is stored (memorized) in the tip input storage area, and if the determination is positive, the process proceeds to step S110, and if the determination is negative, the process proceeds to step S114.

In step S110, the coin-throwing effect superimposition unit 72 superimposes the coin-throwing effect image 60 indicated by the coin-throwing input information stored in the coin-throwing input storage area onto the singing video.

In the next step S112, the tip effect superimposition unit 72 deletes the superimposed tip input information from the tip input storage area.

In the next step S114, the video distribution unit 74 determines whether or not the distribution of the singing video has ended, and if the determination is negative, the process returns to step S102, whereas if the determination is positive, the tipping effect process ends.

As described above, the server 4 accepts a tip input, which is an instruction to display a tip effect image 60 that decorates the singing video being distributed from the viewing user's mobile terminal 3B, superimposes the tip effect image 60 on the singing video at superimposition timing T after accepting the tip input, and distributes the singing video with the tip effect image 60 superimposed to the mobile terminal 3B.

As a result, singing videos that have resonated with viewers are decorated with the tip effect image 60, which creates excitement among viewers and the singing video itself during distribution of the singing video.

[2. Second embodiment]
A second embodiment of the present invention will now be described.

The configurations of the karaoke system 1, mobile terminal 3, and server 4 according to this embodiment are similar to those of the karaoke system 1, mobile terminal 3, and server 4 according to the first embodiment, so a description thereof will be omitted.

[2-1. Timing of coin throwing effect image overlay]
The coin-throwing effect function according to this embodiment superimposes a coin-throwing effect image 60 corresponding to the coin-throwing input on the singing video every time a coin-throwing input is received, in other words, without much time delay from the reception of the coin-throwing input. Note that the coin-throwing effect function according to this embodiment has a preset maximum number of coin-throwing effect images 60 that can be superimposed on the singing video at the same timing (hereinafter referred to as the "maximum number of superimpositions"). Then, the coin-throwing effect function (the coin-throwing effect superimposition unit 72) superimposes the coin-throwing effect images 60 on the singing video in the order of reception of the coin-throwing input so as not to exceed the maximum number of superimpositions.

Figure 11 is a schematic diagram showing the superimposition timing of the tip effect image 60 according to this embodiment. Figure 11 (A) is a schematic diagram of the tip input storage area 80 according to this embodiment, and the tip input storage area 80 stores tip input information in the chronological order in which the tip input was received. Note that the numbers at the end of the "throw input information" in Figure 11 (A) indicate the order in which the tip input was received.

In FIG. 11(B), the horizontal axis indicates the passage of time, and the vertical axis shows a schematic diagram of the superimposed tip effect image 60, with the numbers in FIG. 11(B) corresponding to the order in which the tip inputs shown in FIG. 11(A) were accepted. Also, as an example, in FIG. 11, the maximum number of superimpositions is 4, and the display time for tip input information 3, 4, 5, and 6 is approximately half that of tip input information 1, 2, and 7.

As shown in FIG. 11B, the coin-throwing effect images 60 are superimposed on the singing video in the order in which the coin-throwing input was received (coin-throwing effect images 60-1 to 60-4) until the maximum number of superimpositions is reached. Then, when a coin-throwing effect image 60 ends and the number of coin-throwing effect images 60 superimposed on the singing video falls below the maximum number of superimpositions, the next coin-throwing effect image 60 (coin-throwing effect images 60-5 to 60-7) is superimposed in the order in which the coin-throwing input was received. In other words, the coin-throwing effect function according to this embodiment superimposes the coin-throwing effect images 60 on the singing video in a first-in first-out (FIFO) order so as not to exceed the maximum number of superimpositions, similar to so-called queuing.

In the example of FIG. 11(B), the display time of the coin throwing effect image 60-3 is shorter than the display time of the coin throwing effect image 60-2, so the superimposition of the coin throwing effect image 60-3 ends before the superimposition of the coin throwing effect image 60-2. Therefore, the superimposition of the coin throwing effect image 60-5 starts before the superimposition of the coin throwing effect image 60-2 ends.

In this way, the coin tip effect function according to this embodiment superimposes the coin tip effect image 60 onto the singing video so as not to exceed the maximum number of superimpositions. This prevents the singing video from being filled with a large number of coin tip effect images 60, which would prevent the viewing user from being able to see the singing video itself, while also boosting the viewing user and the singing video itself during the distribution of the singing video.

In addition, if there is a coin throwing effect image 60 (hereinafter referred to as an "unsuperimposed coin throwing effect image") that has received a coin throwing input but has not been superimposed on the singing video at a predetermined timing just before the end of the singing video, the coin throwing effect superimposition unit 72 in this embodiment superimposes all of the unsuperimposed coin throwing effect images on the singing video at the predetermined timing. Note that the predetermined timing is, for example, 10 seconds before the end of the singing video.

As a result, the tip effect function can superimpose all tip effect images 60 corresponding to the received tip inputs onto the singing video, and can liven up the singing video at the end of the singing video.

However, this is not limited to the above, and even if an unsuperimposed coin throwing effect image is present immediately before the end of the singing video, the singing video may be ended without superimposing the unsuperimposed coin throwing effect image on the singing video at a predetermined timing as described above.

[2-2. Flowchart of video distribution processing]
12 is a flowchart showing the flow of the throwing effect process according to this embodiment, which is executed by the CPU 20 of the server 4. A program for executing the throwing effect process according to this embodiment is stored in a predetermined area of the HDD 26 in advance.

First, in step S200, the video distribution unit 74 distributes the singing video to the mobile terminal 3B used by the viewing user.

In the next step S202, the coin input acceptance unit 70 determines whether or not a coin input from the mobile terminal 3B has been accepted, and if the judgment is positive, the process proceeds to step S204, and if the judgment is negative, the process proceeds to step S206.

In step S204, the tip input reception unit 70 stores tip input information indicating the content of the received tip input in the tip input storage area 80 of the HDD 26 (or RAM 24).

In step S206, the tip effect superimposition unit 72 determines whether or not tip input information is stored (memorized) in the tip input storage area 80, and if the determination is positive, the process proceeds to step S208, and if the determination is negative, the process proceeds to step S214.

In step S208, the coin tip effect superimposition unit 72 determines whether the number of coin tip effect images 60 superimposed on the singing video at the current timing is less than the maximum number of superimpositions, and if the determination is positive, the process proceeds to step S210, and if the determination is negative, the process returns to step S202.

In step S210, the coin tip effect superimposition unit 72 superimposes the coin tip effect image 60 onto the singing video in the order in which the coin tip input is received.

In the next step S212, the tip effect superimposition unit 72 deletes the superimposed tip input information from the tip input storage area 80.

In the next step S214, the video distribution unit 74 determines whether or not the distribution of the singing video has ended, and if the determination is negative, the process returns to step S202, whereas if the determination is positive, the tipping effect process ends.

3. Other embodiments
The present invention has been described above using the above-mentioned embodiments, but the technical scope of the present invention is not limited to the scope described in the above-mentioned embodiments. Various modifications or improvements can be made to the above-mentioned embodiments without departing from the gist of the invention, and the forms in which such modifications or improvements are made are also included in the technical scope of the present invention. In addition, the above-mentioned embodiments may be appropriately combined.

For example, in the first embodiment, a form in which a viewing user selects a coin-throwing effect image 60 to be superimposed on a singing video has been described, but the present invention is not limited to this. For example, when a viewing user inputs a coin-throwing input, the viewing user does not select a coin-throwing effect image 60 by himself, but inputs points, and the coin-throwing effect function may superimpose a coin-throwing effect image 60 corresponding to the total points received up to the superimposition timing T on the singing video. Then, for example, the coin-throwing effect image 60 superimposed at the next superimposition timing T1 changes according to the total points input during the non-superimposition period P1, and the coin-throwing effect image 60 superimposed at the next superimposition timing T2 changes according to the total points input during the non-superimposition period P2. Note that the total points are not limited to those input by one viewing user, but are points input by multiple viewing users.

With reference to FIG. 13, the change of the coin throwing effect image 60 according to the points will be described. For example, if the points input in a certain non-overlapping period P are 5 points, the coin throwing effect image 60 shown in FIG. 13(A) is superimposed on the singing video at the next superimposition timing T. Also, if the points input in a certain non-overlapping period P are 10 points, the coin throwing effect image 60 shown in FIG. 13(B) is superimposed on the singing video at the next superimposition timing T. Similarly, if the points are 15 points, the coin throwing effect image 60 shown in FIG. 13(C) is superimposed, if the points are 20 points, the coin throwing effect image 60 shown in FIG. 13(D) is superimposed, and if the points are 25 points, the coin throwing effect image 60 shown in FIG. 13(E) is superimposed. In the example of FIG. 13, the shape of the coin throwing effect image 60 differs according to the points so that the larger the total points, the more gorgeous it becomes, but this is just an example, and the color of the coin throwing effect image 60 may change according to the points.

In addition, in each of the above embodiments, the performance is singing and the video uploaded to the video sharing site is a singing video, but the present invention is not limited to this. For example, the performance may be something other than singing, such as dancing and the video uploaded to the video sharing site is a dance video.

The flow of the coin tip effect processing described in each of the above embodiments is also one example, and unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the present invention.

3B Mobile terminal (terminal device)
4. Server (information processing device)
60 coin throwing effect image (decorative image)
70 Tipping input reception unit (reception means)
72 coin throwing effect superimposition unit (superimposition means)
74 Video distribution unit (distribution means)
76 Image cropping unit (acquisition means)
T Superimposition timing (predetermined timing)

Claims (10)

An information processing device that distributes a video to a terminal device so that a viewing user can view the video,
a reception means for receiving an instruction to display a decorative image that decorates the video being distributed from the terminal device of the viewing user;
a superimposing means for superimposing the decorative image on the video after the receiving means receives the display instruction;
a distribution means for distributing the video on which the decorative image is superimposed to the terminal device. The information processing device according to claim 1, wherein the superimposing means superimposes the decorative image on the video at a predetermined timing of the video. The timing is provided multiple times in the video,
The information processing apparatus according to claim 2 , wherein the superimposing means superimposes the decorative image on the moving image in response to the display instruction at the timing subsequent to the next time the display instruction is received. The video is a video recorded while a piece of music is being played,
4. The information processing apparatus according to claim 2, wherein a plurality of said timings are provided according to said music pieces. A maximum number of the decorative images that can be superimposed on the moving image at the same time is set in advance;
5 . The information processing apparatus according to claim 1 , wherein the superimposing means superimposes the decorative images on the moving image in the order in which the display instructions are received so as not to exceed the maximum number. The information processing device according to claim 5, wherein, if there are any decorative images for which the display instruction has been received but which have not been superimposed on the video at a predetermined timing immediately before the end of the video, the superimposing means superimposes all of the decorative images on the video at the predetermined timing. The information processing device according to any one of claims 1 to 6, wherein the decorative image has a set point and the design varies depending on the point. The information processing device according to any one of claims 1 to 7, further comprising an acquisition means for acquiring from the video an image of a scene in which the decorative image is relatively abundant in the video. A first step of delivering a video to a terminal device so that the video can be viewed by a viewing user;
A second step of receiving an instruction to display a decorative image that decorates the video being distributed from the terminal device of the viewing user;
a third step of superimposing the decorative image on the video after receiving the display instruction;
and a fourth step of distributing the video on which the decorative image is superimposed to the terminal device. A computer included in an information processing device that distributes video to a terminal device so that a viewing user can view the video,
a reception means for receiving an instruction to display a decorative image that decorates the video being distributed from the terminal device of the viewing user;
a superimposing means for superimposing the decorative image on the video after the receiving means receives the display instruction;
a video distribution program for causing the video distribution means to function as a distribution means for distributing the video on which the decorative image is superimposed to the terminal device;

Images