JP5459331B2 - Post reproduction apparatus and program - Google Patents

Description

  The present invention relates to a posting / reproducing apparatus for posting and reproducing posting data including recording data such as karaoke singing sound and musical instrument sound and shooting data of the singer and musical instrument player in an online karaoke system.

  Patent Document 1 discloses a technique for posting musical instrument sound data obtained by recording musical instrument sounds to a server through a communication network using a karaoke apparatus that can be connected to a musical instrument. Patent Document 1 discloses a technique for downloading musical instrument sound data posted by a karaoke apparatus from a server through a communication network and playing it back.

  Furthermore, as a service form of the online karaoke system, posted data including recorded data of singing sounds and musical instrument sounds by the user and moving image data of the user singing and performing is posted on the server. Has been proposed. Thereby, various posting data posted by many users can be viewed. In such a posting system, the appearance of a user singing and playing is recorded by a camera provided in the karaoke apparatus. Moreover, the singing sound and musical instrument sound input with the microphone with which a karaoke apparatus is equipped, or the musical instrument sound input from the musical instrument connected to the karaoke apparatus is recorded. These moving image data and recording data are uploaded as post data to the server of the communication karaoke system via the Internet. The uploaded post data can be reproduced via the Internet by a karaoke apparatus connected to the communication karaoke system, a terminal such as a personal computer or a mobile phone.

  Further, when the posted data is reproduced in the karaoke device connected to the communication karaoke system, the video and audio of the singing and performance performed in real time by the karaoke device are simultaneously output simultaneously with the reproduction of the video and audio of the posted data. By doing so, a service has also been proposed in which non-real-time collaboration can be enjoyed among a plurality of users who are not present. In addition, recording data and video data about singing or performing in collaboration with other post data can be uploaded to the server as post data in which new video and audio are added to the video and audio of other post data collaborated. It is. Furthermore, the number of participants in the collaboration can be increased by creating post data by further singing and performing by another user based on the post data produced by the collaboration.

JP 2010-237533 A

  As described above, when the post data collaborated by adding video / audio for a plurality of persons is reproduced, the images for the plurality of persons participating in the collaboration are simultaneously displayed on a monitor provided in the karaoke apparatus or the like. Moreover, the singing sound and performance sound for a plurality of people participating in the collaboration are simultaneously output from the speaker. Therefore, it is difficult for a user who views post data by collaboration to determine who is producing what sound among a plurality of singers and performers displayed simultaneously.

  An object of the present invention is to solve the above-mentioned problem by visualizing identification information related to a reproduced sound of post data that has been collaborated.

  The post reproduction apparatus of the present invention made to achieve the above object includes post data creation means, determination means, and upload means as a configuration related to upload of post data. In addition, as a function related to the reproduction of post data, a post data acquisition unit, a playback unit, a display control unit, and a sound source identification unit are provided.

  The posted data creating means creates posted data including recorded data obtained by recording a user's singing sound or musical instrument sound and recorded data taken by a camera provided during the singing or performance. The determining means specifies the type of the sound source that generated the sound recorded in the recording data, and determines identification information indicating the specified type of the sound source. The upload means associates the post data created by the post data creation means with the identification information determined by the determination means, and uploads the data to a predetermined posting server through the communication network.

  On the other hand, the posting data acquisition means identifies from the posting server the collaboration posting data that includes a plurality of mutually related posting data and the corresponding posting data included in the collaboration posting data. Information is acquired through a communication network. The reproducing means reproduces a plurality of recorded data and recorded data included in the post data for collaboration acquired by the posted data acquiring means. The display control means displays the reproduced video based on a plurality of recorded data included in the posting data in a predetermined area on the display when the contribution data for collaboration is reproduced by the reproducing means. The sound source identifying means corresponds to each recorded data in each display area corresponding to the reproduced video of each recorded data displayed on the display by the display control means based on the identification information obtained by the posted data obtaining means. Image information indicating the type of sound source indicated by the identification information is also displayed.

  According to the present invention configured as described above, each post is downloaded when a plurality of pieces of post data for collaboration are downloaded and reproduced based on the identification information indicating the type of sound source given when the post data is uploaded. Image information indicating the type of sound source of data can be displayed in association with each post data image. As a result, if the viewer of the post data collaborated sees the type of the sound source displayed together with the video of the participant in the collaboration, it becomes obvious at a glance who is producing what sound.

  Next, in addition to the above configuration, the post reproduction apparatus according to claim 2 further includes an audio output identification unit. The audio output identification means is for a plurality of recorded data included in the post data for collaboration being played, for which the strength of each reproduced sound being output based on the recorded data is more than a specified value The image information indicating that the sound is being output to the display area where the playback video of the recording data corresponding to the target recording data is displayed during the period when the strength of the playback sound is equal to or greater than the specified level. Are also displayed.

  By configuring in this way, during the reproduction of each post data that has been collaborated, only the post data that is actually outputting the singing sound or performance sound is displayed with image information indicating that the audio is being output. The data can be displayed in association with the image. Thereby, the viewer of the collaborated post data can easily grasp who's voice or instrument sound is being heard now. In addition, we are performing performances such as air performance that only imitates playing a musical instrument with a participant who collaborates actually singing and playing, a mouth-pack that moves his mouth according to the singing sound of others Viewers can easily distinguish collaboration participants.

  Next, the posting reproduction apparatus according to claim 3 has the following characteristics. That is, the determining means can specify a voice instrument that expresses a musical instrument sound by one's voice as one of the sound source types of the recording data, and if the voice instrument is specified as the sound source type of the recording data, Identification information indicating a voice instrument is determined. When the identification information acquired by the posted data acquisition unit indicates a voice instrument, the sound source identification unit displays image information indicating the voice instrument with respect to a display area in which a playback image of recorded data corresponding to the identification information is displayed. Are also displayed.

  For example, assume that there is post data in which the post data of a user who sings a song using a microphone and the post data of a user who plays a voice guitar or voice percussion using a microphone are collaborated. Try it. In this case, as long as the viewer sees the video of the participants in this collaboration, both participants use their microphones to input their voices, so it is possible to determine whether they are singing or playing a voice instrument. May not connect.

  Therefore, by adopting identification information indicating a voice instrument as the type of sound source of the posted data, image information indicating that the sound source is a voice instrument is included in the posted data image in the posted data playing the voice instrument. They can be displayed in association with each other. Thus, the viewer can easily determine the participant who is playing the voice instrument in the collaborated post data. That is, the viewer does not confuse the participant who is performing the voice instrument with the other participant who is singing.

  Alternatively, as a contrivance regarding the reproduction function of collaborated post data, a configuration as described in claim 4 is conceivable. That is, in addition to the above-described configuration, an audio output control unit is further provided. The audio output control means mutes the reproduction sound based on the designated sound source data or decreases the volume of the reproduction sound among a plurality of recording data included in the post data for collaboration reproduced by the reproduction means. According to such a configuration, it is possible to listen to other sound sources by muting only the specified recording data or reducing the volume of the recording data for multiple minutes included in the collaborative post data. It can be made easier.

  In addition, as described in claim 5, the present invention can be realized in the form of a program executed in the posting reproduction apparatus or a storage medium storing the program.

The block diagram which shows schematic structure of a karaoke system. Explanatory drawing which shows an example of the collaboration via contribution data. Explanatory drawing which shows an example of the identification display in contribution data. The flowchart which shows the procedure of a contribution data upload process. The flowchart which shows the procedure of a posting data display control process. The flowchart which shows the procedure of a contribution data mute process.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In addition, this invention is not limited to the following embodiment at all, and can be implemented in various aspects.
[Description of configuration of karaoke system 1]
As shown in FIG. 1, the karaoke system 1 includes a karaoke apparatus 10 installed in a karaoke store such as a karaoke box, and an external posting server 30. The karaoke apparatus 10 is connected to a karaoke box LAN (hereinafter simply referred to as “LAN”) 40. The LAN 40 is connected to the router 50. Thereby, the karaoke apparatus 10 can perform data communication with the posting server 30 through the LAN 40 and the router 50 via the Internet 60. In FIG. 1, only one karaoke device 10 is illustrated for easy understanding, but a plurality of karaoke devices 10 may be connected to the LAN 40 in the karaoke store. Moreover, the network in karaoke store which consists of the karaoke apparatus 10, LAN40, and the router 50 which were each installed in the some karaoke store may be connected to the internet 60.

  The karaoke apparatus 10 is mainly configured of a control unit 11 that controls the entire apparatus. The control unit 11 is an information processing device mainly composed of a CPU, ROM, RAM, I / O, and the like, and executes various processes based on programs and data read from the HDD 19. The control unit 11 is connected to the bus line 12, and the audio control unit 13 is connected to the control unit 11.

  An operation processing unit 14, an image input unit 15, an interface unit 16, a video RAM 17, a video reproduction unit 18, and a hard disk drive (HDD) 19 are connected to the bus line 12. In addition, a microphone 20, a MIDI sound source 22, and a speaker 23 are connected to the audio control unit 13. The operation processing unit 14 processes a signal from the operation unit 14a. From the operation unit 14a, information such as a music number for specifying music data to be played, a performance speed of the music data, and a performance key is input by the user. The operation unit 14a is embodied as, for example, a remote controller that performs wireless communication with the apparatus main body.

  A camera 15 a is connected to the image input unit 15. The camera 15a is installed in a room in which the karaoke apparatus 10 is installed, and can shoot a karaoke singer or a musical instrument player. A signal from the camera 15 a is processed by the image input unit 15, and the processed video signal is output through the bus line 12. The interface unit 16 is configured to connect the karaoke apparatus 10 to the LAN 40. Thereby, as described above, data communication with the posting server 30 via the Internet 60 becomes possible. The camera 15a may be installed outside the karaoke apparatus 10 and connected to the karaoke apparatus 10.

  The video RAM 17 temporarily stores video information. The video RAM 17 is connected to the video reproduction unit 18. The video playback unit 18 plays back video based on video information from the video RAM 17 and video data transmitted via the bus line 12. The video played by the video playback unit 18 includes a background and lyrics image displayed together with the karaoke performance, a video of the user's posted moving image downloaded from the posting server 30, and the like. A video control unit 24 is connected to the video playback unit 18. A display 25 is connected to the video control unit 24. The display 25 is embodied as a monitor device. With such a configuration, the video control unit 24 can display the video output from the video playback unit 18 on the display 25.

  The HDD 19 is a so-called hard disk drive device, and stores various data such as a karaoke database in which a large number of karaoke music data are recorded, system programs and application programs for controlling the operation of the karaoke device 10, and data used for reproducing various information contents. Remember. The HDD 19 stores post data acquired from the post server 30.

  Next, a configuration connected to the voice control unit 13 will be described. The microphone 20 is an input device for inputting karaoke singing sound by a user. The MIDI sound source 22 generates musical sounds based on MIDI (Musical Instrument Digital Interface) data. The music data stored in the HDD 19 described above includes MIDI data for generating musical sounds. The speaker 23 converts the sound source signal output from the sound control unit 13 into sound and outputs the sound.

  The sound control unit 13 converts the music data stored in the HDD 19 into a sound source signal via the MIDI sound source 22. The voice control unit 13 converts an external input sound from the microphone 20, an external input sound from the musical instrument 26 input through the control unit 11, and recording data into sound source signals. Then, the audio control unit 13 outputs the converted sound source signal to the speaker 23. The audio control unit 13 digitizes and records the external input sound from the microphone 20 as a single channel monaural sound source signal.

  A musical instrument sound input unit 21 is connected to the control unit 11. Thereby, the control unit 11 acquires the musical instrument sound played by the user using the musical instrument 26. The instrument sound input unit 21 includes a microphone and a terminal directly connected to the instrument 26. Note that the musical instrument 26 includes electronic musical instruments realized by a personal computer, a portable information processing terminal, a game machine, and the like, in addition to string instruments, percussion instruments, wind instruments, and keyboard instruments.

  Next, the configuration of the posting server 30 will be described. The posting server 30 is a well-known server device and constitutes a computer system. The posting server 30 has a database 31. The database 31 stores post video data (post data) uploaded from a plurality of karaoke apparatuses 10 through a communication network.

  This posting data is composed of data recording the video and audio of a contributor who sang or played a musical instrument using the karaoke apparatus 10. The post data is associated with the identification information of the contributor and music. The poster identification information is a user ID. The music identification information is a music number or a music title. The posting data can be searched based on the identification information of the poster or the song. Such posted data is input from the camera 15a through the image input unit 15 and the microphone 20 or the instrument sound input unit 21 when the karaoke device 10 performs a musical performance. It is created from the user's singing sound and performance sound.

  The created post data is temporarily stored in the HDD 19. Then, it may be uploaded to the posting server 30 after obtaining the approval of the user related to the creation of the posting data. In this way, a sufficient number of post data can be stored in the database 31 of the post server 30. Then, the karaoke apparatus 10 downloads the posting data selected by the user from the posting server 30 through the control unit 11 and the interface unit 16.

  And the karaoke apparatus 10 reproduces | regenerates the image | voice / audio | voice of the downloaded contribution data, and the user's singing sound and performance sound acquired from the microphone 20 or the musical instrument sound input part 21, and the user's real time acquired from the camera 15a. Output video. By doing so, it is possible to enjoy non-real-time collaboration (co-production or co-starring) among a plurality of users who are not present.

  Moreover, the karaoke apparatus 1 newly acquires recording / recording data based on the singing or performance performed in collaboration with the posting data downloaded from the posting server 30. The acquired new recording / recording data can be uploaded to the posting server 30 as posting data. Then, a plurality of related post data created by such collaboration is uploaded to the post server 30. Thereby, the karaoke apparatus 10 can be downloaded collectively from the posting server 30 as post data for collaboration, and a plurality of video / audio based on the post data can be reproduced simultaneously. Furthermore, based on the posting data produced by such collaboration, another user can further sing and perform and produce the posting data, thereby increasing the number of participants in the collaboration.

[Explanation on the procedure for creating post data by collaboration]
A procedure for creating post data by collaboration will be described with reference to FIG. In FIG. 2, three users A, B, and C who are participants in the collaboration use the karaoke stores A, B, and C to sequentially produce and upload post data. FIG. 2 assumes a case where users A, B, and C produce post data for collaboration.

(1: Post data A upload)
First, the user A who is a customer of the karaoke shop A performs singing or musical instrument performance in accordance with the performance of the karaoke music by the karaoke apparatus 10 installed in the karaoke shop A. At this time, the real time image of the user A photographed by the camera 15a is displayed on the display 25 of the karaoke apparatus 10. In addition, recording data recording a song or musical instrument performance by the user A, and recording data recording a video of the user A captured by the camera 15a are created.

  In addition to this, identification information regarding the recording data of the user A is created. The identification information of the recording data includes various types of information such as the type of sound source, volume information, technique information, scoring information, user comments, and effect information. The type of sound source is information indicating the source of the sound recorded in the recording data. Examples of the sound source include a microphone, guitar, percussion, voice instrument (guitar voice, etc.), electronic instrument, and the like. The volume information is information relating to the volume of the sound recorded in the recording data. The technique information is information indicating a specific technique (vibrato, shakuri, etc.) performed by the user during the recorded singing or musical instrument performance. The scoring information is information indicating the result of scoring the skill level of the user's singing by a well-known scoring function implemented in the karaoke apparatus 10. The user comment is information indicating the content of the comment input by the user regarding the post data. The effect information is information indicating a setting state at the time of recording of a function for adjusting the performance state of karaoke music such as key control and speed control.

  As described above, the posting data A including the recording data of the user A, the identification information of the recording data, and the recording data of the user A is uploaded from the karaoke apparatus 10 used by the user A to the posting server 30 through the Internet 60.

(2: Post data A download)
Next, the user B who is a customer of the karaoke shop B collaborates with the post data A previously posted from the user A of the karaoke shop A. For this purpose, first, posting data A is downloaded from posting server 30 through Internet 60 in karaoke apparatus 10 installed in karaoke shop B. Then, the recorded / recorded data of the downloaded post data A is reproduced, and the post data A and the user B collaborate.

(3: Post data B upload)
The user B performs singing or musical instrument performance in accordance with the performance of the karaoke music piece by the karaoke apparatus 10 installed in the karaoke shop B and the reproduced voice based on the recorded data of the posted data A reproduced in synchronization with the performance. At this time, on the display 25 of the karaoke apparatus 10, the reproduction image of the user A based on the recorded data of the posting data A (the left side of the screen 25 of the display 25 of the karaoke shop B), and the user B photographed by the camera 15 a. Real-time video (FIG. 2, the right side of the display 25 of the karaoke shop B) is displayed at the same time. Then, recording data recording a song or musical instrument performance by the user B, and recording data recording a video of the user B taken by the camera 15a are created.

  In addition to this, identification information regarding the recording data of the user B is created. The identification information of the recording data includes various types of information such as the type of sound source, volume information, technique information, scoring information, user comments, and effect information. Further, collaboration identification information indicating that the current recording / recording data is created by collaboration with the posted data A is created. This collaboration identification information includes information for identifying post data (collaborated post data) related to the main post data.

  As described above, the posting data B including the recording data of the user B, the recording data identification information, the recording data of the user B, and the collaboration identification information is uploaded from the karaoke apparatus 10 used by the user B to the posting server 30 through the Internet 60. Is done.

(4: Post data A + B download)
Next, the user C who is a customer of the karaoke shop C collaborates with the two pieces of post data A and B posted from the user A of the karaoke shop A and the user B of the karaoke shop B. For this purpose, first, the karaoke apparatus 10 installed in the karaoke shop C downloads two pieces of post data A + B related by the collaboration identification information in the post data from the post server 30 through the Internet 60. The post data A + B includes recording data of the users A and B, identification information of each recording data, and recording data of the users A and B. And the karaoke apparatus 10 installed in the karaoke shop C plays back each recording and video recording data contained in the downloaded contribution data A + B simultaneously, and performs the collaboration with the contribution data A + B and the user C.

  As for the recording data of the user A and the user B, the reproduced video of the recorded data of the posted data A and the reproduced video of the recorded data of the posted data B are displayed separately on one screen. That is, it is conceivable to create recording data for collaboration in which two pieces of recording data are combined in advance on the posting server 30 side. In that case, the posting server 30 attaches the combined recording data to the posting data for collaboration distributed from the posting server 30 to the karaoke apparatus 10. Alternatively, the posting server 30 may be configured such that the two recorded data of the posting data A and B are separately delivered to the karaoke apparatus 10 without being combined.

(5: Post data C upload)
In accordance with the performance of the karaoke music piece by the karaoke apparatus 10 installed in the karaoke shop C and the reproduced sound based on the recorded data of the post data A + B reproduced in synchronization with this performance, the user C performs a song or a musical instrument performance. . At this time, on the display 25 of the karaoke apparatus 10, the playback image of the user A (FIG. 2, the lower left of the screen of the display 25 of the karaoke shop C) and the playback image of the user B (FIG. 2, Karaoke) The lower right screen of the display 25 of the store C) and the real time video of the user C taken by the camera 15a (FIG. 2, upper left screen of the display 25 of the karaoke store C) are displayed at the same time. In addition, recording data recording a song or musical instrument performance by the user C, and recording data recording a video of the user C taken by the camera 15a are created.

  In addition to this, identification information relating to the recording data of the user C is created. The identification information of the recording data includes various types of information such as the type of sound source, volume information, technique information, scoring information, user comments, and effect information. Also, collaboration identification information indicating that the current recording / recording data is created by collaboration with the posted data A and the posted data B is created.

  As described above, the post data C including the recording data of the user C, the recording data identification information, the recording data of the user C, and the collaboration identification information is uploaded from the karaoke apparatus 10 used by the user C to the posting server 30 through the Internet 60. Is done.

  As described above, the post data A, B, and C produced by the collaboration in which the users A, B, and C participate are accumulated in the database 31 of the post server 30. Post data A, B, and C stored in post server 30 are associated with each other based on collaboration identification information. For this reason, the karaoke apparatus 10 can download a plurality of pieces of post data associated with the collaboration identification information as post data for collaboration.

[Explanation regarding display control of identification information in post data for collaboration]
A specific example of the display control of the identification information for the reproduced video displayed based on the post data for collaboration downloaded from the post server 30 by the karaoke apparatus 10 will be described with reference to FIG. FIG. 3 assumes a case where post data for collaboration produced by four users A, B, C, and D who are participants of the collaboration is reproduced.

  As shown in FIG. 3A, a case is considered in which a playback video based on the recording data of each of the users A to D is simply displayed on the same screen. In this case, it is difficult for the viewer to determine who is producing what sound and when, even if he / she listens to the reproduced voices for the four people that are also output.

  In particular, as a sound input to the karaoke apparatus 10, a simple sound such as a sound realized by an application operating on a personal computer or a mobile phone, or a voice instrument that artificially expresses an instrument sound such as a guitar or a percussion by a human voice. There are cases where pseudo sounds other than singing sounds and instrument sounds are included. If the post data collaborated includes such a pseudo sound, it may be more difficult to determine who is producing what sound in the reproduced video of the post data.

  In addition, post data collaborated by performers who perform oral or air performances may be produced by piggybacking on post data by singing or performing by others. In such a case, whether the playback sound based on the post data produced by the collaboration between the mouth-pack and air performance is the sound produced by the original singer or performer, or the performer of the mouth-pack or air performance gave it. There is a possibility that the viewer cannot determine whether the sound is sound.

  Therefore, the type of sound source is specified using the identification information of the recording data of each user in the post data for collaboration. Based on the identified sound source, image information indicating the type of sound source of the recording data of the user is displayed in association with the display area of the playback video of each user. An example is shown in FIGS. 3B and 3C.

  As shown in FIG. 3B, in the playback video of the user B displayed in the display area at the lower right of the screen of the display 25, singing information (microphone indicating that the user B inputs the singing voice with the microphone 20 Is displayed as sound source identification information. Further, highlighting (audio output information) is performed on the outer edge portion of the display area of the reproduced video of user B, indicating that user B is outputting audio.

  In the playback image of the user C displayed in the display area at the upper left of the screen of the display 25, musical instrument information (image information imitating the guitar) indicating that the user C has input the guitar performance sound is used as sound source identification information. It is displayed. Further, highlighting (audio output information) is performed on the outer edge portion of the display area of the playback image of user C, indicating that user C is outputting audio.

  In the playback video of the user D displayed in the display area in the upper right of the screen of the display 25, voice guitar information (simulating a microphone and a guitar) indicating that the user D has input a voice played by the microphone 20 using the microphone 20. (Image information) is displayed as sound source identification information. Further, highlighting (audio output information) is performed on the outer edge portion of the display area of the playback video of user D, indicating that user D is outputting audio.

  On the other hand, in the playback image of the user A displayed in the display area at the lower left of the screen of the display 25, there is no sound information indicating that no sound is currently being output from the recording data of the user A (× × on the speaker). (Image information with a mark) is displayed. This indicates that no sound is currently recorded in the recording data of the user A (not singing or playing).

  Furthermore, when reproducing the post data for collaboration, the karaoke apparatus 10 of the present embodiment individually mutes (silences) the reproduced sound only for the arbitrary recorded data designated by the viewer from among a plurality of recorded data. Or has a function to lower the volume. At that time, image information indicating that the volume is being changed is displayed on the reproduced video corresponding to the recording data whose mute or volume is reduced according to an instruction from the viewer. In the example shown in FIG. 3C, volume change information indicating that the reproduction sound of the recorded data of the user is muted is displayed on the outer edge portion of the reproduction image of the user C displayed on the upper left of the screen. Yes.

  By the way, in this embodiment, when the reproduction | regeneration video for several persons is displayed simultaneously based on the recording data of the contribution data for collaboration, the allocation of the display area on the screen of each recording data depends on the number of participants in the collaboration. Is determined as follows. In the case of post data for collaboration of two participants, the playback videos are arranged in the order of the left side and the right side of the screen of the display 25 in the order of uploading of personal post data. When there are three participants, the playback images are arranged in the order of the lower left, lower right, and upper left of the screen of the display 25 in the order of uploading. When there are four participants, the playback videos are arranged in the order of the lower left, lower right, upper left, and upper right of the screen of the display 25 in the order of uploading. Thereafter, the same applies when there are more participants. Such allocation of the display area is applied to the layout of the screen when the recording data of each individual produced by the collaboration is combined with one recording data for collaboration in the posting server 30. Alternatively, in the case where the recording data is not combined on the posting server 30 side, this is applied when the playback video based on the recording data of each individual is divided and laid out on the screen on the karaoke apparatus 10 side.

  By defining the display area of each user's playback video in the post data for collaboration as described above, the display area of each user's playback video can be easily specified by the number of participants in the collaboration and the upload order. Thereby, the correspondence between the display area of the reproduced video of each user and the display area where the identification information of the recording data of each user should be displayed can be easily specified.

[Explanation of post data upload process]
The procedure of post data upload processing executed by the control unit 11 of the karaoke apparatus 10 will be described with reference to the flowchart of FIG. This process is a process executed when an operation instruction for creating post data is performed together with the performance of the karaoke music piece.

  In S100, the control unit 11 generates recording data obtained by recording the input sound input through the microphone 20 or the musical instrument sound input unit 21 during the performance of the karaoke music, and recording data obtained by recording the video photographed by the camera 15a. .

  In S <b> 102, the control unit 11 identifies the type of sound source that is the source of the sound recorded in the recording data. Here, the control unit 11 determines the input destination of the sound source (whether the microphone 20 or the instrument sound input unit 21) at the time of recording data generation, the type of instrument connected to the instrument sound input unit 21, and the like. It is conceivable to specify the type of sound source. In addition, whether or not a specific application installed in the karaoke apparatus 10 is used, such as a guitar practice mode that provides a function that supports guitar performance, and a voice instrument mode that provides a function that supports the performance of a voice instrument. Based on this, the control unit 11 can also specify the type of sound source (guitar or voice instrument) corresponding to the application.

  In the next S104, the control unit 11 creates identification information regarding the recording data created this time. The recording data identification information includes volume information, technique information, scoring information, user comments, effect information, and the like in addition to the information indicating the type of the sound source specified in S102. The volume information and the effect information reflect the performance state set in the karaoke apparatus 10 at the time of recording. The technique information reflects the technique detected by analyzing the singing sound or instrument sound input at the time of recording. The scoring information reflects the user's singing scoring result by the scoring function of the karaoke apparatus 10. The user comment reflects a comment input by an instruction from the user before or after the posting data is created. The volume information may be acquired in time series along with the performance of karaoke music. Moreover, when singing a song in time series along with the performance of karaoke music, scoring information may be acquired in time series along with the performance of karaoke music.

  In S106, the control unit 11 posts the recorded data and recorded data generated in S100, the posting data including the identification information created in S104, and the like via the Internet 60 in association with the identification information of the poster and the song. Upload to server 30. In addition, when creation of this posting data was performed by collaboration with other posting data, collaboration identification information indicating that the posting data uploaded in S106 is collaboration with other posting data. Further included. After uploading the posted data, this process ends.

[Explanation of post data display control process]
The procedure of post data upload processing executed by the control unit 11 of the karaoke apparatus 10 will be described with reference to the flowchart of FIG. This process is a process executed when the post data for collaboration designated by the viewer user is reproduced.

  In S <b> 200, the control unit 11 downloads the post data for collaboration designated by the user from the post server 30 via the Internet. In next step S202, reproduction of recording data and recording data of each user in the post data for collaboration downloaded in S200 is started.

  The following processes of S204 to S210 will be described on the assumption that the processes are individually performed in parallel for each user's recording data included in the post data for collaboration. In S204, the control unit 11 determines image information indicating the type of the sound source based on the type of the sound source indicated by the identification information of the recording data to be processed. And the control part 11 displays the determined image information on the display area on the screen where the reproduction | regeneration image | video of the user corresponding to sound recording data is displayed. Subsequently, in S206, the control unit 11 determines whether or not the current playback sound intensity (for example, velocity of the recording data) output based on the recording data to be processed is greater than or equal to a specified value. This specified value is a value that makes it possible to distinguish between a state in which a user's singing sound or instrument sound is input and a state in which the user's singing sound or musical instrument sound is not input (silence, mouth-pack, air performance, etc.).

  When the strength of the reproduced sound based on the recording data to be processed is equal to or greater than the specified value (S206: YES), the control unit 11 proceeds to S208. In S208, the control unit 11 displays a highlighted display (sound output information) indicating that sound is being output in a display area on the screen on which the playback video of the user corresponding to the recording data is displayed. . On the other hand, when the strength of the reproduced sound based on the recording data to be processed is less than the specified value (S206: NO), the control unit 11 proceeds to S210. In S210, the control unit 11 has no sound indicating that no sound is output from the recorded data in the display area on the screen on which the playback video of the user corresponding to the recorded data is displayed. Display information superimposed.

  In next step S212, the control unit 11 determines whether or not the reproduction of the post data for collaboration has ended. When the reproduction is continued (S212: NO), the control unit 11 returns to S206. Thereafter, the control unit 11 repeats the processes of S206 to S212 while the reproduction of post data for collaboration is continued. Then, when the reproduction of the post data for collaboration ends (S212: YES), the control unit 11 ends this process.

[Description of post data mute processing procedure]
The procedure of post data mute processing executed by the control unit 11 of the karaoke apparatus 10 will be described with reference to the flowchart of FIG. This process is a process executed when post data for collaboration designated by the viewer user is being reproduced.

  In S300, the control unit 11 determines whether or not an instruction operation for muting any one of the plurality of recording data being reproduced has been received. When a mute instruction is accepted (S300: YES), the process proceeds to S302. On the other hand, when the mute instruction has not been received (S300: NO), this process ends.

  In S <b> 302, which proceeds when a mute instruction is received, the control unit 11 mutes the reproduced sound based on the designated recording data. Then, the control unit 11 changes the volume indicating that the reproduced sound from the recorded data is being muted in the display area on the screen where the reproduced video of the user corresponding to the recorded data designated to be muted is displayed. Display information superimposed. After the display, the control unit 11 ends this process.

[effect]
According to the karaoke system 1 of the above embodiment, the following effects can be obtained.
When the karaoke apparatus 10 posts posted data to the posting server 30, identification information indicating the type of sound source of the recording data included in the posted data can be given. Thereby, when the karaoke apparatus 10 downloads and reproduces post data for collaboration, display on the screen on which the playback video of the participant corresponding to each recording data is displayed based on the identification information of each recording data Image information indicating the type of sound source of the recording data can be displayed in an overlapping manner in the area. When viewers of post data for collaboration see the types of sound sources that are displayed along with the playback video of the participants in the collaboration, it becomes obvious at a glance who is producing what sound.

  During the reproduction of the post data for collaboration, image information indicating that the voice is being output can be displayed in association with the image of the post data in the post data that is actually outputting the singing sound or the performance sound. Thereby, it is possible to easily grasp who's voice or musical instrument sound is being heard by the viewer. In addition, the viewer can easily distinguish between the participants who are actually singing or performing the collaboration and the participants who are performing the performance such as mouth-packing or air performance.

  By using identification information that indicates a voice instrument as the type of sound source in the recording data, image information indicating that the sound source is a voice instrument is displayed in association with the playback video of the participant who plays the voice instrument. Can do. This makes it easy for viewers to identify participants who are playing voice instruments in post data for collaboration, so participants who are playing voice instruments and other participants who are singing And viewers will not be confused.

  It is possible to make it easier to listen to other sound sources by muting (decreasing) only specified recording data or lowering the volume among a plurality of recording data included in post data for collaboration. In addition, the playback sound from the recorded data is being muted or turned down in the display area on the screen where the playback video of the participant corresponding to the recorded data with the mute or volume turned down is displayed. Volume change information indicating can be displayed. Thus, the viewer can easily determine which participant's singing sound or instrument sound volume has been changed.

  In addition, the said embodiment demonstrated the example which applied this invention to the karaoke apparatus installed in a karaoke store etc. The present invention is not limited to this, and the present invention can also be applied to an information processing terminal (for example, a high-function mobile phone terminal, a smartphone, etc.) having a karaoke music reproduction function, an image display function, a recording / recording function, and the like. The present invention can also be implemented in the form of a program installed in an information processing apparatus such as a personal computer, a high-function mobile phone terminal, or a smartphone. Such a program may be downloaded via a network by the information processing apparatus. Alternatively, the program may be read from a portable storage medium such as a CD-ROM storing the program and installed in the information processing apparatus.

[Correspondence with configuration described in claims]
The correspondence between the configuration of the karaoke apparatus 10 according to the embodiment and the configuration described in the claims is as follows. The control unit 11 corresponds to post data creation means, determination means, display control means, sound source identification means, audio output identification means, and audio output control means. The control unit 11 and the interface unit 16 correspond to upload means and post data acquisition means. The control unit 11, the audio control unit 13, and the video playback unit 18 correspond to playback means.

  DESCRIPTION OF SYMBOLS 1 ... Karaoke system, 10 ... Karaoke apparatus, 11 ... Control part, 12 ... Bus line, 13 ... Voice control part, 14 ... Operation processing part, 14a ... Operation part, 15 ... Image input part, 15a ... Camera, 16 ... Interface , 17 ... Video RAM, 18 ... Video playback unit, 19 ... Hard disk drive (HDD), 20 ... Microphone, 21 ... Instrument sound input unit, 22 ... MIDI sound source, 23 ... Speaker, 24 ... Video control unit, 25 ... Display 26 ... musical instrument, 30 ... posting server, 31 ... database, 40 ... LAN in karaoke box, 50 ... router, 60 ... internet.

Claims (5)

Posting data creating means for creating posting data including recording data obtained by recording a user's singing sound or musical instrument sound, and recording data photographed by a camera provided at the time of the singing or playing,
Determining means for identifying the type of sound source that generated the sound recorded in the recording data and determining identification information indicating the type of the identified sound source;
An upload unit that associates the post data created by the post data creation unit with the identification information determined by the determination unit, and uploads it to a predetermined post server via a communication network;
Collaboration data including a plurality of post data related to each other from the posting server, and identification information corresponding to a plurality of pieces of post data included in the post data for collaboration, the communication network Post data acquisition means acquired through,
Reproduction means for reproducing a plurality of recorded data and recorded data included in the contribution data for collaboration acquired by the posted data acquisition means;
Display control means for displaying reproduced video based on a plurality of recorded data included in the posting data in a predetermined area on the display, respectively, when reproducing the posting data for collaboration by the reproducing means;
Based on the identification information acquired by the posted data acquisition means, the identification information corresponding to each recording data in each display area corresponding to the reproduced video of each recording data displayed on the display by the display control means And a sound source identifying means for displaying together image information indicating the type of the sound source indicated by. In the contribution reproduction | regeneration apparatus of Claim 1,
Among the multiple recorded data included in the post data for collaboration during playback, if the strength of each playback sound being output is more than specified based on the respective recording data, the strength of the playback sound Audio for displaying image information indicating that audio is being output in a display area in which a playback video of recorded data corresponding to the target recording data is displayed during a period of time that is greater than or equal to a specified value A post reproducing apparatus, further comprising output identifying means. In the posting reproduction apparatus according to claim 1 or 2,
The determination means can specify a voice instrument that expresses a musical instrument sound by human voice as one of the sound source types of the recording data, and if the voice instrument is specified as the sound source type of the recording data, Determine the identification information that indicates the instrument,
When the identification information acquired by the posted data acquisition unit indicates a voice instrument, the sound source identification unit indicates the voice instrument with respect to a display area in which a playback image of recorded data corresponding to the identification information is displayed. A post reproduction apparatus characterized by displaying image information together. In the contribution reproduction | regeneration apparatus of any one of Claim 1 thru | or 3,
Audio output control means for muting the playback sound based on the designated sound source data or lowering the volume of the playback sound among a plurality of recording data included in the post data for collaboration played back by the playback means is further provided. A post playback device characterized by the above. Uploading post data by a user of a karaoke system to a predetermined post server, and a program executed in a post playback device that plays back post data downloaded from the post server,
A posting data creation step for creating posting data including recording data obtained by recording a user's singing sound or musical instrument sound and recording data photographed by a camera provided during the singing or performance;
Determining the type of sound source that generated the sound recorded in the recording data, and determining identification information indicating the type of the specified sound source;
An upload step in which the post data created in the post data creation step is associated with the identification information determined in the determination step and uploaded to the post server through a communication network;
Collaboration data including a plurality of post data related to each other from the posting server, and identification information corresponding to a plurality of pieces of post data included in the post data for collaboration, the communication network Posting data acquisition step acquired through,
A reproduction step of reproducing a plurality of recording data and recording data included in the posting data for collaboration acquired in the posting data acquisition step;
A display control step of displaying, when the post data for collaboration is played in the playback step, a playback video of a plurality of recorded data included in the post data, respectively, in a predetermined area on the display;
Based on the identification information acquired in the posted data acquisition step, the identification information corresponding to each recording data in each display area corresponding to the reproduced video of each recording data displayed on the display in the display control step A sound source identifying step for displaying together image information indicating the type of sound source indicated by

Images