JP6230068B2 - Program and information processing apparatus - Google Patents

Description

  The present invention relates to a program that can be executed by various information processing apparatuses such as a portable terminal, a personal computer, and a karaoke apparatus, and an information processing apparatus.

  Conventionally, at night stores such as karaoke boxes and snacks, karaoke is enjoyed to enjoy singing along with the performance. Currently, with the spread of mobile terminals and personal computers and the development of communication infrastructure via the Internet, not only karaoke devices installed in karaoke boxes, but also personally owned mobile terminals (smartphones, mobile phones, tablet terminals, etc.), It is possible to enjoy karaoke by using various information processing devices such as a personal computer and a home game machine.

  It is possible to perform karaoke by executing a program on such an information processing apparatus. Currently, in smartphones and tablet terminals, a program is provided by a karaoke dealer, and it is possible to enjoy karaoke by acquiring the program via a communication line and installing it on a corresponding information processing apparatus.

  Patent Document 1 discloses a moving image information distribution system capable of distributing a singing voice and moving image information in which a state of singing is recorded. This moving image information distribution service provides a new way of enjoying posting by a plurality of users by combining moving image information of a plurality of users. Specifically, by uploading the video information recorded using the karaoke device to the host device and publishing it, distributing the released video information to the collaborator user, and creating further recording information, It is possible to create a collaboration video with one song.

JP 2012-209856 A

  As seen in the video information distribution system disclosed in Patent Document 1, in the field of karaoke, not only how to enjoy singing, but also singing and singing to other people (singing posting service) ) Has come to be enjoyed. In the singing posting service, singing voices recorded through one piece of music are usually posted. Therefore, when listening to the posted singing voice, it is necessary to reproduce it from the beginning of the music, and as a result, it takes time to listen to even the singing voice that is not of interest. For this reason, it took time and effort to find a favorite one from among many singing voices.

  In addition, the amount of communication required for posting the singing voice recorded through one song has increased, and it has not been possible to easily post (or listen to the posted singing) using a mobile communication network such as a smartphone. . In addition, the volume of recorded information recorded through one piece of music is large, and it is expected that the load on the host device that stores and publishes the posted singing voice will also increase. Therefore, it is not easy to enjoy posting and listening to information processing devices owned by individuals.

  In view of such circumstances of karaoke singing postings, the present invention aims to improve services related to singing postings and listening to information processing devices owned by individuals such as smartphones. Furthermore, this invention aims at providing the environment which is easy to sing when singing together according to the posted song. Furthermore, when the posted singing and the singing are played against each other, the purpose is to ensure the fairness of the evaluation.

Therefore, the program according to the present invention is characterized by adopting the following configuration.
A program executed in an information processing apparatus capable of communicating with a host device via a network,
A receiving step of receiving recording information including singing voice information corresponding to a partial section of the music from the host device;
Based on the recording information received in the receiving step, the singing voice indicated by the singing voice information corresponding to a partial section of the music is reproduced, and the performance sound of the music is reproduced from a position before the partial section of the music. A reproduction step for starting reproduction is executed by the information processing apparatus.

Furthermore, in the program according to the present invention,
In the reproducing step, the performance sound is reproduced based on the music information corresponding to the music used when recording the recording information.

Furthermore, in the program according to the present invention,
In the reproduction step, a singing voice reproduction start timing corresponding to a partial section of the music is notified.

Furthermore, in the program according to the present invention,
The recording information is included in the posted information transmitted to the host device by the user.

Furthermore, in the program according to the present invention,
The reproduction step starts reproduction of the performance sound from a predetermined position defined for the music.

Furthermore, in the program according to the present invention,
The recording information includes singing voice information corresponding to a plurality of partial sections of the music,
The reproduction step reproduces a singing voice corresponding to a plurality of the singing voice information.

Furthermore, in the program according to the present invention,
The reproduction step displays first evaluation information formed based on a comparison between the singing reference information corresponding to the recording information and the singing voice information that is a recording target of the recording information.

Furthermore, in the program according to the present invention,
The reproduction step displays second evaluation information formed based on a comparison between the singing voice information corresponding to the singing voice input from the microphone and the singing reference information corresponding to the recording information.

Furthermore, in the program according to the present invention,
The reproduction step includes the first evaluation information formed on the basis of the comparison between the singing reference information corresponding to the recording information, the singing voice information that is the recording target of the recording information, and the singing voice input from the microphone. The superiority or inferiority of the second evaluation information formed based on the comparison of the singing voice information corresponding to the singing standard information corresponding to the recording information is displayed.

An information processing apparatus according to the present invention includes:
An information processing apparatus capable of communicating with a host device via a network,
Receiving means for receiving recording information including singing voice information corresponding to a partial section of the music from the host device;
Based on the recording information received by the receiving means, the singing voice indicated by the singing voice information corresponding to a partial section of the music is reproduced, and the performance sound of the music is reproduced from a position before the partial section of the music. Playback means for starting playback.

  According to the program and the information processing apparatus according to the present invention, the recording information recorded in synchronization with the reproduction of the music information is transmitted through a partial section of the music, thereby reducing the capacity of the recording information, the host device, and It is possible to reduce the burden on the communication line.

  When reproducing a partial section of such music, it can be considered to sing together. In such a case, by starting playback of the performance sound of the music from a position before the singing voice corresponding to a part of the music, the beginning of the song is matched to the user who sings along with the recording information. It is possible to provide an environment where it is easy to sing. It is also conceivable to play a match between recording information and singing evaluation. In this case, it is possible to let the user who sings know in advance where the singing voice to be played, that is, the part of the song that is the target of the match, and the fairness of the evaluation at the time of the match It is also possible to ensure that sex is secured.

The schematic diagram which shows the mode of the singing contribution which concerns on embodiment of this invention The figure which shows the portable terminal (information processing apparatus) which concerns on embodiment of this invention. The figure which shows the data structure of the music information which concerns on embodiment of this invention. The flowchart which shows the posting process in the portable terminal which concerns on embodiment of this invention The block diagram which shows the reproduction | regeneration processing (at the time of the first time) which concerns on embodiment of this invention The figure which shows the music selection screen which concerns on embodiment of this invention The figure which shows the music detail screen which concerns on embodiment of this invention The figure which shows the music song screen which concerns on embodiment of this invention. The figure which shows the scoring result display screen which concerns on embodiment of this invention The figure which shows the recording information management screen which concerns on embodiment of this invention The figure which shows the recording information edit screen which concerns on embodiment of this invention The figure which shows the various forms of the edit window which concerns on embodiment of this invention The figure which shows the contribution related information input screen which concerns on embodiment of this invention. The figure explaining the audio | voice reproduction area and performance sound reproduction area which concern on embodiment of this invention. The figure explaining the audio | voice reproduction area and performance sound reproduction area which concern on other embodiment of this invention. The figure explaining the audio | voice reproduction area and performance sound reproduction area which concern on other embodiment of this invention. The figure which shows the data structure of the contribution information which concerns on embodiment of this invention. The figure which shows the scoring ranking screen which concerns on embodiment of this invention The block diagram which shows the reproduction | regeneration processing (at the time of contribution information reproduction | regeneration) concerning embodiment of this invention The figure which shows the recording information reproduction screen which concerns on embodiment of this invention The figure which shows the result display screen which concerns on embodiment of this invention

  FIG. 1 is a schematic diagram showing a state of a singing contribution according to an embodiment of the present invention, and is a diagram showing a configuration of the entire singing contribution system. The singing contribution system according to the present invention has various information processing devices (mobile terminal, karaoke device, personal computer, etc.) installed with a program, with the karaoke host device 5 as the core. The karaoke host device 5 and the various information processing devices are communicatively connected via a network such as the Internet. Various forms such as a smartphone, a tablet terminal, and a mobile phone can be used for the mobile terminal. In the figure, for the sake of convenience, the posting terminal for posting and the listening terminal for listening to posts are described separately. However, the posting function and the listening function for posting may be provided in the same terminal (information processing apparatus). Good.

  A singing contribution when using a mobile terminal will be described. A program is installed in advance on the portable terminal, and the user requests the karaoke host device 5 for a song to be sung using this program. In the case of a portable terminal, music information to be played is not stored, so music is requested from the karaoke host device 5 and performance is performed based on the received music information (non-accumulation type). On the other hand, in the case of a karaoke device, music information is previously received from the karaoke host device 5 and stored, and a performance is performed based on the stored music information (accumulation type). The music to be sung in this way may be acquired as either a non-accumulation type or an accumulation type.

  The portable terminal performs a performance based on the received music information. At that time, the singing voice is recorded in synchronization with the performance to form the recording information, and the posting information including the recording information is transmitted to the karaoke host device 5. The karaoke host device 5 that has received the posted information accumulates it in the storage unit 51 and also publishes various information related to the posted information, such as a song title and a poster, to the listening terminal side through the Web server 52. On the listening terminal side, post information desired to be listened to is selected based on various information disclosed through the Web server 52, and a request is made to the karaoke host device 5. Upon receiving the post information request, the karaoke host device 5 transmits the selected post information to the requested listening terminal. The listening terminal can play the received posted information and listen to the recorded recording information.

  A configuration of the mobile terminal 1 will be described as an example of an information processing apparatus that can execute a program according to an embodiment of the present invention. FIG. 2 is a diagram showing a configuration of the mobile terminal 1 according to the embodiment of the present invention. As the mobile terminal 1, a smartphone, a tablet terminal, a mobile phone, or the like can be used. By installing the program according to the present invention in the portable terminal 1, it becomes possible to execute a posting process capable of posting a singing voice.

  The mobile terminal 1 according to the present embodiment includes a control unit that includes a CPU 11, a RAM 12, a storage unit 13, an image processing unit 14, and an acoustic processing unit 16. The storage unit 13 can employ non-volatile storage means such as a hard disk or SSD. The storage unit 53 stores the program of the present embodiment and various data used in the program.

  The CPU 11 is a unit that performs overall control of the mobile terminal 1 and executes various controls based on programs and data stored in the storage unit 13. The RAM 12 temporarily stores data generated by the control of the CPU 11.

  The acoustic processing unit 16 is a unit that performs input / output of acoustic relations of the mobile terminal 1. This is a means for causing the speaker 17a to emit sound under the control of the CPU 11. The acoustic processing unit 16 is connected to the microphone 17b, and can capture surrounding sounds such as a singing voice from the microphone 17b.

  The touch panel display unit 18 includes a display unit 18b that displays an image and a touch panel 18a disposed on the front surface or the back surface of the display unit 18b. Various types of touch panel 18a that can recognize the touch position of the user, such as a capacitance method, can be employed. The CPU 11 can display on the display unit 18b and determine which part of the displayed image is touch-input by touch input from the touch panel 18a.

  The image processing unit 14 is a means for inputting and outputting image relations of the mobile terminal 1. The image processing unit 14 displays and outputs the image formed by the CPU 11 on the display unit 18b. And it has the function to take in an image from the front camera 15F arrange | positioned at the front surface (touch panel display part 18 side) of the portable terminal 1, or the rear camera 15R arrange | positioned at the back surface of the portable terminal 1. FIG.

  The communication unit 19 can perform wireless communication using a mobile wireless network, a wireless LAN, and the like, and can wirelessly communicate with the access point to connect the mobile terminal 1 to the Internet. The portable terminal 1 can download various programs (also referred to as “applications”) including programs from a server on the Internet. Note that the program may be installed from the external memory EX connected to the external memory connection terminal 22 other than the form using the communication line. The communication unit 19 may be further provided with short-distance communication means such as an infrared communication unit or Bluetooth (trademark).

  The mobile terminal 1 according to the present embodiment is configured to include an input switch 20 in addition to the touch panel 18a as input means. The input switch 20 can be provided with various switches such as a power switch, a volume control switch, and a switch whose input application is changed according to a program.

  In the portable terminal 1 of the present embodiment, it is possible to upload recorded information recorded along with the song to the host device by installing a program.

  FIG. 3 shows a data structure of music information according to the embodiment of the present invention. The music information is information that is stored in the storage unit 51 of the karaoke host device 5 and distributed based on a request from the portable terminal 1. The mobile terminal 1 that has received the music information performs various processes based on the music information, such as playing performance information included in the music information. The music information of this embodiment is divided into meta information, which is various information related to the music information, and actual information that executes various processes such as performance. The meta information includes song identification information for identifying song information, song name, singer name, songwriter name, and composer name.

  The actual information of the music information includes performance information, lyrics information, guide vocal information, and reference pitch information (corresponding to “singing reference information” in the present invention). The performance information is composed of control information for an electronic musical instrument based on the MIDI standard, compressed audio information recording an actual performance, or the like, and is information for playing an accompaniment sound of karaoke. The lyric information is information that is displayed in synchronization with the performance information in order to assist the singing, and may be configured to change the color of the displayed lyrics in synchronization with the performance. Guide vocal information is audio information corresponding to a part to be sung, such as the main melody, and can be turned on / off by the user. The reference pitch information is information used for the evaluation process (scoring), and is information defining the pitch of the part to be sung. The song evaluation process is performed by comparing the pitch of the singing voice signal input from the microphone 17b with the reference pitch information.

  FIG. 4 is a flowchart showing the posting process according to the embodiment of the present invention. Here, the form at the time of using the portable terminal 1 is demonstrated. The posting process is started by starting a program in the mobile terminal 1 and selecting a predetermined menu. In the posting process of the present embodiment, the user can give various instructions on the display screen 31 displayed on the touch panel display unit 18. When the posting process is started, the user can select a desired music via a music selection screen displayed on the display screen 31 (S101).

  FIG. 6 shows an example of a music selection screen according to the embodiment of the present invention. This music selection screen allows you to select the music recommended by the karaoke dealer, and displays music for each item such as “Featured work”, “Like! Ranking”, “Similar! Ranking” Has been. The user can select a song to be sung using this song selection screen. In addition, on the music selection screen, it is also possible to listen to post information posted on the displayed music. When listening is desired, it is possible to reproduce the posted information of the music by operating the playback button displayed on the right side of each music. Note that the selection of music is not limited to the music selection screen of FIG. 6, and it is also possible to search by specifying a music name or singer name.

  When the user decides a song to be sung, the song detail screen shown in FIG. 7 is displayed on the display screen 31. In this music detail screen, various information regarding the music, such as the song name and singer name, as well as various settings during performance can be made. In the present embodiment, it is possible to set the sound emission of guide vocals (guide vocal information described in FIG. 3), the key (performance pitch), and the tempo (performance speed). A user who confirms various information related to music and performs various settings related to performance operates the playback start button 311 labeled “singing” to start performance of the music information.

  On this music detail screen, tabs such as “My Score”, “Ranking”, and “Linked Service” are displayed, and the user can refer to related information by specifying a desired tab. . By selecting “My Score” among these, it is possible to view the highest score or history of the singing score for the displayed music. “Ranking” allows browsing the ranking of scoring information including other users for the displayed music. By operating the ranking detail display button 312 displayed on the right side, a detailed ranking can be viewed.

  By operating the reproduction start button 311 labeled “sing” (S102: Yes), the performance information is started to be played for the designated music piece (S103). In the present embodiment, the song evaluation process S104 and the recording process S105 are performed during the performance of the music. The singing evaluation process S104 is a process of evaluating the singing by comparing the reference pitch information included in the music information and the pitch of the singing voice signal input from the microphone 17b. In this embodiment, the singing evaluation is performed. It is set as the form which performs scoring converted into a score.

  The recording process S105 is a process of recording a singing voice signal input from the microphone 17b to form recording information. In the present embodiment, the recording sound signal and the performance sound signal based on the performance information are mixed and recorded. It has become. It is good also as recording only a song audio | voice signal not only in such a form. In that case, when the recorded information is reproduced, the singing state can be reproduced by performing the performance information of the corresponding music in synchronization.

  FIG. 8 shows a music singing screen displayed on the display screen 31 during music performance. On the song singing screen of the present embodiment, based on the lyrics display area 314 in which lyrics are synchronously displayed based on the lyrics information included in the song information, the reference pitch information and the singing voice input from the microphone 17b. An evaluation display area 313 that can confirm the singing evaluation is provided. The evaluation display area 313 has a horizontal axis as a time axis and a vertical axis as a pitch axis, and a pitch bar 313a based on reference pitch information scrolls to the left as the performance progresses. . The position indicated by the broken line corresponds to the current timing, that is, the singing position. In addition, the pitch of the singing voice input from the microphone 17b with the singing of the singer is sequentially formed on the left side of the broken line as a solid line 313b. In addition, it is good also as changing the display form (a color, a shape, etc.) of the bar of the singing location by the case where the pitch of a song voice adapts to reference | standard pitch information, and the case where it does not adapt.

  Thus, in the present embodiment, as evaluation information, the pitch based on the reference pitch information is displayed as the bar 313a during the singing, and the pitch of the singing voice input from the microphone 17b is displayed as the solid line 313b. It is possible to sing while confirming the singing state.

  Furthermore, a playback management tool 315 is displayed below the song singing screen. This reproduction management tool includes a performance bar that indicates the suspension of performance information and the progress of the performance status. When the performance of the music is finished (S106: Yes), the result of the singing evaluation process (in this case, scoring information) is displayed on the display screen 31.

  FIG. 5 is a block diagram showing a reproduction process (first time) according to the embodiment of the present invention. The reproduction process of this embodiment is a process that is mainly executed by the acoustic processing unit 16 under the control of the CPU 11. In the pitch detection process (P101), the pitch information is extracted by performing an acoustic analysis process such as FFT on the singing voice input from the microphone 17b. In this embodiment, the pitch is used as the evaluation target, but as the evaluation target, various characteristics of the singing voice such as the volume as well as the pitch can be used. The first reproduction process (P104) is a process of reproducing a musical piece as an accompaniment based on performance information in the musical piece information. In the first reproduction process (P104), reference pitch information, that is, information indicating the pitch to be sung is output to the evaluation process (P102) in synchronization with the reproduction of the music.

  In the evaluation process (P102), evaluation information is formed based on the reference pitch information output in the first reproduction process (P104) and the pitch information output in the pitch detection process (P101). In the evaluation information in this embodiment, the pitch bar 313a based on the reference pitch information and the pitch of the singing voice input from the microphone 17b are displayed as a solid line 313b in the evaluation display area 313 described with reference to FIG. Information. The evaluation information formed in the evaluation process (P102) is displayed and output on the touch panel display unit 18 in the display process (P103). Note that the evaluation information can be formed in various forms as long as it is information that can teach the singing evaluation based on the reference pitch information and the pitch of the singing voice, such as the difference of the pitch of the singing voice with respect to the reference pitch information. is there.

  On the other hand, the performance information in the music information reproduced in the first reproduction process (P104) is output as a performance sound and mixed with the singing voice input from the microphone 17b (P106). In the recording process (P105), the singing voice input from the microphone 17b is recorded. The recorded signal is temporarily stored in the RAM 12 or the like of the portable terminal 1 as recording information. Further, the mixed signal (P106) is emitted from the speaker 17a. Thus, the recording process (P105) of this embodiment is intended for recording only singing voice. Therefore, in this embodiment, when recording information is reproduced, the state at the time of singing can be reproduced by synchronizing and reproducing the music information.

  FIG. 9 shows a screen example of the scoring result display screen. At that time, the portable terminal 1 transmits the singing music scoring information to the karaoke host device 5 and compares it with other scoring information related to the music, and acquires and displays the ranking related to the music (S107). The user who has confirmed the scoring information operates the button labeled “Next” to proceed to the management process of the recording information recorded in the recording process S105.

  FIG. 10 shows a recording information management screen for performing various types of management of recording information. On this recording information management screen, the user can select “post singing voice”, “reproduce singing voice”, “save singing voice”, “end”, and the like. When “Reproduce singing voice” is selected, the recording information formed in the recording process of S105 is reproduced. The user can listen to and confirm the recording information of his / her song. When “save” is selected (S108), the formed recording information is saved in the storage unit 13 in the portable terminal 1 (S109). The stored recording information can be read out later for playback or posting. When “Finish” is selected, the recorded recording information is discarded, and a series of posting processes is terminated.

  When “Post a singing voice” is selected (S110: Yes), a process for transmitting the formed voice information to the karaoke host apparatus 5 starts. In the present embodiment, only the section (within a predetermined time) designated by the user in the editing process is included in the posted information and transmitted in the formed recording information. Therefore, the capacity of the recording information is reduced, the burden on the communication line and the karaoke host device 5 can be reduced, and the posting user can also post by specifying the singing place. It becomes. In addition, for the listener who listens to the posted information, it is possible to listen to the singing listening place with respect to the uploaded posted information, and it is possible to efficiently listen to the posted information. Instead of such a form, all the recording information may be transmitted (posted).

  When transmitting the formed audio information to the karaoke host device 5, first, an editing process S111 is executed. As described above, this editing process is an editing (cutting out) process so that the recording information can be stored within a predetermined time (within 90 seconds in the present embodiment), and the operation of the cutting section is performed by the user (contributor) himself. Done. FIG. 11 shows a recording information editing screen according to the embodiment of the present invention. On this recording information editing screen, an editing window 317 for editing the recording information, an editing tool 318, and a “OK” button for designating the end of editing are displayed. The user can specify the section within a predetermined time by operating the operator displayed on the editing window 317 or the editing tool 318 to specify the cut-out section and operating the “OK” button. is there. The editing tool 318 includes two wheel switches that can specify the start time (start position) of the cut-out section and the end time (end position) of the cut-out section, playback of the specified section, and the start of the specified section. It is possible to specify whether to apply fade-in and fade-out at the end. In the present embodiment, it is possible to use a manipulator displayed in the editing window 317 and a wheel switch displayed in the editing tool 318 for specifying a cut-out section (designating a start position and an end position). . By tracing around each wheel switch, the start time and end time can be advanced and retracted.

  FIG. 12 shows various forms of the editing window according to the embodiment of the present invention. FIG. 12A shows an edit window 317 having the same form as the display screen 31 of FIG. In the editing window 317, waveform information indicating the amplitude value of the recording information formed by the recording process is displayed. The waveform information is expected to have a large amplitude in a section such as rust. In addition, the amplitude is expected to be small in the interval such as interlude, prelude, and postlude. The user operates the operation element 317a for designating the section start position and the operation element 317b for designating the section end position based on the amplitude of the waveform information (or the wheel switch in the editing tool 318 in FIG. 11) to cut out. Specify the interval. At this time, the distance between the two operators 317a and 317b is not longer than a predetermined time (90 seconds). The user can specify the section to be cut out using the interface of the editing window 317.

  FIG. 12B shows a form in which the section type is displayed in the edit window 317. This section type is acquired based on the section information of the reproduced music information when performing the recording process. In the case of posting a limited section, it is common to post a singing section among the section types. Therefore, in the present embodiment, a mode is adopted in which only the section type corresponding to the singing section is displayed in the editing window 317 among the section types. The user can specify the section to be posted by operating the operation element 317a and the operation element 317b based on the section displayed in the editing window 317 (or the wheel switch in the editing tool 318 in FIG. 11). It has become. Further, in the present embodiment, the display form (in the present embodiment, the shape of the arrows at both ends) is changed according to the section type, and the user can easily grasp the section type. Thus, in the form using the section information, it may be possible to easily specify the section type separation position. For example, by setting the operation elements 317a and 317b to be attached to the head or end of the section specified by the section type, it is possible to cut out the recording information accurately and easily at the section position of the section type. .

  FIG. 12C shows that a plurality of sections can be specified in the recording information. The user can designate the number of sections using an interface (not shown), and the operators 317a to 317d are displayed according to the designated number of sections. In the example of FIG. 12C, two sections are cut out, and two operators 317a to 317d (four in total) are displayed in each section. When a plurality of sections can be specified in this way, the total of each section needs to be within a predetermined time. Accordingly, the movement range of each of the operators 317a to 317d is limited so as not to exceed a predetermined time.

  FIG. 12A illustrates a form for displaying waveform information, FIG. 12B illustrates a form for displaying section types, and FIG. 12C illustrates a form for allowing a plurality of sections to be specified. Can be performed in appropriate combinations. In addition, a simple interface as shown in FIG. The section designation shown in FIG. 12D is an interface for designating the section start time. The user can set the section start time by matching the rotary drum 317e for specifying the start fraction and the rotary drum 317f for specifying the start seconds to the start position 317g. In this case, a section from the designated section start time to a predetermined time (90 seconds in this embodiment) is designated in the recording information.

  In the present embodiment, a time length limit (90 seconds) is provided for the recording information included in the posted information. However, in the present invention, this time length restriction is not necessarily provided. Further, in the editing process, the section type portion (prelude, interlude, postlude, etc.) corresponding to the non-singing section may be automatically deleted in the recording information. When listening to the recording information included in the posted information, it is conceivable that the part where the user is singing is the main listening target. By performing such deletion processing, it is possible to delete a portion unnecessary for listening and to reduce the capacity of post information to be uploaded. This deletion process may be performed on the recording information in the uploaded post information in the karaoke host device 5.

  When the editing process (S111) for designating the section in the recording information is completed, the user to be posted is authenticated. If the user has already logged in (S112: Yes), the process proceeds directly to related information input (S114). If the user has not logged in (S112: No), the user is logged in. A login process (S113) is executed. At that time, if the user does not have an account, new registration for acquiring an account is performed. When the user login is completed, various post related information to be included in the post information is input (S114). This post-related information is various information that is disclosed together with the recording information, and in the present embodiment, post text and post images can be used.

  FIG. 13 shows a post related information input screen according to the embodiment of the present invention. On this post-related information input screen, a user name registered in the account of the logged-in user, a post text entry field 319, a post image selection button 320, and a post button 321 are displayed. In this embodiment, the user name registered in the account is used as it is, but this user name may be changed by user input. When the post text entry field 319 is designated, it is possible to perform text input using a soft keyboard or the like. The posting user inputs a comment to be communicated to other users regarding the posting. The posted image selection button 320 includes an image selection button 320a and a shooting button 320b. When the image selection button 320a is operated, a posted image is selected from images stored in the storage unit 13 of the portable terminal 1. Is possible. On the other hand, when the shooting button 320b is operated, it is possible to take a picture with the front camera 15F or the rear camera 15R of the portable terminal 1 and use the taken image as a post image.

  The user who has finished inputting the posting related information operates the posting button 321 indicated as “posting” to thereby display the posting information including the edited recording information, the input posting related information, etc. 5 (S116). FIG. 14 shows a data structure of post information according to the embodiment of the present invention. The posting information of this embodiment is composed of meta information and actual information. The actual information includes recording information, pitch information, and reference pitch information. The recording information is recording information at a location designated in the editing process, and in this embodiment, the recording information is composed of a singing voice input from the microphone 17b.

  The pitch information is information indicating a time-series change of the singing voice input from the microphone 17b in the singing evaluation process (S104 or P102) during singing. As with the recording information, this pitch information is obtained by cutting out a section designated by the editing process. The reference pitch information is included in the music information that is the subject of singing and, like the recording information and the pitch information, is a section that is specified by the editing process. As described above, in the present embodiment, the pitch information and the reference pitch information corresponding to the location specified in the section of the recording information are included, and when the recording information is reproduced, the evaluation information is based on the pitch information and the reference pitch information. Can be formed. Therefore, when playing back recorded information, it is possible to confirm what kind of singing the posted user is performing on the reference pitch information by forming and displaying evaluation information based on the pitch information and the reference information. It becomes possible. For example, in addition to being able to confirm in which part the singing is successful or making a mistake, it is possible to check the user's own singing.

  The meta information includes information related to the music used during recording, such as music identification information, music name, singer name, and the like. In addition, user identification information included in the account of the poster is included in order to be able to identify which user posted the posted information. In addition, as information on the performance of music, key (pitch) information and tempo (speed) information set before the start of performance are included. Moreover, final evaluation information such as a scoring value calculated at the end of the performance is included, and can be used for ranking posting information. Also included are edit position information indicating the cut-out section of the recording information, reproduction position information indicating the reproduction position of the performance information when reproducing the recording information, and post-related information input on the post-related information input screen in FIG. .

  Post information transmitted (posted) to the karaoke host device 5 is stored in the storage unit 41 of the karaoke host device 5 and is also disclosed on the Web server 52. Prior to publishing on the Web server 52, it is judged whether or not inappropriate sound has been recorded, or whether or not inappropriate images and words are included in the post-related information. Only posted information may be made public.

  In the present embodiment, posting information can be listened to in a program that allows posting processing. Then, about the listening form of this posting information, the form using the portable terminal 1 is demonstrated. Note that the posting information is not limited to the portable terminal 1 and may be executed in various information processing apparatuses such as a karaoke apparatus, a personal computer, or a game machine.

  When listening to posted information, that is, when recording information corresponding to a part of music is reproduced by editing processing at the time of posting, it is conceivable that the user who listens sings while listening to the singing voice of the recorded information. It is conceivable that singing voice recorded immediately after the reproduction is reproduced from the recording information appropriately cut out by the editing process. Therefore, it is difficult for a user who listens to the recording information to sing the song information in synchronism with the singing voice immediately after the reproduction. Furthermore, in this embodiment, it is supposed that the battle | competition regarding the recording information cut out by the edit process and singing evaluation will be performed. That is, a mode for competing for evaluation of the singing voice of the recorded information and superiority or inferiority of the evaluation of the singing voice of the user who listens to the recorded information is provided. In such a case, if a part that is not a normal start is suddenly played as a singing target, it may be confused about where to sing, and when competing for evaluation superiority or inferiority, it is a disadvantageous condition for the singing user .

  In the present embodiment, when listening to the recording information corresponding to a part of the music cut out by the editing process, the singing sound corresponding to the partial section of the music is reproduced, and the position from the position before the partial section of the music is reproduced. It is supposed to start playing the music performance sound. With such a configuration, a user who sings along with the recording information can listen to the performance sound prior to the singing voice corresponding to a part of the music and confirm where the music can be sung. It becomes possible.

  FIG. 14 (A) shows the relationship between the audio playback section and the performance sound playback section according to the embodiment of the present invention, and displays the music start point to the music end point in time series. In the figure, a section R (start point RS, end point RE) is a section designated in the editing process and corresponds to a section from which recording information is cut out. In such a case, by playing the performance sound from the position before the starting point RS in the music, it is possible to make the user who listens to the posted information know in advance which part of the music should be sung. It becomes. Therefore, at the time of listening to the posted information, the performance sound is reproduced in the section P from the start point PS, which is the position before the start point RS, in the music progression, to the end point PE, which is the same position as the end point RE.

  FIG. 14B shows the reproduction of recorded information and performance information in time series when listening to posted information. The performance information is reproduced in the section P from the start point PS to the end point PE prior to the reproduction of the recorded information. On the other hand, the recording information is reproduced in the section R from the start point RS to the end point RE after the start of reproduction of the performance information. The start point RS and the end point RE of the section R of the recording information can be arbitrarily designated by the user who performs the editing process (the user who makes the posting). It is preferable to set a good position. For example, it is conceivable to set a predetermined position defined in the music, such as a bar break or a lyrics phrase break. By setting in this way, it is possible to suppress a sense of discomfort at the start of listening to the posted information.

  In the present embodiment, the performance sound playback section P is determined during the posting process. The determined playback section is recorded as playback position information in the posted information. When the posted information is reproduced, a part of the performance information corresponding to the music identification information is reproduced with reference to the reproduction position information. On the other hand, the recorded information is reproduced at the reproduction timing based on the editing position information in the posted information, so that reproduction in synchronization with the performance information is performed.

  In the embodiment of FIG. 14, the end point RE of the section R of the recording information and the end point PE of the section P of the performance information are the same time point, but the end point PE of the section P of the performance information is the end point RE of the section R of the recording information. It may be set later. FIGS. 15A and 15B show such a reproduction mode. FIG. 15A is a diagram showing the position of the recording information and performance information in the progression of the music, and FIG. 15B shows the time series relationship between the recording information and the performance information when reproducing the posted information. FIG. As in the case of the start point PS, by setting the end point PE of the performance information at a well-cut position for the performance of the music, it is possible to suppress the uncomfortable feeling at the time of listening even when the reproduction of the posted information is finished.

  In FIG. 14 and FIG. 15, one section is targeted for the progress of the music, but as described with reference to FIG. 12C, a plurality of sections can be designated as posting targets. Even when a plurality of sections are designated, the start points PS1 and PS2 of the performance information are set at positions before the start points RS1 and RS2 of each recording information section. In the embodiment of FIG. 16, similarly to the embodiment of FIG. 15, the end points PE1 and PE2 of the performance information are set to positions that are well cut off after the end points RE1 and RE2 of the recording information, so that the uncomfortable feeling at the time of listening is set. It is suppressed. By setting in this way, it is possible to suppress the uncomfortable feeling at the time of listening to the connection between the section P1 and the section P2, which are performance sections of the performance information. As described above, when the end point PE1 of the performance information is set after the end point RE1 of the recording information, it is conceivable that the section P1 and the section P2 overlap each other. In that case, it is preferable to integrate the performance sections P1 and P2 of the performance information into one section having the start point PS1 and the end point PE2. FIG. 16B is a diagram showing a time-series relationship between the recording information and the performance information when reproducing the manuscript information. The performance information sections P1 and P2 are continuously played, and the recording information sections R1 and R2 are reproduced in synchronization therewith.

  FIG. 18 shows a scoring ranking screen that allows selection of post information to be listened to. This scoring ranking screen is a screen displayed by operating the ranking detail display button 312 displayed in the “ranking” tab in the music detail screen described with reference to FIG. On the scoring ranking screen, scoring values, user names, and playback start buttons 322a to 322c are displayed in descending order of scoring values (evaluation information) in the posted information for the music specified on the music detail screen. It is a screen. The user can listen to the recording information of the user by selecting the playback start buttons 322a to 322c corresponding to the user who wants to listen. Note that a user whose playback start buttons 322a to 322c are not displayed is a user who has not posted and cannot listen to the recording information.

  By operating the playback start buttons 322a to 322c for a certain user, playback of recorded information is started. FIG. 19 is a block diagram showing a reproduction process (at the time of reproduction of posted information) according to the embodiment of the present invention. Although it is possible to reproduce only the posted information, in the present embodiment, the second evaluation information is formed by reproducing the posted information and executing the singing evaluation process on the singing voice input from the microphone 17b. In addition, it is possible to compare with evaluation information based on posted information. Furthermore, in this embodiment, it is possible to perform a further posting process by performing a recording process on the singing voice input from the microphone 17b.

  In the pitch detection process (P101), the second pitch information is detected by performing an acoustic analysis process such as FFT on the singing voice input from the microphone 17b. On the other hand, post information is separated into recording information (singing voice) and pitch information in the separation process (P107). In the first reproduction process (P110), a performance is performed based on performance information corresponding to the music identification information included in the posted information and a performance sound is output. In addition, reference pitch information is output in synchronization with the performance information performance. In the evaluation process (P102), the first evaluation information for the recorded information in the posted information is formed based on the reference pitch information and the pitch information, and the second detected by the reference pitch information and the pitch detection process (P101). The second evaluation information is formed based on the pitch information. 1st evaluation information is evaluation with respect to the recording information of contribution information, and 2nd evaluation information is evaluation with respect to the song sung according to reproduction | regeneration of contribution information this time. In the display process (P103), the first evaluation information and the second evaluation information are displayed on the touch panel display unit 18.

  The recording information separated in the separation process (P107) is reproduced in the second reproduction process (P108). The performance information reproduced in the first reproduction process (P110), the recording information reproduced in the second reproduction process (P108), and the singing voice input from the microphone 17b are mixed in the mixing process (P111). Sound is output from the speaker 17a. The performance information in the first reproduction process (P110) and the recording information in the second reproduction process (P108) can be reproduced in synchronization based on the editing position information and the reproduction position information included in the posted information. is there. About the form of synchronization, it is as having demonstrated in FIGS. 14-16, Comprising: By performing synchronous reproduction, the singing state at the time of recording of recording information is reproduced.

  In the present embodiment, the first evaluation information is formed based on the pitch information and the reference pitch information included in the recording information, and the singing evaluation for the recorded information by the posted user can be reproduced. . A user who receives and reproduces the posted information can check the singing evaluation of the posted user based on the first evaluation information displayed and output on the touch panel display unit 18. Further, in the present embodiment, second evaluation information based on the singing voice and the reference pitch information is formed for the singing voice sung in conjunction with the reproduction of the posted information, and is displayed on the touch panel display unit 18. Yes. According to such a configuration, the user who reproduces the posted information can compare the first evaluation information and the second evaluation information, and can compare and confirm the own song and the posted user song. It becomes.

  FIG. 20 shows an example of a recorded information reproduction screen displayed when the posted information is reproduced. In the scoring ranking screen of FIG. 18, the playback start button 322b is operated for the second-ranked Mr. B. On the recording information playback screen, the song name, singer name, contributor's user name, scoring value (final evaluation information), post image 325, post text 326, and lyrics information 327 input as post related information are displayed. The The lyrics information 327 can be acquired from the corresponding music information based on the music identification information included in the posted information. At that time, all the lyrics information may be displayed for the music to be played back, but based on the editing position information in the posted information, the lyrics information 327 at the location corresponding to the recording information cut out by the editing process is displayed. It is good.

  An evaluation display area 313 is provided at the center of the recorded information reproduction screen. In the evaluation display area 313, the pitch based on the reference pitch information is displayed as a bar 313a, and the (first) pitch information included in the posted information is displayed as a broken line 313b. The display form related to the evaluation information is substantially the same as the evaluation display area 313 described with reference to FIG. 8, and the horizontal axis is the time axis and the vertical axis is the pitch axis. Scroll to the left along with it. The position indicated by the broken line corresponds to the current timing, that is, the singing position. Thus, in this embodiment, when reproducing the posted information, the first display information is displayed in the evaluation display area 313 based on the first pitch information included in the posted information and the reference pitch information. It becomes possible to confirm what kind of singing is performed on the reference pitch information by the posted user.

  In the present embodiment, as described with reference to FIGS. 14 to 16, the performance is started from a position (starting point PS in the case of FIG. 15) before the section in which the recording information is reproduced (section R in the case of FIG. 15). The sound playback is started. In this case, it is preferable that the user who listens to the posted information can be notified whether or not the current playback position is the playback section of the recording information. For example, in the evaluation display area 313 where the progress of the performance is displayed, the playback section of the recording information is displayed so that the playback section of the recording information can be recognized by changing the display form (color and shape) of the bar 313a. It is possible to do. Further, the arrival of the start point of the recording information may be notified by a countdown display, or may be notified by an acoustic output such as voice. Moreover, it is preferable to display lyrics in synchronization with the reproduction of the performance information. By displaying the lyrics in synchronization with the performance information, the listening user can recognize in advance which section the recording information sings in the music.

  When reproducing a partial section of a music piece cut out by editing processing as in the present embodiment, it may be possible to finish the reproduction in a short time. Furthermore, in this embodiment, it is supposed that the evaluation with respect to the recording information in contribution information and the evaluation with respect to the song sung according to reproduction | regeneration of contribution information are performed. In such a form, it is conceivable to perform a so-called singing battle that competes for superiority or inferiority of both evaluations. Conventionally, singing battles have been performed in karaoke apparatuses, but in order to perform a battle through one song, it is necessary to sing all one song, and it has not been possible to perform it easily in a short time. As in this embodiment, it is possible to perform a singing battle in a short time by targeting a part of the song as a battle target, and it is also possible to compete for superiority or inferiority in some sections such as a chorus section. It is possible to provide new services that can be performed.

  In the present embodiment, the section edited by the editing process of the posting user is the target of the battle. In such a case, if a part of the music is suddenly reproduced, it is considered that the user as the opponent is confused about where to sing, from which singing evaluation is disadvantageous. In the present embodiment, as described above, it is possible to maintain fairness in the battle with the poster by notifying the arrival of the start point of the recording information by various means.

  In addition, when playing recorded information, the bar display format (color, shape, etc.) of the sung portion is determined depending on whether the pitch information (first pitch information) of the recorded information matches the reference pitch information or not. ) May be changed. At that time, a plurality of reference pitch information may be provided for each difficulty level, and may be switched by a user who performs playback. When performing evaluation based on pitches as in the present embodiment, it is conceivable that the range of compatible pitches is different, for example, the higher the difficulty level, the narrower the range of compatible pitches. When reproducing the posted information, the user who plays the posted information can set the degree of difficulty using a user interface (not shown) to switch the reference pitch information and evaluate the posted information based on different evaluation criteria. In this case, not only the display of the evaluation display area 313 in FIG. 20 but also the final evaluation information (scoring value) described in FIG. 9 may be changed according to the set difficulty level.

  Further, in the present embodiment, the second pitch information detected in the pitch detection process (P101) is displayed by a solid line 313c. In the present embodiment, the second evaluation information is formed by the bar 313a based on the reference pitch information and the solid line 313c based on the second pitch information. By displaying the second evaluation information together with the first evaluation information, the user's singing singing in accordance with the reproduction of the posted information is the reference pitch information, and the pitch information of the user who has posted (the first pitch information). It is possible to confirm how they differ with respect to the pitch information.

  A playback management tool 323 including a progress bar for specifying a playback position and a pause button is displayed at the bottom of the recording information playback screen. Below the playback management tool 323, a voting button 324a labeled “similar !!” and a voting button 324b labeled “like!” Are displayed. The user who has listened to the recorded information can evaluate the recorded information by operating these voting buttons 324a and 324b. Note that these voting buttons 324a and 324b may be displayed or operable when the recording information is listened to for a certain period of time or when the listening is completed. It is possible to suppress fraud in which voting is performed without listening to recorded information.

  The voting operation by the voting buttons 324a and 324b is transmitted to the karaoke host device 5 as voting information regarding the post information being listened to. The karaoke host device 5 counts voting information for each voting item and uses it to rank each item described in FIG. 6 (“Like!”, “Similar!”) To introduce post information. It becomes possible.

  After the reproduction of the posted information is completed, it is possible to compare the first evaluation information and the second evaluation information output in the evaluation process (P102) and determine the superiority or inferiority. FIG. 21 shows a result display screen according to the embodiment of the present invention. After the reproduction of the posted information, the first evaluation information and the second evaluation information are finally compared, so that the result of the singing battle between the user who posted the posted information and the user who reproduced the posted information is Determined. In the result display screen of FIG. 21, a button labeled “re-challenge” is displayed. By operating this button, the user can reproduce the same posted information again and play a battle. As described above, in the present embodiment, it is possible to perform a singing battle in a short time and in a partial section such as a rust section by setting a partial section of the music as an evaluation target. Moreover, in this embodiment, it determines by the edit process by the user who contributed some sections of the music, and it becomes possible for the user who contributes to arbitrarily designate the section to be a battle target.

  As described above, the posting process and the listening process for the posted information have been described using the portable terminal 1 as an example. However, the posting process and the listening process are not limited to the portable terminal 1, but also a karaoke device, a personal computer, It can be performed in various information processing devices such as game machines, and can provide postings by singing recording to a wide range of users.

DESCRIPTION OF SYMBOLS 1 ... Portable terminal (information processing apparatus) 17b ... Microphone 11 ... CPU 18 ... Touch panel display part 12 ... RAM 18a ... Touch panel 13 ... Memory | storage part 18b ... Display part 14 ... Image control part 19 ... Communication part 15F ... Front camera 20 ... Input Switch 15R ... Rear camera 21 ... External input terminal 16 ... Sound processing unit 22 ... External memory connection terminal 17a ... Speaker

Claims (10)

A program executed in an information processing apparatus capable of communicating with a host device via a network,
A receiving step of receiving recording information including singing voice information corresponding to a partial section of the music from the host device;
Based on the recording information received in the receiving step, the singing voice indicated by the singing voice information corresponding to a partial section of the music is reproduced, and the performance sound of the music is reproduced from a position before the partial section of the music. A program for causing an information processing apparatus to execute a playback step for starting playback. The program according to claim 1, wherein the reproduction step reproduces a performance sound based on music information corresponding to the music used when recording the recording information. The program according to claim 1, wherein the reproduction step notifies a singing voice reproduction start timing corresponding to a partial section of the music. The program according to any one of claims 1 to 3, wherein the recording information is included in the posted information transmitted to the host device by the user. The program according to any one of claims 1 to 4, wherein the reproduction step starts reproduction of a performance sound from a predetermined position defined for the music. The recording information includes singing voice information corresponding to a plurality of partial sections of the music,
The program according to any one of claims 1 to 5, wherein the reproduction step reproduces a singing voice corresponding to a plurality of the singing voice information. The said reproduction | regeneration step displays the 1st evaluation information formed based on contrast of the song reference | standard information corresponding to the said recording information, and the song audio | voice information used as the recording object of the said recording information. 6. The program according to any one of 6. The said reproduction | regeneration step displays the 2nd evaluation information formed based on contrast of the song audio | voice information corresponding to the song audio | voice input from the microphone, and the song reference | standard information corresponding to the said recording information. Program. The reproduction step includes the first evaluation information formed on the basis of the comparison between the singing reference information corresponding to the recording information, the singing voice information that is the recording target of the recording information, and the singing voice input from the microphone. 7. The superiority or inferiority of the second evaluation information formed based on the comparison between the singing voice information corresponding to the singing information and the singing reference information corresponding to the recording information is displayed. program. An information processing apparatus capable of communicating with a host device via a network,
Receiving means for receiving recording information including singing voice information corresponding to a partial section of the music from the host device;
Based on the recording information received by the receiving means, the singing voice indicated by the singing voice information corresponding to a partial section of the music is reproduced, and the performance sound of the music is reproduced from a position before the partial section of the music. And an information processing apparatus.