JP2014123085A - Device, method, and program for further effectively performing and providing body motion and so on to be performed by viewer according to singing in karaoke - Google Patents

Abstract

PROBLEM TO BE SOLVED: To further effectively perform and provide an interlude such as a body motion, a shout, musical instrument performance to be performed by a viewer according to rhythm of a musical piece in Karaoke by using an electronic device.SOLUTION: Video data, image data, and sound data are reproduced S14 according to performance of a musical piece on the basis of interlude data S6 and model data S8 specified as pieces of data to be used for the musical piece to be performed in Karaoke, information on an interlude performed by a viewer is collected S16 by an interlude input device and a sound input device, the interlude data is generated S18, the video data, the image data, and the sound data corresponding to the generated interlude data are reproduced S20, the generated interlude data is evaluated S22, and the generated interlude data and evaluation information, etc. are transmitted to a server device, and transmitted S26 according to collation from another Karaoke device.

Description

  The present invention relates to an apparatus, a method, and a program for collecting information on a matching hand performed by a person who views a karaoke singer in an electronic apparatus and reproducing an image, video, and sound corresponding to the content.

  In general, in singing with an audience, those who listen to the singing and choreography of those who sing, from small things like karaoke to large things like concerts There are often so-called “matching hands” that perform specific body movements, make voices, or play musical instruments in conjunction with singing.

  Matching hands not only have an effect of raising the sense of exhilaration by satisfying the approval desire that `` others are responding to their singing all at once '' to those who sing to receive matching hands, but also according to the singing Even for those who are watching the audience, they are not just passive listeners of the music, but are placed in an active position to participate in the music together with other viewers. Has an effect to increase. In this respect, the mate is an important component in entertainment where people who sing, such as live performances and concerts, and viewers unite.

  Usually, in order to perform a match, the viewer needs to be in front of the singer, so the singer and the viewer must share the same place for the same time (time / place) Constraints). In other words, the feeling of exaltation that can be enjoyed by a person who sings by receiving the hand of a singer who receives the majority of the viewers at the same time can gather the majority of the viewers at the same place at the same time. It is not possible to achieve this without providing a sense of excitement that can be experienced by those who perform singing in a live performance gathering a large audience, for example, in a small space such as karaoke. Have difficulty. In addition, hand gestures, vocalization, musical instrument performance, etc. are performed by the viewer's own body or by the instrument owned by the viewer, so there are limitations and limitations on the content and scale (physical constraints). ). In other words, it is impossible to achieve a match of hands that a large number of viewers perform at the same time unless a large number of viewers do a match, and the volume and type of sound that can be produced by a watcher must be viewed. It is limited to the voices and sounds of musical instruments that can be produced by their ability and skill. Furthermore, in karaoke, there is no example of how to improve the feeling of uplifting and immersive feelings for the viewer.

  In the present invention, in the karaoke, the above time, place, and physics are presented by presenting a model of a good match by combining information collection / distribution by server technology and network technology and production of video and audio by electronic equipment. Regardless of technical restrictions, those who sing in karaoke (hereinafter referred to as “singer”) and those who listen to singing in karaoke (hereinafter referred to as “viewers”) more effectively and more easily. ”) (Hereinafter referred to as“ user ”inclusive of singers and viewers).

  In the present invention, an apparatus and method for displaying an example of a matching hand corresponding to a song sung at karaoke, and an apparatus for detecting an expression of a matching hand performed by a viewer, recording it as information, and transmitting / receiving it to / from a server device And a method, and an apparatus, method, and program for reproducing images / videos / audio corresponding to the information on the matching hand are provided.

  That is, a method for more effectively directing and providing physical movements performed by the viewer in accordance with the rhythm of the music in electronic karaoke, a voice, a musical instrument performance, and the like, and reading the model data into the karaoke device A step of determining a display method of the sample data in accordance with conditions set in advance in the karaoke device or designated by the user, and the sample data in accordance with the progress of the music being played back from the audio playback device The part of the display device according to the display method in which the part in which the contents of the matching hand to be performed by the viewer are recorded and the part of the part in which the contents of the matching hand to be performed by the viewer after that point are recorded are determined. Information about the steps displayed on the screen and the matching hand performed by the viewer is collected by the voice input device or the matching hand input device. Detecting the contents of the hand of the match performed by the viewer using the manual input device, generating the hand data of the match based on the detected content of the match hand, and recording the data in the karaoke device in advance according to the contents of the generated hand data of the match A step of displaying image data or video data recorded in the karaoke device on the screen of the display device according to the conditions registered in the device or instructed by the user, or reproducing the audio data recorded in the karaoke device with the sound reproducing device When the user instructs the karaoke device, the content of the image or video to be displayed or the sound to be reproduced is changed in accordance with the instruction content, and the display on the screen of the display device or the reproduction by the sound reproducing device is performed. Step and generated hand data from karaoke device via network line Based on the conditions registered in the karaoke device in advance from the matching hand data recorded in the karaoke device or the server device based on the step of transmitting to the server device and accumulating in the server device and the evaluation of the singing content of the singer An image recorded in the karaoke device in accordance with the progress of the music being played from the audio playback device in accordance with the conditions registered in advance in the karaoke device based on the step of extracting the extracted data and the contents of the extracted matching hand data The step of displaying data or video data on the screen of the display device or reproducing the audio data recorded in the karaoke device from the audio reproduction device, and the karaoke device when a condition registered in advance in the karaoke device is satisfied Or the condition is specified from the hand data recorded in the server device An image recorded on the karaoke device in accordance with the progress of the music being played from the audio playback device in accordance with the conditions registered in the karaoke device in advance based on the content of the extracted match hand data The step of displaying data or video data on the screen of the display device or reproducing the sound data recorded in the karaoke device from the sound reproducing device, and the instruction from the user or the condition registered in advance in the karaoke device is satisfied In this case, the image or video data to be displayed on the screen of the display device or the audio data to be reproduced by the audio reproduction device is determined from the data recorded in the karaoke device according to the instruction from the user or the contents of the registered conditions. Preset in the karaoke device or specified by the user. A step of determining a reproduction method of the image / video and audio data according to the conditions, and a screen of the display device displaying the image or video data according to the reproduction method determined in accordance with the progress of the music being reproduced from the audio reproduction device Reproducing the voice data displayed on or processed from the voice reproducing device; reading out the matching hand model information from the matching hand model data shown to the viewer; A step of performing evaluation based on an evaluation standard registered in advance in karaoke equipment in contrast to information in a book, and when there is an instruction from a user or a condition registered in karaoke equipment is satisfied in advance, When scoring, there is a step that takes into account the content of the scoring for the above hand data and there is an instruction from the user If the conditions registered in the karaoke device are satisfied, the method or program including the step of taking into account the content of the singing in the scoring of the matching hand data, or using the electronic device to match the rhythm of the music There is provided an apparatus for providing and providing more effective hands such as physical movements, voice calls, and musical instrument performances performed by a viewer, and including the above steps as means.

  The electronic device provided with the present invention does not need to be composed of a single device, and may be a set of a plurality of electronic devices having a function of exchanging information with other electronic devices by wire or wireless.

  The present invention is provided to an electronic apparatus having a function of recording and reproducing image / video and audio data.

  The karaoke device is connected to each device to be described later by wire or wireless, and can record image data, video data and audio data, and analyzes the information on the singing of the singer and the matching hand of the viewer according to the result. Image data, video data and audio data can be reproduced.

  The image / video data and audio data used in the present invention need only be recorded in the electronic device when the data is reproduced, and the necessary data is transmitted via the network line every time reproduction is performed as in the so-called streaming reproduction. The data may be always stored in a device capable of recording data such as a hard disk or a flash memory built in the karaoke device.

  In the present invention, a voice input device such as a microphone is used to collect information on a singer's singing and a matching hand by the viewer's voice. The voice input device only needs to be able to collect voices uttered by a singer and a viewer as information, and may be a microphone generally used in karaoke. In addition, in the present invention, it is normal that separate voice input devices are used for the singer and the viewer, but the voice input device or the karaoke device has a function that allows the voices of the singer and the viewer to be collected or recorded separately. If it is provided, a single voice input device may be shared and used by the singer and the viewer.

  In the present invention, an input device (hereinafter referred to as “match hand input device”) is used for collecting information on a match hand performed by the viewer and detecting the content of the match hand performed by the viewer. The information collected by the matching manual input device can include not only information collected by the matching manual input device itself but also information collected by the voice input device. The matching hand input device only needs to be able to collect information on the actions performed by the viewer on the singing and detect it as a matching hand. From a sophisticated device such as a sensor device that detects the action content of the human body from the change in position to a simple device such as a button device that detects only when the viewer presses which button at what timing It doesn't matter.

  In the present invention, an instruction input device is used for the user to give instructions to the karaoke device regarding the method of performing various functions and the like. The instruction input device is typically a touch panel device or an input device with operation buttons, but is not limited thereto.

  The display device used by the present invention is not limited to this as long as it has a screen capable of displaying image data and video data transmitted from a karaoke device, for example, a liquid crystal display.

  The sound reproducing device used by the present invention is not limited to this as long as it can reproduce sound data transmitted from the karaoke device as sound, for example, a speaker is typical.

  The image data used by the present invention is, for example, an electronic image in an arbitrary data format such as bitmap, JPEG, JPEG2000, GIF, TIFF or other unique data format, and records a pre-produced image. The data including the parameter information for generating the data of the image to be displayed by rendering in the karaoke apparatus may be used.

  The video data used by the present invention is, for example, an arbitrary video format such as MPEG2, FLV, AVCHD, and other unique data formats, such as an electronic video. Data including parameter information for generating video data to be displayed by rendering in the karaoke apparatus may be included.

  The audio data used by the present invention is, for example, any audio format such as MP3, WAVE, AAC, and other unique data formats. Data including parameter information for generating sound data to be reproduced each time by a sound source device built in the karaoke device, such as MIDI data, may be used.

  The model data refers to the contents of the image or video shown as a model of the matching hand to the viewer and the corresponding relationship in playback time with the corresponding karaoke music, and information on the matching hand as a model, that is, which of the music This is data in which information on which match is ideal for timing is recorded. When the karaoke equipment plays the image or video included in the sample data in accordance with the performance of the karaoke music, the viewer performs not only the contents of the matching hand corresponding to the currently played part of the karaoke music, but also after that All or a part of the content of the hand of the power can be displayed simultaneously on the screen of the display device. This enables the viewer to recognize in advance the contents of the matching hand to be performed after the currently played portion of the karaoke music, so that even a viewer who does not store the contents of the matching hand can easily It is a device to make it possible to perform a matching hand according to.

  Matching hand data refers to information on what kind of physical movement and screaming hands the viewer detected by the matching hand input device performed at which timing of the song, as well as information accompanying this (for example, This data includes the contents of the evaluation performed by the karaoke machine for the match hands, the number of viewers who participated in the match hands, and the gender. The karaoke device refers to the matching hand data and reproduces image data, video data and audio data according to the contents of the matching hand that the viewer is supposed to perform at the same timing as the viewer did the matching hand, The contents of the matching hand performed by the viewer can be expressed at the same time as the viewer performs the matching hand, or can be reproduced after the fact.

  In the model data, in addition to image / video data to be displayed on the screen of the display device as a model, information on karaoke music corresponding to the data, information on which match should be performed at which point of the music being played, etc. It is included. There is not necessarily one example data for one piece of music, and which example data is to be used for performing the matching hand may be set in advance in the karaoke apparatus or determined according to the conditions specified by the user. This allows, for example, more advanced viewers to perform more precise and difficult-to-demonstrate hands, and beginner viewers to perform simpler and easier-to-execute hands. It is possible to present an example of a match of difficulty. Since the model data includes the correspondence relationship with the corresponding karaoke song in terms of the playback time, that is, which part of which image / video data should be displayed at which point of the song, the viewer performs it. Images and videos can be displayed in synchronization with the performance of music.

  The present invention can be used in combination with an evaluation function for singing provided in a general karaoke apparatus. The evaluation function for singing that can be combined with the present invention only needs to have a function of analyzing the content of the singer's voice and performing a score or other evaluation according to a predetermined evaluation criterion.

  The evaluation function for the match hand used in the present invention is a record in which the content of the viewer's match hand detected by the match hand input device and the timing at which the match hand is performed are recorded in the sample data that is modeled on the match hand. Preliminary evaluation of the karaoke device to determine how well the matching hand performed by the viewer matched the information about the matching hand to be used as a book, that is, the information on which matching hand should be used at which timing in the music Can be evaluated according to criteria. The evaluation criteria set for the karaoke device do not necessarily have to be a single one, and according to the skill of each user, for example, from a plurality of evaluation criteria that differ in the degree of strictness with respect to the model and factors emphasized in the evaluation. It can also be made selectable. The evaluation of the matching hand can be performed not only after the performance of the music is completed, but also as an intermediate evaluation at that time during the performance of the music.

  When the matching hand input device detects that the viewer has performed a specific matching hand during the performance of the karaoke music, the karaoke device receives the contents of the matching hand from the matching hand input device and records them as matching hand data. According to the conditions set in the device or specified by the user, the image / video data and audio data recorded in the karaoke device are assigned to the contents of the matching hand, and the data is allocated on the screen of the display device or on the audio reproduction device. Can be played. For example, if the viewer detects that the hand is clapping, it displays the “hand clapping” image or video on the display device screen, and the audio playback device displays “hand clapping”. Sound effects can be played. Note that the content of the video / image and audio to be played does not necessarily reflect the content of the matching hand performed by the viewer, and if it is set in advance in the karaoke device or designated by the user In addition, it is possible to exaggerate and adapt the hand of the viewer performed as a production. For example, by setting the audio playback device to play the sound effect of “cheers” when the viewer detects that the hands are clapping hands, the feeling of exhilaration is different from that of ordinary clapping hands. And can give the user a sense of immersion. Further, the image / video data and audio data corresponding to one matching hand are not necessarily one, and the timing for reproducing the data does not have to be exactly the same as when the viewer performs the matching hand. Thus, for example, a large number of viewers can feel as if a singer is able to play a number of images, video data, and audio data at the same time by shifting the display position on the screen and the playback time for a single hand. It can also give a sense of exhilaration as if you had sent all of your friends together.

  When the matching hand input device detects the matching hand performed by the viewer, which video data / image data and audio data the karaoke device plays as the expression corresponding to the matching hand is determined based on the condition set in the karaoke device in advance or the user However, it can also be determined in consideration of the result of the evaluation performed by the matching hand evaluation function described in [0025]. For example, when a viewer's hand is highly rated, the viewer's willingness to get a higher rating by playing a larger video / image / sound than when a low rating is given. It can stimulate and enhance the sense of exaltation that has been highly evaluated.

  After the performance of the karaoke music piece, the karaoke device displays information on the contents of the matching hand performed by the viewer and information accompanying it (for example, the name of the song provided by the matching hand, the number of viewers who sent the matching hand, gender, age) Final matching hand data including the degree of evaluation of the matching hand, etc.) can be generated, and this can be transmitted to and accumulated in the server device via the network line. When a karaoke device connected to a server device via a network line plays a karaoke song, the server device can inquire the server device to transmit a match hand data recorded for a match performed on the song. The karaoke apparatus receives the hand data of the match, and reproduces video data, image data, and audio data based on the received hand data of the match in accordance with the performance of the music, and expresses the contents of the match hand to the singer. It is possible to give a feeling of uplifting as if there were viewers in front of you and sending your hands to your singing. Note that the matching hand data accumulated in the server device does not necessarily need to be based on the matching hand information actually performed by the viewer, and the matching hand video / image and sound are reproduced according to the production intended by the creator. The data produced by adjusting the data numerical values may be used.

  The karaoke apparatus can reproduce video / images and sounds on a display device and sound reproducing device by using a plurality of hand data simultaneously. For example, it is possible to simultaneously use the match hand data generated by collecting the match hand information currently being performed by the viewer and the match hand data transmitted from the server device, or to simultaneously use multiple match hand data transmitted from the server device. Thus, video / image and audio based on the respective data can be reproduced simultaneously. As a result, it is possible to produce an effect as if various hands of many viewers are being sent to the singer.

  When a plurality of matching hand data are candidates for the server device to transmit matching hand data in response to a request from the karaoke device, the karaoke equipment can receive all or a part of the plurality of data. When there are multiple candidates for matching hand data, which data is used is determined by the conditions set in advance in the karaoke device or specified by the user prior to the performance of the karaoke music, but is necessary during the performance of the karaoke music. Corresponding hand data used to express a matching hand with video / image or audio can be switched accordingly. For example, if the singer's singing is evaluated during the performance of a karaoke song, and the evaluation is above a certain level, the content of the matching hand data is switched to the matching hand data in which the demonstration of the original matching hand was highly evaluated By reproducing the video data / image data and audio data based on the above, it is possible to give the singer an exhilarating feeling as if the viewers are more excited by their singing and are sending their hands together.

  In the match data, not only the content of the match hand but also accompanying information (for example, the evaluation result of the original match hand performance, the number of viewers who sent the match hand, gender, etc.) can be recorded and displayed. It can be displayed on the screen of the device. As a result, when there are multiple candidates for the match hand data, and when it is left to the user to select which match hand data is used to play the video / image / audio of the match hand, the user confirms the contents of each match hand data. Then, you can sing by selecting the matching hand that matches your taste and mood. In addition, the evaluation of the performance of the match hand that is the source of the match hand data recorded in the server device is aggregated for each song, and the ranking is sent to the karaoke device. It can also be shown. The ranking is calculated for the entire piece of music sent by the matching hand, as well as for each viewer attribute information included in the accompanying information of the matching hand data (for example, by difficulty level, by gender, by age, by region, etc.). It can also be done.

  Even when it is left to the user to select which match hand data to use when there are a plurality of match hand data candidates, by setting in advance in the karaoke device, it is possible to set a certain condition to be satisfied by the user for the selection. . Thus, for example, it is possible to provide a hand that can be used only by a user who has a history of singing a certain number of times, or a hand that can be used only by a user who has purchased a separate usage right.

  The method in which the karaoke apparatus reproduces the video / image / sound according to the contents of the matching hand performed by the viewer can be changed according to the designation from the user. The playback method can be changed by selecting the data to be played back from the video / image data or audio data candidates presented by the karaoke device, or by the user-selected processing for specific video / image data or audio data. (For example, in the case of video / image, the display size is enlarged / reduced, the tone is changed, and the animation effect is applied, and in the case of voice, the volume and pitch are changed, but not limited thereto). However, it is not limited to these. Further, by combining a plurality of these methods at the same time, more various reproduction methods can be provided.

  In the present invention, as described in [0021], when the model data is displayed on the screen of the display device, the viewer performs the contents of the matching hand corresponding to the portion of the karaoke song currently being played, and thereafter. All or part of the content of the right hand can be displayed on the screen of the display device at the same time, but at that time, it can be displayed so that the user can easily distinguish between them. Such a display method is not limited as long as the user can visually distinguish between the two. For example, a video / image showing the contents of the matching hand currently being played, and a video showing the contents of the matching hand to be performed by the viewer thereafter.・ The image is displayed in a different color, and the video / image that indicates which of the displayed video / images shows the contents of the hand that is currently being played back is supplemented, or is actually played back on the screen An image / video display area showing the contents of the matching hands and a video / image display area showing the contents of the matching hands to be performed by the viewer after that are provided. However, it is not limited to these. In addition to determining according to a condition preset in the karaoke apparatus, it is also possible to determine by a user selecting a candidate presented by the karaoke apparatus to determine the method by which the two can be distinguished.

  The karaoke equipment can take into account the content of the evaluation of the singer's song when evaluating the hand of the viewer, and the content of the evaluation of the viewer's hand when evaluating the song of the singer. Thereby, for example, when the viewer performs a highly appreciated hand, the singing of the singer can be further evaluated. This is to express to the user clearly and concretely the effect of the match hands on the psychology of the singer and viewer, which is essentially conceptual and abstract, in the form of evaluation results for the match hands and singing. This is a device to raise the user's consciousness about the matching hands and make karaoke more exciting.

  In the present invention, an apparatus, program, or method that allows a viewer who performs a match in karaoke to easily perform a match according to the performance while viewing the example on the screen without storing the content of the match Is provided.

  In the present invention, the singer is evaluated by linking the evaluation with the evaluation of the singer's singing while evaluating how much the matching hand performed by the viewer in karaoke is compared to the matching hand as a model. There is provided an apparatus, a program, or a method capable of providing an effect that further enhances the viewer's sense of unity.

  In the present invention, the effect of the match hands on the psychology of the singer and the viewer is expressed by expressing the match hands performed by the viewer in karaoke with video / images and sounds, and adding various effects in the expression of the match hands. A device, program or method is provided that enhances karaoke.

  In the present invention, by recording information on the hand of the viewer performed in the karaoke on the server device and distributing it to the karaoke device, the singer can also sing to a singer who sings in a state where the viewer is not on the spot. An apparatus, program, or method capable of providing a match is provided.

It is a block diagram which shows the structure of the electronic device concerning embodiment of this invention. It is a flowchart which shows the flow of the processing step concerning embodiment of this invention. It is explanatory drawing which shows the method of making a viewer understand the content of the match hand which should be performed now, and the match hand which should be performed after that using the image data concerning embodiment of this invention. It is explanatory drawing which shows the method of making a viewer understand the content of the match hand which should be performed now and the match hand which should be performed after that using the video data concerning embodiment of this invention. It is explanatory drawing which shows the mode of the input and expression of a match hand which used the camera apparatus concerning embodiment of this invention as a match manual input device. It is explanatory drawing which shows the mode of the input and expression of the matching hand which made the acceleration sensor concerning embodiment of this invention the matching manual input apparatus. It is explanatory drawing which shows the method of adding the effect of the viewer according to embodiment of this invention by adding a production. It is explanatory drawing which shows the mode of the input and expression of the match hand which used the tablet type terminal concerning embodiment of this invention as the match manual input device. It is explanatory drawing which shows the method of sharing a match hand data via the network line concerning embodiment of this invention.

Embodiments of the present invention will be described below with reference to the drawings.
The structure of the electronic device of the present invention is shown in FIG. The electronic device 1 includes a display device 2, a voice reproduction device 3, an instruction input device 4, a matching hand input device 5, a voice input device 6, a karaoke device 7, model data 8, a matching hand input device 5 and a voice input device. 6 shows the match hand data 9 generated based on the information of the contents of the viewer's match hands collected, the video data 10 / image data 11 and audio data 12 to be played back as the match hands recorded in the karaoke device, and performances as the karaoke is performed. In addition to the music data 13 of the music to be played and the server device 14 connected to the karaoke device 7 over the network and the hand data 15 generated by the karaoke device 7 and transmitted to the server device 14, the karaoke device 7 connected to the server device 14 over the network A karaoke device 16 which is a separate device is provided.

  Next, an embodiment of the processing of the present invention will be described with reference to a flowchart in FIG. First, the music data 13 corresponding to the music to be played is determined by designating the music that the user desires to play to the karaoke device 7 using the instruction input device 4 (S2).

  The karaoke device 7 inquires the server device 14 of the matching hand data 15 corresponding to the music piece, and receives all or a part thereof (S4). However, this procedure is omitted when the hand data already owned by the karaoke apparatus 7 is used. If there are a plurality of matching hand data corresponding to the music, the matching hand data 9 to be used is determined by the user's designation by the instruction input device 4 or the judgment of the karaoke device 7 based on a preset criterion (S6). However, this procedure is omitted when there is only one matching hand data corresponding to the music.

  The sample data 8 to be used is specified by the user's designation by the instruction input device 4 or the judgment of the karaoke device 7 based on a preset criterion (S8).

  The display method of the model data 8 and the matching hand data 9 and the matching hand data 15 according to the user's designation by the instruction input device 4 or the judgment of the karaoke device 7 based on the preset criteria, the video data 10 and the image used for the display thereof Data 11 and audio data 12 are determined (S10).

  The sound recorded in the music data 13 of the music specified by the user is reproduced from the audio output device 3 (S12).

  The karaoke device 7 reproduces the video data 10 and the image data 11 corresponding to the sample data 8 and the matching hand data 9 on the screen of the display device 2 in accordance with the conditions determined in S10, and also reproduces the sound. The audio data 12 corresponding to the matching hand data 9 is reproduced by the device 3 (S14).

  Information on the contents of the matching hand that the viewer is performing in accordance with the performance of the karaoke music is collected by the matching hand input device 5 and the voice input device (S16).

  The karaoke device 7 generates the matching hand data 15 based on the information collected by the matching hand input device 5 and the powerful voice device (S18).

  The karaoke device 7 adjusts the video data 10 and the image data 11 corresponding to the match hand data 15 on the screen of the display device 2 according to the condition determined by the viewer according to the condition determined in S10, and the audio playback device 3 as well. The voice data 12 corresponding to the matching hand data 15 is reproduced (S20).

  The karaoke device 7 should determine which hand should be used at which point of the musical piece being played recorded in the reference and sample data 8 set in advance in the karaoke device 7 during or after the musical piece is played. Based on the information, an evaluation is performed on the matching hand performed by the viewer (S22). In addition, if a user directs with the instruction | indication input device 4, the content of the evaluation with respect to a suitable hand can be added to the evaluation with respect to a singer's song, or the content of evaluation with respect to a suitable hand can be added to the evaluation with respect to a singer's song.

  The karaoke device 7 displays on the screen of the display device 2 the contents of the evaluation for the matching hand performed by the viewer during the performance of the music or after the performance of the music is completed (S24).

  The karaoke device 7 determines the match hand data including the content of the final match hand data and accompanying information such as evaluation of the match hand after the performance of the music is completed, and transmits it to the server device 14. The server device 14 records the received data as matching hand data 15, and when there is an inquiry from another karaoke device 16 different from the karaoke device 7, the whole matching hand data 15 or a part of the matching matching hand data 15 is transmitted to the karaoke device 16 ( S26).

  FIG. 3 and FIG. 4 show the portion of the sample data realized by claim 1 in which the hand of the match to be performed by the viewer at that time and the portion of the hand of the match to be performed by the viewer after that point are recorded It is a specific example of the method of displaying on the screen of a display apparatus.

  FIG. 3 shows an example of a method of using model data in which the contents of a matching hand are recorded as an image. An image 310 is displayed on the screen 300 of the display device in which part of the contents of the matching hand recorded in the model data are arranged from left to right. The character 320 is the lyrics of the karaoke song being played, and the color tone of the character changes with the progress of the song, as in general karaoke. According to the present invention, the image of the matching hand data can be changed in color as the music progresses, like the image 310 to the image 330, like the lyrics. Can be understood to be “hands clapping hands” when reaching “Sky ~”, and when the performance of the music reaches “We ~” by the image 330, It can be understood that "FLY!" In addition, the viewer can understand from the image 310 that the performance of the musical piece reaches “We ~” before “the arm is raised and“ FLY! ” Can be prepared for the timing when the performance of “We” is reached, it is possible to easily perform the match without storing the content of the match. When the change in the color tone of the image of the matching hand model reaches the right end of the image, the display is switched to a position indicating the subsequent matching hand of the image included in the model data.

  FIG. 4 shows an example of a method of using example data in which the contents of a matching hand are recorded as an image. A video 410 that is the contents of the matching hand recorded in the model data is displayed on the screen 400 of the display device. The video 410 is recorded so that the contents of the matching hands are arranged from left to right, and the speed of the transition of the video so that the content of the choreography to be performed by the viewer is just inside the figure 420 displayed at the fixed position. Has been adjusted. The viewer can be made to understand that when the performance of the musical piece reaches “Sky ~” by the video 410 and the graphic 420, the “musical hand” is performed. When "We ~" is reached, it can be understood that "the arm is pushed up and" FLY! " In addition, the viewer can understand from the video 410 that the song performance is “a hand that raises his arms and raises“ FLY! ”” Before the performance of the music reaches “We”. By the video 430, you can understand that the performance of the music will be “can fly up” two more times before reaching the “can fly ~” timing. Since it can be provided, it is possible to easily perform a match without storing the contents of the match.

  FIG. 5 shows an example of a method of using a camera device that captures, analyzes, and converts data of a human body as a matching hand device. The camera device 500 shoots and analyzes the motion of the viewer 510 during the performance of the karaoke music piece, and transmits the information to the karaoke device 520 when it is detected that the viewer 510 is clapping. The karaoke device 520 that has received the information displays the video data 530 corresponding to the clapping hand on the screen of the display device 540 and reproduces the sound data 550 corresponding to the clapping hand on the sound reproducing device 560.

  FIG. 6 shows an example of a method of using a sensor device that is attached to a human body as a matching hand device and calculates the movement of the human body from the position change. The sensor device 610 worn by the viewer 600 on the wrist has a wristband shape, and the position change and tilt of the device can be detected by a built-in acceleration sensor. Prior to the performance of karaoke music, the sensor device 610 causes the viewer to take the same pose as the image 630 displayed on the screen of the display device 620, thereby providing a basis for detecting the change in position and inclination of the viewer's human body. The initial position of the sensor device can be determined. When the performance of the karaoke music is started and the viewer performs a “hand clapping” hand movement, the sensor device 610 moves both hands of the viewer based on the information on the change in position and tilt of the sensor device compared to the initial position. Is calculated as “hand clapping” and transmitted to the karaoke device 640. The karaoke device 640 that has received the information that the viewer has performed the “hand clapping” from the hand input device displays video data 650 corresponding to the “hand clapping” hand on the screen of the display device 620, and “hand clapping”. The audio reproduction device 670 reproduces the audio data 660 corresponding to the matching hand.

  FIG. 7 shows an example of a method for performing exaggerated expression as a performance from the hand of the viewer. The matching hand input device 710 that has detected that the viewer 700 is clapping hands transmits the information to the karaoke device 720. The karaoke apparatus 720 that has received the information displays a large number of video data 730 corresponding to the time signature on the screen of the display device 740 at the same time and a large number of audio data 750 corresponding to the time signature at the same time in accordance with the presetting or the designation from the user. Playback is performed by the audio playback device 760. As a result, it is possible to produce an effect as if a large number of viewers performed matching hands at the same time only by one viewer performing matching hands.

  FIG. 8 shows an example of a method for allowing the user to specify the contents of images / videos and audio to be played in response to the hand of the viewer. The matching hand input device 800 has a touch panel display device and a communication function with the karaoke device 810. When the performance of the karaoke music is started, an image 820 showing the type of the matching hand and an image 830 showing the method of executing the matching hand as contents of the matching hand image / video and audio that can be designated by the viewer on the screen of the matching hand input device 800. Is displayed. When the “right” image of “Maracas” displayed on the touch panel is designated as the type of matching hand desired by the viewer 840 and the execution method, the information is transmitted from the matching hand input device 800 to the karaoke device 810. The karaoke device 810 that has received the information displays the video data 850 corresponding to the motion of shaking the maracas with the right hand on the screen of the display device 860, and the audio data 870 corresponding to the operation of shaking the maracas with the right hand of the audio playback device 880. Play with. Note that the types of matching hands described as the image 820 and the image 840 and the execution method are merely examples, and the contents thereof are not limited thereto.

  FIG. 9 provides a matching hand for a singer singing at another time at another place by transmitting the matching hand data generated by one karaoke device to another karaoke device via a network line. It is an example of the method to do. When the singer 900 sings the karaoke song and the viewer 905 performs a match, the match hand input device 910 detects the content of the match hand and sends the information to the karaoke device 915 for viewing on the screen of the display device 920. The video data 925 corresponding to the matching hand performed by the viewer 905 is displayed, and the audio data 935 corresponding to the matching hand performed by the viewer 905 is reproduced by the audio playback device 930. When the singing of the singer 900 is completed, the karaoke apparatus 915 generates matching hand data 940 including accompanying information such as evaluation of the matching hand from the matching hand information performed by the viewer 905, and transmits it to the server apparatus 945 via the network line. . Then, a singer 950 who is trying to sing the same karaoke music using another karaoke device 955 at a different place / time from the singer 900 and the viewer 905 instructs the karaoke device 955 to use matching hand data. Then, the karaoke device 955 makes an inquiry to the server device 945 to transmit the hand data of the karaoke song match, and the server device 945 that has received the inquiry sends the matching hand data 940 to the karaoke device 955 via the network line. . The karaoke device 955 displays video data 965 corresponding to the content of the matching hand on the screen of the display device 960 in accordance with the performance of the karaoke music based on the content of the received matching hand data 940, and the content of the matching hand is displayed by the audio playback device 970. Audio data 975 corresponding to is reproduced. Thus, the singer 950 can sing the karaoke music while receiving the same match as the viewer actually performed even if there was no viewer in front of him.

  Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and various modifications, changes and combinations can be made based on the technical idea of the present invention. .

DESCRIPTION OF SYMBOLS 1 Electronic device 2 Display apparatus 3 Audio | voice output apparatus 4 Instruction input apparatus 5 Match hand input apparatus 6 Voice input apparatus 7 Karaoke apparatus 8 Model data 9 Match hand data 10 Image data 11 Video data 12 Audio data 13 Music data 14 Server apparatus 15 Match hand data 16 Karaoke equipment

Claims (8)

A method for producing and providing more effective hands such as body movements, voices, and musical instrument performances performed by viewers in karaoke using electronic devices,
In accordance with the conditions set in advance in the karaoke device or specified by the user, specifying the sample data to be displayed from the sample data recorded in the karaoke device;
Determining a display method of the model data in accordance with conditions set in advance in the karaoke apparatus or designated by the user;
In accordance with the progress of the music, all or a part of the above-mentioned data in which the hand of the match to be performed by the viewer at that time and the part of the data of the match to be performed by the viewer after that time are recorded. Displaying on the screen of the display device according to the determined display method. According to the conditions set in advance in the karaoke device or specified by the user, image data, video data, and audio data to be used for expressing the hand of the viewer on the display device and the audio reproduction device are specified, and these reproduction methods are A step to determine;
Collecting information on the contents of the matching hand performed by the viewer with a voice input device or a matching manual input device;
Generating matching hand data based on the collected information and recording it in a karaoke device;
According to the content of the matching hand data, the image data or video data recorded in the karaoke device is displayed on the screen of the display device according to the determined reproduction method, or the audio data recorded in the karaoke device is determined. Reproducing from the audio reproduction device according to the reproduction method;
The method according to claim 1, further comprising: transmitting the data from the karaoke device to the server device via a network line and accumulating the data in the server device. Extracting data determined to be appropriate based on conditions registered in advance in the karaoke device from the hand data recorded in the karaoke device or server device, based on the evaluation information to be sung by the singer,
Image data recorded on the karaoke device according to the progress of the music being played from the audio playback device in accordance with the conditions set in advance on the karaoke device or specified by the user based on the contents of the extracted hand data Or displaying the video data on the screen of the display device or reproducing the audio data recorded in the karaoke device from the audio reproduction device;
The method of claims 1 and 2, further comprising: Extracting the hand data specified by the condition from the hand data recorded in the karaoke device or the server device when a condition registered in advance in the karaoke device is satisfied;
Display image data or video data recorded on the karaoke device in accordance with the progress of the music being played from the audio playback device in accordance with the pre-registered conditions in the karaoke device based on the contents of the extracted hand data Reproducing audio data displayed on the screen of the apparatus or recorded in the karaoke apparatus from the audio reproducing apparatus;
The method of claims 1 and 2, further comprising: When there is an instruction from the user or when the conditions set in the karaoke device are satisfied in advance, from the data recorded in the karaoke device according to the instruction from the user or the conditions set in the karaoke device, Determining image data or video data to be displayed on the screen of the display device, or audio data to be reproduced by the audio reproduction device;
Determining a reproduction method of the image data, video data, and audio data in accordance with conditions set in advance in the karaoke device or designated by the user;
Reproducing the image data, video data, and audio data by the determined reproduction method according to the progress of the music being reproduced from the audio reproduction device;
The method of claims 1 and 2, further comprising: Reading out the matching hand information as a model from the matching model data shown to the viewer;
The step of comparing the match hand data with the information of the match hand, which is the above example, and performing an evaluation based on the evaluation criteria registered in advance in the karaoke device, and the condition that the user has given an instruction or has been registered in the karaoke device in advance is satisfied In the case of scoring the singing content, the step of taking into account the content of the evaluation for the matching hand data,
In the case where there is an instruction from the user or the condition registered in the karaoke device is satisfied in advance, the step of taking into account the content of the evaluation for the singing in the evaluation for the matching hand data,
The method of claims 1 and 2, further comprising:   The program which makes a computer perform the method of any one of Claims 1 thru | or 6.   7. An apparatus for producing and providing more effective hands such as body movements, voices, and musical instrument performances performed by a viewer in accordance with the rhythm of music in electronic karaoke. Comprising the steps as a means.

Images