JP2015155973A - Karaoke device and karaoke singing image data editor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a karaoke device and karaoke singing image data editor capable of imaging a moving image of a singing person who uses a karaoke and providing the moving image to the singing person.

SOLUTION: Editing means 32 has a zoom table 73 in which a singing score value and a zoom rate of an image are associated each other, face area recognition means 71 recognizes a face area of a singing person, to singing image data acquired by singing image acquisition means from a moving image imaging camera, and zoom processing means 72 edits the recognized face area of the singing person for every singing sections of music based on the zoom table according to the score value for every singing sections which is marked by marking means, with a corresponding scale factor for creating a singing moving image.

COPYRIGHT: (C)2015,JPO&INPIT

Description

  The present invention relates to a karaoke apparatus and a karaoke singing video data editing program for taking a video of a karaoke user's song and providing it to a singer.

  In recent years, services that record and record a user's song in a karaoke room and provide the song to the singer have become commonplace, and the enjoyment of allowing the user to view the video data by a third party has become widespread. In such a case, it is desired to provide a singing moving image with higher satisfaction to the user.

  Conventionally, various techniques for providing a karaoke user with a moving image during singing are known. For example, Patent Document 1 discloses a technique for obtaining an optimal thumbnail image when a user uploads a singing video shot in a karaoke room to a server. Patent Document 2 discloses a technique for displaying interrupt video data in place of a background video for each singing section of a karaoke song according to the scoring result. Furthermore, in Patent Document 3, a singing video shot with a digital video camera installed in a karaoke room and a video shot using a mobile phone in the room are synthesized and displayed as a background image or as moving image data. A technique of synthesizing at a time position based on a singing scoring result or a music climax determination result is disclosed together with a technique of posting.

JP 2012-027340 A JP 2011-215196 A JP 2013-210464 A

  By the way, it is natural for a user who enjoys posting a singing video to shoot and post a better singing video. Such a user often enters a karaoke room by himself so that he can concentrate on singing, and after rehearsal, tries to shoot his own singing video. However, in the technique of Patent Document 1, the shooting camera in the room can only be fixed aiming at the user with a certain amount of long shot, and even if it can be satisfied with singing and choreography, only a monotonous singing video can be obtained There is a problem.

  In addition, the technology of Patent Document 2 is an edited video in which animation prepared in advance that is not related to the singer's singing video is inserted, and can be enjoyed as a variation of the background video. It cannot meet the demands of users who want to shoot and post good singing videos.

  Furthermore, the technology of Patent Document 3 is based on the premise of the existence of a third party who operates the mobile phone in addition to the singer, and can meet the demand for singing intensively and creating a better singing video. In other words, even if this technology is used, regardless of what is photographed by a third party mobile phone and what kind of composition is taken, It simply synthesizes the shot image with the shot image of the digital video camera installed in the karaoke room.

  Accordingly, the present invention has been made in view of the above problems, and provides a karaoke device and a karaoke song video data editing program that can simplify a karaoke user's singing camera setting and post a highly edited video to a web server. The purpose is to provide.

  In order to solve the above-mentioned problem, in the invention of claim 1, a karaoke apparatus is provided in a karaoke room and includes a moving image shooting camera for shooting a user's song, and obtains song video data from the moving image shooting camera. Singing video acquisition means, scoring means for analyzing and scoring the singing voice for each predetermined singing section of the music with respect to the song singing, and singing video data acquired by the singing video acquisition means for each singing section of the music And editing means for creating a song video by editing a singer at a corresponding magnification according to the scoring value for each singing section scored by the scoring means.

The inventions of claims 2 to 4 are configured such that the editing means recognizes a singer's face area in the singing video data and edits the face area at the corresponding magnification.
“The editing means includes a zoom table that sets the corresponding magnification with the scoring value to be higher as the scoring value of the singing section is higher”,
“Each song has background video data to be displayed when a music performance is performed, and the editing means corresponds to the singing video data when the scoring value by the scoring means falls below a predetermined value for each singing section. The singing section is replaced with the video of the corresponding section in the background video data of the music ”.

  In invention of Claim 5, based on the singing score value data for every singing section of the music by the scoring means with which a karaoke apparatus is equipped, and the singing video data which image | photographed the karaoke user's song by a video camera, the said singing video data are obtained. A karaoke singing video data editing program that edits for each singing section based on a zoom table in which a singing scoring value and a zoom rate are associated with each other, wherein the singing scoring value for each singing section and the singing video data photographed by the video camera The step of acquiring, the step of recognizing the face area of the singer from the acquired singing video data, and the recognition of the singer based on the zoom table according to the score value for each acquired singing section for each singing section of the music Editing the face area thus created at a corresponding magnification to create a singing video.

  The invention of claim 6 is the case where the acquired scoring value falls below a predetermined value for each singing section when the background video data displayed at the time of playing the corresponding music is acquired and the singing video is created. Further, the corresponding singing section of the singing video data is replaced with the video of the corresponding section in the background video data of the music.

  According to the present invention, the singing video data acquired from the video camera corresponds to a singer, for example, a face area that has been recognized and processed, for each singing section of the music, according to the scoring value for each singing section. By creating a singing movie by editing at a magnification, it is possible to simplify the camera settings for karaoke users' singing shooting and to post highly edited videos to the web server, and to improve user satisfaction It can be improved.

It is a block block diagram of the karaoke apparatus which concerns on this invention. FIG. 2 is a configuration explanatory diagram of an editing unit in FIG. 1. It is explanatory drawing of the music selection of FIG. It is explanatory drawing of the moving image edit process in the edit means of FIG. It is a flowchart of the moving image edit process in the edit means of FIG. It is a display explanatory drawing regarding provision of the produced edit animation.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 shows a block configuration diagram of a karaoke apparatus according to the present invention, and FIG. 2 shows a configuration explanatory diagram of the editing means of FIG. In FIG. 1, a karaoke device 11 is connected to a karaoke main body 12 as a main device by wire or wirelessly, and includes a display unit 13, a mixing amplifier 14, a microphone 15, a speaker 16, a remote input / output device 17, a moving image. A photographing camera 18 is connected.

  The display unit 13 displays a normal music selection display, a background image during karaoke performance, and the like. For example, a liquid crystal display (LCD), a plasma display (PDP), and other various displays can be employed. The mixing amplifier 14 mixes the audio signal from the microphone 15 with the music performance signal sent from the karaoke main body 12, amplifies it, and outputs it from the speaker 16. The mixing amplifier 14 is provided with an output terminal through which the audio signal from the microphone 15 is passed, and the output terminal is connected to an A / D conversion section (29) of the karaoke main body 12 described later.

  The remote input / output device 17 transmits / receives data to / from the karaoke main body 12 using a wired system or a wireless system (IR system, Bluetooth (registered trademark) mechanism piconet connection system, etc.) by a terminal transmission / reception unit (not shown). For this purpose, at least a music selection music registration means 17A is provided as appropriate, and a display unit 17B and a display control unit 17C are provided. This music selection music registration means 17A is a program provided with a table for searching for music on the display unit 17B and selecting music, and the music selection is registered in a reservation queue (41) in the karaoke main body 12 described later. The music selection screen will be described with reference to FIG.

  Further, the display unit 17B of the remote input / output device 17 stacks a liquid crystal display (LCD) and a touch sensor for input / output, and data such as selection of music by the touch sensor corresponding to the displayed icon or the like. Is provided with a GUI user interface function. The display control unit 17C controls various displays in addition to the music selection screen and the like on the display unit 17B. Here, the display control unit 17C is shown as reproducing and displaying edited moving image data edited by the editing means (32) described later. ing.

  The video camera 18 is installed so as to be able to shoot the entire stage where the user sings in the karaoke room, records the singing of the singer, records the singing state, and outputs it as singing video data. It is a camera. In the present invention, the camera settings by the user need only be set within a range that can cover the entire stage.

  The karaoke main body 12 includes a bus 20, a central control unit 21, a ROM 22, a RAM 23, a video display control unit 24, a music performance control unit 25, a sound source (synthesizer) 26, transmission / reception units 27A and 27B, a scoring unit 28, and A / D conversion. Unit 29, storage unit 30, singing video acquisition unit 31, editing unit 32, and singing video reproduction unit 33. In the RAM 23, a storage area for a reservation queue 41, singing video data 42, scoring data 43, and edited moving image data 44 is formed, and a music DB 51 and a video DB 52 are stored in the storage unit 30. In addition, about each said structure, even if it is an element part which is not directly related to the summary of this invention, in order to show that most can be applied also in the conventional karaoke system, the whole component is demonstrated.

  The central control unit 21 is a physical CPU that performs overall processing control of the system, and performs algorithm processing based on a program stored in the ROM 22. The RAM 23 forms a storage area for the reservation queue 41, user information 42 and user singing adjustment value information 43, and serves as a work area for developing and executing the various programs. Thus, for example, it is a concept including a case where it is configured by a semiconductor memory and is virtually built on a hard disk.

  The video display control means 24 is a program or an electronic circuit that outputs the background video extracted from the video DB 52 and the lyrics data of the music extracted from the music DB 51 to the display unit 13 during performance. The music performance control unit 25 includes, for example, a sequence program, and drives a sound source (synthesizer) 26 according to the note data extracted from the music DB 51 by a music code. The output of the sound source 26 is sent to the mixing amplifier 14 as a performance signal. Is output.

  The transmission / reception unit 27A is used to exchange data with the remote input / output terminal 17 using a wired system or a wireless system (such as an IR system or a Bluetooth (registered trademark) mechanism piconet connection system). Electronic circuit and program. The transmission / reception unit 27B is configured by software such as a physical communication circuit and a platform for exchanging data between the karaoke main body 12, the distribution server 62, and the singing video server 63 via a communication network 61 such as the Internet. Is.

  The distribution server 62 includes a database that stores music data, user information, background video data, and other data related to the karaoke apparatus. In addition, the singing video server 63 uploads and stores the singing video data directly from each karaoke device 11 and provides services such as browsing only the person himself / herself and publishing on the net (viewing to a third party). It is a net site. The distribution server 62 and the singing video server 63 may be a single server.

  The scoring means 28 is for analyzing and scoring the singing voice for each predetermined singing section of the music with respect to the music singing, and the audio signal from the microphone 15 passes through the mixing amplifier 14 and passes through the A / D converter 29. Is a program that performs a scoring process for each singing section of the music based on reference data for scoring a song included in the music data with respect to the digitally converted audio signal. Specifically, for example, a technique described in Japanese Patent No. 4229915 can be used.

  Here, the basic analysis and scoring is performed, for example, with a singing voice waveform having a length of 10 to several tens of msec as one unit (hereinafter referred to as a minute section). On the other hand, the singing section is a section having a length of, for example, four measures or eight measures. In the scoring process of an arbitrary singing section, a scoring value obtained by summing the scoring results of all the minute sections included in the singing section is obtained. Shall. More specifically, as the singing score value, an added value or an average value of the score results of those minute sections is obtained. The scoring value including the scoring value for each singing section processed by the scoring means 28 is stored in the RAM 23 in association with the song music as scoring data 43.

  The music DB 51 stored in the storage unit 30 stores note data, lyric data, reference data for singing scores, and the like for each music. Regarding performance, specifically, there is a music table in which a music ID, a music title, and an artist ID (artist name) are associated, and for each music, note data of a karaoke music in a predetermined data format managed by the music ID (for example, , MIDI (registered trademark) note data) and other music data (files), each of which stores a music code as a file name, and displays a background video for each music stored in the video DB 52. Allocation data for assigning a scene video is associated with each singing section for the purpose. The video DB 52 stored in the storage unit 30 is a database that stores a predetermined number of scene video data corresponding to a singing section for displaying a background video, with a music code as a file name for background video data corresponding to each song. is there.

  The singing video acquisition means 31 is a program that acquires the singing video data output from the video camera 18 and stores it in the RAM 23 as the singing video data 42.

  The editing means 32 sings the singing video data acquired by the singing video acquisition means 31 and stored in the RAM 23 for each singing section of the music by the grading means 28 and stored in the RAM 23. It is a program that creates a singing video by editing a singer at a corresponding magnification according to the scoring value for each section. Specifically, as shown in FIG. 2A, a face area recognition unit 71, a zoom processing unit 72, and a zoom table 73 are provided.

  In this embodiment, the editing process by the editing means 32 is started after singing. However, the process may be a so-called chasing process when a singing section is scored during singing.

  The face area recognizing means 71 is a program for recognizing a singer's face area in singing video data, and a conventionally known program can be used. The zoom processing means 72 edits the face area recognized by the singer at the corresponding magnification based on the zoom table 73 in accordance with the acquired score value for each singing section for each singing section of the music, and the final singing video It is possible to use a previously known program.

  The zoom table 73 associates scoring values and zoom rates, and is shown in FIGS. 2B and 2C as an example. In FIG. 2B, 95 points or more with respect to the scoring value S is set to a maximum magnification of 3 times, 50 points or more are set to the same magnification, the magnification is sequentially determined for each predetermined range, and less than 50 points are displayed as a song. Among the background videos, the scene video (BGR) corresponding to the singing section is used. FIG. 2 (C) shows that the higher the scoring value in the singing section, the higher the magnification. For example, the highest scoring value in each singing section is specified. A scene corresponding to the singing section of the background video displayed as a song when the difference between the maximum scoring value exceeds 20 points, for example, is determined based on this, and the magnification is set for each difference range. Video (BGR).

  When the BGR is to be replaced for editing, the editing unit 32 acquires the background video data of the music from the video DB 52 stored in the storage unit 30 and specifies the scene video of the corresponding singing section. Here, the zoom table 73 in FIG.

  The program of the editing unit 32 including the program of the face area recognition unit 71 and the program of the zoom processing unit 72 is a karaoke song video data editing program. In this case, the data of the zoom table 73 may be included in the karaoke song video data editing program, or may be stored in a separate storage area and read out for reference.

  Returning to FIG. 1, the singing moving image reproducing means 33 is a program for reproducing the edited moving image data 44 stored in the RAM 23 edited by the editing means 32 in accordance with the user's instruction (see FIG. 6).

  Here, FIG. 3 is an explanatory diagram of the music selection of FIG. FIG. 3A shows an example of a screen for music selection of the remote input / output terminal 17, and an item for selecting normal shooting or zoom-up editing for song shooting is provided in addition to the song name and the like.

  In the karaoke main body 12, by receiving the music selection signal from the remote input / output terminal 17 via the transmission / reception unit 27A, the music IDs and the like are sequentially stored in the reservation queue 41 of the RAM 23 as shown in FIG. To do. In this case, at the time of music selection, a flag is attached to the selection between normal shooting and zoom-up editing.

  FIG. 4 is an explanatory diagram of the moving image editing process in the editing unit of FIG. 1, and FIG. 5 is a flowchart of the moving image editing process in the editing unit of FIG. First, after the user sings, the editing unit 32 acquires the scoring data (singing scoring value) 43 for each singing section from the RAM 23, and is shot by the video camera 18 and acquired by the singing video acquiring unit 31. Is acquired (step S1 (S1) in FIG. 5).

  The acquired singing video data is video data as shown in FIG. Therefore, the face area recognition means 71 recognizes the singer's face area 81 from the acquired singing video data (S2). Then, for each singing section of the music, the face area 81 recognized by the singer is edited at a corresponding magnification based on the zoom table 73 shown in FIG. 2B according to the scoring value for each singing section of the acquired scoring data. A singing video is created (S3).

  As shown in FIG. 4B, according to the scoring value of each singing section of the acquired scoring data, for example, a 79-point section 1 is set to an equal-size video scene, and an 85-point section 2 is doubled in zoom ratio. A video scene with a zoom ratio of 1.5 times in a section 3 of 84 points, a video scene with a zoom ratio of 2.5 times in a section 9 of 94 points, and a zoom ratio of the highest in a section 10 of 98 points The video scene is tripled. Then, video scenes in all singing sections are determined, and edited moving picture data as a whole is created.

  If the singing section is less than 50 points, the background video data corresponding to the singing section of the music is read from the video DB 52 and replaced with the video scene of the singing section. Note that the video scene of each singing section may be a video scene that is zoomed up or down to the corresponding magnification from the video scene of the previous singing section without suddenly setting the corresponding magnification.

  Next, FIG. 6 shows a display explanatory diagram relating to provision of the created edited moving image. In FIG. 6, when the singing of the music is finished and an edited moving image is created by the editing means 32, as shown in FIG. 6A, the singing moving image reproduction means 33 sends the edited moving image to the singer by confirming the singing. For example, the remote input / output device 17 (which may be the display unit 13 connected to the karaoke main body 12) is displayed to select whether or not to reproduce.

  When the singer selects reproduction, the edited moving image edited by the editing means 32 is reproduced as shown in FIG. When the reproduction is completed, as shown in FIG. 6C, a display for selecting whether or not to save this in the singing moving image server 63 is performed. If “do not save” is selected, the process ends and the created edited moving image is deleted. On the other hand, when “save” is selected, as shown in FIG. 6D, a display for selecting whether or not to release the saved edited moving image on the network is performed.

  When “not open” is selected, the edited video is associated with the user and the song song (music ID) under the condition setting that only the user can view, and is transmitted from the karaoke device 11 to the song video server 63. To save. On the other hand, when “publish” is selected, the edited video is associated with the user and the song song (music ID) with the condition setting for release to a third party, and transmitted from the karaoke device 11 to the song video server 63. And save it.

  Incidentally, the singing video server 63 and the distribution server 62 may be provided with the editing means 32 constituting the karaoke singing video data editing program. In this case, when the singing video data and scoring data stored in the RAM 23 are acquired from the karaoke device 11 to create edited moving image data and the BGR is used, the same video DB 52 as the video DB 52 provided in the karaoke device 11 is used. In addition, the background video data of the music may be acquired from the karaoke apparatus 11 together with the singing video data and the scoring data.

  Further, when the scoring value is lower than a predetermined value, the scene video of the singing video data may be reduced by an equal magnification according to the low score, instead of using the BGR scene video. Further, in the present embodiment, only one type of zoom table 73 is prepared in advance. However, a plurality of types of zoom tables are prepared, and a zoom table selection unit is provided in the remote input / output device 17 or the like to provide a singer. It is good also as a structure which makes it select. Alternatively, a zoom table setting input means is provided in the remote input / output device 17 or the like so that the singer can freely set the correspondence between the scoring value and the magnification, and the setting information can be stored in the distribution server together with the singer ID and can be called at any time. It is good also as such a structure.

  As described above, since the zoom processing editing is performed, it is only necessary to set the video camera 18 so that the entire singing stage can be captured, and it is possible to simplify the karaoke user's camera setting for singing and sing well. Zoom in for a good facial expression when you make it, and replace it with a different video when you make a mistake. For example, you can post a video of a highly edited singing video to the web server (singing video server 63). It is possible to improve user satisfaction.

  The karaoke apparatus of the present invention can be used in the industry of manufacturing and using an apparatus including means for capturing a singing image of a singer.

DESCRIPTION OF SYMBOLS 11 Karaoke apparatus 12 Karaoke main body 17 Remote input / output apparatus 18 Movie shooting camera 28 Scoring means 31 Singing image acquisition means 32 Editing means 33 Singing movie reproduction means 63 Singing movie server 71 Face area recognition means 72 Zoom processing means 73 Zoom table 81 Face area

Claims (6)

A karaoke apparatus equipped with a video camera that is installed in a karaoke room and photographs a user's song,
Singing video acquisition means for acquiring singing video data from the video camera;
Scoring means for analyzing and scoring the singing voice for each predetermined singing section of the music for song singing,
The singing video data acquired by the singing video acquisition means is created for each singing section of the music, and the singer is edited at a corresponding magnification according to the scoring value for each singing section scored by the scoring means to create a singing video. Editing means,
A karaoke apparatus comprising: The karaoke apparatus according to claim 1, wherein the editing unit recognizes a singer's face area in the singing video data and edits the face area at the corresponding magnification. 3. The karaoke apparatus according to claim 1, wherein the editing unit includes a zoom table that sets a corresponding magnification with the scoring value to a higher magnification as the scoring value of the singing section is higher. . It is a karaoke apparatus in any one of Claims 1-3,
Each song has background video data to be displayed when playing the song,
The editing means replaces the corresponding singing section of the singing video data with the video of the corresponding section in the background video data of the song when the scoring value by the scoring means falls below a predetermined value for each singing section. Karaoke device characterized by. Based on the singing score data for each singing section of the song by the scoring means of the karaoke device and the singing video data of the karaoke user's singing by the video camera, the singing score data and the zoom rate are A karaoke song video data editing program for editing for each song section based on an associated zoom table,
Obtaining a singing score value for each singing section and singing video data photographed by a video camera;
Recognizing a singer's face area from the acquired singing video data;
For each song section of the music, editing a face area recognized by the singer based on the zoom table according to the obtained scoring value for each song section, and creating a song video,
A karaoke song video data editing program characterized by including: A karaoke song video data editing program according to claim 5,
Get the background video data displayed when playing the corresponding music,
When creating the singing video, if the acquired scoring value falls below a predetermined value for each singing section, the corresponding singing section of the singing video data is changed to the video of the corresponding section in the background video data of the song. A karaoke song video data editing program characterized by replacement.

Images