JP6547551B2 - Karaoke device, program - Google Patents

Description

  The present invention relates to a karaoke apparatus having a function of photographing and displaying a karaoke singer.

  Patent Document 1 describes a karaoke apparatus that shoots a singer singing with a camera and displays it on a display. According to the karaoke apparatus described in Patent Document 1, it is possible for the camera to track and shoot moving singers.

Japanese Patent Application Laid-Open No. 10-240276

  According to the prior art described in Patent Document 1, it is supposed that the place can be enhanced by tracking and projecting a singer. However, only by projecting the singers, the added value to the singers and the listeners is not sufficient, and further ingenuity in the stage of presentation is required.

  The present invention has been made to solve the above problems. An object of the present invention is to provide a technique for performing high-value-added presentation using a video of a singer.

  A karaoke apparatus according to an aspect of the present invention includes setting means, recognition means, and control means. The setting means sets a determination area, which is a part of the area serving as a reference for determining whether or not visual effects are to be performed, to the area of the captured image acquired by the imaging means as the karaoke music to be played. Set accordingly. The recognition means recognizes an image area representing a specific region of a singer who sings a karaoke piece from among the captured images acquired by the imaging means during the performance of the karaoke piece. The control means causes the display means to display the captured image acquired by the imaging means during the performance of the karaoke music. Then, the control means takes an effect image representing a visual effect on condition that the image area representing the specific part of the singer recognized by the recognition means falls within the judgment area set by the setting means. To the display means.

  According to the present invention, when the specific region of the singer in the captured image enters the determination area set in the captured image corresponding to the karaoke music to be played, the image of the singer being captured The effect image can be displayed in the image. In this way, during the performance of karaoke, for example, the singer can unexpectedly display the effect image at a timing unintended by the singer or the listener, or the singer can intentionally display the effect image. It can provide added value such as arousing motivation to actively move the body.

  In addition, it is possible to recognize the image area showing a face part as an example of the specific part of the singer who recognizes with a recognition means. By doing this, it is possible to display the effect image when the face part of the singer enters the determination area.

  In addition, the determination area may be configured to be sequentially changed according to the progress degree of the performance, instead of being fixedly set consistently throughout the performance of one song. By doing this, it is possible to diversify the method of displaying the effect image and to enhance the added value to the singer and the listener.

  Alternatively, evaluation information representing the evaluation of singing during the performance of the karaoke music may be acquired, and the determination region may be changed during the performance according to the evaluation of the singing represented by the acquired evaluation information. . By doing this, for example, the method for displaying the effect image is diversified to change the position and the size of the judgment area according to the level of the evaluation of the singing during the performance, and the singer or the listener can listen Can add value to

  In addition, it is possible to set it as the area | region which has a coordinate position and magnitude | size corresponding to the coordinate system of the pixel which comprises a captured image as a detail of the determination area | region set by the setting means. By doing this, it is possible to perform image processing corresponding to the coordinate system of the pixels that make up the captured image.

  The present invention can also be realized in the form of a program executed by a computer or a storage medium storing the program.

FIG. 1 is a block diagram showing the configuration of a karaoke apparatus 10. The flowchart showing the procedure of a special effect image display process. Explanatory drawing showing the outline | summary of the determination area | region. (A) Display example 1 of effect image, (b) Display example 2 of effect image.

Hereinafter, embodiments of the present invention will be described based on the drawings. The present invention is not limited to the following embodiments, and can be implemented in various aspects.
[Description of configuration of karaoke apparatus]
The configuration of the karaoke apparatus 10 according to the embodiment will be described with reference to FIG.

  The karaoke apparatus 10 is, for example, a commercial karaoke apparatus installed in a karaoke shop, and has a function of playing karaoke music requested by a user, a song evaluation function of evaluating a user's singing, and various images at the time of performance It has a function to be displayed on the display 16. As exemplified in FIG. 1, the karaoke apparatus 10 includes a control unit 11, a storage unit 13, an operation unit 14, and a voice input / output unit 15. In addition, peripheral devices such as a display 16, a microphone 17, a speaker 18, and a camera 19 are connected to the karaoke apparatus 10.

  The control unit 11 is an information processing apparatus mainly configured of a CPU, a RAM, a ROM, and the like (all not shown), and controls the entire apparatus. The control unit 11 implements various functions by executing arithmetic processing in accordance with a program stored in a storage medium. When the control unit 11 receives a request for a karaoke song from a user via an electronic preview device (not shown) such as a remote control terminal, the controller 11 registers the song number of the karaoke song in the reservation list. The control unit 11 sequentially reads the music data corresponding to the reserved music from the karaoke database in accordance with the playing order registered in the reservation list, and performs the playing process. Further, the control unit 11 is known to score the skill level of singing by comparing the melody of the singing voice inputted from the microphone 17 during the performance of the karaoke music with the singing standard data which is data representing the singing evaluation criteria. Perform the singing scoring function of. In the present embodiment, it is assumed that an evaluation in the middle of the performance of the karaoke music is calculated by this song scoring function.

  Furthermore, the control unit 11 causes the display 16 to display the image of the singer captured by the camera 19 during the performance of the music as a characteristic function of the present invention. At that time, control is performed to display the special effect image according to the position of the face image of the singer recognized from the captured image. A detailed procedure of control for displaying the special effect image will be described later.

  The storage unit 13 is, for example, a storage device such as a hard disk drive or a semiconductor memory. The storage unit 13 stores a program executed by the control unit 11 and data used at the time of the execution. In addition, the storage unit 13 is music data corresponding to each karaoke music, lyric data for displaying telops representing the lyrics of the music, singing reference data representing evaluation criteria for singing, background image data displayed when playing the music It has a karaoke database that contains a large number of etc. Each piece of music data is composed of performance data (MIDI data) corresponding to the MIDI (Music Instrument Digital Interface) standard. The MIDI data includes performance data for playing the melody of the accompaniment part of the karaoke music.

  The operation unit 14 is an input device for inputting an operation from the user, and includes a touch panel, a key switch, and the like provided in the karaoke apparatus 10 main body. The voice input / output unit 15 is a device that controls voice input / output. A microphone 17 is connected to the input side of the voice input / output unit 15. Thereby, the voice input / output unit 15 acquires the singing sound of the singer. A speaker 18 is connected to the output side of the voice input / output unit 15. As a result, the voice input / output unit 15 outputs to the speaker 18 the sound source signal of the performance sound of the music reproduced by the control unit 11 and the singing sound of the singer. The speaker 18 converts the sound source signal output from the audio input / output unit 15 into sound and outputs the sound.

  The display 16 is a display device that displays a video based on the video data sent from the control unit 11. The camera 19 is an imaging device installed in a place (for example, a karaoke room) where the karaoke apparatus 10 is installed, and images and captures an image of a place where a singer who performs karaoke is using the karaoke apparatus 10 The image data is output to the control unit 11. The control unit 11 causes the display 16 to display an image and a lyrics telop captured by the camera 19 in parallel with the performance process of the karaoke music. The other functions and configurations of the karaoke apparatus 10 are in accordance with known techniques, and thus detailed description thereof is omitted here.

[Description of special effect image display processing]
The procedure of the special effect image display process executed by the control unit 11 of the karaoke apparatus 10 will be described with reference to the flowchart of FIG. This process is started when the performance process of the karaoke music requested by the user is started, and is performed in parallel with the performance process of the karaoke music. As a premise, the control unit 11 causes the display 16 to display the captured image input to the camera 19 as a background image of karaoke in parallel with the performance of the karaoke music. At this time, the control unit 11 performs an effect of displaying various symbols such as a character image on the captured image displayed as the background image.

  In S100, the control unit 11 sets a determination area for the coordinate area of the captured image input from the camera 19 according to the performance of the karaoke music. The determination area in the present embodiment is a coordinate area serving as a reference for determining whether or not to execute visual effects using a special effect image. The determination area is, for example, configured of information representing an area having a coordinate position and a size corresponding to the coordinate system of the pixels constituting the captured image.

  In the present embodiment, it is conceivable to set the determination area in accordance with the performance of the karaoke music as follows. A determination area corresponding to each performance section obtained by dividing the performance of the karaoke music into a plurality of sections is defined in advance, and a determination area corresponding to the performance section currently being played is set to the captured image. For example, it is conceivable to set determination areas of different positions and sizes for each performance section such as A-melody, B-melody, or rust that composes a refrain format often adopted for karaoke music. Then, the control section 11 determines in S100 which performance section the present performance belongs to in accordance with the time progression of the performance of the music data, whereby the position and the size of the determination area according to the progress of the performance are set. Ru. Furthermore, by adding random numbers to the setting values of the position and the size of the judgment area corresponding to each performance section, variation in the position and the size of the judgment area can be provided for each performance of the karaoke music. it can.

  In addition, it is possible to set it as the structure which included the setting value of the determination area | region in the music data of each karaoke music as setting information for every karaoke music. In this case, at the time of playing the karaoke music, the control unit 11 extracts setting information on the judgment area from the music data used for the music, and sets the judgment area in S100 based on the extracted setting information. Alternatively, the setting information in which the setting value of the determination area for each karaoke song is associated with the song identification information which is the identification information of the karaoke song may be stored separately from the song data in the storage unit 13 . In this case, at the time of playing the karaoke music, the control unit 11 reads the setting information of the judgment area corresponding to the music identification information of the music data together with the music data, and sets the judgment area at S100 based on the read setting information. .

  In addition, according to the progress of the performance of the karaoke music, the pitch value and volume value at the time of performance are extracted from the music data, and the position and the size of the determination area are set in S100 based on the extracted values. May be By doing this, every time S100 to S108 are repeatedly executed between the start of performance and the end of performance, the position and size of the determination area are determined based on the change in pitch and volume due to the time progression of the performance of music data. Can be varied in many ways. Alternatively, a different determination area may be set for each individual karaoke song or each genre of karaoke song.

  Also, using a known song scoring function that evaluates the skill of the user's singing in the middle of a performance, also called real-time scoring, acquires evaluation information representing the user's singing midway evaluation calculated at the current moment of playing, Judgment regions of different positions and sizes may be set according to the degree of evaluation during evaluation. For example, it is conceivable to set the determination area wider and near the center of the captured image as the evaluation on the way at this time is higher, and to set the determination area closer to the edge of the captured image as the evaluation in the middle is lower.

  A specific example of the set determination area will be described with reference to FIG. In FIG. 3, reference numeral 30 represents a coordinate area of a captured image. Further, reference numeral 31 denotes an image of a singer appearing in the image area 30. It is assumed that the control unit 11 sets the rectangular determination area 32 in the left part of the image area 30 with respect to the image area 30 of such a captured image. The shape of the determination area is not limited to a rectangle, and may be a curved figure such as another polygon or a circle. Although the judgment area 32 is visualized and expressed in FIG. 3 for convenience of explanation, in actuality, the display 16 is displayed as a background in order to prevent the singer from understanding the set judgment area. It is assumed that the determination area is not clearly indicated on the captured image. The same applies to FIGS. 4A and 4B described later.

  It returns to description of the flowchart of FIG. In S102, the control unit 11 recognizes an image area of a portion (an example of a specific portion in the present invention) corresponding to the face of the person corresponding to the singer from the captured image at the current time input from the camera 19. In addition, as a method of recognizing the face part of a person from the image, human body detection technology such as JP-A-2014-50901 or "Kinect (registered trademark)", JP-A 2010-218060 and JP-A 2013-131209 It is conceivable to use a known image recognition technology such as a face detection technology such as H. Further, as a method of identifying the singer, for example, the microphone 17 in the captured image is recognized by pattern matching or the like, and a person who holds the microphone 17 or a person closest to the microphone 17 It is possible to identify as a singer.

  In S104, the control unit 11 determines whether the face image area of the singer recognized in S104 is within the determination area set in S100. In S104, a positive determination may be made on condition that the entire face image area of the singer is included in the determination area, or at least a part of the singer's face image area is in the determination area. It may be configured to make a positive determination on the condition that it is included. When it is determined that the face image area of the singer is not within the determination area (S104: NO), the control unit 11 returns to S100.

  On the other hand, when it is determined that the face image area of the singer is within the determination area (S104: YES), the control unit 11 proceeds to S106. In S106, the control unit 11 adds a special effect image prepared in advance for display when the singer's face image area is within the determination area on the captured image and causes the display 16 to display this. .

  A display example of the special effect image in S106 will be described with reference to FIGS. 4 (a) and 4 (b). In addition, in FIG. 4 (a), (b), the same code | symbol is attached | subjected to the same structure. First, FIG. 4A is a display example of a state in which the special effect image in S106 is not displayed. The code 31 represents the image of the singer appearing in the image area 30, and the code 32 represents the determination area set in S100. And the code | symbol 33 represents the face image area corresponded to the singer's face part recognized in S102. Further, reference numeral 34 denotes a character image (not a special effect image) for a normal effect added on the captured image by the control unit 11 during the performance of the karaoke music.

  As illustrated in FIG. 4A, since the face image area 33 recognized in the image area 30 is outside the determination area 32, special effects are produced except for the character image 34 for normal effect at this time. The image is not displayed. Then, from the situation illustrated in FIG. 4A, it is assumed that the singer moves to the imaging range of the camera 19 to transition to the situation illustrated in FIG. 4B.

  As exemplified in FIG. 4B, when the face image area 33 recognized in the image area 30 is in the determination area 32, the control unit 11 adds a number of special effect images 35, 36 on the captured image. Be done. Among them, the special effect image 35 is a character image that newly appears on the captured image. Further, the special effect image 36 is an effect image that represents that a change has occurred in the existing character image 34 by being added in association with the existing character image 34 that has been displayed for normal effect.

  It returns to description of the flowchart of FIG. In S108, the control unit 11 determines whether or not the performance of the karaoke music has ended. When the performance of the karaoke music continues (S108: NO), the control unit 11 returns to S100. Then, when the performance of the karaoke music is finished (S108: YES), the control unit 11 ends the special effect image display process.

[effect]
According to the karaoke apparatus 10 of the embodiment, the following effects are achieved.
When the image of the singer's face portion captured in the captured image acquired by the camera 19 enters the determination area set in the coordinate area of the captured image, the special image is captured in the captured image captured by the singer An effect image can be displayed. By doing this, for example, the singer positively feels the surprise due to the special effect image being displayed at a timing not intended by the singer or the listener, or the singer actively displays the special effect image. Can provide added value such as motivating to

  In addition, the method of displaying a special effect image is diversified by sequentially changing the determination area according to the progress degree of the performance, instead of setting the determination area consistently throughout the performance of one song in a consistent manner. it can. In addition, by changing the determination area during the performance according to the evaluation of the singing during the performance of the karaoke music, for example, changing the position or the size of the determination area according to the level of the evaluation of the singing during the performance Can.

[Modification]
In the above-described embodiment, an example is described in which the special effect image is displayed on the condition that the face image area representing the face part of the singer corresponds to the determination area. The special effect image is displayed under the condition that the specific part (for example, the hand, the arm, etc.) of the singer's body other than the face part is recognized and the recognized specific part corresponds to the determination area. May be

  As a condition for displaying the special effect image, in addition to the face image area representing the singer's face portion corresponding to the determination area, the size of the recognized singer's face image area is not less than the specified May be added to the condition.

  The function of one component in the above embodiment may be distributed as a plurality of components, or the function of a plurality of components may be integrated into one component. In addition, all the aspects contained in the technical thought specified only by the words described in the claim are an embodiment of the present invention.

[Correspondence with the configuration described in the claims]
Correspondence between each configuration of the embodiment and the configuration described in the claims is as follows.
The camera 19 corresponds to an imaging unit. S100 of FIG. 2 which the control part 11 performs corresponds for a setting means. S102 of FIG. 2 which the control part 11 performs is corresponded to a recognition means. S104 and S106 of FIG. 2 which the control part 11 performs are corresponded to a control means. The display 16 corresponds to display means. The song evaluation function performed by the control unit 11 corresponds to an evaluation acquisition unit.

  DESCRIPTION OF SYMBOLS 10 ... Karaoke apparatus, 11 ... Control part, 13 ... Storage part, 14 ... Operation part, 15 ... Sound input / output part, 16 ... Display, 17 ... Microphone, 18 ... Speaker, 19 ... Camera.

Claims (6)

The judgment area, which is a part of the area used as a reference for judging whether or not visual effects are to be executed, is set corresponding to the karaoke music to be played with respect to the area of the captured image acquired by the imaging means Setting means for
Recognition means for recognizing an image area representing a specific region of a singer who sings the karaoke music from among the captured images acquired by the imaging means during the performance of the karaoke music;
A determination area in which an image area representing a specific part of the singer recognized by the recognition means is displayed by the setting means, the display means displays the captured image acquired by the imaging means during the performance of the karaoke music piece provided on condition that the appropriate, and control means for displaying on the display means the effect image representing the visual effect is added to the captured image, to,
The determination area is configured not to be displayed on the display means;
Karaoke device characterized by. The judgment area, which is a part of the area used as a reference for judging whether or not visual effects are to be executed, is set corresponding to the karaoke music to be played with respect to the area of the captured image acquired by the imaging means Setting means for
Recognition means for recognizing an image area representing a specific region of a singer who sings the karaoke music from among the captured images acquired by the imaging means during the performance of the karaoke music;
A determination area in which an image area representing a specific part of the singer recognized by the recognition means is displayed by the setting means, the display means displays the captured image acquired by the imaging means during the performance of the karaoke music piece Control means for adding an effect image representing a visual effect to the captured image and displaying the effect image on the display means under the condition that
An evaluation acquisition unit that acquires evaluation information representing an evaluation of singing during the performance of the karaoke music;
The setting means changes the determination area during the performance of the karaoke music according to the evaluation of the song represented by the evaluation information acquired by the evaluation acquisition means.
A karaoke apparatus comprising: In the karaoke apparatus according to claim 1 or 2 ,
The recognition means recognizes an image area representing a singer's face as the specific part;
The control means causes the effect image to be added to the captured image and displayed on condition that an image area representing the face part of the singer recognized by the recognition means falls within the determination area.
Karaoke device characterized by. The karaoke apparatus according to any one of claims 1 to 3 .
The setting means sequentially sets the determination area according to the progress degree of the performance of the karaoke music.
Karaoke device characterized by. The judgment area, which is a part of the area used as a reference for judging whether or not visual effects are to be executed, is set corresponding to the karaoke music to be played with respect to the area of the captured image acquired by the imaging means Configuration steps, and
A recognition procedure for recognizing an image area representing a specific region of a singer who sings the karaoke music from among the captured images acquired by the imaging means during the performance of the karaoke music;
A determination area in which the image area representing the specific part of the singer recognized in the recognition procedure is set in the setting procedure, the display unit displays the captured image acquired by the imaging device during the performance of the karaoke music piece And causing the computer to execute a control procedure of adding an effect image representing a visual effect to the captured image and displaying the effect image on the display means, under the condition that
Being configured to control the computer such that the determination area is not displayed on the display means;
A program characterized by The judgment area, which is a part of the area used as a reference for judging whether or not visual effects are to be executed, is set corresponding to the karaoke music to be played with respect to the area of the captured image acquired by the imaging means Configuration steps, and
A recognition procedure for recognizing an image area representing a specific region of a singer who sings the karaoke music from among the captured images acquired by the imaging means during the performance of the karaoke music;
A determination area in which the image area representing the specific part of the singer recognized in the recognition procedure is set in the setting procedure, the display unit displays the captured image acquired by the imaging device during the performance of the karaoke music piece A control procedure for adding an effect image representing a visual effect to the captured image and displaying the effect image on the display means on condition that the image corresponds to
And causing a computer to execute an evaluation acquisition procedure for acquiring evaluation information representing an evaluation of singing during the performance of the karaoke song.
Furthermore, the setting procedure may cause the computer to execute a procedure of changing the determination area during the performance of the karaoke music according to the evaluation of the song represented by the evaluation information acquired by the evaluation acquisition procedure. Being configured,
A program characterized by

Images