JP2016071185A - Karaoke device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform performance for notifying of a person who sings next, during musical performance of music, using a face image of a participant of karaoke.SOLUTION: A karaoke device acquires a face image of a person who reserved music corresponding to music whose musical performance reservation is received, then registers the face image in a reservation list. The karaoke device specifies (S304) the face image of the person who reserved music, corresponding to the music corresponding to a next musical performance order of music which is being musical performed, from the reservation list. Then, the karaoke device recognizes (S306) a person corresponding to the face image of the person who reserved music which is musical performed next, out of captured images in which a range where participants exist, is imaged. Then, the karaoke device displays (S310) a performance image to which visual performance is performed to the image of the person who is recognized as a person who reserved music which is musical performed next, from the captured image, on predetermined display means when the musical performance of music which is musical performed proceeds to a predetermined part.SELECTED DRAWING: Figure 4

Description

  The present invention relates to a karaoke apparatus having a function of photographing and displaying karaoke participants.

  Patent Document 1 describes a karaoke device that takes a picture of a singer singing with a camera and displays it on a display. According to the karaoke apparatus described in Patent Document 1, a camera can track a singing singer and take a picture.

JP-A-10-240276

  By the way, the karaoke apparatus has a function of reserving a plurality of songs desired to be sung. Thereby, when a plurality of participants use the karaoke apparatus jointly, each participant reserves a song that he / she wants to sing, and each participant sequentially sings according to the order in which the reserved songs are played. Be able to. However, in a situation where an unspecified number of customers, such as kids rooms and karaoke cafes, use a karaoke device jointly, a stranger tells the reservation person of the next performance song that the order of singing has arrived. Not necessarily. Therefore, the reservation person was inconvenient because he / she did not know when the order of singing would come.

  On the other hand, in the prior art as described in Patent Document 1, it is said that the place can be excited by tracking and projecting a singer. However, in a system that shows only the singer, the next singer cannot be announced from an unspecified number of participants.

  The present invention has been made to solve the above problems. An object of the present invention is to provide a technique for performing an effect of notifying a next singer during performance of a music piece using a participant's face image.

  The karaoke apparatus of the present invention includes face image acquisition means, list creation means, performance means, identification means, and display control means. The face image acquisition means acquires the face image of the reservation person corresponding to the music for which the reservation is accepted. The list creation means registers the face image of the reservation person acquired by the face image acquisition means in association with the music identification information for identifying the music corresponding to the reservation person in the reservation list. The performance means sequentially plays the music pieces registered in the reservation list according to the performance order in the reservation list. The specifying means specifies a face image associated with the music corresponding to the next performance order of the music currently being played by the performance means based on the reservation list. The display control means recognizes a person corresponding to the face image of the reservation person specified by the specifying means from the picked-up images picked up by the image pickup means for picking up the range where the participant is present. The display control means displays an effect image in which a visual effect is applied to the image of the recognized person on a predetermined display means when the performance of the currently played music has progressed to a predetermined position. Let

According to the present invention, based on the reservation list including the face image acquired at the time of music reservation, the singer of the music scheduled to be played next to the music currently being played is obtained from the captured image. An effect image using the image of the reservation person can be displayed. Thus, by notifying the next singer using the participant's image, the singer can know that the order of singing is close. In addition, the following singers can be introduced or appealed to an unspecified number of partners participating in karaoke.

  The karaoke apparatus according to claim 2 has the following characteristics. That is, the display control means causes the display means to display an image in which a lyrics telop is superimposed on a predetermined background image corresponding to the music before the performance of the music currently being played progresses to a predetermined position. Then, when the performance of the currently played music has progressed to a predetermined position, the display control means switches the background image to the effect image relating to the reservation person specified by the specifying means. In this way, by switching from the normal background image to the effect image that announces the next singer during the performance of the music, the display content during the performance is changed, and the place for the performance of the next music is played. It can be excited.

  It should be noted that the specific content of the effect image may be as described in claim 3. That is, the display control means displays the effect image arranged on the captured image in association with the image of the predetermined notification character with respect to the image of the person recognized as the reservation person in the captured image. In this way, for example, the next singer can be introduced happily, for example, by displaying a unique character (character) in the vicinity of the next singer's image on the captured image.

  The karaoke apparatus according to claim 4 has the following characteristics. That is, if the display control unit cannot recognize a person corresponding to the face image of the reservation person specified by the specifying unit from the captured images captured by the imaging unit, the display control unit acquires the reservation acquired by the face image acquisition unit. An effect image in which a visual effect is applied to the person's face image is displayed. In this way, even if the person corresponding to the reservation person cannot be recognized from the captured image for some reason, the reservation person previously acquired when the reservation of the music is accepted. An effect image for notifying the next singer can be displayed using the face image.

  In addition, as described in claim 5, the present invention can be realized as a program executed in a computer or a storage medium for storing the program.

1 is a block diagram illustrating a schematic configuration of a karaoke apparatus 10. FIG. (A) The flowchart showing the procedure of a reservation process, (b) The flowchart showing the procedure of a reservation list registration process. Explanatory drawing showing an example of a reservation list. The flowchart showing the procedure of effect image display processing. (A) Production image display example 1 (b) Production image display example 2

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In addition, this invention is not limited to the following embodiment, It is possible to implement in various aspects.
[Description of configuration of karaoke apparatus 10]
As shown in FIG. 1, the karaoke apparatus 10 includes a control unit 11, a storage unit 12, a music playback unit 13, an audio control unit 14, a video control unit 15, a video input unit 16, an operation unit 17, and a communication unit 18. The karaoke apparatus 10 has a function of playing karaoke music requested by the user and a function of displaying various images on the display 23 when the music is played.

The control unit 11 includes a CPU, a ROM, a RAM, and the like, and controls the operation of the karaoke apparatus 10 by executing various programs. The control part 11 performs the program of the reservation list registration process (refer FIG. 2) and the effect image display process (refer FIG. 4) which are mentioned later, and participation which is going to sing the next performance music during performance of a music is performed. An effect image for notifying the person is created (see FIG. 5).
Then, the control unit 11 causes the display 23 to display the created effect image.

The storage unit 12 is a storage device such as a hard disk drive. The storage unit 12 stores a program executed by the control unit 11 and data used at the time of execution. Further, the storage unit 12 has a karaoke database in which a lot of music data corresponding to each karaoke music, lyrics data for displaying a telop representing the lyrics of the music, background image data to be displayed when the music is played, and the like are recorded. ing. Each piece of music data is MIDI (Music Instrument
It consists of performance data (MIDI data) corresponding to the (Digital Interface) standard. The MIDI data includes performance data for playing the melody of the accompaniment part of karaoke music.

  The music playback unit 13 plays back music based on the music data stored in the storage unit 12. The voice control unit 14 is a device that controls voice input / output, and includes an input unit 14a, an output unit 14b, and the like. A microphone 19 is connected to the input unit 14a. Thereby, the input part 14a acquires a user's song sound. A speaker 21 is connected to the output unit 14b. The output unit 14 b outputs the sound source signal of the music reproduced by the music reproducing unit 13 and the sound source signal of the singing sound output from the input unit 14 a to the speaker 21. The speaker 21 outputs the sound source signal output from the output unit 14b in place of sound.

  The video control unit 15 reproduces and outputs video based on the video data sent from the control unit 11. A display 23 for displaying video is connected to the video control unit 15. As a result, the video control unit 15 displays the reproduced video on the display 23. The video input unit 16 is a control device that controls video input. A camera 25 is connected to the video input unit 16. Thereby, the video input unit 16 acquires image data captured by the camera 25. The camera 25 is an imaging device installed in a place (for example, a karaoke room) where the karaoke device 10 is installed, and images the entire state where the participants who perform karaoke using the karaoke device 10 are present, and images it. Output the image data.

  The operation unit 17 is an input device for performing various operations on the karaoke apparatus 10. The operation unit 17 includes a touch panel and various input buttons, and outputs an operation signal corresponding to the input operation to the control unit 11. The communication unit 18 is a communication interface for communicating with the remote control terminal 30. In addition, since the other structure and function of the karaoke apparatus 10 are based on a well-known technique, detailed description here is abbreviate | omitted.

  The remote control terminal 30 is a terminal device for remotely operating the karaoke apparatus 10. The remote control terminal 30 has a communication function, an information display function, and an input function for a user to perform an input operation. The remote control terminal 30 receives information such as identification information (song number) of music to be reserved for performance from the user. The remote control terminal 30 transmits information input from the user to the karaoke apparatus 10.

[Description of reservation processing and reservation list registration processing]
The procedure of the reservation process executed by the remote control terminal 30 will be described with reference to the flowchart of FIG. This process is repeatedly executed at predetermined control intervals when the remote control terminal 30 is operating.

In S100, the remote control terminal 30 determines whether or not a reservation operation for music to be performed by the karaoke apparatus 10 has been received. When the reservation operation is accepted (S100: YES), the remote control terminal 30 proceeds to S102. On the other hand, when the reservation operation is not accepted (S100: NO), the remote control terminal 30 ends this process. In S <b> 102, the remote control terminal 30 transmits reservation information including music identification information (for example, song number) to the karaoke apparatus 10. After S102, the remote control terminal 30 ends this process.

  Next, the procedure of the reservation list registration process executed by the control unit 11 of the karaoke apparatus 10 according to the reservation process by the remote control terminal 30 will be described with reference to the flowchart of FIG. This process is repeatedly executed at predetermined control intervals when the karaoke apparatus 10 is operating.

  In S200, the control unit 11 initializes a counter for counting the reservation order to 1. In S202, the reservation information transmitted from the remote control terminal 30 is received. This reservation information is information transmitted in S102 of the above-described reservation place process (see FIG. 2A). The reservation information includes a song number that is identification information of a song designated by the reservation person. In S204, the control unit 11 registers the reservation order currently counted and the music number represented by the reservation information received in S202 in a reservation list (details will be described later).

  In S206, the control unit 11 specifies the reservation person corresponding to the reservation information received in S202 from the captured images of all the participants captured by the camera 25 when the reservation information is received in S202, and the reservation is made. An image of the person's face (face image) is acquired. Specifically, the control unit 11 detects an image of the remote control terminal 30 from a captured image input from the camera 25 using a known image recognition algorithm. Then, an image of a person existing closest to the detected image of the remote control terminal is detected by image recognition. As a known technique for recognizing a person's face from an image, for example, Japanese Patent No. 5002311 can be cited. Moreover, as a well-known technique for recognizing a specific object, for example, Japanese Patent No. 4830696 can be cited. The detected image of the person is specified as the image of the reservation person of the music, and the image of the face portion is extracted from the image of the reservation person detected from the captured image.

  In S208, the control unit 11 registers the reservation person's face image acquired in S208 in the reservation list in association with the reservation order and the song number registered in S204. In the next S210, the control unit 11 adds 1 to the counter that counts the reservation order. After S210, the control unit 11 returns to S202.

  Here, the detailed contents of the reservation list will be described with reference to FIG. As illustrated in FIG. 3, information corresponding to each information item of “reservation order”, “song number”, and “face image” is registered in the reservation list. The reservation order represents the order in which the music is reserved. The controller 11 sequentially plays the music according to the reservation order. The song number is identification information for identifying a song. The face image is obtained by extracting an image of the face portion of the participant who reserved the music, that is, the participant who plans to sing the music.

[Description of effect image display processing]
The procedure of the effect image display process which the control part 11 of the karaoke apparatus 10 performs is demonstrated, referring the flowchart of FIG. This process is executed in parallel with the process of sequentially playing music registered in the reservation list in the order of reservation. As a premise, the control unit 11 displays a background image on the display 23 based on predetermined background image data corresponding to the performance target music in parallel with the performance of the music. Further, the control unit 11 causes the display 23 to display a telop representing the lyrics in synchronism with the progress of the performance based on the lyrics data corresponding to the musical piece to be played.

In S300, the control unit 11 determines whether or not the music performance by the music reproduction unit 13 has progressed to a predetermined notification timing. The notification timing may be a specific timing from the second half to the end of the music (for example, a melody break in the refrain format). When the performance of the music has not progressed to the notification timing (S300: N
O), the control unit 11 repeats S300. On the other hand, when the music performance has progressed to the notification timing (S300: YES), the control unit 11 proceeds to S302.

  In S302, the control unit 11 determines whether or not there is a music corresponding to the next reservation order of the music currently being played, that is, a music to be played next. If there is no next performance song (S302: NO), the control unit 11 ends this process. When there is a next performance song (S302: YES), the control unit 11 proceeds to S304.

  In S304, the control unit 11 acquires the face image of the person who reserves the next musical piece from the reservation list (see FIG. 3). In S306, the control unit 11 detects an image of a person corresponding to the face image of the reservation person acquired in S304 from the captured images obtained by capturing the current state of all participants by the camera 25 by image processing. As a method for detecting a person corresponding to the reservation person from the captured image, for example, the face image of the reservation person is applied as a recognition reference model for pattern matching, and an image having a high similarity to the recognition reference model is detected from the captured image. The method of detecting is mentioned.

  In S308, the control unit 11 determines whether or not the image of the reservation person can be specified from the current captured image as a result of S306. When the image of the reservation person can be identified from the captured images (S308: YES), the control unit 11 proceeds to S310. In S310, the control unit 11 uses the current captured image captured by the camera 25 to create an effect image in which a visual effect is applied to the reservation person image recognized in S306. And the control part 11 switches and displays the background image of karaoke currently displayed on the display 23 in the production image created from the captured image. At this time, the telop of the lyrics corresponding to the musical piece being played is superimposed and displayed on the effect image.

  Here, a display example of the effect image in S310 will be described with reference to FIG. As illustrated in FIG. 5A, the effect image 50 includes real images 51 and 52 of a plurality of participants captured by the camera 25. Among these, it is assumed that the image 51 corresponds to the reservation person specified based on the face image acquired from the reservation list. The control part 11 arrange | positions the image of the notification character 53 showing the virtual character who announces the singer of the music played next in the vicinity of the image 51 of a reservation person. Moreover, the control part 11 arrange | positions the notification message 54 to the effect of notifying the singer of the music played next in the vicinity of the notification character 53 as a dialog which the notification character 53 speaks. In addition, the control unit 11 displays a telop representing the lyrics of the music superimposed on the effect image 50.

  Returning to the flowchart of FIG. When the image of the reservation person cannot be specified in S308 (S308: NO), the control unit 11 proceeds to S312. In this case, it is considered that the reservation person was not shown in the captured image because the reservation person was out of the imaging range of the camera 25 for some reason. In S <b> 312, the control unit 11 creates an effect image with a visual effect using the face image of the person who reserves the next performance song registered in the reservation list. And the control part 11 switches and displays the background image of the karaoke currently displayed on the display 23 in the production image created from the face image of the reservation person. At this time, the telop of the lyrics corresponding to the musical piece being played is superimposed and displayed on the effect image.

Here, a display example of the effect image in S312 will be described with reference to FIG. As illustrated in FIG. 5B, the control unit 11 arranges the face image 61 of the reservation person acquired from the reservation list in the center part of the effect image 60. The control part 11 arrange | positions the image of the notification character 62 showing the virtual character who announces the singer of the music played next in the vicinity of the face image 61 of a reservation person. Moreover, the control part 11 arrange | positions the notification message 63 to the effect of notifying the singer of the music played next in the vicinity of the notification character 62 as a speech which the notification character 62 speaks. The control unit 11 also displays a telop representing the lyrics of the music superimposed on the effect image 60.

[effect]
According to the karaoke apparatus 10 of the embodiment, the following effects can be obtained.
An effect image in which an effect such as a notification character or an announcement message is applied to an image of a singer of a song that is scheduled to be played next, based on a reservation list (see FIG. 3) in which face images acquired at the time of music reservation are registered. Can be displayed. Thus, by notifying the next singer using the captured images of the participants, the singer can know that the singing order is close. Moreover, the next singer can be introduced in an easy-to-understand manner to an unspecified number of participants participating in karaoke.

  In addition, by switching from a normal background image to a production image that announces the next singer at a specific timing in the performance, the display content during the performance will change, and the venue will be excited for the performance of the next song Can do.

  Moreover, even if the person corresponding to the face image of the next singer could not be recognized from the captured images of all participants captured by the camera 25 for some reason, the reservation of the music was accepted. An effect image for notifying the next singer can be displayed using a face image acquired in advance as a reservation person.

[Modification]
In the above-described embodiment, when the karaoke apparatus 10 receives reservation information from the remote control terminal 30, the reservation person is specified from the captured images acquired from the camera 25 connected to the karaoke apparatus 10, and the reservation person An example of acquiring a face image was described. Apart from this, a configuration may be used in which the face image of the reservation person is acquired using a camera provided on the remote control terminal 30 side.

  Specifically, a camera for imaging the user's face is provided in the vicinity of the display included in the remote control terminal 30. When receiving a music reservation operation from the user, the remote control terminal 30 images the face of the user with the camera. Then, when transmitting the reservation information to the karaoke apparatus 10, the remote control terminal 30 also transmits the captured face image data. When the karaoke apparatus 10 receives the reservation information and the face image of the reservation person from the remote control terminal 30, the information is registered in the reservation list. By doing in this way, the face image of the music reservation person can be acquired by methods other than the method of specifying the music reservation person from the captured image of the camera 25 connected to the karaoke apparatus 10.

[Correspondence with configuration described in claims]
The correspondence between each configuration of the embodiment and the configuration described in the claims is as follows.
S206 (control unit 11) in FIG. 2B corresponds to a face image acquisition unit. S208 (control unit 11) in FIG. 2B corresponds to the list creation means. The control unit 11 and the music playback unit 13 correspond to performance means. S306 (control unit 11) in FIG. 4 corresponds to the specifying unit. S310 (control unit 11) in FIG. 4 corresponds to display control means.

  DESCRIPTION OF SYMBOLS 10 ... Karaoke apparatus, 11 ... Control part, 12 ... Memory | storage part, 13 ... Music reproduction part, 14 ... Audio | voice control part, 14a ... Input part, 14b ... Output part, 15 ... Image | video control part, 16 ... Image | video input part, 17 DESCRIPTION OF SYMBOLS ... Operation part, 18 ... Communication part, 19 ... Microphone, 21 ... Speaker, 23 ... Display, 25 ... Camera, 30 ... Remote control terminal.

Claims (5)

Face image acquisition means for acquiring a face image of the reservation person corresponding to the music for which the reservation has been accepted;
A list creation means for associating and registering in the reservation list the face image of the reservation person acquired by the face image acquisition means and the music identification information for identifying the music corresponding to the reservation person;
According to the performance order in the reservation list, performance means for sequentially playing the music registered in the reservation list;
A specifying means for specifying a face image associated with the music corresponding to the next performance order of the music currently being played by the performance means based on the reservation list;
A person corresponding to the face image of the reservation person specified by the specifying means is recognized from the picked-up images picked up by the image pickup means for picking up the range where the participant is present, and the image of the recognized person is recognized. Display control means for displaying on the predetermined display means when the performance of the music currently being played has progressed to a predetermined position, an effect image that has been visually produced;
A karaoke apparatus comprising: The karaoke apparatus according to claim 1,
The display control means causes the display means to display an image in which a lyrics telop is superimposed on a predetermined background image corresponding to the music before the performance of the music currently being played progresses to a predetermined position. Switching the background image to the effect image related to the reservation person specified by the specifying means when the performance of the currently playing music has progressed to a predetermined position;
Karaoke device characterized by. In the karaoke apparatus of Claim 1 or Claim 2,
The display control means displays an effect image arranged on the captured image in association with an image of a predetermined notification character with respect to an image of a person recognized as a reservation person in the captured image;
Karaoke device characterized by. The karaoke apparatus according to any one of claims 1 to 3,
The display control means is acquired by the face image acquisition means when the person corresponding to the face image of the reservation person specified by the specifying means cannot be recognized from the captured images picked up by the image pickup means. Displaying an effect image with a visual effect on the reservation person's face image,
Karaoke device characterized by. A face image acquisition procedure for acquiring the face image of the reservation person corresponding to the music for which the reservation is accepted;
A list creation procedure for registering the face image of the reservation person acquired in the face image acquisition procedure and the music identification information for identifying the music corresponding to the reservation person in association with the reservation list;
According to the performance order in the reservation list, a performance procedure for sequentially playing the music registered in the reservation list;
Based on the reservation list, a specific procedure for identifying a face image associated with a music corresponding to the next performance order of the music currently being played in the performance procedure;
A person corresponding to the face image of the reservation person specified in the specific procedure is recognized from the picked-up images picked up by the image pickup means for picking up the area where the participant is present, and the image of the recognized person is recognized. A display control procedure for displaying an effect image with a visual effect on a predetermined display means when the performance of the currently playing music has progressed to a predetermined position;
A program characterized by causing a computer to execute.