JP2006126300A - Karaoke machine - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a Karaoke machine capable of displaying a result of evaluation of a singing voice to a singer by using simpler image processing. <P>SOLUTION: The Karaoke machine 1 is equipped with playing means 13, 16, 17, and 18, a singing input means 15, an evaluating means 341 and 342 for evaluating an input singing voice, a photographing means 41 for photographing the singer, image processing means 343 and 40 for processing a photographed image of the singer according to evaluation results of the evaluating means, and a display means 44 for displaying photographed image of the singer processed by the image processing means 343 and 40, and further equipped with an additional image storage means 112 for storing an additional image to be added to the photographed image of the singer. The image processing means 343 and 40 are equipped with an extracting means 343 for extracting a featured part of the singer in the photographed image of the singer and an image composing means 404 for performing composition processing of the additional image to the featured part extracted by the extracting means 343 according to the evaluation results of the evaluating means 341 and 342. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a karaoke apparatus that evaluates a singing voice of a singer and displays a photographed image of the singer that has been subjected to image processing in accordance with the evaluation.

Conventionally, a karaoke apparatus having a function of evaluating a singer's singing voice and a function of showing the evaluation result to the singer has been put into practical use. As a karaoke apparatus equipped with such an evaluation function, there has been proposed a karaoke apparatus that captures a singer's face with a video camera, for example, and transforms the image of the face according to the evaluation result (for example, Patent Documents). 1). In such a karaoke apparatus, since the degree of deformation processing is changed according to the evaluation result of the singing voice, the evaluation result of the singing voice can be shown to the singer in an interesting manner.
Japanese Patent Laid-Open No. 11-231881

  However, since the conventional karaoke apparatus uses complex image processing in which deformation processing is performed on a photographed image of a singer, the load of image processing is large, and a high-speed image processing apparatus (DSP: Digital Signal Processor or the like) There was a problem of having to have. In particular, during the singing of the singer, when a configuration is adopted in which the singing voice is evaluated sequentially and the display content of the photographed image of the singer is changed according to the evaluation result during the singing, the singing is performed sequentially during the singing. Since image processing is performed on a photographed image of a singer, a higher-speed image processing device is required.

  Accordingly, an object of the present invention is to provide a karaoke apparatus that can display an evaluation result for a singing voice interesting to a singer using simpler image processing in order to solve the above-described problems.

  The invention of the present application evaluates the singing voice inputted by the singing input means, the singing voice inputting means for inputting the singing voice of the singer, as the means for solving the above-mentioned problems. Evaluation means, photographing means for photographing the singer, image processing means for performing image processing on the photographed image of the singer according to the evaluation result by the evaluation means, and photographing of the singer image-processed by the image processing means A karaoke apparatus comprising a display means for displaying an image, further comprising additional image storage means for storing an additional image to be added to the photographed image of the singer, wherein the image processing means is a singer in the photographed image of the singer Extracting means for extracting the feature portion of the image processing means, and image synthesizing means for synthesizing the additional image with the feature portion extracted by the extracting means according to the evaluation result by the evaluating means, Characterized in that it comprises.

  According to the configuration of the present invention, when a karaoke song is played by the performance means and the singer sings along with the karaoke song, the singing voice is input by the singing input means, and the input singing voice is evaluated by the evaluation means. Rated by. Then, the singer is photographed by the photographing means, and the photographed image of the singer is subjected to image processing by the image processing means according to the evaluation result by the evaluating means, and the photographed image of the singer that has undergone the image processing is Displayed by display means. Here, as the image processing by the image processing unit, the extraction unit extracts the characteristic part of the singer from the photographed image of the singer, and the additional image stored in the additional image storage unit is the image of the characteristic part. The synthesis process is performed by the synthesis means. As a result, it is possible to reflect the evaluation result of the singing voice on the photographed image of the singer while using simpler image processing as compared with the conventional technique in which the photographed image of the singer is subjected to the deformation process. .

  Further, in the karaoke apparatus described above, the evaluation means evaluates the singing voice at predetermined time intervals during the performance of the karaoke song by the performance means, and the image processing means outputs an image every time the evaluation result is obtained. It is good also as performing processing.

  According to this configuration, the singing voice is evaluated by the evaluation unit at predetermined time intervals during the performance of the karaoke song, and the image processing is executed by the image processing unit each time an evaluation result is obtained. By displaying the captured images of the subsequent singers sequentially by the display means, for example, the evaluation result of the singing in the middle of the singing can be displayed to the singers and the viewers of the singers. It becomes possible.

  Moreover, in the karaoke apparatus mentioned above, the said extraction means is good also as extracting the component of a singer's face as a singer's characteristic part. According to this configuration, the constituent elements of the singer's face (for example, eyes, nose, mouth, etc.) are extracted as characteristic portions of the singer, and an additional image (for example, a pattern of ink with feathers, for example) For example, it is possible to display the evaluation result of the singing voice with higher taste for a singer, a viewer of the singer, and the like.

  And in the karaoke apparatus mentioned above, the said image synthetic | combination means may synthesize | combine many additional images with the characteristic part of a singer, so that evaluation by the said evaluation means is low. According to this configuration, as the evaluation of the singing voice is lower, the evaluation result of the singing voice is obtained, for example, by using a simple configuration in which many additional images are synthesized with respect to the characteristic part of the singer. It is possible to display visually and easily for the viewer of the singer.

  In the karaoke apparatus described above, the image processing means further includes effect processing means for performing at least one of mosaic processing and color change processing on the characteristic part of the singer extracted by the extraction means. It is good also as providing. According to this configuration, in addition to the synthesis of the additional image according to the evaluation result of the singing voice, the effect processing is performed on the characteristic part of the singer according to the evaluation result of the singer, It becomes possible to express the evaluation result of the singing voice in more detail.

  Further, the karaoke apparatus described above may further include a composite image output unit capable of writing a photographed image of the singer on the recording medium, which has been subjected to the additional image combining process. According to this configuration, since the photographed image of the singer that has been subjected to the synthesis process of the additional image can be written to the recording medium by the synthesized image output unit, the evaluation result of the singing voice can be stored. Thus, for example, in the performance of karaoke songs from the next time onward, it is possible to use a configuration in which a photographed image of a singer that has undergone additional image synthesis processing can be used, and singing over multiple karaoke song performances. It is possible to obtain the evaluation result of the person and display the evaluation result.

  According to the present invention, the evaluation result of the singing voice can be reflected in the photographed image of the singer while using simpler image processing as compared with the conventional technique in which the photographed image of the singer is subjected to the deformation process. Therefore, the evaluation result of the singing voice can be displayed to the singer or the like by the photographed image of the singer while using simpler image processing as compared with the prior art.

  Hereinafter, a karaoke apparatus according to an embodiment of the present invention will be described with reference to the drawings.

  In the karaoke apparatus according to the embodiment, one mode can be selected from a plurality of modes including the evaluation mode in addition to the normal mode by a selection operation of a user (a singer or a viewer who views the singer). It has become. The normal mode is a mode in which the singing voice is not evaluated. The evaluation mode is a mode for evaluating the level of the singing voice of the singer in accordance with the performance of the singer's karaoke. In the evaluation mode, the karaoke apparatus executes an evaluation process. With the evaluation process, when the singer sings, the singer is photographed and the singing voice is evaluated, and according to the evaluation result, the photographed image of the singer includes, for example, ink with feathers such as a kite. It is a series of processes that synthesize and display simulated objects.

(First embodiment)
FIG. 1 is a block diagram of a karaoke apparatus 1 according to the first embodiment of the present invention. The karaoke apparatus 1 is composed of a CPU (Central Processing Unit) 10 that controls the operation of the entire apparatus, and various devices connected thereto via buses and connectors (not shown). A hard disk (hereinafter referred to as “HDD”) 11, a RAM (Random Access Memory) 12, and an operation unit 20 are connected to the CPU 10. Further, a sound source 13, an A / D converter 14, a mixer 16, a sound system (hereinafter referred to as “SS”) 17, and a vocal adapter 19 are connected to the CPU 10. Further, the CPU 10 is connected to a singer image processing unit 40, an MPEG (Moving Picture Experts Group) decoder 42, and a synthesis circuit 43. Moreover, the karaoke apparatus 1 is configured so that a microphone 15, a speaker 18, a video camera 41, and a monitor 44 can be connected.

  The HDD 11 stores a predetermined program such as performance of a karaoke song, input / output of singing voice and displaying an image corresponding to the performance of the karaoke song on the monitor 44, and also stores image data and sound data. Functionally, the music data storage unit 110, the video data storage unit 111, and the additional image storage unit 112 are provided. The song data storage unit 110 is an area for storing song data for playing karaoke songs. The video data storage unit 111 is an area for storing video data of a background video to be displayed on the monitor 44. The additional image storage unit 112 is an area for storing a plurality of types of additional image data. The additional image data is image data of an object (hereinafter referred to as “additional image”) to be added to the photographed image of the singer in the evaluation mode. The additional image is, for example, an ink brush drawn as a penalty on the face of the loser when defeated by a battledore or an object imitating a figure such as ◯ ×.

  The RAM 12 is a work area of the CPU 10, and in addition to an area for reading programs, song data, and additional images from the HDD 11, a scoring log area 120 is set. The grading log area 102 is an area for recording a singing voice grading result and the like scored in the evaluation mode.

  The sound source 13 forms a musical sound signal in accordance with music data (note event data or the like) input from the CPU 10 (a music sequencer 32 described later), and outputs it to the mixer 16. The A / D converter 14 converts a singer's singing voice signal input by the microphone 15 (an example of a singing input unit) into a digital signal and outputs the digital signal to the mixer 16 and the vocal adapter 19.

  The mixer 16 gives an effect such as echo to the plurality of musical sound signals generated by the sound source 13 and the singing voice signal of the singer input via the microphone 15 -A / D converter 17. Mix the signal with an appropriate balance. The mixer 16 outputs the mixed digital audio signal to the SS 17. The effect that the mixer 16 gives to each audio signal and the balance of mixing are controlled by the CPU 10. The SS 17 includes a D / A converter and a power amplifier, converts an input digital signal into an analog signal, amplifies it, and emits sound from the speaker 18.

  The vocal adapter 19 receives a singing voice signal from the A / D converter 14 and a reference from the CPU 10 (a music sequencer 321 described later). As the reference, for example, guide melody data included in music data is used. The vocal adapter 19 calculates the voice volume and singing frequency of the singing voice from the input singing voice signal, and also calculates the input reference frequency and voice volume, and the calculated information for each sampling interval. It outputs to the scoring mode process part 34) mentioned later.

  The operation unit 20 includes, for example, a panel switch interface and a remote control receiving circuit. The operation unit 20 is connected to a panel switch or a remote control device (not shown) so as to be able to transmit and receive signals. The panel switch and the remote control device are provided with various key switches. When the user presses the key switch, the user selects a song or selects one mode from a plurality of modes. Accept. When an operation from a user is received by these devices, the operation unit 20 receives an operation signal indicating the operation content from these devices, and outputs the operation signal to the CPU 10.

  The CPU 10 functionally includes an operation input processing unit 31, a sequencer 32, a background video reproduction unit 33, and an evaluation mode processing unit 34. The operation input processing unit 31 executes processing according to the operation signal input from the operation unit 20. For example, when a song selection operation signal is input, the selected karaoke song is notified to the sequencer 32. When a mode selection operation signal is input, for example, in a predetermined flag storage area in the RAM 12. The selected mode is set by storing a flag or the like. Note that the initial setting for mode selection is the normal mode, and when the operation signal for selecting the evaluation mode is not input, the normal mode is set.

  The sequencer 32 executes processing necessary for playing a karaoke song, and includes a song sequencer 321, a lyrics sequencer 322, and a character pattern creation unit 322a. The song sequencer 321 executes processing for playing a karaoke song. As a process for playing this karaoke song, the song sequencer 321 reads, for example, the song data of the karaoke song selected by the user notified from the operation input processing unit 31 from the song data storage unit 110 and stores it in the RAM 12. Let The song sequencer 321 sequentially reads song data from the RAM 12, and uses the data included in the read song data (for example, data of tracks such as performance data tracks, guide melody tracks, etc.) to control the sound generation processing of the sound source 13. By doing so, the performance sound of the karaoke song is generated. The music sequencer 321, the sound source 13, the mixer 16, the SS 17 and the speaker 18 constitute a performance means.

  The lyrics sequencer 322 reads the lyrics track corresponding to the position of the karaoke song processed by the song sequencer 321 from the karaoke song read by the song sequencer 321, and based on the data of the lyrics track, the character pattern creation unit 322a. Then, an image pattern is generated and output to the synthesis circuit 43.

  For example, when the normal mode is set, the background video reproduction unit 33 sequentially reads background image data corresponding to the selected karaoke song in accordance with an instruction from the sequencer 32 and outputs the background image data to the synthesis circuit 43. .

  When the evaluation mode is set, the evaluation mode processing unit 34 comprehensively executes, for example, the following evaluation processing, evaluates the singing voice of the singer, and uses the singer image processing unit 40. Then, the singer is photographed by the video camera 41, and the photographed image is subjected to image processing according to the evaluation result and displayed on the monitor 44. The evaluation mode processing unit 34 includes an evaluation engine 341, a point extraction unit 342, and an object extraction unit 343.

  When the evaluation mode is set, the evaluation engine 341 executes a process for evaluating the accuracy and inflection of the singing voice from the pitch, voice volume, and the like of the singing voice input via the microphone 18. For example, the evaluation of the singing voice is performed using the singing frequency, the singing voice volume, the reference frequency, and the reference voice volume output from the vocal adapter 19 (details of the singing voice evaluation will be described later). The point extraction unit 342 scores the singing voice based on the evaluation result of the evaluation engine 341 (details regarding the singing voice scoring will be described later). The point extraction unit 342 stores the singing voice scoring result in the scoring log storage area 120 and outputs it to the object extraction unit 343. The evaluation engine 341 and the point extraction unit 342 constitute an evaluation unit.

  The object extraction unit 343 outputs a singing voice scoring result from the point extraction unit 342. When the scoring result is lower than a predetermined level, the object extraction unit 343 reads out from the additional image storage unit 112 with reference to the RAM 12, and stores it in the RAM 12. An additional image corresponding to the scoring result is read out from a plurality of types of additional images stored, and is output to the singer image processing unit 40 (an image composition unit 404 described later). On the other hand, when the scoring result is higher than the predetermined level, the object extraction unit 343 does not extract the additional image.

  The singer image processing unit 40 includes, for example, a DSP, and includes an image storage unit 401, a video analysis unit 402, a feature portion extraction unit 403, and an image composition unit 404. The image storage unit 401 includes, for example, a RAM or the like, stores the additional image output from the object extraction unit 343, and stores the photographed image of the singer output from the video camera 41 (an example of the imaging unit). Store the data.

  When an additional image is extracted by the object extraction unit 343, the video analysis unit 402 analyzes the captured image data stored in the image storage unit 401 and determines the singer's face from the color information of the singer's captured image data. And the facial components (for example, facial contours, eyes, nose, mouth, ears, nose, eyebrows, hair, etc.) are specified as the characteristic parts of the singer. The feature part extraction unit 403 corresponds to the extraction means of the present invention, and the result of specifying the singer's feature part by the video analysis unit 402 is output. Based on the specification result, the feature part extraction unit 403 is stored in the image storage unit 401. A feature portion corresponding to the image is extracted. For example, when the additional image is a wrinkle object, a nose, a mouth, or the like is extracted as a characteristic portion corresponding to the wrinkle object.

  The image composition unit 404 corresponds to the image composition unit of the present invention, and for the feature part extracted by the feature part extraction unit 403, an additional image corresponding to the feature part is read from the image storage unit 401 and synthesized. (The composition of the feature portion of the additional image will be described in detail later). The image composition unit 404 outputs the photographed image of the singer after the image processing to the composition circuit 43. The object extraction unit 343 and the singer image processing unit 40 constitute an image processing unit.

  The MPEG decoder 42 converts the input MPEG data into an NTSC (National Television System Committee) video signal and inputs it to the synthesis circuit 43. For example, when the normal mode is set, the MPEG decoder 42 performs signal conversion of the background video data encoded in the MPEG2 format output from the background video reproduction processing unit 33 and outputs it to the synthesis circuit 43.

  The synthesizing circuit 43 displays the lyrics telop and the scoring result on the video signal of the background video output from the MPEG decoder 42 in the normal mode and on the singer synthesized image output from the image synthesizing unit 404 in the evaluation mode. This is a circuit for synthesizing the OSD (On Screen Display). The monitor 44 is, for example, a CRT (Cathode Ray Tube) display, an LCD (Liquid Crystal Display), or the like, and displays an image output from the synthesis circuit 43.

  FIG. 2 is a flowchart showing an example of an evaluation process executed by the karaoke apparatus 1 shown in FIG. This process is started when the evaluation mode processing unit 34 receives an operation signal for selecting an evaluation mode from the operation input processing unit 31.

  First, the evaluation mode processing unit 34 instructs the singer image processing unit 40 to start shooting a singer using the video camera 41. The singer image processing unit 40 causes the video camera 41 to start shooting a singer at predetermined time intervals, outputs a shot image of the shot singer, and stores the output shot image of the singer as an image. The data is stored in the unit 401 and updated sequentially (S1).

  Next, the sequencer 32 receives a song selection instruction from the operation input processing unit 31, and starts playing a karaoke song corresponding to the song selection (S2). And the singer image process part 40 performs the display process of an initial image (S3). In the initial image display process, the singer image processing unit 40 starts outputting the photographed image of the singer stored in the image storage unit 401 to the synthesis circuit 43, and the lyrics telop output from the sequencer 32; The photographed image of the singer is synthesized with the synthesis circuit 43, and this synthesized image is displayed on the monitor 44 as an initial image.

  Here, when the singer sings along with the performance of the karaoke song with reference to the displayed lyrics telop, the singing voice is input to the microphone 15. The input singing voice is mixed with the karaoke music by the mixer 16 and output from the speaker 18 via the SS 17 and also to the vocal adapter 19 via the A / D converter 14.

  Next, the evaluation mode processing unit 34 determines the arrival of the evaluation timing of the singing voice (S4). The evaluation mode processing unit 34 includes, for example, a time measuring unit for measuring time, and the time measuring unit is notified of arrival at every predetermined time interval. It is determined that there is an evaluation timing.

  When the arrival of the evaluation timing of the singing voice is not determined (NO in S4), the evaluation mode processing unit 34 repeatedly executes step S4. When judging the arrival of the evaluation timing of the singing voice (YES in S4), the evaluation mode processing unit 34 performs the singing voice evaluation process (S5). In the singing voice evaluation process here, the evaluation mode processing unit 34 instructs the vocal adapter 19 to output the frequency and volume of the reference and the singing voice to the evaluation engine 341. The evaluation engine 341 evaluates the singing voice based on the frequency and volume of the input reference and the singing voice. Specifically, the evaluation engine 341 compares the singing frequency with the reference frequency to obtain a difference, compares the singing voice volume with the reference voice volume, obtains a difference, and the difference between the two is within an allowable range. If the number of times is greater than or equal to the predetermined number of times within a predetermined time, it is determined to be acceptable, and otherwise it is determined to be unacceptable. This pass / fail decision is made for each note.

  The point extraction unit 342 scores the singing voice based on the determined pass / fail determination result. The scoring here is, for example, by adding a predetermined score (plus points) for success and subtracting a predetermined score (minus points) for failure from a reference score (for example, 50 points, etc.) For example, the score is 100 points.

  The object extraction unit 343 executes a process of extracting a specific additional image from a plurality of types of additional images according to the scoring result of the point extraction unit 342 (S6). Specifically, extraction of a specific additional image is performed by comparing a scoring result with a plurality of threshold values. For example, when the plurality of threshold values are 10, 20, 30, 40, and 50 points, and the score of the singing voice is 32 points, the threshold values are 50 points or less. Therefore, the additional image corresponding to the threshold value of 50 points and the additional image corresponding to the threshold value of 40 points are specified and extracted. Thus, the lower the singing voice evaluation result, the more additional images are extracted. On the other hand, if the scoring result is a value larger than any threshold value, the additional image is not extracted.

  Conversely, the higher the evaluation result, the more the additional image may be extracted. For example, in the case of a perfect score, a crown object to be displayed above the singer image may be extracted.

  The point extraction unit 342 outputs a control signal for instructing execution of image processing to the singer image processing unit 40. Here, when the additional image is extracted, the control signal includes image data of the extracted additional image.

  When the control signal is received, the singer image processing unit 40 causes the image storage unit 401 to store the additional image included in the control signal. The video analysis unit 402 determines whether there is an additional image extraction (S7). The determination of whether or not the additional image is extracted is performed based on, for example, whether or not the additional image is stored in the image storage unit 401. If the additional image is not stored, it is not determined that the additional image is extracted (S7). NO).

When it is determined that there is an additional image extraction (YES in S7), the video analysis unit 402 refers to the image storage unit 401, analyzes the photographed image of the singer, and determines the characteristic part of the singer. Then, the feature portion extraction unit 403 extracts a feature portion corresponding to the additional image from the specified feature portions (S8). For example, when the additional image is a heel object, the bottom of the nose is extracted as a feature portion corresponding to the heel object.

  Then, the image composition unit 404 synthesizes an additional image with the extracted feature portion (S9). Thereafter, the image composition unit 404 outputs a photographed image of the singer to the composition circuit 43. Here, as for the photographed image of the singer to be output, when there is no extraction of the additional image, an image in which the additional image is not combined is output, and when there is extraction of the additional image, a composite image of the additional image is output. Is output.

  The singer image processing unit 40 updates the display of the monitor 44 by causing the synthesis circuit 43 to synthesize the photographed image of the singer and the lyrics telop and displaying the synthesized image on the monitor 44 (S10). .

  Thereafter, the evaluation mode processing unit 34 determines whether or not the karaoke performance has ended (S11), and when determining that the performance of the karaoke song has ended (YES in S12), the mode setting is changed to the normal mode. This process is terminated. On the other hand, if it is not determined that the performance of the karaoke song has ended (NO in S11), the evaluation mode processing unit 34 determines whether or not the evaluation timing has arrived (S12). If it is not determined that the evaluation timing has arrived (NO in S12), the evaluation mode processing unit 34 returns the process to step S8. On the other hand, when it is determined that the evaluation timing has arrived (YES in S12), the evaluation mode process 34 returns this process to step 5.

  FIG. 3 is an example of a screen diagram D displayed on the monitor 44 in the evaluation process shown in FIG. 2, and (a) shows a screen diagram D1 in which no additional image is synthesized with the photographed image G of the singer, b) shows a screen diagram D2 with the composition of the additional images O1, O2.

  With reference to Fig.3 (a), the case where the scoring score based on evaluation of the singing voice by step S5 is 60 points | pieces by the evaluation process shown in FIG. 2, for example. In step S6, a plurality of threshold values are compared with 60 points which are the scoring points. Here, it is assumed that the plurality of threshold values are 10, 20, 30, 40, and 50 points. Since the scoring score is 60 points, it is a value larger than any threshold value, no additional image is extracted, NO is determined in step S8, and the screen diagram D1 is displayed in step S11. Become.

  With reference to FIG.3 (b), the case where the score is 32 points | pieces is demonstrated. Since the scoring number is 32 points, it is a value larger than 40 points and 50 points among a plurality of threshold values. Therefore, in step S6, additional images (in the example, additional images O1 and O2) corresponding to the threshold values 40 and 50 are extracted. Then, YES is determined in step S8, and in step S9, a feature portion P1 corresponding to the additional image O1 and a feature portion P2 corresponding to the additional image O2 are extracted, and in step S10, the additional image O1 is added to the feature portion P1. Are combined, the additional image O2 is combined with the feature portion P2, and the screen diagram D2 is displayed in step S11. Next, when the number of scoring points is changed by executing step S5, for example, when the number of scoring points is 42, the threshold value 40 is exceeded, so that an additional image O1 corresponding to the threshold value 40 points is extracted. Not. Therefore, in the updated screen of the monitor 44, only the additional image O2 is synthesized with the singer's image G, and the display on the monitor 44 corresponds to the change in the evaluation of the singing voice during the performance of the karaoke song. Are changed sequentially.

  Thus, since the additional images are combined with the singer's image G as the score is lower, it is easier for the user to understand and the evaluation of the singing voice can be displayed. Moreover, since the transition of the change in the evaluation of the singing voice during the performance of the karaoke song can be displayed on the monitor 44, it is possible to give a tension to the singer in the middle of the singing and playability with respect to the user's singing voice. Can be increased.

  In this figure, for convenience of explanation, the characteristic portions P1 and P2 are displayed by dotted lines, but they are not displayed in the actual screen view. In the figure, the lyrics telop is omitted.

  According to the configuration of the present embodiment described above, the singer's singing ability (singing level) is achieved by using simple image processing called image synthesis as compared with the conventional technique in which the photographer's photographed image is subjected to deformation processing. Can be expressed and displayed. In particular, in this embodiment, during the performance of a karaoke song, the singer's singing voice is evaluated at predetermined time intervals, and image processing is performed on the photographed image of the singer based on the evaluation result each time it is evaluated. As a result, the display on the monitor 44 is updated, so that image processing can be performed quickly and the display on the monitor 44 can be updated relatively quickly as compared with the prior art.

(Second Embodiment)
Hereinafter, a second embodiment of the present invention will be described with reference to FIG. The second embodiment is different from the first embodiment in that effect processing is performed on the feature portion extracted by the feature portion extraction unit 403. The karaoke apparatus 1a is different from the karaoke apparatus 1 in that the singer image processing unit 40 is functionally provided with an effect processing unit 405 indicated by a broken line in FIG. 1, and the other configurations are the same. The effect processing unit 405 is an example of an effect processing unit. The scoring result is output from the point extraction unit 342, the extraction result is notified from the feature portion extraction unit 403, and the notified singing is performed according to the scoring result. The effect process including at least one of the mosaic process and the color change process is executed on the feature portion of the person.

  According to the configuration of the present embodiment, since the expression of the scoring result can be expressed not only by the synthesis of the additional image but also in the effect processing for the characteristic portion, the more detailed evaluation result can be displayed. Can do.

(Third embodiment)
Hereinafter, a third embodiment of the present invention will be described with reference to FIGS. The third embodiment is different from the first and second embodiments in that a photographed image of a singer processed by the singer image processing unit 40 can be recorded on a recording medium. . The karaoke apparatus 1b according to the third embodiment is different from the karaoke apparatus 1 and the karaoke apparatus 1a in that it further includes a composite image input / output unit 51 as shown by a broken line in FIG. It is the same. The composite image input / output unit 51 corresponds to the composite image output means of the present invention, and is realized by, for example, a Memory Stick (registered trademark) drive, a CD-R (CD Recordable) drive, or the like. The composite image input / output unit 51 receives a user's instruction using the operation unit 20 and, for example, the singer's stored in the image storage unit 401 in a writable recording medium 511 such as a memory stick or a CD-R. The captured image and the score stored in the scoring log area 102 are output.

  Further, when the evaluation mode is set, the composite image input / output unit 51 receives an instruction using the operation unit 20 of the user, and from the recording medium 511 on which the photographed image of the singer is written, Read out the captured image and the number of points. The composite image input / output unit 51 outputs the read singer to the singer image processing unit 40 and outputs the read score points to the scoring log area 102 and the point extraction unit 342. The singer image processing unit 40 uses a photographed image of the singer output from the composite image input / output unit 51. In addition, the point extraction unit 342 performs the subtraction and addition of points on the output scoring score as the first reference score in the evaluation process.

  According to the configuration of the above-described embodiment, the singing voice of the singer can be evaluated over a plurality of karaoke performances, and the singing voice can be evaluated with high playability.

  In the first to third embodiments, one mode can be selectively set from a plurality of modes. However, the present invention is not limited to this, and the evaluation mode is always set. Good.

  Moreover, in 1st-3rd embodiment, although the evaluation result of a singing voice is displayed only with the picked-up image of a singer, it is good also as a structure which displays a scoring result in addition to this. According to this, the evaluation result can be notified in detail by the user, which is more preferable.

  In the first to third embodiments, scoring is performed based on the evaluation result of the singing voice, but it is not always necessary to score, and the plus points and minus points obtained by the evaluation engine are as follows. It may be simply expressed by addition or deletion of an additional image.

  In the first to third embodiments, a live-action image of the singer is displayed. However, the present invention is not limited to this, and the “photograph of the singer” is an image of the singer. And those obtained by singing a photographed image of a singer, or those represented by animals or the like. In the first to third embodiments, a still image is displayed. However, the present invention is not limited to this, and a moving image may be displayed.

  In the evaluation mode, the background image is not displayed. However, the present invention is not limited to this, and a configuration in which a photographed image of the singer is displayed on a part of the background image may be used.

1 is a block diagram of a karaoke apparatus according to a first embodiment of the present invention. It is a flowchart which shows an example of the evaluation process which the karaoke apparatus shown in FIG. 1 performs. It is an example of the screen figure displayed on a monitor by the evaluation process shown in FIG. 2, (a) shows a screen figure without the synthesis | combination of an additional image, and (b) is a synthesis | combination of an additional image. The screen figure with is shown.

Explanation of symbols

  1, 1a, 1b-Karaoke apparatus 112-Additional image storage unit (an example of additional image storage means) 13-Sound source (an example of performance means) 15-Microphone (an example of singing input means) 16-Mixer (an example of performance means) 17-sound system (an example of performance means) 18-speaker (an example of performance means) 341-evaluation engine (an example of evaluation means) 342-point extraction unit (an example of evaluation means) 343-object extraction unit (an image processing means) Example) 40-Singer image processing unit (an example of image processing means) 403-Feature portion extraction unit (an example of extraction means) 404-Image composition unit (an example of image composition means) 405-Effect processing unit (of the effect processing means) Example) 41-video camera (an example of photographing means) 51-composite image input / output unit (an example of composite image input means)

Claims (6)

Performance means for playing karaoke music, singing input means for inputting the singing voice of the singer, evaluation means for evaluating the singing voice input by the singing input means, photographing means for photographing the singer, In accordance with the evaluation result by the evaluation means, image processing means for performing image processing on the photographed image of the singer, and display means for displaying the photographed image of the singer image-processed by the image processing means. A karaoke device provided,
An additional image storage means for storing an additional image to be added to the photographed image of the singer;
The image processing means includes
Extracting means for extracting a characteristic portion of the singer in the photographed image of the singer;
A karaoke apparatus comprising: an image synthesis unit that synthesizes the additional image with respect to the feature portion extracted by the extraction unit in accordance with an evaluation result by the evaluation unit.   The evaluation means evaluates the singing voice at predetermined time intervals during the performance of the karaoke song by the performance means, and the image processing means executes the image processing every time the evaluation result is obtained. The karaoke apparatus according to claim 1.   The karaoke apparatus according to claim 1, wherein the extraction unit extracts a constituent element of a singer's face as a characteristic part of the singer.   The karaoke apparatus according to any one of claims 1 to 3, wherein the image synthesizing unit synthesizes many additional images with the characteristic portion of the singer as the evaluation by the evaluation unit is lower.   The image processing means further comprises effect processing means for performing at least one of mosaic processing and color change processing on the characteristic part of the singer extracted by the extraction means. Item 5. The karaoke apparatus according to any one of items 1 to 4.   6. The karaoke apparatus according to claim 1, further comprising a composite image output unit that is capable of writing the photographed image of the singer that has been subjected to the composite processing of the additional image to a recording medium.

Images