JP5131291B2 - Karaoke equipment, karaoke system, set-top box - Google Patents

Description

  The present invention relates to a karaoke apparatus, a karaoke system, and a set-top box that can record a video of a singer.

  Conventionally, a karaoke apparatus capable of recording the singer's singing content has been proposed. For example, the karaoke recording device described in Patent Document 1 records a singer's singing voice together with accompaniment music. The karaoke recording apparatus is configured to record only accompaniment music.

Japanese Patent Laid-Open No. 2003-167582

  Here, in the field of karaoke devices, services can be diversified and enriched by recording not only the singing voice of the singer but also the state of the singing of the singer as a video. Providing such a service is preferable for a karaoke company from the viewpoint of improving the ability to attract customers.

  Usually, in a karaoke system, a singer sings while watching a lyrics telop displayed on a display. Also, during playback of the music, a predetermined background video is displayed on the display. Here, when the state of the singing of the singer is photographed, the content of the display can be photographed and recorded together with the singer. In this regard, when karaoke singing is performed using content for which recording of lyrics telop or background video is prohibited, it is necessary to have a configuration capable of prohibiting recording of the lyrics telop and background video (including uploading to a server or the like). is there.

  The present invention has been made to solve the above-described problems, and is a karaoke apparatus that can shoot a singer's singing state and record it as a video, and uses karaoke content for which recording is prohibited. In such a case, an object is to provide a karaoke apparatus or the like that can appropriately limit recording of photographing contents.

  A karaoke apparatus according to an aspect of the present invention includes a display, a camera, a photographing recording unit, a video output control unit, a voice output unit, a photographing video recognition unit, a transmission control unit, and karaoke data composed of video data and music data. The karaoke apparatus comprises: the audio output means outputs an audio signal based on music data included in the karaoke data; the shooting recording means records a shooting signal shot by the camera; and the video output The control means can output the video data included in the karaoke data to the display as a video signal as the music data progresses, and the photographed video recognition means can control information associated with the karaoke data. In response, the video signal included in the karaoke data is compared with the shooting signal recorded in the shooting recording means, A result signal based on the comparison result is output to the transmission control unit, and the transmission control unit transmits information related to the imaging signal to an external apparatus connected via a communication network based on the result signal. It is prohibited to do.

  The karaoke device compares the video signal constituting the karaoke data with the photographic signal recorded in the photographic recording means via the camera in accordance with the control information associated with the karaoke data. Based on the result signal is generated. In other words, the karaoke device can determine how much the video content related to the shooting signal matches the video based on the video signal related to the karaoke data, and whether the video related to the karaoke data was mainly shot. It can be determined whether or not. And the said karaoke apparatus prohibits transmitting the information which concerns on the said imaging | photography signal to an external device via a communication network based on the produced | generated result signal. Therefore, when the karaoke apparatus uses karaoke content or the like for which recording (including transmission to an external apparatus) is prohibited while providing the charm that a singing state of a singer can be taken and recorded as a video May properly limit recording.

  The “shooting signal recorded in the shooting and recording means” in the present invention is not limited to a signal that has been shot, but includes a signal that is temporarily stored during shooting. That is, the shooting data temporarily stored in the image memory or the like during execution of shooting corresponds to the “shooting signal recorded in the shooting recording means” in the present invention.

  A karaoke apparatus according to another aspect of the present invention displays and outputs on a display a character range constituting lyrics telop data in which a predetermined character size is defined by video output control means as the music data progresses. The karaoke apparatus calculates the ratio of the display to the photographic signal based on a comparison between the type and size of the character range displayed and output on the display and the lyrics telop in the photographic signal. Furthermore, the karaoke apparatus prohibits transmission of information related to the captured signal to an external apparatus when the calculated display ratio exceeds a predetermined ratio. Therefore, the karaoke device determines whether or not the image captured by the camera is an image obtained mainly by photographing the display content on the display. Transmission may be prohibited. On the other hand, when the display content of the display is not photographed actively, the karaoke apparatus can allow the information related to the photographed signal to be transmitted to the external device. As a result, the karaoke apparatus used the karaoke content etc. for which recording (including transmission to an external apparatus) is prohibited while providing the charm that the singer's singing state can be photographed and recorded as a video. If so, recording may be appropriately limited.

  A karaoke apparatus according to another aspect of the present invention extracts a still image from a shooting signal and extracts character information from the still image at a timing accompanying the progression of music indicated in the timing data. Further, the karaoke apparatus acquires a character range corresponding to the lyrics telop output to the display at the same timing as the still image extraction. The karaoke apparatus compares the character information extracted from the still image with the acquired character range. Therefore, the karaoke apparatus can calculate the ratio of the display to the photographing signal with higher accuracy, and can appropriately determine whether or not the display content on the display is being photographed independently. As a result, the karaoke apparatus used the karaoke content etc. for which recording (including transmission to an external apparatus) is prohibited while providing the charm that the singer's singing state can be photographed and recorded as a video. In some cases, recording may be more appropriately restricted.

  A karaoke system according to one aspect of the present invention is a karaoke system including a commander and a set-top box, and the set-top box includes a photographing recording unit that is connected to a camera and records a photographing signal photographed by the camera. The commander is connected to the display, and based on the karaoke data composed of the video data and the music data, the video data included in the karaoke data is displayed on the display as a video signal along with the music progress of the music data. Further, the set-top box compares the video signal included in the karaoke data with the photographic signal recorded in the photographic recording means in accordance with the control information associated with the karaoke data. And output a result signal based on the comparison result to the commander, , On the basis of the result signal, the information relating to the photographing signal, and inhibits the sending to an external device connected via a communication network.

  The karaoke system includes a commander and a set top box. The set top box compares the video signal constituting the karaoke data with the photographic signal recorded in the photographic recording means via the camera according to the control information associated with the karaoke data, and based on the comparison result The result signal is output to the commander. Based on the result signal, the commander prohibits transmission of information related to the imaging signal to an external device connected via the communication network. Accordingly, the karaoke system can determine how much the video content related to the shooting signal matches the video based on the video signal related to the karaoke data by the cooperation of the commander and the set top box. It can be determined whether or not the video related to the data has been independently photographed. As a result, the karaoke system uses karaoke contents that are prohibited from being recorded (including transmission to an external device) while providing the charm of being able to shoot and record a singer's state of singing as a video. In that case, recording can be limited appropriately.

  A set-top box according to an aspect of the present invention is a set-top box connected to a display and connected to a commander having video output control means. The set-top box is connected to a camera and captures a photographing signal taken by the camera. A recording unit for recording; a transmission unit for transmitting a shooting signal recorded in the shooting recording unit to an external device connected via a communication network; and a shooting signal recorded in the shooting recording unit. Photographed image recognizing means for performing predetermined image processing, and the photographed image recognizing means further includes the karaoke data according to control information associated with karaoke data composed of image data and music data. Along with the music progression of the music data related to the data, the video signal output to the display based on the video data and the video recording unit recorded A shadow signal is compared, and based on the comparison result, a result signal is generated that is prohibited from being transmitted to the external device based on information relating to the imaging signal, and the result signal is transmitted to the transmission unit. It is characterized by being output to.

  The set-top box includes a video signal output to the display based on the video data as the music data of the karaoke data progresses according to control information associated with the karaoke data, and the shooting and recording unit. And a result signal for prohibiting transmission to the external device based on the information related to the imaging signal based on the comparison result. Output to the transmission means. Accordingly, the set top box can determine how much the video content related to the shooting signal matches the video based on the video signal related to the karaoke data, and the video related to the karaoke data is mainly shot. It can be determined whether or not it has been done. As a result, the set-top box uses karaoke content that is prohibited from being recorded (including transmission to an external device) while providing the charm of being able to shoot and record a singer's state of singing as a video. If so, the recording can be appropriately limited.

  According to the present invention, a karaoke device or the like shoots a singer's singing and provides the charm of recording it as a video, while using karaoke content that is prohibited from being recorded, shooting for an external device Prohibit content transmission. Thus, the present invention can contribute to diversification and enhancement of services in the karaoke business.

It is explanatory drawing regarding a karaoke system. It is explanatory drawing regarding the control structure of a karaoke system. It is explanatory drawing regarding karaoke data. It is explanatory drawing regarding a copyright flag. It is explanatory drawing regarding a frame extraction signal. It is a flowchart of a video data reproduction processing program. It is a flowchart of a recognition signal confirmation processing program. It is a flowchart of a captured video upload processing program. It is a flowchart of an STB side control processing program. It is explanatory drawing regarding the control content performed by STB. It is explanatory drawing which shows the relationship between display display content, imaging | photography content, etc. FIG.

Hereinafter, an embodiment of the present invention will be described.
(System configuration)
First, the configuration of a karaoke system according to an embodiment of the present invention will be described with reference to FIGS. 1 and 2. The karaoke system according to the present embodiment is basically composed of a karaoke apparatus 100 installed in a karaoke store or the like in each place and a server 600 connected via a communication network N (see FIG. 2).

(Composition of karaoke equipment)
The karaoke apparatus 100 includes a commander 200, an STB 300 (Set Top Box), a camera 410, a first display 421, a second display 422, a microphone 430, and a speaker 450. As shown in FIG. 1, the singer S performs karaoke singing using a microphone 430 in accordance with the accompaniment related to the desired music output from the speaker 450. At this time, the lyrics telop and the background image are displayed on the first display 421 and the second display 422 (see, for example, FIGS. 11A and 11C). Therefore, the singer S can perform karaoke singing according to the display of the lyrics telop on the second display 422 or the like. And the camera 410 is normally arrange | positioned in the position which can image | photograph the singer S. FIG. Therefore, the karaoke apparatus 100 can photograph the singer S who performs karaoke singing. Since the camera 410 can be moved freely, the second display 422 may be omitted and only the first display 421 may be configured.

(Composition of commander)
The commander 200 includes a control unit 210, a video data reproduction unit 220, a karaoke data storage unit 230, an instruction input unit 240, an audio reproduction unit, and an auxiliary storage unit.
The control unit 210 has a CPU, ROM, and RAM. The CPU of the control unit 210 executes various programs (for example, a video data reproduction processing program described later). The ROM of the control unit 210 stores various data. Specifically, the ROM of the control unit 210 stores a video data reproduction processing program and the like (see FIGS. 6 to 8). The RAM of the commander 200 temporarily stores various data.
And the control part 210 receives the input of the copyright signal Cr based on the copyright flag mentioned later based on various control programs. Further, the control unit 210 outputs a shooting start signal St, a shooting end signal En, and an upload signal U based on the operation signal from the instruction input unit 240. Furthermore, the control unit 210 outputs a frame extraction signal Fe described later based on various control programs. In addition, the control unit 210 receives a recognition signal Re output from the image recognition unit 325 of the STB 300.

The video data playback unit 220 reads the playback video data Pv of the karaoke data stored in the karaoke data storage unit 230, and plays back the video based on the playback video data Pv. That is, the video data playback unit 220 has an MPEG decoder. The MPEG decoder decodes reproduced video data Pv (that is, background video data Bi and lyrics telop data Ly described later) generated in the MPEG format, and outputs a video signal Vi.
Further, the video data reproducing unit 220 changes the display color of the lyrics telop displayed based on the lyrics telop data Ly based on the telop color conversion signal Cc output according to the progress of the music related to the karaoke data to white. To a predetermined color (for example, pink, light blue, etc.) and output. That is, the telop color conversion signal Cc is output according to the progress degree of the music related to the karaoke data, and indicates the timing for changing the display color of the lyrics telop displayed on the first display 421 or the like. Further, the video data playback unit 220 outputs the audio signal Vo input from the microphone 430 to the first display 421 and the second display 422 together with the video signal Vi related to the playback video data Pv.

  And as shown in FIG. 2, the karaoke data storage part 230 has memorize | stored the karaoke data corresponding to the music used for a karaoke song. The karaoke data includes musical sound data Mu and reproduced video data Pv. The musical sound data Mu is accompaniment music data relating to a certain musical piece, and is used for karaoke singing by outputting the musical sound data Mu from the speaker 450. The reproduced video data Pv is video data output to the first display 421 and the second display 422, and is composed of background video data Bi and lyrics telop data Ly.

  The background video data Bi is data constituting a background video of a screen displayed on the first display 421 or the like when karaoke singing. For example, the background video data Bi constitutes a thundercloud video in the case shown in FIG. It should be noted that a promotion video image of an artist who sings a song related to karaoke data can be used as the background image data. The lyrics telop data Ly is data constituting a lyrics telop displayed on the first display 421 and the like when karaoke singing. The lyrics telop data is character data indicating the lyrics of music related to karaoke data, and the size of the characters constituting the lyrics telop is defined. Here, the background video based on the background video data Bi and the lyrics telop based on the lyrics telop data Ly are permitted to be “reproduced” for karaoke singing, but “memory” may be prohibited. In the singing mode shown in FIG. 1, it is assumed that the background video and the lyrics telop appear in the video taken by the camera 410 (see FIG. 13B).

  In the karaoke system according to this embodiment, when recording of lyrics telop and background video is prohibited, a copyright flag is attached to the karaoke data (see FIGS. 3 and 4). As shown in FIG. 4, when the copyright flag for the karaoke data is “1”, the copyright flag indicates that storage of the background video and lyrics telop related to the karaoke data is prohibited. On the other hand, when the copyright flag is “0”, the copyright flag indicates that storage of the background video and the lyrics telop relating to the karaoke data is permitted. Therefore, the CPU of the commander 200 can recognize whether the background video and the lyrics telop related to the karaoke data are prohibited by checking the copyright flag. In this embodiment, the copyright flag is attached to the karaoke data. However, the copyright flag for each piece of music (karaoke data) may be defined as a table. In this case, the table is stored in the storage device of the commander 200 (for example, the karaoke data storage unit 230 or an auxiliary storage unit described later).

  The instruction input unit 240 receives input of various instructions related to the karaoke apparatus 100. The instruction input unit 240 receives a user operation input to a control panel disposed on the commander 200 or a remote controller (not shown). The remote controller is wirelessly connected to the instruction input unit 240. Then, the instruction input unit 240 outputs a shooting start signal St, a shooting end signal En, and an upload signal U based on a user operation to the STB 300.

  The voice playback unit of the commander 200 plays back the accompaniment music of the music based on the musical sound data Mu constituting the karaoke data. The accompaniment music is mixed with the audio signal Vo input from the microphone 430 and emitted from the speaker 450 as a mixed audio signal.

(STB configuration)
As shown in FIG. 2, the STB 300 includes a CPU, a ROM, a RAM, an A / D converter 310, a still image extraction unit 320, an image recognition unit 325, an MPEG encoder 330, and a storage unit 340.
The CPU of the STB 300 controls the entire STB 300 based on various control programs (for example, an STB side control processing program described later). The ROM of the STB 300 stores various data. Specifically, the ROM of the STB 300 stores an STB side control processing program (see FIG. 9). The RAM of the STB 300 temporarily stores various data.
The A / D converter 310 outputs a captured video frame Rf by converting a captured video signal R (that is, an analog signal) captured by the camera 410 into a digital signal. Then, the A / D converter 310 outputs the captured video frame Rf to the MPEG encoder 330 and the still image extraction unit 320.
The still image extraction unit 320 extracts the captured video frame Rf input from the A / D converter 310 as a still image at a timing based on the frame extraction signal Fe input from the commander 200. Then, the still image extraction unit 320 inputs the captured video frame Rf extracted according to the frame extraction signal Fe to the image recognition unit 325.

  Here, the frame extraction signal Fe will be described with reference to FIG. The frame extraction signal Fe is output from the commander 200 in accordance with a video data reproduction processing program described later. As shown in FIG. 5, the frame extraction signal Fe includes a frame extraction instruction and unit lyrics telop data. The frame extraction instruction is a control instruction that instructs the still image extraction unit 320 to extract one captured video frame Rf as a still image. The unit lyrics telop data constitutes a part of the lyrics telop data Ly and is included in one screen displayed according to the progress of the music on the first display 421 or the like (for example, in the case of FIG. 13A, “ ○ △ × □ ○ △△ ”). The unit lyrics telop data includes character size data. The character size data indicates the character size defined for each character constituting the unit lyrics telop data. The unit lyrics telop data is input to the image recognition unit 325 and used for image processing (extraction content comparison processing (S35) described later).

  As illustrated in FIG. 2, the image recognition unit 325 performs predetermined image processing on the captured video frame Rf extracted as a still image by the still image extraction unit 320. The image recognition unit 325 performs character recognition processing on the captured video frame Rf from the still image extraction unit 320. Then, the image recognizing unit 325 includes character information (character type and character size) extracted from the captured video frame Rf by character recognition processing, and lyrics telop data Ly (unit lyrics telop data described later) input from the commander 200. Compare the character type and character size. When the comparison result between the extracted character information and the lyrics telop satisfies a predetermined condition, the image recognition unit 325 outputs a recognition signal Re to the commander 200.

  The MPEG encoder 330 encodes the captured video frame Rf input from the A / D converter 310 and the audio signal Vo input from the microphone 430 in the MPEG format, and generates posted moving image data Co. Here, the format of the posted moving image data Co can be selected as appropriate.

  The storage unit 340 stores the posted moving image data Co generated by the MPEG encoder 330. Thereby, the captured video signal R from the camera 410 and the audio signal Vo from the microphone 430 are captured by the STB 300. Then, the posted moving image data Co stored in the storage unit 340 is to be uploaded to the server 600 described later. That is, the posted moving image data Co is uploaded to the posted moving image database 610 via the communication network N based on the upload signal U.

  Note that the STB 300 is connected to the commander 200. The STB 300 functions as an external connection device for the commander 200. That is, the present invention can be realized by connecting the STB 300 to the existing commander 200. The commander 200 and the STB 300 are connected via a control line, and can control each other's devices using various control signals.

(Camera configuration)
The camera 410 is a digital video camera shown in FIG. 2 and includes an image forming unit 411. The camera 410 controls the shooting by the camera 410 according to the shooting start signal St and the shooting end signal En output from the commander 200. Based on the light incident on the imaging surface, a photographic video signal R that is an analog signal is generated. Then, the image forming unit 411 outputs the generated captured video signal R to the A / D converter 310 of the STB 300. Since the configuration of the digital video camera is already known, a detailed description thereof will be omitted.

(Other configurations)
Next, other configurations constituting the karaoke apparatus 100 will be described with reference to FIGS. 1 and 2. The first display 421 and the second display 422 draw and display on the screen a video related to the video signal input from the karaoke apparatus 100.
The microphone 430 inputs the voice signal Vo of the singer S to the commander 200 and the MPEG encoder 330.

  As shown in FIG. 2, the server 600 has a posted moving image database 610. The posted moving image database 610 stores posted moving image data Co uploaded from the karaoke apparatus 100. The posted moving image database 610 can be accessed via a personal computer, a commander, or the like. A user of the posted captured video (posted video data Co described later) reads the posted video data Co from the posted video database 610 and plays it back via a personal computer or other commander, thereby reproducing the posted captured video. Can browse.

  The server 600 has a karaoke database (not shown). The karaoke database stores karaoke data used for reproducing karaoke music. The server 600 distributes karaoke data stored in the karaoke database to each karaoke store.

(Contents of commander side control processing)
Next, a video data reproduction processing program executed by the control unit 210 of the commander 200 related to the karaoke apparatus 100 will be described in detail with reference to FIG.
First, in S1, the control unit 210 determines whether or not there is a music reproduction instruction based on an operation signal from a remote controller or the like. When there is a music reproduction instruction (S1: YES), the control unit 210 shifts the process to S2. When there is no music reproduction instruction (S1: NO), the control unit 210 stands by for processing.

  If transfering to S2, the control part 210 will start reproduction | regeneration of a user desired music. At this time, the control unit 210 reads the karaoke data related to the music from the karaoke data storage unit 230, and the video signal Vi based on the reproduction video data Pv (that is, background video data Bi and lyrics telop data Ly) related to the karaoke data. Are output to the first display 421 and the second display 422 via the video data reproducing unit 220. At the same time, the control unit 210 outputs the musical sound data Mu related to the karaoke data from the speaker 450 by the voice reproduction unit. Thereafter, the control unit 210 shifts the process to S3.

  In S3, the control unit 210 sequentially outputs unit lyrics telops on the first display 421 and the second display 422 as the music progresses. Here, the unit lyrics telop means a lyrics telop that constitutes a part of the lyrics telop data Ly and is displayed on one screen such as the first display 421. That is, the character string related to the unit lyrics telop corresponds to the contents of the unit lyrics telop data described above. After the unit lyrics telop is displayed and output on the first display 421 or the like in white characters of a predetermined size, the control unit 210 proceeds to S4. The display color of the unit lyrics telop is changed from white to a predetermined color based on the telop color conversion signal Cc output as the music progresses. Thereby, the singer S can grasp the singing timing of the lyrics related to the unit lyrics telop.

  In S4, control unit 210 determines whether or not there is a posting instruction based on an operation signal from a remote controller or the like. When there is a posting instruction (S4: YES), the control unit 210 outputs an upload signal U to the STB 300, and the process proceeds to S5. Here, when it is determined that there is a posting instruction in the first S4 after the reproduction of the music is started (S4: YES), the control unit 210 outputs a shooting start signal St to the STB 300 and the camera 410. Thereby, the camera 410 starts the imaging | photography regarding the mode of the singer's S singing. On the other hand, when there is no posting instruction (S4: NO), the control unit 210 shifts the process to S8.

  In S5, the control unit 210 determines whether or not a copyright flag is attached to the karaoke data related to the music. That is, the control unit 210 determines whether or not the copyright flag of the karaoke data is “1” based on the copyright signal Cr. When the copyright flag is attached (S5: YES), the control unit 210 shifts the process to S6. When the copyright flag is not attached (S5: NO), the control unit 210 shifts the process to S8.

  If transfering to S6, the control part 210 will judge whether the unit lyrics telop currently displayed on the 1st display 421 etc. was updated with the music progress of the said karaoke data. When the displayed unit lyrics telop has been updated (S6: YES), the control unit 210 shifts the process to S7. On the other hand, when the unit lyrics telop has not been updated (S6), the control unit 210 proceeds to S8.

  In general, the lyrics telop in a karaoke apparatus or the like is displayed on the first display 421 or the like when the lyric telop for one screen (that is, unit lyrics telop) is displayed on the first display 421 or the like. Be changed. Then, when all the display colors of the displayed lyrics telop are changed, the first display 421 newly displays the lyrics telop for one screen. That is, in S6, the control unit 210 determines whether the unit lyrics telop displayed on the first display 421 or the like has been changed to a new unit lyrics telop. The determination in S6 can be made based on the elapsed time from the start of music playback.

  In S7, the control unit 210 performs a frame extraction signal output process. In the frame extraction signal output process (S7), the control unit 210 determines that recording of the playback video data Pv is prohibited because the copyright flag is attached to the karaoke data of the music, and the frame extraction signal Generate and output Fe. Specifically, the control unit 210 first generates a frame extraction signal Fe based on the unit lyrics telop data related to the updated unit lyrics telop and a frame extraction instruction. Then, the control unit 210 outputs the generated frame extraction signal Fe to the STB 300. At this time, the control unit 210 outputs a frame extraction instruction related to the frame extraction signal Fe to the still image extraction unit 320, and outputs unit lyrics telop data related to the frame extraction signal Fe to the image recognition unit 325. After outputting the frame extraction signal Fe, the control unit 210 shifts the process to S8.

  In S8, control unit 210 determines whether or not the reproduction of the music related to the karaoke data has been completed. When the reproduction of the music is completed (S8: YES), the control unit 210 shifts the process to S9. On the other hand, when the music is still being reproduced (S8: NO), the control unit 210 returns the process to S3 and continues to reproduce the music.

  After shifting to S9, the control unit 210 executes a photographing end signal output process. In the shooting end signal output process (S9), the control unit 210 determines that the singing of the singer S has ended with the end of reproduction of the music, and outputs a shooting end signal En to the STB 300 and the camera 410. Thereby, the camera 410 complete | finishes imaging | photography regarding the mode of the singer's S singing. When the shooting end signal output process (S9) ends, the control unit 210 ends the video data reproduction processing program.

  Next, the recognition signal confirmation processing program executed by the control unit 210 of the commander 200 will be described in detail with reference to FIG. Here, in the STB 300, the recognition signal Re has a ratio of the first display 421 or the like that displays the reproduced video data Pv occupying the captured video frame Rf is equal to or higher than a predetermined ratio, and the first display 421 or the like that displays the reproduced video data Pv. Is output to the commander 200 from the still image extraction unit 320 of the STB 300. That is, the recognition signal confirmation processing program is a program for confirming whether the screen of the reproduced video data Pv attached with the copyright flag is being photographed by the camera 410.

  First, in S11, the control unit 210 determines whether or not the reproduction of the reproduction video data Pv related to the karaoke data is started based on an operation signal from a remote controller or the like. When the reproduction of the reproduced video data Pv has been started (S11: YES), the control unit 210 shifts the process to S12. On the other hand, when the reproduction of the reproduced video data Pv has not been started (S11: NO), the control unit 210 waits for processing.

  In S <b> 12, the control unit 210 resets the recognition signal detection memory formed in the RAM constituting the control unit 210. After initializing the storage contents of the recognition signal detection memory, the control unit 210 shifts the process to S13.

  After shifting to S13, the control unit 210 determines whether or not the recognition signal Re has been received. The recognition signal Re has a predetermined comparison result of the character type and size included in the captured video frame Rf and the character type and size constituting the unit lyrics telop data in the STB side control processing program (FIG. 9) described later. When the condition is satisfied, the image is output from the image recognition unit 325. When the recognition signal Re is received (S13: YES), the control unit 210 proceeds to S14. On the other hand, when the recognition signal Re is not received (S13: NO), the control unit 210 shifts the process to S15.

  In S14, the control unit 210 sets the recognition signal detection memory in response to receiving the recognition signal Re. The fact that the recognition signal detection memory is set means that the comparison result between the character extracted from the photographed video frame Rf and the character related to the unit lyrics telop data satisfies a predetermined condition, It shows that the video including the data Pv is mainly recorded. After setting the identification signal detection memory, the control unit 210 proceeds to S15.

  In S15, the control unit 210 determines whether or not the reproduction of the reproduced video data Pv is finished. When the reproduction of the reproduced video data Pv has been completed (S15: YES), the control unit 210 ends the recognition signal confirmation processing program. On the other hand, when the reproduction of the reproduced video data Pv has not been completed yet (S15: NO), the control unit 210 returns the process to S13.

Next, a captured video upload processing program executed by the control unit 210 of the commander 200 will be described in detail with reference to FIG. The captured video upload processing program is executed when the posted moving image data Co captured by the camera 410 is uploaded.
First, in S21, the control unit 210 determines whether or not there is a posting instruction from the user. Specifically, the control unit 210 performs the determination process of S21 depending on whether or not the upload signal U is output. When there is a posting instruction (S21: YES), the control unit 210 shifts the process to S22. On the other hand, when there is no posting instruction (S21: NO), the control unit 210 waits for processing.

  In S22, control unit 210 determines whether or not a recognition signal detection memory formed in the RAM is set. When the recognition signal detection memory is set (S22: YES), the control unit 210 shifts the process to S23. On the other hand, when the recognition signal detection memory is not set (S22: NO), the control unit 210 shifts the process to S24.

  After shifting to S23, the control unit 210 executes upload non-permission processing. In the upload disapproval process (S23), it is determined that the video of the reproduction video data Pv whose recording is prohibited is included in the posted video data Co, and the control unit 210 uploads the posted video data Co. A control signal is output to the STB 300 so as to delete the posted moving image data Co from the storage unit 340 without performing it. Further, in the upload non-permission process (S23), the control unit 210 displays an image (hereinafter referred to as “non-permission notification image”) that “the upload of the post video data Co is prohibited and cannot be uploaded”. And displayed on the first display 421 or the like (see FIG. 11D). When the upload disapproval process ends, the control unit 210 ends the captured video upload process program.

  In S24, the control unit 210 executes upload permission processing. In the upload permission process (S24), the control unit 210 outputs a control signal to the STB 300 in order to upload the posted moving image data Co to the posted moving image database 610 of the server 600 via the communication network N. When the upload permission process ends, the control unit 210 ends the captured video upload process program. When receiving the control signal output in the upload permission process, the STB 300 reads the posted moving image data Co indicated by the control signal from the storage unit 340 and uploads it to the posted moving image database 610 via the communication network N.

(Contents of STB side control processing)
Next, the STB side control processing program executed by the STB 300 according to the karaoke apparatus 100 will be described in detail with reference to FIG. The STB side control processing program is executed by the CPU of the STB 300.
First, in S31, the STB 300 determines whether or not the imaging start signal St output from the commander 200 has been received. When the imaging start signal St is received (S31: YES), the STB 300 shifts the process to S32. When the imaging start signal St has not been received (S31: NO), the STB 300 waits for processing.

  In step S32, the STB 300 determines whether or not the frame extraction signal Fe is received. As described above, the frame extraction signal Fe reproduces the reproduction video data Pv whose recording is prohibited on the screen of the first display 421 or the like, and the contents of the unit lyrics telop displayed on the screen are updated. In this case, it is output from the commander 200 (S7). When receiving the frame extraction signal Fe (S32: YES), the STB 300 shifts the process to S33. On the other hand, when the frame extraction signal Fe is not received (S32: NO), the STB 300 shifts the process to S39.

  In S33, the STB 300 executes a captured video frame extraction process. In the captured video frame extraction process (S33), the STB 300 extracts one captured video frame Rf based on the captured video signal R from the camera 410 as a still image based on the received frame extraction signal Fe, and the image recognition unit 325. To enter. After extracting the captured video frame Rf as a still image, the STB 300 shifts the process to S34.

  In S34, the STB 300 executes a character information extraction process. Specifically, the STB 300 controls the image recognition unit 325 to perform character recognition processing on the captured video frame Rf extracted as a still image in S33, and extracts characters included in the captured video frame Rf. Then, the STB 300 generates character information based on the extracted character type and size. That is, the character information indicates the character type and character size extracted by the character recognition process. Since the processing content of the character recognition process in the character information extraction process (S34) is already known, detailed description thereof will be omitted. When the analysis result of the character information extraction process is stored in the RAM of the STB 300, the STB 300 ends the character information extraction process and proceeds to S35.

  Here, the captured video frame Rf extracted as a still image in the captured video frame extraction process (S33) is extracted based on the frame extraction signal Fe. The frame extraction signal Fe is output to the STB 300 when the unit lyrics telop displayed on the first display 421 or the like is updated (S6: YES). As described above, the display color of the unit lyrics telop is initially white when displayed, and is changed to a predetermined color as the music progresses. Accordingly, when the captured video frame Rf extracted as a still image includes a screen such as the first display 421, the unit lyrics telop is included as white characters. In general, for a background video related to background video data Bi configured using various colors, character recognition processing can be performed with high recognition accuracy when the lyrics telop is a white character. That is, since the character recognition process is also performed on the captured video frame Rf extracted at the timing related to the frame extraction signal Fe, the STB 300 can extract the character information with higher accuracy.

  After shifting to S35, the STB 300 executes an extracted content comparison process. In the extracted content comparison process (S35), the STB 300 compares the character information extracted from the captured video frame Rf in S34 with the unit lyrics telop data input in response to the reception of the frame extraction signal Fe. Specifically, the STB 300 performs a comparison regarding the type of characters constituting the character information and the type of characters constituting the unit lyrics telop data. Furthermore, the STB 300 calculates the size ratio by comparing the character size related to the character information with the character size based on the character size data (see FIG. 5) included in the unit lyrics telop data. Thereafter, the STB 300 stores the character type comparison result and the size ratio in the RAM of the STB 300, and the process proceeds to S36.

  The size ratio calculated in the extracted content comparison process (S35) indicates the ratio of the character size related to the character information to the character size related to the unit lyrics telop data. As described above, the character size included in the unit lyrics telop data is a specified value, and the screen sizes of the first display 421 and the second display 422 in the karaoke system can also be specified. Accordingly, the STB 300 can specify the size of the first display 421 and the like in the extracted captured video frame Rf based on the size ratio calculated in the extracted content comparison process (S35). Furthermore, since the size of the captured video frame Rf captured by the camera 410 can be specified, the STB 300 calculates the ratio of the first display 421 and the like in the captured video frame Rf based on the calculated size ratio. It can be used as an indicator of whether or not the display content on the first display 421 or the like is being photographed actively.

  In S36, the STB 300 reads the comparison result of the extracted content comparison process (S35), and determines whether or not the character type related to the character information matches the character type constituting the unit lyrics telop data. If the character types match (S36: YES), the STB 300 determines that the first display 421 or the like that displays the reproduced video data Pv is being photographed, and the process proceeds to S37. On the other hand, if the character types do not match (S36: NO), the STB 300 moves the process to S39.

In S37, the STB 300 determines whether or not the size ratio calculated in the extracted content comparison process (S35) is greater than or equal to a predetermined ratio. Here, the predetermined ratio is a ratio in which the reproduced video is reflected in the screen, and if a design numerical value (for example, the captured video screen size 10: the reproduced video screen size 1) is determined. Although it is good, it is often different for each author depending on the way of thinking about video. In such a case, the predetermined ratio is determined in advance as the author of the reproduced video data Pv, and information indicating the predetermined ratio is attached to the reproduced video data Pv together with the copyright flag.
When the control unit 210 detects the copyright flag in S5, the commander 200 side reads the attached predetermined ratio, and sets the predetermined ratio to the STB 300 in the period from S5: YES to S7. By transmitting using a signal line, a predetermined ratio is set on the STB 300 side. Since this period is before the frame extraction signal output process (S7), the STB 300 side has not reached (S32), so the process is not hindered.
The determination of the predetermined ratio for each playback video by the configuration of the playback video data Pv, the processing on the commander 200 side, and the processing on the STB 300 side reflects the intention of the playback video data Pv for each playback video. The effect to obtain is produced.
When the size ratio is equal to or greater than the predetermined ratio (S37: YES), the STB 300 determines that the first display 421 or the like that displays the reproduced video data Pv is being photographed independently, and the process proceeds to S38. . On the other hand, when the size ratio is smaller than the predetermined ratio (S37: NO), the STB 300 moves the process to S39.

  After shifting to S38, the STB 300 executes a recognition signal output process. In the recognition signal output process (S38), the STB 300 determines that the first display 421 or the like that displays the reproduced video data Pv is being photographed mainly by the processes of S33 to S37, and controls the control unit 210 of the commander 200. A recognition signal Re is output. When the recognition signal output process (S38) ends, the STB 300 proceeds to S39.

  In S39, the STB 300 determines whether or not the photographing end signal En has been received. When the imaging end signal En is received (S39: YES), the STB 300 ends the STB side control processing program. On the other hand, when the photographing end signal En has not been received (S39: NO), the STB 300 returns the process to S32.

(Contents of control during upload prohibited video playback)
Next, in the karaoke apparatus 100 according to the present embodiment, the control contents when the STB side control program is executed will be described with reference to FIG. In the following description, a case will be described in which a video based on the reproduced video data Pv of the karaoke data with the copyright flag (hereinafter referred to as upload prohibited video) is output to the first display 421 or the like.

  First, the control contents related to the output of the upload prohibited video performed on the commander 200 side will be described. As described above, the karaoke data storage unit 230 attaches the copyright flag “1” to the karaoke data related to the upload prohibited video (see FIGS. 3 and 4). Then, when playback of the upload prohibited video (for example, see FIG. 13C) is started, the control unit 210 draws and outputs the upload prohibited video on the first display 421 or the like. At this time, the control unit 210 outputs the frame extraction signal Fe to the STB 300 at a timing of updating the unit lyrics telop in the upload prohibited video as the music progresses (S5 to S7).

  Here, the karaoke apparatus 100 which concerns on this embodiment can image | photograph the mode of singing of the singer S with the camera 410. FIG. When shooting the state of the singing of the singer S, the first display 421 or the like displaying the upload prohibited video may be reflected.

Subsequently, in the karaoke apparatus 100 according to the present embodiment, the control on the STB 300 side when the first display 421 or the like (for example, refer to FIG. 11C) that displays the upload prohibited video is reflected at the time of shooting by the camera 410. Will be described.
As described above, when the camera 410 receives the shooting start signal St, the camera 410 shoots the singing state of the singer S and the like. Then, the STB 300 converts the captured video signal R captured by the camera 410 into a captured video frame Rf using the A / D converter 310. Here, when the frame extraction signal Fe is received from the commander 200, the STB 300 inputs a frame extraction instruction related to the frame extraction signal Fe to the still image extraction unit 320 and unit lyric telop data to the image recognition unit 325 (FIG. 10). reference).

  The STB 300 inputs a frame extraction instruction to the still image extraction unit 320, thereby extracting the captured video frame Rf at the time when the unit lyrics telop is updated as a still image (S33). Since the frame extraction signal Fe is output at the timing when the unit lyrics telop is updated, the extracted captured video frame Rf may include the unit lyrics telop displayed in white characters. Thereafter, the STB 300 inputs the captured video frame Rf extracted as a still image to the image recognition unit 325.

  When the extracted captured video frame Rf is input to the image recognition unit 325, the STB 300 first performs character recognition processing on the input captured video frame Rf, and converts characters included in the captured video frame Rf into the captured video. Extract from frame Rf. For example, when the captured video frame Rf includes the content shown in FIG. 11C, the STB 300 performs a character string (that is, a unit of “XXX × ΔΔΔ”) from the captured video frame Rf by character recognition processing. Extract lyrics telop). After that, the STB 300 generates character information by specifying the character type and character size of the extracted character, and stores the character information in the RAM of the STB 300 (S34).

  After that, the STB 300 compares the extracted character information with the unit lyrics telop data related to the frame extraction signal Fe by the extracted content comparison process (S35). The STB 300 determines whether the character related to the character information is derived from the unit lyrics telop displayed on the first display 421 or the like by determining whether the character type matches or does not match between the character related to the character information and the character of the unit lyrics telop. Is confirmed (S36). Further, the STB 300 calculates a size ratio based on the character size related to the character information and the character size indicated by the character size data of the unit lyrics telop (S37). By calculating the size ratio, the STB 300 can determine whether the captured video frame Rf has been captured primarily by the first display 421 or the like that displays the upload prohibited video.

  When the STB 300 determines from the comparison result of the extracted content comparison process (S35) that the first display 421 or the like that displays the upload-prohibited video has been photographed mainly (S36: YES, S37: YES), the image recognition unit The recognition signal Re is output to the control unit 210 of the commander 200 via 325 (S38).

  When receiving the recognition signal Re, the control unit 210 sets the recognition signal detection memory (S14). Then, when the recognition signal detection memory is set (S22: YES), the controller 210 executes an upload non-permission process (S23). In this upload non-permission process (S23), the control unit 210 outputs a control signal to the STB 300, and deletes the posted moving image data Co from the storage unit 340 without uploading it to the posted moving image database 610. At this time, the control unit 210 displays a non-permission notification image (see FIG. 11D) on the first display 421 and the like.

  Even if the captured video frame Rf includes the display content of the first display 421 or the like, if the first display 421 or the like has not been photographed based on the size ratio (S37: NO), The STB 300 does not output the recognition signal Re to the commander 200. That is, when the upload prohibited video is not photographed, the karaoke apparatus 100 uploads the posted video data Co including the captured video frame Rf to the posted video database 610 (S24).

(Contents of control during upload permitted video playback)
Next, in the karaoke system according to the present embodiment, details of the control when the upload permission video is reproduced on the first display 421 or the like will be described in detail with reference to the drawings. Here, the upload-permitted video is a video that permits recording of lyrics telop and background video, and means a video related to karaoke data whose copyright flag indicates “0”.

  Here, when photographing the state of the singer S singing the music related to the upload permission video, the first display 421 and the like include the lyrics telop and the background video based on the reproduction video data Pv related to the upload permission video. An image is output (see FIG. 13A). Since the copyright flag indicates “0”, the control unit 210 does not output the frame extraction signal Fe to the STB 300 when the upload permission video is output to the first display 421 or the like. Therefore, the STB 300 does not output the recognition signal Re to the commander 200. Thereby, in the case of a music piece related to the upload permission video, the karaoke apparatus 100 can upload the posted moving image data Co to the posted moving image database 610 even if the karaoke apparatus 100 has mainly photographed the first display 421 and the like.

  As described above, the present invention is a karaoke system including the karaoke apparatus 100 and the server 600, and the karaoke apparatus 100 includes the commander 200, the STB 300, the camera 410, and the like. When shooting the state in which the singer S sings the song related to the karaoke data with the copyright flag attached, the karaoke system outputs the frame extraction signal Fe to the STB 300 at the timing when the unit lyrics telop is updated ( S7), the captured video frame Rf is extracted as a still image (S33).

  Then, the karaoke system extracts character information included in the photographed video frame Rf by character recognition processing on the extracted photographed video frame Rf (S34), character type and character size related to the character information, unit The character type and the character size related to the lyrics telop data are compared (S35). If it is determined from this comparison result that the captured video frame Rf has been captured mainly by the first display 421 or the like (S37: YES), the karaoke system uploads the posted moving image data Co to the posted moving image database 610. Is prohibited (see FIGS. 7 to 9).

  Thereby, the karaoke system can reliably prohibit the uploading of the posted moving image data Co mainly including the upload prohibited video to the server 600. In the upload non-permission process (S23), the karaoke apparatus 100 displays a non-permission notification image (see FIG. 11D) on the first display 421 and the like. Thereby, the singer S can grasp that uploading of the karaoke data is prohibited and uploading of the photographed posted video data Co has not been performed.

  In addition, the karaoke system does not include the upload prohibition video even if the posted video data Co includes the upload prohibition video (for example, when the first display 421 is accidentally reflected). Then, uploading of the posted moving image data Co is permitted (S24). Therefore, the karaoke system can provide a karaoke service capable of photographing the state of the singer's singing and recording it as a video while appropriately protecting the upload prohibited video.

  Furthermore, when the singer S sings a song related to karaoke data not accompanied by a copyright flag with the camera 410 and uploads the contents, the karaoke system outputs a frame extraction signal Fe. The reproduced video data Pv is output to the first display 421 or the like. As a result, the karaoke system does not extract the captured video frame Rf as a still image and does not output the recognition signal Re. Therefore, the karaoke system uploads the posted moving image data Co including the captured video frame Rf to the server 600 (S24). Thereby, the karaoke system can record the video data including the content of the upload-permitted video and upload the video data to the server 600.

  As a result, when the upload prohibition video is used, the karaoke system and the karaoke apparatus appropriately prohibit the shooting of the singing state of the singer S and the upload according to the relationship of the upload prohibition video in the captured video frame Rf. Can do. And when the upload permission video is used, it is possible to shoot and upload the display contents of the first display 421 and the like along with the state of the singing of the singer S. Can contribute to enrichment.

  Although the present invention has been described based on the embodiments, the present invention is not limited to the above-described embodiments, and various improvements and modifications can be made without departing from the spirit of the present invention. For example, in the present embodiment, the STB 300 is configured to perform extraction of the captured video frame Rf (S33), extraction of character information (S34), and comparison of character information and unit lyrics telop data (S35). However, the present invention is not limited to this embodiment. For example, the commander 200 may be configured to execute the processes of S33 to S37. In this case, if the commander 200 acquires the captured video frame Rf from the STB 300 at the update timing of the unit lyrics telop and executes the character information extraction process by the image recognition unit arranged in the commander 200, the commander 200 , S34 to S37 can be performed.

  In the present embodiment, when color information is detected from the captured video frame Rf, uploading is prohibited and deleted from the storage unit 340 when the posted video data Co including the captured video frame Rf is uploaded to the server 600. However, the present invention is not limited to this mode. For example, it is possible to configure so that the photographing by the camera 410 is stopped when the recognition signal Re is output in S37.

  Furthermore, each flowchart described above is merely an example, and the processing may be realized by another flowchart as long as the same result as the processing of each flowchart can be obtained.

  And this invention is realizable also as a karaoke apparatus mentioned above, a karaoke system, the method concerning STB, the program for making a computer perform this method, the recording medium which recorded this program, etc.

100 Karaoke device 200 Commander 210 Control unit 220 Video data playback unit 230 Karaoke data storage unit 300 STB
320 Still image extraction unit 325 Image recognition unit 340 Storage unit 410 Camera 421 First display 422 Second display 450 Speaker Pv Playback video data Cr Copyright signal Fe Frame extraction signal Rf Shooting video frame Ly Lyric telop data Re recognition signal

Claims (5)

A karaoke apparatus comprising a display, a camera, a photographing recording means, a video output control means, a sound output means, a photographed video recognition means, a transmission control means, and karaoke data composed of video data and music data,
The audio output means outputs an audio signal based on music data included in karaoke data,
The photographing recording means records a photographing signal photographed by the camera,
The video output control means can output video data included in the karaoke data to the display as a video signal as the music data progresses.
The photographed video recognition means compares the video signal included in the karaoke data with the photographed signal recorded in the photography recording means according to the control information associated with the karaoke data, and based on the comparison result Outputting a result signal to the transmission control means;
The karaoke apparatus, wherein the transmission control unit prohibits transmission of information related to the photographic signal to an external apparatus connected via a communication network based on the result signal. The karaoke apparatus according to claim 1,
The video data includes lyrics telop data composed of characters having a predetermined character size,
The transmission control means includes
As the music data progresses, a predetermined character range constituting the lyrics telop data is instructed to sequentially display the video output control means, and the character range is sent to the photographed video recognition means.
The photographed image recognition means includes
The display occupies the photographic signal recorded in the photographic recording means based on the comparison between the character type of the character range and the size of the character range and the lyrics telop in the photographic signal recorded in the photographic recording means. To calculate the percentage of
A karaoke apparatus characterized in that, when the calculated ratio exceeds a predetermined ratio, a result signal for prohibiting transmission of information relating to the photographing signal to the external apparatus is output to the transmission control means. A karaoke apparatus according to claim 2,
The karaoke data further includes timing data indicating the display timing of the lyrics telop,
The transmission control means includes
Sending a still image extraction instruction and a character range corresponding to the lyrics telop output to the display to the photographed video recognition means at the timing accompanying the music progression indicated in the timing data,
The photographed image recognition means includes
In accordance with the still image extraction instruction, extract a still image from the shooting signal,
Extract text information from the extracted still image,
A karaoke apparatus, wherein the extracted character information is compared with a character range sent from the transmission control means. A karaoke system consisting of a commander and a set-top box,
The set top box is
Connected to the camera,
A photographing recording means for recording a photographing signal photographed by the camera;
The commander
Connected to the display,
Based on karaoke data consisting of video data and music data, the video data included in the karaoke data can be output to the display as a video signal as the music data progresses,
Furthermore,
The set top box is
In accordance with the control information associated with the karaoke data, the video signal included in the karaoke data is compared with the photographic signal recorded in the photographic recording means, and a result signal based on the comparison result is sent to the commander. Output,
The commander
Based on the result signal, the karaoke system is characterized by prohibiting transmission of information relating to the photographing signal to an external device connected via a communication network. A set-top box connected to a display and connected to a commander having video output control means,
Connected to the camera,
Photographing recording means for recording a photographing signal photographed by the camera;
A transmission means for transmitting a photographing signal recorded in the photographing recording means to an external device connected via a communication network;
Photographic video recognition means for performing predetermined image processing on the photographic signal recorded in the photographic recording means,
Further, the photographed image recognition means includes
According to the control information associated with karaoke data consisting of video data and music data,
Along with the music progression of the music data related to the karaoke data, the video signal output to the display based on the video data is compared with the shooting signal recorded in the shooting recording means, based on the comparison result, A set top box characterized in that, based on information relating to the photographing signal, a result signal that is prohibited from being transmitted to the external device is created, and the result signal is output to the transmission means.

Images