KR20210037891A - System and method for providing karaoke service using mask content - Google Patents

Abstract

The present invention relates to a system and a method for providing a karaoke service using mask content. The mask content-based karaoke system includes: a karaoke device providing a song accompaniment, outputting mask content pieces that can be selected by a user, imaging the singing user, synthesizing the mask content selected by the user with the facial area of the user in the taken image, outputting the result of the synthesis in real time, and transmitting the output image to a karaoke service providing server; and the karaoke service providing server receiving the mask content-synthesized singing user image from the karaoke device, performing matching and storage by user account, exposing the singing user images of users to the karaoke device and a connected user terminal, conducting an audition for user selection-based ranking setting, and sending the singing user image of a user for personal collection or sending and uploading the image to an interworking external server depending on the user's selection.

Description

Karaoke system using mask contents and method of providing karaoke service {SYSTEM AND METHOD FOR PROVIDING KARAOKE SERVICE USING MASK CONTENT}

The present invention relates to a karaoke system for providing a karaoke service with a masked singer concept by applying mask content to a user's image, and a method for providing the same.

One of the domestic TV programs, “The Masked Singer”, is gaining popularity among the public by appealing to the fun of inferring a singer with only a voice without knowing the singer and the advantages of being able to concentrate on music without prejudice.

The concept of this masked singer program has been exported to many foreign countries as well as in Korea, and the overseas market is also growing in size.

Accordingly, there is a demand for a service that allows the concept of a masked singer to be enjoyed by a large number of general people, not limited to content on broadcast media.

On the other hand, the singer concept content that the public can easily access in the modern society is karaoke. In general, the content provided through karaoke equipment is only about the accompaniment sound source required for the user to sing a song, background screens such as music videos, and lyric text provided with the background screen so that the user can refer to the beat and lyrics of the song. Do.

In addition, in the case of the existing karaoke service, only the level of service that the karaoke service provider stores data recorded by the karaoke equipment itself and provides it directly to the user was possible.

In addition to these basic karaoke services, there is a growing demand for a new concept karaoke system that allows users to use karaoke services with a masked singer concept using karaoke equipment. Furthermore, it is also necessary to develop various contents that can be provided through a karaoke system with the concept of a masked singer.

Japanese Patent No. 65630302 (Name of invention: Karaoke system, Karaoke device, video display method and video display control program)

An embodiment of the present invention is to provide a karaoke system and a service providing method for providing a karaoke service of a masked singer concept by synthesizing and providing mask content with an image that a user sings.

In addition, an embodiment of the present invention is to provide a karaoke system and a method for providing the service so that a user's song image, in which mask content is synthesized, can be used as various contents such as audition participation, individual competition, personal collection, and SNS sharing. .

However, the technical problem to be achieved by the present embodiment is not limited to the technical problem as described above, and other technical problems may exist.

As a technical means for achieving the above-described technical problem, the system for providing a karaoke service using mask content according to an embodiment of the present invention provides a song accompaniment, but outputs a plurality of mask contents that can be selected by the user, and A karaoke apparatus for photographing a user singing a song, synthesizing the mask content selected by the user to the user's face region in the photographed image, and outputting the image in real time, and transmitting the output image to a karaoke service providing server; And receiving the user song image synthesized with the mask content from the karaoke device, matching and storing the user account, and exposing the user song images of a plurality of users to the connected user terminal and the karaoke device to set the ranking according to the user's selection. It may include a karaoke service providing server for performing an audition to perform an audition, and transmitting the user's song image for personal collection or uploading to a previously linked external server according to the user's selection.

According to any one of the above-described problem solving means of the present invention, an individual can use a karaoke service with a masked singer concept by synthesizing and providing mask content with an image that the user sings.

In addition, according to any one of the problem solving means of the present invention, the song image of the user in which the mask content is synthesized is provided to an external server for sharing through a main server, personal collection or SNS, etc. , The song image can be reproduced and utilized as various contents.

In addition, according to any one of the problem solving means of the present invention, in addition to the mask content basically provided by the karaoke system itself, various models of the mask content self-produced by individuals or companies can be provided so that the user using the karaoke system can select. I can. Through this, users can select and enjoy various new mask contents.

In addition, according to any one of the problem solving means of the present invention, it is not limited to servicing songs by one song through a karaoke system, but by providing the sound sources of the latest Korean Wave idols, which are trending around the world, by part, A song image may be serviced, or a clip-based song image corresponding to a choreography image matched with a sound source of a Hallyu idol may be provided.

1 is a block diagram of a karaoke system according to an embodiment of the present invention.
2 is a block diagram of a karaoke apparatus according to an embodiment of the present invention.
3 is a diagram illustrating a graphic user interface (GUI) displayed on a second display according to an embodiment of the present invention.
4 is an example of a user interface for selecting mask content according to an embodiment of the present invention.
5 is an example of a karaoke service user interface output through a user terminal according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily implement the present invention. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In order to clearly describe the present invention, parts irrelevant to the description are omitted in the drawings, and similar reference numerals are attached to similar parts throughout the specification. In addition, while describing with reference to the drawings, the drawing numbers may vary depending on the drawings even if the configuration is indicated by the same name, and the drawing numbers are only described for convenience of description, and the concept, features, and functions of each configuration Or the effect is not to be interpreted as limiting.

Throughout the specification, when a part is said to be "connected" with another part, this includes not only "directly connected" but also "electrically connected" with another element interposed therebetween. . In addition, when a part "includes" a certain component, it means that other components may be further included, and one or more other components are not excluded unless specifically stated to the contrary. It is to be understood that the presence or addition of features, numbers, steps, actions, elements, parts, or combinations thereof, does not preclude the possibility of preliminary exclusion.

In this specification, the term'section' or'module' includes a unit realized by hardware or software, a unit realized using both, and one unit is realized using two or more hardware. Alternatively, two or more units may be realized by one piece of hardware.

1 is a block diagram of a karaoke system according to an embodiment of the present invention.

As shown in FIG. 1, the karaoke system 10 according to an embodiment of the present invention may include a karaoke apparatus 100 and a karaoke service providing server 200. Additionally, the karaoke system 10 may further include a user terminal 300 and an external server (eg, an SNS server, a video sharing service server, etc.) 400.

The karaoke apparatus 100 may be connected to each network with the karaoke service providing server 200, the user terminal 300, and the external server 400 to transmit and receive data.

Specifically, the karaoke apparatus 100 provides a song accompaniment and song lyrics to the user, and combines a screen photographed by the user while the user sings a song, and a screen created by combining the user's preselected mask content in real time. You can print it on the screen. In this case, the karaoke apparatus 100 may receive and store various mask content information, newly added song accompaniment, and song lyrics information from the karaoke service providing server 200, or provide data stored by itself. .

Meanwhile, in an embodiment of the present invention, the mask content is an image that is synthesized on a region recognized as the user's face in an image photographed by the user, and may be implemented in various forms such as a plurality of mask images, character images, and avatar images.

In addition, the karaoke apparatus 100 transmits and stores a screen (i.e., video content) generated by synthesizing the mask content previously selected by the user on a screen photographing the user while the user is singing to the karaoke service providing server 200. I can.

The user terminal 300 may access the karaoke service providing server 200 and play content stored in the server 200. In this case, the content to be played may be content generated through the karaoke apparatus 100 and then uploaded to the server 200 and stored. In addition, the user terminal 300 may directly receive and play content from the karaoke apparatus 100.

The user terminal 300 personally owns the content acquired from the karaoke apparatus 100 or the karaoke service providing server 200 (that is, an image in which mask content is combined with an image photographing a user singing a song), or It may be selectively uploaded to an external server 400 such as an SNS (Social Network Service), a streaming server, or other platform server. In this case, the user terminal 300 may access the karaoke service providing server 200 and receive a user interface for uploading its own content to the external server 400.

For reference, in an embodiment of the present invention, the "user terminal" may be implemented as a computer or portable terminal capable of accessing a server or other terminal through a network. Here, the computer includes, for example, a web browser or a notebook equipped with a mobile web browser, a desktop, a laptop, and the like, and the portable terminal is a wireless communication device that guarantees portability and mobility, For example, IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), LTE (Long Term Evolution), IMT (International Mobile Telecommunication)-2020 communication It may include all types of handheld-based wireless communication devices such as mobile terminals, smartphones, and tablet PCs.

In addition, in an embodiment of the present invention, the "user terminal" is described as an example of a smartphone, but the user terminal 300 includes a user for using karaoke service contents provided by the karaoke apparatus 100 or the server 200 A program for outputting a user interface including a service page on a screen may be mounted.

For reference, the user service page may be provided in any one of a web page, a mobile web page, a web app, a progressive web application, a hybrid web, and a native app.

In addition, a dedicated mobile app (APP) that outputs a graphic user interface (GUI) provided by the server 200 may be installed in the user terminal 300, and the user service page may be provided through such a dedicated mobile app. have.

In addition, in an embodiment of the present invention, a "network" in which data is transmitted and received between the user terminal 300 and the server 200, the karaoke apparatus 100 and the server 200, and the server 200 and the external server 400 May be implemented as any type of wireless network or wired network such as a mobile radio communication network or a satellite communication network.

Hereinafter, the configuration and operation of the karaoke apparatus 100 will be described with reference to FIGS. 2 and 3.

2 is a block diagram of a karaoke apparatus according to an embodiment of the present invention.

Referring to FIG. 2, the karaoke apparatus 100 includes a first display 110, a second display 120, a control device 130, a sound equipment 140, and a photographing unit 150.

Operations and processing for each component to be described below may be performed under the control of the control device 130.

The first display 110 includes a song service screen in which information such as song title, singer, and lyrics is sequentially updated on a basic screen related to karaoke service, that is, a background screen set to be output while accompaniment is provided, and a song service screen in which accompaniment is not provided. A background screen set as default at the time point and a song service preparation screen in which various information for selecting a song are output may be output.

The first display 110 may output a mask content service screen provided under the control of the control device 130. In this case, the service screen may be a screen matched by various service types to be described below.

For example, on the first display 110, a mask content previously selected by the user is displayed on the entire screen or at least a partial area of the screen, in an image (hereinafter referred to as “user song image”) of the user singing a song. The synthesized masked singer concept image can be output in real time.

In this case, when the masked singer concept image is output in a partial area of the first display 110, the basic screen described above may be output in the remaining area.

In addition, a basic screen may be output on the first display 110 and a masked singer concept image may be separately output on the second display 120 to be described below.

The photographing unit 150 is installed to photograph a preset area, but may be an exemplary camera. The photographing unit 150 may be disposed on the top of the first display 110 as shown in FIG. 2 so that the entire body can be photographed around the user's face as much as possible.

As described above, the first display 110 may display a screen obtained by synthesizing the mask content previously selected by the user on the user image captured by the photographing unit 150. Specifically, when the user starts singing after selecting the mask content, the first display 110 may display a screen generated by synthesizing the mask content with an image photographed by the user at the same time as the song starts in real time.

The second display 120 may be configured with a touch screen or the like to enable input by user manipulation.

The second display 120 may receive input information through a user's touch by displaying an icon in the form of a graphic user interface (GUI).

Specifically, the input through the user's touch may include selecting a mask and selecting a song provided by the karaoke system 10. A detailed GUI of the second display 120 and a karaoke service process by user selection will be described later.

3 is a diagram illustrating a graphic user interface (GUI) displayed on a second display according to an embodiment of the present invention.

3A, the GUI screen provided through the second display 120 includes an advertisement (P10), a karaoke service mode (P20), other functions (P30), and settings (P40) in the form of icons. It can be marked with. In this case, the karaoke service mode P20 may include a general song mode, an audition mode, a game mode, and a Korean wave idol mode.

When the user selects an icon on the GUI, additional information related to the selection may be displayed or proceed to the next step.

In the case of the advertisement P10, a screen may be provided to display advertisements of a pre-contracted or contracted partner company, or to expose information such as notices related to the system.

When the general song mode of the karaoke service mode P20 is selected, the process proceeds to a stage in which a song can be selected, and accompaniment of the selected song is provided so that a song can be sung. In this case, the lyrics of the song and a screen (eg, a music video screen) corresponding to the song may be displayed on the first display 110. After the song is finished, the karaoke device 100 uploads the user's song video to an external server 400 such as a karaoke service providing server 200 or an SNS (Social Network Service), a streaming server, or other platform server according to the user's selection. can do. As such, the uploaded image can be checked through the user terminal 300.

In the case of selecting the audition mode, first, the process may proceed to a step in which mask content can be selected.

In this case, the mask may be selected from a common mask that can be selected by all users for free and a personal mask that can be purchased through individual payment. When purchasing a mask through payment, payment is made through the user terminal 300, or credit card, mobile phone micropayment, money/pay payment, cash payment, point payment, QR code, on the second display 120, By outputting related icons such as cryptocurrency payment, payment can be made by user selection. To this end, the karaoke device 100 or the user terminal 300 accesses the karaoke service providing server 200, and the karaoke service providing server 200 processes payment for the corresponding user by interlocking with a separate payment server (not shown). You can do it.

After the mask is selected, a song selection step, song lyrics and screen output step, and a photographed image upload step in the above-described general song mode may be performed.

In addition, when the audition mode is selected, an event can be conducted in the form of an audition, such as a TV program, to compete with the competence of participating users or to obtain prizes and prizes. In this case, the karaoke service providing server 200 may process an audition competition for selecting a ranking by exposing the uploaded user song images for each user to a plurality of unspecified users including the corresponding user to select one or more images. .

In the case of selecting the game mode, it may proceed to a step in which a preset song confrontation with one or more other users on the offline in which the karaoke apparatus 100 is installed or one or more other users on the online connected through the other karaoke apparatus 100 is played.

For example, the type of confrontation may perform confrontation such as obtaining a high score, singing a song containing a high note, or making a louder sound than an opponent. Even in the case of such a game mode, after selecting another online or offline user selection and a confrontation method, a song selection step, a song lyrics and screen output step, and a photographed image upload step in the above-described general song mode may be performed. Additionally, you can upload the results of the match in addition to the recorded video. The video generated through this game mode is uploaded on the karaoke service control server 200 or the external server 400, which is the main server, so that the user can use a mobile app for karaoke service or a dedicated channel opened on the external server 400. You can check it by connecting.

When selecting the Hallyu idol mode, the user first selects an idol group that the user wants from among a plurality of preset idol groups, selects one member or multiple members of the idol group, and then selects a song or rap in the part sung by the selected member. Can be called. For reference, the part of the member not selected by the user can be played back with the original song of the idol group.

At this time, on the first display 110, the song lyrics of the part sung by the member selected by the user may be highlighted and displayed, or only the corresponding part may be displayed. Also, on the first display 110, a screen in which a user is photographed in a part of a group member selected by the user may be combined and displayed in a performance or broadcast image of an idol group selected by the user. In this case, the screen in which the user is photographed may be a screen which is actually photographed by the user and a screen in which a mask previously selected by the user is combined with the screen actually photographed by the user. Thereafter, the step of storing and uploading the screen output on the first display 110 may be additionally performed.

Among the GUIs provided through the second display 120, the other function P30 may output a screen including information such as a popular song chart and a popular/recommended video. In addition, a screen for the user to sign up for membership or log in may be displayed.

Among the GUIs provided through the second display 120, the setting (P40) is to set a separate accessory device related to the karaoke device 100 (eg, a microphone, amplifier, speaker, display, etc.), or a pitch related to a song. , You can display a screen that allows you to adjust the volume, beat, etc. In addition, a user interface for performing a setting for uploading an image to the user terminal 300 through the setting P40 may be provided.

In addition, referring to FIG. 3B, the GUI provided through the second display 120 further includes a recommendation function P50 for recommending popular videos among masked singer concept images created and uploaded by other users. I can.

Each operation of the first and second displays 110 and 120 and the photographing unit 150 described above, and the operation of the sound equipment 140 to be described later, may be performed through the karaoke service control apparatus 130.

Referring back to FIG. 2, the karaoke service control apparatus 130 of the karaoke apparatus 100 may be accommodated in a housing of the karaoke apparatus 100.

The karaoke service control apparatus 130 may include a communication module 131, a memory 132, and a processor 133.

The communication module 131 may transmit and receive data such as the karaoke service providing server 200 and the user terminal 300 through a network.

A karaoke service program for providing a karaoke service using mask contents through the karaoke apparatus 100 is stored in the memory 132. In addition, the memory 132 stores data related to the karaoke service, such as a song list, accompaniment information, and lyrics information.

In addition, the memory 132 may store user information input through the second display 120 or the user terminal 300 by a user using the karaoke apparatus 100, and may additionally store mask information selected by the user.

The processor 133 executes a program stored in the memory 132 and performs the following operations and processing by executing a karaoke service program.

The processor 133 performs a control related to the operation of the karaoke apparatus 100 and simultaneously performs a process of synthesizing a user song image captured by the photographing unit 150 and a mask previously selected by the user in real time.

At this time, the processor 133 processes the image received from the photographing unit 150 in real time to recognize the user's facial area included in the image, and mask content preset (ie, the user selected) in the recognized facial area. Is synthesized in real time. In this case, the processor 133 performs a process of synthesizing mask contents by tracking the user's face according to the user's movement.

As described above, the control related to the operation of the karaoke apparatus 100 performed by the processor 133 is specifically, a control operation related to song accompaniment playback, lyrics output, song-related image selection and output, and the mask content is synthesized into the user song image. And storing and providing a user image including the mask content to the main server (ie, the karaoke service control server 200) and the user terminal 300.

The sound equipment 140 of the karaoke apparatus 100 may include, for example, an accompaniment, a microphone, an amplifier, a speaker, etc., and as shown in FIG. 2, it is mounted in one area inside and outside the housing of the karaoke apparatus 100 Can be. However, a microphone and a speaker may be connected to the karaoke device 100 by wired/wireless, but may be provided separately from the device housing itself.

The above-described karaoke apparatus 100 may be disposed in place of the currently commercialized karaoke equipment, such as a karaoke business place and coin karaoke, which are generally operated by large rooms or booths. In addition, it is also possible to be disposed in a specific space such as an unmanned coffee shop in combination with a device or system having a separate purpose such as a bending machine. Through this, users can not only visit for karaoke services, but can also enjoy events by exposing them to use karaoke services using mask content simply while using other cultural facilities or convenience facilities.

In addition, the karaoke system 10 according to an embodiment of the present invention can be expanded to perform multifunctionality by combining with other IoT devices. In this way, when the karaoke system 10 is combined with IoT devices, the owner of the karaoke device 100 (ie, a business operator or a manager, etc.) can remotely manage and control the device through a smartphone or IoT device.

Returning to FIG. 1, each configuration and operation of the karaoke service providing server 200 and the user terminal 300 will be described in detail.

The karaoke service providing server 200 may include a communication module, a memory, and a processor. The communication module may transmit and receive data by communicating with the karaoke apparatus 100, the user terminal 300, and the external server 400, respectively, through a network.

A karaoke service management program for karaoke service using mask contents is stored in the memory. In addition, a song list, accompaniment information, and lyrics information provided for each of one or more karaoke devices 100 may be matched and stored for each karaoke device 100 in the memory.

In addition, user information using the karaoke apparatus 100 may be stored in the memory, and mask information selected by the user may be additionally stored. In addition, user information and mask information may be matched and stored. In addition, the uploaded image, score, and payment information of the user may be matched and stored.

The processor executes a program stored in the memory, but may perform the following operations and processing through the execution of the karaoke service management program.

The processor receives and stores user song images including mask content uploaded within a certain period according to a preset audition schedule (eg, monthly, seasonal audition competition, etc.), but exposes it to an unspecified number of users.

In this case, the processor may process first or more image filtering according to a preset criterion, and sorts the images by time and date for the images that have passed the filtering to form a database.

The processor exposes each image stored in the database, and calculates a ranking based on the user's selection result. In this case, the processor may reflect the result of the ranking calculation in real time to preferentially expose the user's song image up to the preset rank number to the upper rank. In order to secure more active audition results, you can change and set the number of rankings selected as the highest ranking for each specific section within the audition period, and videos that are not included in the ranking are deleted from the audition list after a certain period of time, so that only the videos in the ranking required for competition are deleted. You can proceed with the audition by leaving a message.

The processor may expose these audition competition results in real time to web pages (including mobile web pages) operated by the karaoke apparatus 100 and the server 200 itself, and the images included in the audition competition results can be displayed by the user. Each user interface may be provided so as to check and select directly on the karaoke apparatus 100 or check and select online through the user terminal 300.

In addition, the processor provides a user interface for uploading personally created mask content produced by an individual or a company to the connected user terminal 300. In response to this, the processor may provide a payment system that allows the user to select and purchase desired mask content by checking the uploaded personal mask content at the karaoke apparatus 100 and the user terminal 300. To this end, the karaoke service control server 100 may interwork with an external payment server.

4 is an example of a user interface for selecting mask content according to an embodiment of the present invention.

The processor may provide a mask content selection page through the second display 120 of the karaoke apparatus 100 or a mobile app installed on the user terminal 300.

As shown in FIG. 4, the mask content selection page includes a list P40 including a plurality of mask contents, a selection mask item P50 showing the mask contents selected by the user, and a selection confirmation for determining the selected mask contents. Item P60 may be included. In FIG. 4, it is shown as an example that the selection confirmation item P60 is a “purchase” item that enables payment processing.

Meanwhile, the user terminal 300 may directly check the user's own image through an application for a karaoke service using the installed mask content, and may process a basic function of purchasing a mask or participating in an event. For reference, a user may download an application through a market supported by the user terminal 300, and such an application may be provided from the karaoke service control server 200.

5 is an example of a karaoke service user interface output through a user terminal according to an embodiment of the present invention.

Referring to FIG. 5, a GUI capable of processing individual member registration and login steps through a karaoke service application executed on the user terminal 300 is output.

After membership registration and login steps are processed through this GUI, GUI icons such as HOME, my information, video view, usage history, market access, sharing, announcements, and events are displayed.When selecting an icon, information related to the icon is displayed. Can be printed.

The HOME icon performs a function of moving to the first screen, My information outputs information related to the user, and the video view displays a video uploaded or stored on the karaoke service providing server 200, karaoke device 100, SNS, etc. And the usage history allows you to check the user's access history and karaoke device usage history, and the market can perform functions such as saving to the personal purchase avatar market and uploading a personal avatar to the market, and sharing Applications and user videos can be shared with other users, streaming service servers, and SNS, and related information can be output for announcements and events.

Through the application installed in the user terminal 300, various guide information for using a karaoke service using mask content may be provided, and as described above, membership registration and digital mask content selection and Various services such as purchase, payment, song selection, and point accumulation are all available.

The above description of the present invention is for illustrative purposes only, and those of ordinary skill in the art to which the present invention pertains will be able to understand that other specific forms can be easily modified without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and are not limiting. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as being distributed may also be implemented in a combined form.

The scope of the present invention is indicated by the claims to be described later rather than the detailed description, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be construed as being included in the scope of the present invention. do.

10: Karaoke system using mask content
100: Karaoke device
200: Karaoke service control server
300: user terminal
400: external server

Claims (1)

In the system for providing a karaoke service using mask contents,
Provides accompaniment to songs, but outputs a plurality of mask contents selectable by the user, photographs the user singing the song, synthesizes the mask contents selected by the user into the user's facial area in the photographed image, and outputs it in real time. A karaoke apparatus for transmitting the output image to a karaoke service providing server; And
Receives a user song image synthesized with mask content from the karaoke device, matches and stores the user account, and exposes the user song images of a plurality of users to the connected user terminal and karaoke device to set the ranking according to the user's selection. A karaoke service providing system comprising a karaoke service providing server for performing an audition and delivering a user's song image for personal collection or uploading to a previously linked external server according to a user's selection.

Images