JP6315475B2 - Karaoke management system, program and karaoke system - Google Patents

Description

  The present invention is a karaoke device for enjoying singing, a server device connected to the karaoke device via a network, a karaoke management system provided with a program executable on a mobile terminal, and executable on a mobile terminal The present invention relates to a karaoke system including a program and a karaoke device.

  A karaoke apparatus that enjoys singing along with accompaniment has various functions in addition to the accompaniment function. In recent years, there is also a karaoke apparatus equipped with a posting function that uses an image taken by a camera to take an image and release the state of singing on the Internet. Patent Documents 1 and 2 disclose various karaoke systems that perform imaging using such a camera.

  In Patent Document 1, a singer at the time of singing is photographed as a video, and the reservation information of the singing video specified at a predetermined timing accompanying the music selection information of the reserved music by the karaoke user and the music selection information is a column of the reservation queue. When the reserved music is played, if the shooting presence / absence information is shooting specified, the shooting means is operated from the start to the end of the performance, the video data from the shooting means is acquired, and the recorded data is stored. A karaoke system is disclosed in which recording is automatically recorded from the start to the end of the performance of all music pieces by storing in the recording section.

  Patent Document 2 receives video data and audio data transmitted from a karaoke terminal device, and evaluation results obtained by evaluating the singing voice at a plurality of different playback points, and receives a plurality of received video data, audio data, and evaluation results. A karaoke system is disclosed that stores and associates and stores karaoke content data that selectively reproduces one audio data related to each video data according to an evaluation result for each audio data of a song. Yes.

JP 2008-139374 A JP 2011-59620 A

  As disclosed in Patent Documents 1 and 2, when imaging is performed in a karaoke apparatus, it is usual to use a camera connected to the karaoke apparatus. Since the camera is installed at the store side, it is fixed at a predetermined position, and it is not possible to select a shooting position, a shooting angle, or the like. In addition, there is a model that does not have an imaging function or a karaoke apparatus that has an imaging function but is not connected to a camera for convenience of a store. Conventionally, with such a karaoke apparatus, it has been impossible to use a service for photographing and viewing a state of singing.

  The present invention has been made in view of such a situation, and by using a mobile terminal possessed by a user for imaging a karaoke song image, the karaoke apparatus can easily be used even if it does not have a function related to imaging. The purpose is to form video works of karaoke singing.

Therefore, the karaoke management system according to the present invention is
In a karaoke management system comprising a karaoke device, a program for causing a mobile terminal to execute processing, and a server device,
The karaoke device can execute reservation reception processing, transmission processing, reproduction processing, and recording processing,
The reservation reception process registers reservation information including the music identification information of the music designated by the user and the user identification information of the user who designated the music in the reservation table,
The transmission process sends the reservation table to the mobile device,
The reproduction process reads the reservation information registered in the reservation table, reproduces the music information corresponding to the music identification information in the reservation information,
The recording process records the singing voice signal input from the microphone, forms the recording information in association with the reservation information,
The program causes the mobile terminal to execute reception processing, designation processing, and imaging processing,
The reception process receives the reservation table from the karaoke device,
The designation process accepts designation of reservation information registered in the reservation table,
In the imaging process, when reservation information is specified, the imaging information captured by the imaging unit is stored in association with information included in the specified reservation information,
In any of the karaoke device, the program, and the server device, the video work forming process can be executed,
In the video work formation process, the video work information is formed by associating the recording information corresponding to the reservation information with the imaging information corresponding to the reservation information.

Furthermore, in the karaoke management system according to the present invention,
The program causes the portable terminal to execute notification processing for notifying the karaoke device of the reservation information specified by the specification processing,
The recording process is executed with the start of the reproduction process corresponding to the reservation information notified by the notification process.

Furthermore, in the karaoke management system according to the present invention,
The program executes a notification process for notifying the user before the reproduction process corresponding to the reservation information designated by the designation process is started.

Furthermore, in the karaoke management system according to the present invention,
The program activates an imaging function for executing the imaging process before the reproduction process corresponding to the reservation information designated by the designation process starts.

Furthermore, in the karaoke management system according to the present invention,
The imaging process is executed in synchronization with the reproduction process corresponding to the reservation information designated by the designation process.

Furthermore, in the karaoke management system according to the present invention,
The designation process controls designation of the reservation information when the user identification information included in the reservation information in the reservation table matches the user identification information corresponding to the mobile terminal on which the designation process is executed.

Furthermore, in the karaoke management system according to the present invention,
The reservation acceptance process includes, in the reservation table, prohibition information that is prohibited from being specified by the specification process, and registers it in the reservation table.
The designation process prohibits designation of reservation information including prohibition information.

Furthermore, in the karaoke management system according to the present invention,
The reservation acceptance process includes request information that is requested to be specified by the specification process, registered in the reservation table in the reservation information,
The designation process controls designation of reservation information including request information.

The program according to the present invention is
A program that is executed in a portable terminal that can communicate with a karaoke device,
A first receiving step of receiving a reservation table in which reservation information including the music identification information of the music specified by the user and the user identification information of the user specifying the music is registered;
A designation step for accepting designation of reservation information registered in the reservation table;
When reservation information is designated, an imaging step for storing imaging information captured by the imaging unit in association with information included in the designated reservation information;
A recording information formed by recording a singing voice signal input from a microphone, wherein the recording information stored in association with the reservation information is received from the karaoke device;
By associating the recording information corresponding to the reservation information with the imaging information corresponding to the reservation information, the portable terminal is caused to execute the forming step of forming the video work information.

In addition, the karaoke system according to the present invention is
A karaoke system that can communicate with a mobile terminal,
Reservation accepting means for registering reservation information including the music identification information of the music designated by the user and the user identification information of the user who designated the music in the reservation table;
A transmission means for transmitting the reservation table to the mobile terminal;
Reproducing means for reading the reservation information registered in the reservation table and reproducing the music information corresponding to the music identification information in the reservation information;
Recording means for recording a singing voice signal input from a microphone, and forming recording information in association with reservation information;
Receiving means for receiving imaging information associated with information included in the reservation information registered in the reservation table, which is information captured by the mobile terminal;
Forming means for forming video work information by associating the recording information corresponding to the reservation information with the imaging information corresponding to the reservation information.

  According to the karaoke management system according to the present invention, in this embodiment, it is possible to capture an image with a free imaging angle by performing imaging using a mobile terminal possessed by the user. Also, even if the karaoke device is not connected to a camera, by using the camera of the mobile terminal owned by the user, it remains as video work information including video that enjoys karaoke, and video work information such as a posting service It can be used for various services for viewing.

  Furthermore, in this embodiment, the video work information is formed by associating the imaging information captured by the portable terminal with the recording information obtained by recording the user's singing voice signal by the karaoke apparatus. Here, since the recording information is a singing voice signal input from the microphone in the karaoke apparatus, it is possible to achieve high sound quality. As described above, in the present invention, a high-quality video work that combines imaging information captured at a high-definition imaging angle using a mobile terminal and high-quality sound recording information recorded by a karaoke device. Information can be formed.

Diagram showing the configuration of the karaoke system The figure which shows the constitution of the portable terminal Flow chart showing the association process in the karaoke management system Schematic diagram showing the association process in the karaoke management system Flow chart showing remote operation processing in karaoke management system Schematic diagram showing remote operation processing in a karaoke management system The figure which shows the music confirmation screen which is displayed on the portable terminal The figure which shows the music confirmation screen which is displayed on the remote control device The figure which shows the reservation table which is managed with the karaoke device Flow diagram showing imaging and processing performed on a mobile terminal The figure which shows the reservation list screen which is displayed on the portable terminal Flow chart showing music playback process executed in karaoke device Flow chart showing imaging processing executed on portable terminal The figure which shows the data structure of portable imaging information and karaoke recording information The figure which shows the imaging screen displayed on the portable terminal Flow chart showing video work formation process executed on mobile terminal Schematic diagram for explaining the video work formation process The figure for demonstrating the image work formation process of other embodiment. Diagram showing various forms of video work information

  FIG. 1 is a diagram showing a configuration of a karaoke system according to an embodiment of the present invention. The karaoke system in this embodiment includes a karaoke device 2 (commander) and a remote control device 1. The karaoke device 2 and the remote control device 1 are connected so as to form a network using the LAN 100 and the access point 110.

  The karaoke apparatus 2 installed in a store such as a karaoke box includes an acoustic control unit 25 as a playing means for playing music. Moreover, the karaoke apparatus 2 includes an operation unit 21 as input means on the karaoke apparatus 2 side, which includes switches that accept various inputs from the user. The karaoke apparatus 2 includes an operation processing unit 22 that interprets an input from the operation unit 21 and transmits the input to the control unit 30. The karaoke apparatus 2 includes an HDD 32 (hard disk) as karaoke apparatus side storage means for storing various information. The karaoke apparatus 2 includes a LAN communication unit 24 as karaoke apparatus side communication means for connecting to the LAN 100 and joining the network.

  Further, the karaoke apparatus 2 includes video reproduction means for displaying lyrics video and background video on the monitor 41. The video playback means includes a video playback unit 29 that plays back video based on video information, a video RAM 28 that temporarily stores video to be played back, video that performs superimposed display of lyrics telop on the played back video, video effects, and the like. A control unit 31 is provided.

  Furthermore, in this karaoke apparatus 2, various information can be displayed on the touch panel monitor 33 in addition to the monitor 41 connected to the outside. The touch panel monitor 33 includes a display unit 33 a that displays video information input from the video control unit 31 and a touch panel 33 b that outputs a touch input position to the operation processing unit 22. Various operations on the karaoke apparatus 2 can also be performed via the operation unit 21 provided on the front panel of the karaoke apparatus 2 or the like.

  The touch panel monitor 33 of the present embodiment can also function as an operation unit similar to the touch panel monitor 11 that is an operation unit of the remote control device 1. The user can function as an operation means equivalent to the remote control device 1 such as selecting music from the touch panel monitor 33 and causing the karaoke device 2 to make a reservation directly.

  Furthermore, the karaoke apparatus 2 includes a control unit 30 constituted by a CPU and a memory 27 for temporarily storing information necessary for executing various programs as control means for controlling and controlling each component. I have.

  With such a configuration, the karaoke apparatus 2 executes various processes, but as a main function of the karaoke apparatus 2, a music designation process, a reproduction process, and the like can be performed. The music designation process is a process for designating and reserving music based on designation from the user, and is executed in cooperation with the remote control device 1. The reservation information transmitted from the remote control device 1 is registered in the reservation table stored in the memory 27. The reproduction process is a process for reproducing a reserved music piece, and the performance process and the lyrics reproduction process are executed in synchronization with each other.

  The performance process is a process for causing the sound control unit 25 to perform the performance data included in the music data. The music played by the acoustic control unit 25 is emitted from the speaker 42 together with the singing voice input from the singing microphones 44a and 44b. The lyrics reproduction process is a process for assisting the singing by displaying the lyrics data included in the music data on the monitor 41. A background video display process for superimposing a background video on the lyrics displayed in the lyrics reproduction process may be executed.

  On the other hand, the remote control device 1 transmits various instructions such as reservation information to the karaoke device 2 and receives various information from the karaoke device 2 or the server device 5 connected to the Internet. In the present embodiment, an operation unit 17 such as a button and the touch panel monitor 11 are provided as user interfaces. The touch panel monitor 11 includes a display unit 11a and a touch panel 11b, displays various interfaces on the display unit 11a, and can accept a touch input from a user.

  Further, the remote control device 1 controls the memory 14 as a remote control side storage means for storing a database, various programs required for music search, and various information generated in accordance with the execution of the program, and these configurations. Remote control side control means. The remote control side control means includes a control unit 15 constituted by a CPU, a video control unit 13 for forming a video to be displayed on the display unit 11a, a video RAM 12 for temporarily storing video information to be displayed, and an operation unit 17. The operation processing unit 18 that interprets and transmits the input to the control unit 15 is included.

  The remote control device 1 is connected to a network constituted by the LAN 100 by being wirelessly connected to the access point 110 by the wireless LAN communication unit 16. Each remote control device 1 is associated with a specific karaoke device 2 in advance. Various commands output from the remote control device 1 are received by the associated karaoke device 2.

  Next, the portable terminal 6 possessed by the user in the present invention will be described. FIG. 2 is a diagram showing a configuration of the mobile terminal 6 according to the embodiment of the present invention. As the mobile terminal 6, a smartphone, a tablet terminal, a mobile phone, or the like can be used. The portable terminal 6 can be used as a remote controller for the karaoke device 2 by installing the program (application) according to the present invention in the portable terminal 6 and executing the association process on the program.

  The mobile terminal 6 according to the present embodiment includes a control unit that includes a control unit 61, a RAM 62, a storage unit 63, an image processing unit 64, and an acoustic processing unit 66. The storage unit 63 can employ nonvolatile storage means such as a hard disk or a NAND flash memory. The storage unit 63 stores a program capable of executing the posting process in the present embodiment and various data used in the program.

  The control unit 61 is a unit that controls the entire portable terminal 6 such as a CPU, and executes various controls based on programs and data stored in the storage unit 63. The RAM 62 temporarily stores data generated under the control of the control unit 61.

  The sound processing unit 66 is a unit that performs input / output of sound related to the mobile terminal 6. This is means for causing the speaker 67a to emit sound under the control of the control unit 61. In addition, the acoustic processing unit 66 is connected to the microphone 67b and can capture surrounding sounds from the microphone 67b.

  The touch panel display unit 68 includes a display unit 68b for displaying an image and a touch panel 68a disposed on the front surface or the back surface of the display unit 68b. Various types of touch panel 68a that can recognize a user's touch position, such as a capacitance method, can be used. The control unit 61 can display on the display unit 68b and determine which part of the displayed image is touch-input by touch input from the touch panel 68a.

  The image processing unit 64 is a means for inputting and outputting image relations of the mobile terminal 6. The image processing unit 64 displays and outputs the image information formed by the control unit 61 on the display unit 68b. The image processing unit 64 has a function of capturing an image from the front camera 65F arranged on the front surface (touch panel display unit 68 side) of the mobile terminal 6 or the rear camera 65R arranged on the back surface of the mobile terminal 6.

  The mobile terminal 6 of the present embodiment includes a mobile network communication unit 69a and a wireless LAN communication unit 69b in order to communicate with the outside. The mobile network communication unit 69a can perform wireless communication via a mobile wireless network, and can perform communication or communication using the Internet by performing wireless communication with a wireless base station. The wireless LAN communication unit 69b can perform wireless LAN communication by wirelessly connecting to an access point connected on the LAN. By connecting to the access point wirelessly, it is possible to connect to a network on the LAN or the Internet.

  The mobile terminal 6 uses the mobile network communication unit 69a or the wireless LAN communication unit 69b to execute various programs (also referred to as “apps”) including a program that can execute a posting process from a server on the Internet. It is possible to download. The program may be installed from the external memory EX connected to the external memory connection terminal 72 other than the form using the communication line. In addition to the mobile network communication unit 69a and the wireless LAN communication unit 69b, near field communication means (infrared communication, Bluetooth (trademark), etc.) may be further provided.

  The portable terminal 6 of this embodiment is configured to include an input switch 70 in addition to the touch panel 68a as input means. The input switch 70 can be provided with various switches such as a power switch, a volume adjustment switch, and a switch whose input application is changed by a program to be executed.

  The mobile terminal 6 according to the present embodiment associates the mobile terminal 6 with the karaoke device 2 to be operated by installing and executing a program provided by the karaoke dealer, so that the music from the mobile terminal 6 to the karaoke device 2 is associated. It is possible to send various commands such as reservation commands.

  In the karaoke system installed in the karaoke store described with reference to FIG. 1, a user who visits the karaoke store can cause the mobile terminal 6 possessed by the user to function as a remote control for the karaoke system to be used. Therefore, in the karaoke management system according to the present embodiment, the association process is performed for associating the portable terminal 6 possessed by the user with the karaoke system to be used. Here, the karaoke management system in the present embodiment includes the server device 5, a program that can be executed by the mobile terminal 6, and a karaoke system that includes at least the karaoke device 2. The mobile terminal 6 possessed by the user is not included in the karaoke management system.

  Now, an example of association processing for causing the mobile terminal 6 to function as a remote controller for the karaoke apparatus 2 will be introduced. There are various forms of association processing in addition to the examples introduced here. In the present invention, as long as the associating process for associating the mobile terminal 6 possessed by the user with the karaoke system to be used, various forms of associating processes can be adopted. It does not ask various conditions such as the use of 5.

  FIG. 3 is a flowchart showing the association process in the karaoke management system of the present embodiment. Moreover, the schematic diagram which shows the matching process in a karaoke management system is shown by FIG. In the association process of the present embodiment, the user logs in to the karaoke system to be used, and associates the user's portable terminal 6 with the karaoke system by logging in with the portable terminal 6 using the same account. I am going to do that.

  The user who has visited the karaoke store uses his / her input means such as the remote control device 1 to input his / her account information (S301). The karaoke system transmits a device-side login request to the server device 5 based on the input of account information (S301). This device-side login request includes a karaoke device ID (karaoke device identification information) in order to specify which karaoke system the request is from. The server device 5 that has received the device-side login request (S201) performs authentication using the account information included in the received terminal-side login request (S202). Authentication is performed based on whether or not the pair of the user ID and password included in the account information matches that stored in the storage unit 51 on the server device 5 side. When the authentication is successful (S203: Yes), the completion of login is notified to the requested karaoke system (S204). In the karaoke system in which the login is completed, it is possible to receive various services using user information corresponding to the logged-in user (S302). On the other hand, when the authentication fails (S203: No), the server device 5 transmits authentication error information to the requested karaoke system.

  If the login from the karaoke system is successful, the server device 5 adds the karaoke device ID and the user ID to the association table stored in the storage unit 51 (S205). With this association table, on the server device 5 side, it is possible to determine which user is logged into which karaoke system.

  On the other hand, the user activates the program in the mobile terminal 6 possessed by the user, and transmits a terminal-side login request to the server device 5 (S101). This terminal-side login request includes user account information (user ID and password). The server device 5 that has received the terminal-side login request performs authentication using the account information included in the received terminal-side login request as in the case of the device-side login request (S206). When the authentication is successful (S207: Yes), the server device 5 determines whether or not the association table stored in the storage unit 51 includes the user ID of the user who has made the terminal-side login request. If there is a user ID (S208: Yes), the association between the portable terminal 6 and the karaoke system is completed (S209). Then, an association completion notification is transmitted to the mobile terminal 6 that requested it (S210). In the portable terminal 6 that has been associated, remote operation processing (S110) for remotely operating the associated karaoke system can be executed.

  The association processing of this embodiment has been described above, but it does not ask which of the device-side login request and the terminal-side login request is first. Next, details of the remote operation process (S110) executed in the associated portable terminal 6 will be described.

  FIG. 5 is a flowchart showing remote operation processing in the karaoke management system of the present embodiment, and FIG. 6 is a schematic diagram showing remote operation processing in the karaoke management system of the present embodiment. As shown in the example of the schematic diagram of FIG. 6, the karaoke management in which the mobile terminal 6 possessed by the user (user A) and the karaoke device 2 (karaoke device a) to be used by the user are associated with each other. In the system, various command information for remotely operating the karaoke system such as music reservation can be transmitted from the portable terminal 6 to the karaoke system.

  In the remote operation process, after the association process is completed, the karaoke system is operated by the portable terminal 6 by repeatedly executing the processes of S401 to S403. FIG. 7 shows a music confirmation screen displayed on the touch panel display unit 68 of the mobile terminal 6. The music confirmation screen is a screen showing details of the music designated based on the music name, singer name, etc. designated by the user, and the user can reserve the music on the music confirmation screen. On the music confirmation screen, music setting information 602 such as key setting, music type setting, posting function setting and the like is displayed in addition to music related information 601 such as singer name, music title, singing, and thumbnail images related to music. The user can reserve the displayed music piece in the associated karaoke apparatus 2 by operating the reservation button 603.

  In the music confirmation screen of the present embodiment, when singing, an imaging setting button for specifying whether or not to allow shooting is provided, and by operating the imaging setting button, a permission state for allowing imaging, It is possible to set a prohibited state in which imaging is prohibited. The reservation information includes a prohibition flag indicating the setting state of the imaging setting field. In the imaging setting column, in addition to such a form, it may be possible to set a requested state for the user who has selected the music to actively request imaging. In this case, the imaging setting is a specification that can be set to request, allow, and prohibit. The request is set when the user wants to take an image of himself / herself, the permission is set when the user does not need to take an image even if the user is taken, and the prohibition is set when the user does not want to take an image. Set to.

  In the portable terminal 6, when it is determined that there is a command input by an action such as touching the reservation button 603 on the music confirmation screen (S <b> 401: Yes), transmission command information is transmitted to the server device 5. (S402). The transmission command information includes a command (command A) corresponding to the command input and a user ID (user A) stored in the mobile terminal 6.

  The server device 5 performs a transfer process based on the association table. In the transfer process, first, it is determined whether or not the user ID (user A) in the transmission command information sent from the mobile terminal 6 exists on the association table. When there is a user ID in the transmission command information on the association table, the transmission command information is transmitted to the karaoke device 2 to be transferred. The transfer command information includes a command (command A) in the transmission command information, a karaoke device ID as a destination (karaoke device a), and a user ID (user A) of the user who has input the command. The karaoke apparatus 2 (karaoke apparatus a) that has received the transfer command information executes processing according to the command (command A) included in the transfer command information.

  The remote operation process using the portable terminal 6 is repeatedly executed until an end instruction (logout or association release instruction) is input in the portable terminal 6 (S403: Yes). When the end instruction is input (S403: Yes), the mobile terminal 6 transmits an association release instruction to the server device 5 (S404). The server device 5 that has received the association release instruction deletes the corresponding item (a pair of the karaoke device ID and the user ID) from the association table.

  In the present embodiment, since the user is logged in to the karaoke system, remote operation processing using the remote control device 1 can be performed. FIG. 8 shows a music confirmation screen displayed on the touch panel monitor 11 of the remote control device 1 when logging into the karaoke system. The user's login status is always displayed at the top of the screen. In a state where a plurality of people have logged in, the logged-in user field 103 displays the logged-in images 103a to 103e (face portions in the present embodiment) of the logged-in user. In the logged-in user column 103, the user whose background is highlighted at the right end is the active user 103e. In the state shown in the figure, the service for the active user 103e, that is, the user information of the active user 103e is used. This is a state in which music reservation is being made.

  After the music search, the user designates one music and transitions to this music confirmation screen. On the music confirmation screen, the music name, singer name, singing, etc. are displayed for the designated music, and the user can confirm whether there is no mistake in the music that he sings. When the reservation button 113 displayed on the music confirmation screen is operated, reservation information including the music identification information (music ID) and the user identification information (user ID) of the active user is transmitted to the karaoke apparatus 2. Similar to the music confirmation screen of the mobile terminal 6, in this embodiment, when singing, an imaging setting field for specifying whether or not to allow shooting is provided, and shooting is permitted by operating the change button. It is possible to set a permission state and a prohibition state in which imaging is prohibited. The reservation information includes a prohibition flag indicating the setting state of the imaging setting field. In the imaging setting column, as in the case of the mobile terminal 6 shown in FIG. 7, the user may be able to set a request state for actively requesting imaging.

  FIG. 9 shows a reservation table in which reservation information transmitted from the mobile terminal 6 or the remote control device 1 is stored and managed. As shown in the reservation table, the reservation information includes music identification information (music ID), a reservation user ID that is a user ID of the user who made the reservation, a prohibition flag (a blank indicates a permission state, and “1” indicates a prohibition state. Is included). If imaging request setting is possible, the reservation information further includes a request flag. Of the reserved music whose prohibition flag is blank (permitted state), “1” is set in the request flag field of the reserved music for which the request is set. When the reservation information is received, the karaoke apparatus 2 gives reservation identification information (reservation ID) and a scheduled start time to the reservation information. The reservation ID is information given to identify each reservation information, and is configured so that the order of reservation can be identified. In the present embodiment, the last four digits are the serial number, and 1 is added to the last serial number in the newly added reservation information. In the present embodiment, for example, the upper digit is constituted by a karaoke device ID and the lower digit is constituted by a serial number, so that it can be distinguished from a reservation ID of another karaoke device 2. The scheduled start time is a scheduled time when the music reserved in each reservation information is started to be played back, and is calculated using the current time, the playback time of the previously reserved music, and the like.

  The command (command B) such as the input reservation information is transmitted to the karaoke apparatus 2 as command information together with the user ID (user A) by the operation of the remote control device 1. The karaoke apparatus 2 (karaoke apparatus a) that has received the command information executes processing according to the command (command B) included in the command information.

  As described above, in the present embodiment, it is possible to perform a remote operation on a karaoke system to be used from a karaoke system in which a user logs in or a portable terminal. Now, main processing performed by the karaoke apparatus 2 in the karaoke system in which the user has logged in will be described.

  In this embodiment, it is possible to take an image of a user who sings using the mobile terminal 6 possessed by the user. At present, in the karaoke apparatus 2, there is a model equipped with a camera as an imaging means, but a model that does not have an imaging function or a camera that has an imaging function but is not connected for convenience of a store. There is a karaoke device. Conventionally, in such a karaoke apparatus 2, the state of singing could not be photographed and posted, but in this embodiment, imaging cannot be performed by performing imaging using the mobile terminal 6 possessed by the user. Even the karaoke device 2 can use a service using an imaging function such as posting. In addition, the camera connected to the karaoke apparatus 2, that is, the camera prepared on the karaoke store side is fixed at a predetermined position and cannot select an imaging angle or the like. By using the portable terminal 6 possessed by the camera, it is possible to take an image at any imaging angle.

  In the present embodiment, the video work information is formed by associating the imaging information captured by the mobile terminal 6 with the recording information obtained by recording the user's singing voice signal by the karaoke apparatus 2. Here, since the recording information uses the singing voice signal input from the singing microphones 44a and 44b in the karaoke apparatus 2, it is possible to achieve high-quality sound. Therefore, in the present embodiment, it is possible to form video work information that combines imaging information and high-quality sound recording information at an imaging angle with a high degree of freedom.

  Now, the formation of video work information in the karaoke management system will be described. FIG. 10 shows a flowchart of the imaging designation process executed by the mobile terminal 6. In the present embodiment, the mobile terminal 6 associated with the karaoke system can designate reservation information to be imaged in advance with reference to the reservation table of FIG. The imaging designation process of this embodiment is executed in the mobile terminal 6 possessed by the user. In a program executed on the portable terminal 6, when a reservation confirmation instruction is input (S501: Yes), a reservation table is requested from the karaoke device 2 (S502) and received from the karaoke device 2 (S503). A reservation list screen is displayed based on the table (S504). In addition, transmission / reception of various information, such as a reservation table, between the portable terminal 6 and the karaoke apparatus 2 is performed via the server apparatus 5 similarly to the transmission / reception of the command demonstrated in FIG. The server device 5 refers to the association table, and the information transmitted from the karaoke device 2 is associated with the portable terminal 6 associated with the information or the karaoke device 2 associated with the information transmitted from the portable terminal 6. Forward to.

  FIG. 11 shows a reservation list screen displayed on the touch panel display unit 68 of the mobile terminal 6. The reservation list screen is formed using the reservation table of FIG. 9, and the reservation list screen of the present embodiment includes the song name, singer name, and reserved user name of the currently playing music. The displayed reproduction music display field 604 and the reserved music display field 605 displaying the music name, singer name, and reserved user name are displayed for the reserved music. In the reserved music display column 605, each reserved music is displayed in the order of reservation. In the reserved music display field 605, an imaging button is provided for reserved music that can be imaged. The user can make an imaging reservation for the reserved music by operating an imaging button of the reserved music to be imaged.

  By the way, it is difficult for the user who sings to image his own song. Therefore, in the present embodiment, the imaging button is not displayed so that the reserved music reserved by the user is not designated for imaging. The example of FIG. 11 is a reservation list screen on the mobile terminal 6 of the user A, and no imaging button is displayed for the music title D reserved by the user A and the reserved music of the music name G. Note that the reserved music reserved by the user A may also be designated for imaging. In this case, by changing the wording displayed on the shooting button to “self-shooting” (shooting by yourself) instead of “shooting” so that it can be distinguished from the song name B and the song name E, If you select a song that you sing, you can identify that you are shooting. In addition, as described with reference to FIGS. 7 and 8, it is possible to specify permission / prohibition of imaging on the reservation confirmation screen. This imaging permission / prohibition designation is recorded as a prohibition flag in the reservation table. In the present embodiment, when the user who sings prohibits imaging, the imaging button is grayed out so that the imaging cannot be designated, and the prohibition display is changed (music name C in FIG. Reserved song with song name F). In addition, when there is a reserved music set with a request flag set by the user who wants to actively take an image, a column in which the music name is displayed in the reservation table so as to stand out from the other music names May be executed such as changing the color of the field to gold or red, or making the field itself larger than the other reserved music fields.

  In the reservation list screen, when the imaging button is operated and imaging designation is performed (S505: Yes), the mobile terminal 6 stores the designated reservation information in the storage unit 63 as imaging designation information in the reservation table. (S506). Then, the reserved karaoke device 2 is notified of which reserved music is designated (S507). The notification to the karaoke apparatus 2 is performed by transmitting the reservation ID in the reservation table and the user ID of the user who has designated imaging to the karaoke apparatus 2. The karaoke apparatus 2 that has received the notification changes the recording flag to “1” for the reservation information corresponding to the reserved music for which imaging has been designated, and registers the user ID for which imaging has been designated as the imaging user ID. When the display end of the reservation list screen is input by the user, such as operating the return button on the top screen button displayed in the lower left of the reservation list screen (S508: Yes), the imaging designation process is terminated.

  The imaging designation process for designating the reserved music to be imaged on the reservation list screen has been described with respect to the case where the portable terminal 6 performs the imaging designation process. It is also possible to use means. When the karaoke system is used, the user who does not use the mobile terminal 6 is not allowed to specify the imaging for all reserved music such as not displaying the imaging button as described in the mobile terminal 6. It is preferable.

  FIG. 12 is a flowchart showing the music reproduction process executed by the karaoke apparatus 2. In the music playback process, the karaoke apparatus 2 refers to the reservation table and determines whether or not there is a music to be played next (S601). If there is a music to be played next in the reservation table (S602: Yes), the recording flag in the reservation information is confirmed, and if there is a recording flag ("1") (S603: Yes), the music is played back. (S604) and a recording process for recording the singing voice signal input to the singing microphones 44a and 44b is started (605). When the reproduction of the music is finished (S606: Yes), the karaoke recording information including the recording information obtained by the recording of the recording process is transmitted to the server device 5 (S607). FIG. 14A shows a data structure of karaoke recording information formed by the karaoke apparatus 2. The karaoke recording information includes the reservation ID of the music to be recorded, the recording information recording the singing voice signal, the recording start date and time, the recording time indicating the timing of the recording process, the imaging user ID indicating the user who performs imaging, etc. , Including.

  On the other hand, if there is no recording flag in the reservation information for the music to be reproduced next (S603: No), the recording process is not performed and the reproduction of the reserved music is started (S608), and the reproduction of the music is finished. In the case (S609: Yes), the next reserved music is processed.

  Thus, in the karaoke apparatus 2, the recording process which forms the audio | voice of video work information is performed. On the other hand, in the portable terminal 6 that has designated imaging, imaging processing for forming an image of the video work information is executed. FIG. 13 is a flowchart showing the imaging process executed by the mobile terminal 6. In the imaging process of the present embodiment, the reserved music to be imaged is specified using the imaging designation information stored in the portable terminal 6 in the imaging designation process (S506). Further, in the present embodiment, the start of imaging is notified to the user possessing the portable terminal 6 before the reserved music for which imaging is designated is started to be reproduced. Reference is made to the imaging designation information (S701), and if one minute before the start of reproduction of the reserved music designated for imaging (S702: Yes), the mobile terminal 6 notifies the user of a notice of the start of imaging (S703). The notice of notice of start of imaging can be determined by transmitting the notice of start notice from the karaoke apparatus 2 to the mobile terminal 6 in addition to the form using the designated start time recorded in the reservation table.

  The start notice may be generated by sounding an alarm sound from the speaker 67a of the mobile terminal 6. However, since it is considered that the alarm sound cannot be heard or the singing is hindered during singing, the mobile terminal It is preferable to adopt other forms such as operating the vibrator function 6 or performing display on the touch panel display unit 68 or the like.

  In the present embodiment, in accordance with the notification, the imaging function of the mobile terminal 6 is activated and an imaging screen is displayed (S704). By activating the imaging function of the portable terminal 6 before the reproduction of the reserved music is started, the user who captures images can cope with the imaging of the user who sings quickly. FIG. 15 shows an example of an imaging screen when the imaging function is activated on the mobile terminal 6. On the imaging screen, a live view area 607 for displaying the video imaged by the rear camera 65R, and an imaging music related information display for displaying information related to the reserved music to be imaged (user name, music name in this example) A field 608, an image capture button 609 for instructing image capture, is displayed.

  In the present embodiment, the imaging information captured by the mobile terminal 6 is a still image. The imaging information may be a moving image, or a still image and a moving image may be selected and captured. When an imaging instruction is input to the mobile terminal 6 by operating the imaging button 609 (S705: Yes), the mobile terminal 6 stores the imaging information captured by the rear camera 65R (S706), and the mobile imaging information Is stored in the storage unit 63 (S707). FIG. 14B shows the data structure of the portable imaging information. The mobile imaging information includes a reservation ID, imaging information, imaging start date and time, and imaging time indicating imaging timing, and an imaging user ID that is a user ID of the user who performed imaging ing.

  In the present embodiment, in order to form video work information, a reservation ID is used as meta information for associating imaging information and recording information. The imaging information is obtained by the mobile terminal 6, while the recording information is obtained by the karaoke device 2. Therefore, in order to form the video work information, it is necessary to specify the correspondence between both by the meta information. As meta information, in addition to the form of using a reservation ID in this way, karaoke device ID and recording time as meta information are attached to recording information, and karaoke device ID and imaging time as meta information are attached to imaging information. Thus, the correspondence relationship may be specified. In addition, for the same reserved music, various forms can be used as long as the meta information can identify the correspondence between the imaging information and the recording information.

  The user can perform imaging by operating the imaging button 609 until the music reproduction ends (S708: Yes). Note that the end of music reproduction can be determined by including an end designation time in the reservation table or by transmitting an end notification from the karaoke apparatus 2 to the portable terminal 2.

  It is possible to perform a video work formation process for forming video work information for the reserved music that has been imaged, after the music playback process and the imaging process have been completed. The video work information is formed by associating the recording information recorded by the karaoke apparatus 2 with respect to a certain reserved music and the imaging information captured by the portable terminal 6 with respect to the reserved music, and the singing voice at the time of singing The state of singing can be viewed. In the present embodiment, the video work forming process is executed in the mobile terminal 6 that has performed imaging. In addition to performing the video work forming process in the mobile terminal 6 as described above, the server device 5 constituting the karaoke management system, the karaoke system (including the karaoke device 2), or imaging was performed. It is possible to use a form performed by the portable terminal 6 other than the above, or a form performed by the portable terminal 6, the server device 5, and the karaoke system in cooperation.

  FIG. 16 is a flowchart showing a video work forming process executed on the mobile terminal. FIG. 17 is a schematic diagram for explaining the video work forming process. In the present embodiment, for the reserved music that is to be imaged, a video work forming process for forming video work information is performed when there is an instruction to form a video work from the mobile terminal 6 after the music reproduction process and the imaging process are completed. Executed.

  In the portable terminal 6, it may be considered that imaging processing is performed for a plurality of reserved songs. Therefore, in the video work formation process, it is preferable that it is possible to specify for which reserved music the video work formation process is to be executed. Although it is conceivable to execute the video work forming process subsequent to the imaging process, in this case, it is not necessary to designate a reserved music piece. When the video work formation process starts, the server apparatus 5 is requested for karaoke recording information of the reserved music to be processed (S801). The request for the karaoke recording information includes a reservation ID of the target mobile imaging information and an imaging user ID. As shown in FIG. 17, the karaoke apparatus 2 has already transmitted karaoke recording information to the server apparatus 5 in the music reproduction process (S607, first circle in FIG. 17). The server device 5 searches for the karaoke recording information corresponding to the reservation ID and the imaging user ID received from the portable terminal 6 and transmits the karaoke recording information to the requested portable terminal 6 (S802, second circle in FIG. 17). Then, the target mobile imaging information is read from the storage unit 63 (S803), and the video work information is formed by associating the imaging information in the mobile imaging information with the recording information in the karaoke recording information (S804, FIG. 17). The third circle).

  FIG. 19 shows various forms of video work information. FIG. 19A shows a case where a still image is used for image information captured by the mobile terminal 6 and corresponds to this embodiment. Video work information is formed by associating imaging information with recording information. Since the recording information is a singing voice signal recorded by the singing microphones 44a and 44b, the performance sound based on the music information is not included. Therefore, the video work information may be formed including the performance sound based on the music information. For example, the performance sound based on the music information may be transmitted from the server device 5 to the mobile terminal 6 that forms the video work information, or the server device 5 may record the performance sound over the recording information. Alternatively, in the recording process, a singing voice signal and a performance sound based on a reproduction sound of music information may be recorded together.

  The imaging information P1 to P6 is displayed and reproduced in synchronization with the progress of the reproduction time of the recording information. When still images are used, playback is performed in a slide show format. In the present embodiment, the title information Q is displayed at the start of reproduction of the video work information. In the title information Q, various information related to singing such as a song name, a user name, and a singing time (date and time) are displayed. In the present embodiment, the recording information and the imaging information are based on the recording time (information indicating the start timing of the recording process) included in the karaoke recording information and the imaging information (information indicating the imaging timing) included in the portable imaging information. Are associated. In the case of FIG. 19A, the recording time corresponds to T1, and T2 to T6 can be determined as the recording information progresses. On the other hand, the imaging time of each imaging information is t1 to t6. By arranging the imaging times t1 to t6 in the vicinity of the same times T1 to T6 in the progress of the recording information, it is possible to form video work information close to the actual state of singing. In the present embodiment, the imaging times t2 to t6 of the imaging information P2 to P6 are made to coincide with the recording times T2 to T6. The first imaging information P1 is made to coincide with the start time T1 of the recording information so that the video is not displayed at the start of the video work information.

  In FIG. 19A, a mode in which a still image is used as imaging information has been described, but a moving image can also be used as imaging information. FIG. 18B shows an example of correspondence between recording information and imaging information for video work information when a moving image is used. When a moving image is used for the imaging information, it is preferable to display the recording information and the imaging information in synchronization in order to synchronize the movement of the mouth of the singing user. The example of FIG. 18B is a case where the recording time T0 of the recording information and the imaging time t0 of the imaging information are substantially the same, and the video and the audio are synchronized by making T0 and t0 coincide. It becomes possible to reproduce the state of singing. In addition, when using a moving image for imaging information, it is preferable that a video and an audio | voice are synchronized from the reproduction | regeneration of music to completion | finish. Therefore, in the imaging process for capturing a moving image, the mobile terminal 6 may start and end the capturing of the moving image in synchronization with the start and end of the music reproduction. In the case of FIG. 19B, one piece of music information is used for one piece of music, but a plurality of pieces of image information can also be used. In that case, it is conceivable that a period during which no image is taken occurs between the pieces of imaging information. In that case, it is conceivable to compensate for a period during which no image is captured, for example, by displaying a still image extracted from the imaging information. In the imaging process, when a moving image and a still image can be selected and captured, the moving image and the still image may be displayed together in the video work information.

  As described above, when the video work information formed by associating the imaging information with the recording information is completed, the user performs preview processing (S806) for reproducing the completed video work information on the portable terminal 6 and confirming it. Is possible. Further, in the present embodiment, it is possible to perform a posting process for publishing completed video work information to other users. When posting is selected (S807: Yes), the mobile terminal 6 transmits the completed video work information to the server device 5 (S808). The server device 5 can widely release the received video work information via the Internet. In addition, in this posting process, the user to be disclosed may be disclosed by designating a user in addition to a form generally disclosed to the public. Alternatively, a user who participates in the singing (in the case of FIG. 11, users A to C) may be able to view and enjoy the inner ring.

  In the above-described embodiment, the form in which the video work information is formed on the mobile terminal 6 has been described. However, the formation of the video work information is not limited to such a form, and various configurations in the karaoke management system are possible. Can be done using. FIG. 18 shows a form in which the video work information is formed in the server device 5. In this case, the karaoke recording information obtained by the recording of the karaoke device 2 is transmitted to the server device 5 (first circle in FIG. 18). Also, portable imaging information obtained by imaging with the portable terminal 6 is also transmitted to the server device 5 (second circle in FIG. 18). In the server device 5, the received karaoke recording information and the reservation ID given to the portable imaging information are used to associate the two and form video work information (third circle in FIG. 18). The video work information formed by the server device 5 can be viewed through the Internet as it is, downloaded to the mobile terminal 6 or the karaoke system, or the like.

  As described above, formation of video work information can be performed by the mobile terminal 6 (example in FIG. 17) or the server device 5 (example in FIG. 18) that has performed the imaging process. In addition, it is good also as performing with the portable terminal 6 which is not performing imaging information, or a karaoke system (the karaoke apparatus 2 or the remote control apparatus 1). Furthermore, in the configuration of the karaoke management system, it is conceivable to perform processing in a distributed manner.

  In the present embodiment, the video work information is formed by using the imaging information captured by one mobile terminal 6, but a plurality of mobile terminals 6 are used as shown in FIG. If the image information is captured, the image information captured by each portable terminal 6 may be used to form one piece of video work information. In such a case, the video work information is formed by associating a plurality of pieces of imaging information captured by each mobile terminal 6 with the recording information recorded by the karaoke apparatus 2.

DESCRIPTION OF SYMBOLS 1a, 1b ... Remote control device 5 ... Server apparatus 11 ... Touch panel monitor 51 ... Memory | storage device 11a ... Display part 6 ... Portable terminal 11b ... Touch panel 61 ... Control part 12 ... Video RAM 62 ... RAM
DESCRIPTION OF SYMBOLS 13 ... Video control part 63 ... Memory | storage part 14 ... Memory 64 ... Image control part 15 ... CPU 65F ... Front camera 16 ... Wireless LAN communication part 65R ... Rear camera 17 ... Operation part 66 ... Sound processing part 18 ... Operation processing part 67a ... Speaker 19 ... Infrared communication unit 67b ... Microphone 2 ... Karaoke device (commander) 68 ... Touch panel display unit 21 ... Operation unit 68a ... Touch panel 22 ... Operation processing unit 68b ... Display unit 23 ... Infrared communication unit 69a ... Mobile network communication unit 24 ... LAN communication unit 69b ... Wireless LAN communication unit 25 ... Acoustic control unit 70 ... Input switch 27 ... Memory 71 ... External input terminal 28 ... Video RAM 41 ... Monitor 29 ... Video playback unit 72 ... External memory connection terminal 30 ... Control unit 81 ... Sound control unit 31 ... Video control unit 82 ... Speaker 32 ... HDD (hard disk) 100 ... L AN
42 ... Speaker 110 ... Access points 44a, 44b ... Singing microphone (microphone) 120 ... Router

Claims (10)

In a karaoke management system comprising a karaoke device, a program for causing a mobile terminal to execute processing, and a server device,
The karaoke device can execute reservation reception processing, transmission processing, reproduction processing, and recording processing,
The reservation reception process registers reservation information including the music identification information of the music designated by the user and the user identification information of the user who designated the music in the reservation table,
The transmission process sends the reservation table to the mobile device,
The reproduction process reads the reservation information registered in the reservation table, reproduces the music information corresponding to the music identification information in the reservation information,
The recording process records the singing voice signal input from the microphone, forms the recording information in association with the reservation information,
The program causes the mobile terminal to execute reception processing, designation processing, and imaging processing,
The reception process receives the reservation table from the karaoke device,
The designation process accepts designation of reservation information registered in the reservation table,
In the imaging process, when reservation information is specified, the imaging information captured by the imaging unit is stored in association with information included in the specified reservation information,
In any of the karaoke device, the program, and the server device, the video work forming process can be executed,
The video work formation process is a karaoke management system that forms video work information by associating recording information corresponding to reservation information with imaging information corresponding to the reservation information. The program causes the portable terminal to execute notification processing for notifying the karaoke device of the reservation information specified by the specification processing,
The karaoke management system according to claim 1, wherein the recording process is executed along with the start of the reproduction process corresponding to the reservation information notified by the notification process. The karaoke management system according to claim 1 or 2, wherein the program executes a notification process for notifying a user before a reproduction process corresponding to the reservation information specified by the specification process is started. The karaoke according to any one of claims 1 to 3, wherein the program activates an imaging function for executing the imaging process before the reproduction process corresponding to the reservation information designated by the designation process is started. Management system. The karaoke management system according to any one of claims 1 to 4, wherein the imaging process is executed in synchronization with a reproduction process corresponding to the reservation information designated by the designation process. The designation process controls designation of reservation information when the user identification information included in the reservation information in the reservation table matches the user identification information corresponding to the mobile terminal on which the designation process is executed. Item 6. The karaoke management system according to any one of items 5. The reservation acceptance process includes, in the reservation table, prohibition information that is prohibited from being specified by the specification process, and registers it in the reservation table.
The karaoke management system according to any one of claims 1 to 6, wherein the designation process prohibits designation of reservation information including prohibition information. The reservation acceptance process includes request information that is requested to be specified by the specification process, registered in the reservation table in the reservation information,
The karaoke management system according to any one of claims 1 to 7, wherein the designation process controls designation of reservation information including request information. A program that is executed in a portable terminal that can communicate with a karaoke device,
A first receiving step of receiving a reservation table in which reservation information including the music identification information of the music specified by the user and the user identification information of the user specifying the music is registered;
A designation step for accepting designation of reservation information registered in the reservation table;
When reservation information is designated, an imaging step for storing imaging information captured by the imaging unit in association with information included in the designated reservation information;
A recording information formed by recording a singing voice signal input from a microphone, wherein the recording information stored in association with the reservation information is received from the karaoke device;
A program for causing a mobile terminal to execute a forming step of forming video work information by associating recording information corresponding to reservation information with imaging information corresponding to the reservation information. A karaoke system that can communicate with a mobile terminal,
Reservation accepting means for registering reservation information including the music identification information of the music designated by the user and the user identification information of the user who designated the music in the reservation table;
A transmission means for transmitting the reservation table to the mobile terminal;
Reproducing means for reading the reservation information registered in the reservation table and reproducing the music information corresponding to the music identification information in the reservation information;
Recording means for recording a singing voice signal input from a microphone, and forming recording information in association with reservation information;
Receiving means for receiving imaging information associated with information included in the reservation information registered in the reservation table, which is information captured by the mobile terminal;
A karaoke system comprising: forming means for forming video work information by associating recording information corresponding to reservation information and imaging information corresponding to the reservation information.