JP2012233997A - Karaoke device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow each terminal searching for a song according to a user's operation to display a moving image following the playback of the song.SOLUTION: In a karaoke device, a control part 11 searches a contents table DB121 for a song number according to an operation of a user on an operation part 13, identifies the person image number related to the song number by referring to a correspondence table 122, and sends an instruction signal to a main device 2 using a short-distance communication part 15. A control part 21 locates, from a song DB221, song data related to the song number indicated by the instruction signal that the main device 2 receives, reads out performance data to play the song, locates, from a person image DB222, person image data related to the person image number, generates spacial data using the person image data and action data included in the song data, and sends the spacial data to a terminal 1. The control part 11 locates eye point data depending on the posture of a frame detected by a sensor 133, and displays, on a display screen 141, an image according to the spacial and eye point data.

Description

  The present invention relates to a technique for displaying an image in accordance with the reproduction of a song by a karaoke apparatus.

  As the performance progresses, a technology has been developed that allows the karaoke apparatus to produce an effect to excite the performance. For example, Patent Document 1 describes a karaoke device that displays a polygon image accompanied by a motion as well as a lyric display that matches the progress of a performance.

Japanese Patent No. 3668547

  By the way, a terminal used when a user selects a karaoke song is known. This terminal presents information of a table of contents for selecting a song to the user, accepts an operation for selecting a song from the user, and transmits a signal corresponding to the operation to the karaoke apparatus by wireless communication or the like. However, even if this terminal is used for music selection, it has not been used for the effects as described above.

  The present invention has been made in view of the above-described circumstances, and an object of the present invention is to display an image that moves with the reproduction of a song for each terminal that identifies the song in accordance with a user operation.

  In order to solve the above-described problems, a karaoke apparatus according to the present invention includes a terminal and a main device, and the terminal operates in a virtual space along with song identification information for identifying a song and reproduction of the song. Storage means for associating and storing image identification information for identifying an image to be recorded, first specifying means for specifying song identification information of a song designated by the user, and song identification information specified by the first specifying means Second identification means for identifying image identification information corresponding to the storage means based on the stored contents of the storage means, third identification means for identifying a viewpoint in the virtual space in accordance with a user operation, and the first identification means The main body apparatus transmits the specified music identification information to the main body apparatus, and requests the reproduction of the music indicated by the music identification information, and the image identification information specified by the second specifying means. Send to us Second request means for requesting transmission of spatial data indicating the position and orientation of the image in the virtual space when the image indicated by the image identification information is operated in the virtual space as the music is reproduced; When viewing the image in the virtual space from the viewpoint based on the spatial data transmitted from the main body device in response to the request from the second request means and the viewpoint specified by the third specifying means Display means for displaying the image, and the main body device reproduces the music indicated by the music identification information in response to the request from the first request means, and responds to the request from the second request means. In response, the apparatus includes a generation unit that generates the spatial data and a transmission unit that transmits the spatial data generated by the generation unit to the terminal.

  The karaoke apparatus according to the present invention includes a terminal and a main device, and the terminal identifies music identification information for identifying a song and an image for identifying an image that operates in a virtual space along with the reproduction of the song. Storage means for associating and storing identification information, first specifying means for specifying song identification information of a song designated by the user, and image identification information corresponding to the song identification information specified by the first specifying means Based on the stored contents of the storage means, second specifying means for specifying a viewpoint in the virtual space according to a user operation, and the song identification specified by the first specifying means First information means for transmitting information to the main body device and requesting reproduction of the music indicated by the music identification information; the music identification information; and the image identification information specified by the second specification means. Sent to the device, Transmission of operation data indicating an operation to be performed on the image indicated by the image identification information in the virtual space in association with reproduction of the music indicated by the identification information, and transmission of shape data indicating the shape of the image indicated by the image identification information Based on the second request means to request, the operation data and the shape data transmitted from the main body device in response to the request by the second request means, and the viewpoint specified by the third specification means, Display means for displaying the image when the image in the virtual space is viewed from the viewpoint, and the main body device stores shape data indicating the shape of the image identified by the image identification information. Shape data storage means for storing information in association with each other, reproduction means for reproducing the music indicated by the music identification information in response to a request from the first request means, and response to a request from the second request means And a sending means for sending said the operation data and generating means for generating the shape data, the operation data and the shape data the generating unit has generated to the terminal.

  Moreover, the karaoke apparatus which concerns on this invention comprises the said terminal in multiple in the aspect mentioned above, The said transmission means of the said main body apparatus receives the data requested | required by the said 2nd request means from each of these terminals. It transmits to the said each terminal, It is characterized by the above-mentioned.

  ADVANTAGE OF THE INVENTION According to this invention, the image which moves with the reproduction | regeneration of the music can be displayed for every terminal which specifies music according to a user's operation.

It is a figure which shows the structure of the karaoke system which concerns on embodiment of this invention. It is a figure which shows the structure of the karaoke apparatus which concerns on embodiment of this invention. It is a figure which shows an example of a table of contents database. It is a figure which shows an example of the music selection menu displayed on a display screen. It is a figure which shows an example of the list | wrist of the singer name displayed on a display screen. It is a figure which shows an example of the song name list according to singer displayed on a display screen. It is a figure which shows an example of a correspondence table and an instruction signal. It is a figure which shows an example of a music database. It is a figure which shows a data structure. It is a figure which shows an example of a person image database. It is a figure for demonstrating the outline of the person image displayed on a terminal. It is a figure which shows an example of the tree structure of person image data. It is a block diagram which shows an example of a functional structure of a karaoke apparatus. It is a figure which shows the flow of the calculation regarding the display of the person image in a karaoke apparatus. It is a figure which shows the structure which performs the calculation in a karaoke apparatus, and its timing. It is the figure which showed the example of an image each displayed by a karaoke apparatus. It is a figure which shows the structure which performs the calculation in the modification 2, and its timing. It is a figure which shows the structure which performs the calculation in the modification 4, and its timing. It is a figure which shows the structure which identifies the light source in the modification 5, and its timing.

<Embodiment>
(Composition of karaoke system)
FIG. 1 is a diagram showing a configuration of a karaoke system 9 according to an embodiment of the present invention. The karaoke system 9 includes a karaoke device 3, a network 4 such as the Internet, and a server 5. The karaoke device 3 includes a terminal 1 and a main device 2.
The server 5 supplies song data representing a song to the main device 2 of the karaoke device 3 via the network 4. The main device 2 stores the music data supplied from the server 5. The terminal 1 receives an operation for specifying a song from the user, and transmits a signal corresponding to the operation to the main body device 2. The main device 2 receives a signal from the terminal 1, specifies the music data of the music indicated by this signal, and performs the performance according to the music data.

(Composition of karaoke equipment)
FIG. 2 is a diagram showing a configuration of the karaoke apparatus 3 according to the embodiment of the present invention. The terminal 1 of the karaoke apparatus 3 includes a control unit 11, a storage unit 12, an operation unit 13, a display unit 14, and a short-range communication unit 15. The main device 2 of the karaoke device 3 includes a control unit 21, a storage unit 22, an operation unit 23, a display unit 24, a short-range communication unit 25, a communication unit 26, and a sound emission unit 27.

(Terminal configuration)
Based on FIG. 2, the structure of the terminal 1 is demonstrated.
The control unit 11 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory), and the CPU stores a boot loader stored in the ROM and a program stored in the storage unit 12. Are read into the RAM and executed to control each part of the terminal 1. The storage unit 12 is a large-capacity storage unit such as an EEPROM (Electrically Erasable Programmable Read Only Memory), and stores a program used by the control unit 11.

  The display unit 14 includes a display screen 141, and displays a person image and the like on the display screen 141 based on data sent from the interactive window that prompts the operation and the main body device 2 under the control of the control unit 11. The display screen 141 of the display unit 14 is configured by, for example, a display element using liquid crystal.

  The operation unit 13 includes an operation element 131 for inputting various instructions such as buttons, and a transparent pressure-sensitive member arranged on the display screen 141 of the display unit 14. A touch panel 132 that detects a position touched by the touch panel and outputs a signal indicating coordinates according to the position, and a sensor 133 that detects a motion state and a posture of the casing of the terminal 1 and outputs a signal corresponding to the motion state. Is provided. Specifically, the sensor 133 includes an acceleration sensor, an angular velocity sensor, and the like. The user uses the terminal 1 by pressing a button or the like of the operation element 131, touching the touch panel 132, or holding the casing of the terminal 1 to change its posture and causing the sensor 133 to detect this. Operation with respect to the karaoke apparatus 3 is performed. The operation unit 13 receives a user operation from the operation element 131, the touch panel 132, the sensor 133, and the like, and sends a signal corresponding to the operation content to the control unit 11.

  The near field communication unit 15 is an interface that communicates control information and various data with the main body device 2. For example, a near field communication circuit compliant with IEEE 802.11, IEEE 802.15, etc., IrDA (Infrared Data Association) Wireless interface etc.

  The storage unit 12 includes a table of contents database (hereinafter, the database is also referred to as “DB”) 121 and a correspondence table 122. The table of contents DB 121 stores table of contents data for allowing a user to specify a song. The correspondence table 122 stores a song number that is identification information unique to each song and a person image number that is identification information of a person image in association with each other. A person image is an image that moves along with the reproduction of a song, for example, an image imitating or deforming a singer or dancer. The person image number associated with each song number in the correspondence table 122 represents the person image number of the person image displayed on the terminal 1 when the user specifies the song indicated by the song number.

  Here, the contents stored in the storage unit 12 will be described with reference to FIGS. FIG. 3 is a diagram illustrating an example of the table of contents DB 121. The table of contents DB 121 has fields indicating a song number, a song name, a singer name, a genre name, and an announcement year, and associates data described in each field. The song number is identification information for identifying a song, and is a unique number assigned to each song. The song name is a character string indicating the name of the song, that is, the title. The singer name is a character string indicating the name of a singer who sings the song or a group performing the song. The genre name is a character string indicating the genre of music to which the song belongs. The release year is a number that indicates the year in which the song was publicly announced.

  In general, songs are identified by the title of the song, but the same songwriter writes the lyrics, and the song composed by the same composer may be sung by multiple singers. Sometimes a completely different song is created that completely matches. In some cases, the same singer re-releases his own songs that have been released in the past, such as recording again later. The table of contents DB 121 describes that a unique record is always specified when a song name and any one of the fields other than the song name are combined, but this unique record is associated with a unique song number. The song is identified by the song number. As shown in FIG. 3, for example, the song number “P01” is associated with the song name “○ ▽ ○”, the singer name “A ○ △”, the genre name “pop”, and the announcement year “2002”. .

  FIG. 4 is a diagram showing an example of the music selection menu displayed on the display screen 141. The song selection menu is a menu that is displayed first when the user specifies a song using the terminal 1. The music selection menu shown in FIG. 4 has four items of “by singer name”, “by song name”, “by genre name”, and “by announcement year”. The user selects one of these four items and searches for a song he desires.

  FIG. 5 is a diagram showing an example of a singer name list displayed on the display screen 141. This list of singer names is a list displayed on the display screen 141 when the user selects the “by singer name” item when the music selection menu shown in FIG. 4 is displayed. In this list, singer names are classified and arranged according to their initials. In the example shown in FIG. 5, a plurality of windows corresponding to each line of the Japanese syllabary are superimposed and displayed. A tab for selecting the window is provided at the top of each window. In this tab, characters “a”, “ka”, “sa”, “ta”, “na”, and “ha” representing each line of the Japanese syllabary are described. The tab described as “next page” functions as a button for displaying a window having tabs describing characters “ma”, “ya”, “ra”, and “wa”. In this example, since the tab “A” is selected, the names of the singers having the initials of the “A” line among the lines of the Japanese syllabary are arranged vertically in the window. If the list of singer names does not fit in the vertical direction in the window, the list can be moved up and down using a scroll bar at the right end of the window. In addition, when the user touches the position of a button described as “Romaji”, the classification using the initial letters of the singer's name is switched from alphabet to alphabet.

  FIG. 6 is a diagram illustrating an example of a song name list for each singer displayed on the display screen 141. This song name list is a list displayed on the display screen 141 when the user selects the item of the singer name “A ○ Δ” when the singer name list shown in FIG. 5 is displayed. In this list, the names of songs owned by “AOO” are arranged in the vertical direction in the window. If the list of music titles does not fit in the vertical direction in the window, it can be moved up and down using a scroll bar at the right end of the window.

  When the user selects the item “○ ▽ ○” when the song name list for each singer shown in FIG. 6 is displayed, the singer name “A ○ △” and the song name “○ ▽ ○” are displayed. Therefore, the music number “P01” associated with these sets is determined in the table of contents DB 121.

  FIG. 7 is a diagram illustrating an example of the correspondence table 122 and the instruction signal. In the correspondence table 122 shown in FIG. 7A, for example, the song number “P01” is associated with the person image number “M01”. Therefore, when the user selects the song indicated by the song number “P01”, the control unit 11 identifies the person image number “M01” associated with the song number “P01” in the correspondence table 122.

  As a result, the control unit 11 of the terminal 1 generates a signal in which the person image number “M01” is added to the music number “P01” as shown in FIG. 7B, and indicates the instruction based on the user's music selection. An instruction signal is transmitted to the main device 2 using the short-range communication unit 15. As a result, the person image data of the person image identified by the person image number “M01” is sent from the main unit 2 to the terminal 1, and this person image is displayed on the display screen 141 of the terminal 1.

(Configuration of main unit)
Returning to FIG. 2, the configuration of the main device 2 of the karaoke apparatus 3 will be described.
The control unit 21 includes a CPU, a ROM, and a RAM, and the CPU reads out the boot loader stored in the ROM and the program stored in the storage unit 22 to the RAM and executes the program, whereby each unit of the main body device 2 is executed. To control. The storage unit 22 is a large-capacity storage unit such as a hard disk drive, and stores a program used by the control unit 21. In addition, the storage unit 22 includes a song DB 221 that stores song data indicating a song, and a person image DB 222 that stores person image data indicating a person image.

  Here, the contents stored in the storage unit 22 will be described with reference to FIGS. FIG. 8 is a diagram illustrating an example of the music DB 221. The song data includes performance data, background data, lyrics data, and motion data. The performance data is data in which a musical sound that is generated when the music is played is associated with a timing at which the musical sound is generated. The background data is data in which a still image or a moving image displayed on the main device 2 as the performance progresses and the display timing are associated with each other. The lyric data is data in which lyric information to be displayed is recorded in time series in accordance with the progress of the performance, and describes a character string representing the lyrics, timing of color change, and the like. The motion data is data in which the motion of the person image displayed on the terminal 1 as the performance progresses is associated with the display timing. The song DB 221 has fields indicating song numbers, performance data, background data, lyrics data, and motion data, and associates data described in each field. Accordingly, the song DB 221 stores the song number and the song data in association with each other.

  FIG. 9 is a diagram showing data structures of performance data, background data, lyrics data, and motion data. As shown in FIG. 9A, each data is divided into a header part and a data body part. In the header part, parameters used for handling information stored in the data body part, the type of description format of the information, and the like are described. As shown in FIG. 9B, the information stored in the data body has a list structure in which timing information indicating “time” and event information indicating “event” are associated with each other.

  If the data is performance data, this event information represents an event related to musical performance such as note-on and note-off. Also, if this event information is background data, events related to background display such as still images and inter-frame prediction information used for predicting a frame image to be newly drawn from the image of the immediately preceding frame are included. It represents. In addition, if the event information is lyrics data, in the case of indicating by changing the color of each character in the character string indicating the lyrics and which part of the character string is being sung, It represents an event related to the color change position. In addition, if the event information is motion data, it represents an event related to the motion of the person image, such as information on the moving direction of the person image and the motion direction around the joints constituting the person image. The timing information represents the timing at which these events should be generated.

  As shown in FIG. 9B, the event information “event α” is associated with the timing information “123”. This represents that the event α is generated at the timing indicated by the timing information “123” (for example, the timing at which 123 milliseconds have elapsed from the start of the music performance, the timing at which the frame is redrawn at the 123rd time, or the like).

  FIG. 10 is a diagram illustrating an example of the person image DB 222. The person image DB 222 stores a person image number that is a number for identifying a person image and person image data that is data of the person image in association with each other. FIG. 11 is a diagram for explaining an outline of a person image displayed on the terminal 1. As shown in FIG. 11, the human image is arranged in an xyz coordinate space (virtual space) that is a virtual orthogonal coordinate space. The human image is configured by expressing the shape of the human body as a polygonal part CH for each part and combining these parts CH. The plurality of parts CH are stored in a tree structure in the person image data. FIG. 12 is a diagram illustrating an example of a tree structure of person image data. As shown in FIG. 12, in the human image data, the plurality of parts have a tree-like structure with the part indicating the waist as a root. That is, three parts of “chest”, “upper right thigh”, and “upper left thigh” are connected to the “hip” that is the root part, and “head”, “upper right arm”, “upper left” are further connected to “chest”. Three parts of "arms" are connected. In the person image data, polygon data representing the shape of each part and the connection position and angle between the parts in the standard state are described. The person image data may further include texture data indicating the texture, color, pattern, etc. of the surface of each part. In short, the person image DB 222 stores person image data (shape data) indicating the shape of the person image identified by the person image number (image identification information) in association with each person image number.

  In the motion data, parameters necessary for causing the human figure to perform a motion in accordance with the progress of the performance are described. These parameters are, for example, a numerical value indicating the arrangement of the human image in the xyz coordinate space, a numerical value indicating a change in the connection position or angle between the parts of the human image, and the like. In the example shown in FIG. 11, the numerical value indicating the arrangement of the human image is the coordinates of the center of gravity PO of “waist”. Further, the numerical value indicating the connection angle between the parts is, for example, an angle JA between the upper left arm and the left forearm of the person image as shown in FIG.

Returning to FIG. 2, the description of the main unit 2 will be continued.
The display unit 24 has a display screen 241, and under the control of the control unit 21, lyrics that match the progress of the performance of the song based on the lyrics data and the background data among the song data read from the song DB 221. And a background video are displayed on the display screen 241. The display screen 241 of the display unit 24 is configured by, for example, a display element using liquid crystal.

  The operation unit 23 includes an operation element 231 for inputting various instructions such as buttons, and a microphone 232 that transmits a sound signal obtained by collecting a singing voice uttered by the user to the sound emission unit 27. The operator 231 sends a signal indicating an instruction input by the user to the control unit 21.

  The short-range communication unit 25 is an interface that communicates control information and various data with the short-range communication unit 15 of the terminal 1. For example, a short-range wireless communication circuit compliant with IEEE 802.11, IEEE 802.15, or the like, IrDA Wireless interface etc.

  The communication unit 26 is an interface for receiving music data from the server 5 via the network 4, for example, various modems, wireless communication circuits compliant with IMT-2000, or USB (Universal Serial Bus) standards Serial interface.

  The sound emitting unit 27 includes a sound source circuit (TG), a mixer, an amplifier, and a speaker. The tone generator circuit synthesizes musical sounds based on performance data among the music data read from the music DB 221. The mixer mixes the musical sound synthesized by the sound source circuit and the sound signal transmitted from the microphone 232 at a predetermined ratio. The amplifier amplifies the signal mixed by the mixer. The speaker emits a sound corresponding to the signal amplified by the amplifier.

(Functional configuration of karaoke equipment)
FIG. 13 is a block diagram illustrating an example of a functional configuration of the karaoke apparatus 3. As illustrated in FIG. 13, the control unit 11 of the terminal 1 and the control unit 21 of the main body device 2 provided in the karaoke apparatus 3 each realize various functions. Specifically, the control unit 11 functions as a first specifying unit 111, a second specifying unit 112, a third specifying unit 113, a first request unit 114, a second request unit 115, and a display control unit 116. Further, the control unit 21 functions as a reproduction unit 211, a generation unit 212, and a transmission unit 213. Hereinafter, these functions will be described. In addition, although communication with the terminal 1 and the main body apparatus 2 is performed via the near field communication part 15 and the near field communication part 25 as shown in FIG. 2, these are abbreviate | omitted in FIG.

  The first specifying unit 111 (first specifying unit) specifies the identification information of the song designated by the user in response to the user's operation received by the operator 131 or the touch panel 132 of the operation unit 13. Specifically, the first specifying unit 111 displays the above-described menu screen as shown in FIG. 4 on the display screen 141 of the display unit 14, receives a user instruction via the operator 131 or the touch panel 132, and stores the menu screen. Any one song number (song identification information) stored in the table of contents DB 121 of the unit 12 is specified.

  The second specifying unit 112 (second specifying unit) specifies the person image number (image identification information) corresponding to the song number specified by the first specifying unit 111 based on the correspondence table 122. Specifically, the second specifying unit 112 searches the correspondence table 122 for the song number specified by the first specifying unit 111, and specifies the person image number associated with the song number.

  The third specifying unit 113 (third specifying unit) specifies the viewpoint in the above-described xyz coordinate space in accordance with the user operation detected by the sensor 133. This viewpoint specifies the position, direction, angle of view, and the like of viewing the xyz coordinate space, and an image when the xyz coordinate space is found from this viewpoint is displayed on the display unit 14 of the terminal 1. The viewpoint will be described later.

The first requesting unit 114 (first requesting unit) transmits the music number specified by the first specifying unit 111 to the main device 2 and requests the reproduction of the music indicated by the music number.
The reproduction unit 211 (reproduction unit) reproduces the song indicated by the above-described song number in response to the request from the first request unit 114. Specifically, when the playback unit 211 receives a playback request for a song for which a song number is specified from the terminal 1, the playback unit 211 searches the song DB 221 in the storage unit 22 for song data associated with the song number, and stores the song data in the song data. Read the performance data included. Then, the reproduction unit 211 generates a musical sound signal corresponding to the read performance data, and uses the sound emitting unit 27 to emit sound according to the musical sound signal.

The second requesting unit 115 (second requesting unit) transmits the person image number specified by the second specifying unit 112 to the main body device 2, and the person image indicated by the person image number is associated with the reproduction of the music described above. Requesting the transmission of spatial data indicating the position and orientation of the person image in the xyz coordinate space when operated in the xyz coordinate space.
The generation unit 212 (generation unit) generates spatial data in response to a request from the second request unit 115. Specifically, when the generation unit 212 receives a person image number from the terminal 1, the generation unit 212 searches the person image DB 222 of the storage unit 22 for person image data associated with the person image number. In addition, the generation unit 212 reads operation data from the above-described song data searched for by the reproduction unit 211. Then, the generation unit 212 generates spatial data using the operation data and the person image data. That is, the generation unit 212 generates spatial data by arranging each part of the human image based on the read human image data at each position in the xyz coordinate space based on the motion data.

  The transmission unit 213 (transmission unit) transmits the spatial data generated by the generation unit 212 to the terminal 1. When the display control unit 116 (display unit) views the person image in the xyz coordinate space described above from the viewpoint based on the spatial data transmitted from the main device 2 and the viewpoint specified by the third specifying unit 113. These images are displayed using the display screen 141 of the display unit 14.

(Karaoke device operation)
FIG. 14 is a diagram showing a flow of calculation regarding display of a person image in the karaoke apparatus 3. First, the karaoke apparatus 3 performs a first calculation using the person image data and the operation data. The first calculation is a calculation for placing the person image indicated by the person image data in the xyz coordinate space based on the operation data and causing the person image to take a pose (posture). By the first calculation, a pose to be taken by the person image in the xyz coordinate space is taken, and spatial data indicating the space arranged at the position to be arranged is generated. Accordingly, the first calculation is performed by the generation unit 212 in FIG. As indicated by a broken line in FIG. 14, the first calculation may generate new spatial data by further using the already generated spatial data. In this case, the operation data only needs to describe data indicating a difference in operation from the previous timing to the current timing, for example.

  Next, the karaoke apparatus 3 irradiates light on the person image in the xyz coordinate space, the spatial data obtained by the first calculation, the viewpoint data indicating the viewpoint coordinates and the angle of view of the person image in the xyz coordinate space, and the like. The second calculation is performed using light source data representing the position of the light source and the color of the light. The viewpoint data is, for example, viewpoint coordinates PV and angle of view VA in FIG. The viewpoint coordinate PV indicates the coordinates of the viewpoint in the xyz coordinate space. The angle of view VA indicates the range of the visual field that can be seen from the viewpoint coordinate PV in the xyz coordinate space. The position of the light source included in the light source data is, for example, the light source PL in FIG. The second operation includes a so-called three-dimensional rendering. In the example shown in FIG. 11, the second calculation calculates the ratio that the light emitted from the light source PL is reflected by the surface of each part CH constituting the human image data, and out of the reflected light reflected from the surface Then, the object that goes to the viewpoint coordinate PV is extracted. Then, by arranging the gradation value of the color corresponding to the extracted reflected light in the image area determined by the angle of view VA, an image that can be seen when viewing the xyz coordinate space in the range of the view angle VA to the angle of view VA is obtained. The shown image data is generated. Accordingly, the second calculation is performed by the display control unit 116 in FIG.

  FIG. 15 is a diagram showing a configuration for performing calculation in the karaoke apparatus 3 and its timing. The control unit 11 of the terminal 1 specifies the song number from the table of contents DB 121 of the storage unit 12 based on the operation received by the operation unit 13 (step S101), and is associated with the song number with reference to the correspondence table 122. A person image number is specified (step S102). And the control part 11 transmits the instruction | indication signal as shown in FIG.7 (b) to the near field communication part 25 of the main body apparatus 2 using the near field communication part 15 (step S103).

  When the control unit 21 of the main device 2 receives the instruction signal, it finds the music data associated with the music number indicated by the instruction signal from the music DB 221 and reads the performance data included therein to perform the music (step) S104). Further, the control unit 21 finds out the person image data associated with the person image number indicated by the instruction signal from the person image DB 222, and uses the person image data and the operation data included in the music data described above to perform the first calculation. Is performed (step S105). The control unit 21 causes the display screen 241 of the display unit 24 to display background video and lyrics to be displayed as the performance progresses based on the background data and lyrics data included in the song data.

  The sensor 133 of the terminal 1 detects the attitude of the casing of the terminal 1, and sends a signal corresponding to this attitude to the control unit 11 (step S106). The control unit 11 processes the viewpoint data in the xyz coordinate space according to the posture of the casing detected by the sensor 133, and specifies the current viewpoint (step S107). For example, when the housing of the terminal 1 is tilted by the user, the control unit 11 analyzes the signal sent from the sensor 133 to obtain the tilt angle, and the angle of view included in the viewpoint data according to this angle. The data contents are rewritten so that is inclined. In addition, the control unit 11 may process the viewpoint data in accordance with an operation received by the operator 131 of the operation unit 13 or the touch panel 132. In this example, it is assumed that the light source data is fixed in the xyz coordinate space.

  The control unit 21 transmits the spatial data generated by performing the first calculation to the terminal 1 using the short-range communication unit 25 (step S108). The control unit 11 of the terminal 1 performs the second calculation using the spatial data transmitted from the main device 2, the above-described light source data, and viewpoint data (Step S109). And the control part 11 displays the image according to the image data produced | generated by 2nd calculation on the display screen 141 of the display part 14 (step S110).

  As described above, since the karaoke apparatus 3 identifies a person image according to the song specified by the terminal 1, the display unit 14 of the terminal 1 according to the set of the terminal 1 that identifies the song and the identified song. The person image to be displayed is determined. Therefore, the karaoke apparatus 3 provides a service for displaying a unique person image for each terminal 1 used for music selection.

  Further, the karaoke apparatus 3 displays individual images on the display unit 14 of the terminal 1 and the display unit 24 of the main body device 2. FIG. 16 is a diagram showing an example of images displayed on the display unit 14 and the display unit 24 by the karaoke apparatus 3. The image shown in FIG. 16 (a) is an example of an image displayed on the display unit 14 of the terminal 1, and a person image specified together with music selection dances according to operation data as the performance progresses. It is expressed. On the other hand, the image shown in FIG. 16B is an example of an image displayed on the display unit 24 of the main device 2, and is expressed by superimposing the background corresponding to the background data and the lyrics corresponding to the lyrics data. Has been. Therefore, the karaoke device 3 provides the terminal 1 with an image representing the behavior of the person image as an image unique to the terminal 1 used for music selection, in addition to the image provided to the main body device 2.

Moreover, since the karaoke apparatus 3 displays an image as shown in FIG. 16A on the terminal 1 used by the user for music selection, the user looks at the line of sight toward the display unit 14 of the terminal 1 at the time of music selection. Without moving, you can see how the figure moves as the song plays. In other words, even when shifting from music selection to music playback, the user does not need to move his line of sight.
In the karaoke apparatus 3, since the person image that operates in accordance with the reproduction of the song is displayed on the display unit 14 of the terminal 1 even while the song is being reproduced after the music selection, the device (terminal 1) Can be used effectively.

<Modification>
As mentioned above, although embodiment of this invention was described, this invention can be implemented in various aspects as follows.
(Modification 1)
In the embodiment described above, the correspondence table 122 included in the storage unit 12 may be a read-only table, or may be a table that can be updated. For example, the user may be configured to be able to edit the correspondence table 122 using the operation unit 13 of the terminal 1. Further, the main device 2 stores a correspondence table in the storage unit 22, and the control unit 11 of the terminal 1 acquires the correspondence table from the main device 2 using the short-range communication unit 15 and updates the correspondence table 122. May be. In this case, the main device 2 may update the correspondence table stored in the storage unit 22 in accordance with information transmitted from an external device such as the server 5.

(Modification 2)
In the above-described embodiment, the first calculation is performed by the generation unit 212, that is, the control unit 21 of the main device 2, but may be performed by the control unit 11 of the terminal 1. FIG. 17 is a diagram illustrating a configuration for performing calculations in the karaoke apparatus 3 according to the second modification and the timing thereof. The steps common to the operation flow shown in FIG. 17 and the operation flow shown in FIG.

  When receiving the instruction signal, the control unit 21 of the main device 2 finds out the person image data associated with the person image number indicated by the instruction signal from the person image DB 222 and transmits the person image data to the terminal 1 (step S201). ). In addition, the control unit 21 finds music data associated with the music number indicated by the instruction signal from the music DB 221, reads performance data included in the music data 221, plays the music (step S 104), and the music data described above. Is transmitted to the terminal 1 (step S202). The terminal 1 performs a first calculation using the person image data obtained from the main device 2 in step S201 and the operation data obtained from the main device 2 in step S202 (step S203). Thereby, compared with the case where the 1st calculation is performed by the control part 21 of the main body apparatus 2, the load of the control part 21 is reduced.

In this case, the second request unit 115 illustrated in FIG. 13 transmits the song number specified by the first specifying unit 111 and the person image number specified by the second specifying unit 112 to the main body device 2. . Then, the second request unit 115 transmits operation data indicating an operation to be performed on the person image indicated by the person image number in the xyz coordinate space in association with the reproduction of the song indicated by the song number, and the person image indicated by the person image number. Requesting transmission of human image data indicating a shape.
In this case, the generation unit 212 illustrated in FIG. 13 generates motion data and human image data instead of the spatial data in response to a request from the second request unit 115. Specifically, the generation unit 212 generates the operation data by extracting the operation data from the searched music data, and at the same time, the person image associated with the person image number included in the request by the second request unit 115 The person image data is generated by finding the data from the person image DB 222.

In this case, the transmission unit 213 illustrated in FIG. 13 transmits the operation data and the person image data generated by the generation unit 212 to the terminal 1.
In this case, the display control unit 116 illustrated in FIG. 13 is specified by the third specifying unit 113 and the operation data and the person image data transmitted from the main body device 2 in response to a request from the second request unit 115. Based on the viewpoint, an image when the above-described person image in the xyz coordinate space is viewed from the viewpoint is displayed. Therefore, in this case, the display control unit 116 performs the first calculation shown in FIG. 14 and further performs the second calculation.

(Modification 3)
In the above-described embodiment, the person image DB 222 is stored in the storage unit 22 of the main device 2, but content corresponding to this may be stored in the storage unit 12 of the terminal 1. For example, as indicated by a broken line in FIG. 2, the storage unit 12 may store a person image DB 123 having the same storage contents as the person image DB 222 described above. In this case, the terminal 1 may read out the person image data specified by the person image number from the person image DB 123 and transmit it to the main body device 2 in addition to the instruction signal described above. The control unit 21 of the main device 2 that has received the person image data may perform the first calculation based on the person image data and the operation data described above. In the third modification, when the control unit 11 of the terminal 1 performs the first calculation as shown in the second modification, it is necessary to exchange the human image data between the main body device 2 and the terminal 1. Absent. In this case, the control unit 11 of the terminal 1 performs the first calculation using the person image data read from the person image DB 123 and the operation data received from the main device 2.

In this case, the second request unit 115 illustrated in FIG. 13 may not transmit the person image number specified by the second specifying unit 112 to the main body device 2. The second requesting unit 115 transmits the song number specified by the first specifying unit 111 to the main body device 2, and the person indicated by the above-described person image number in the xyz coordinate space with the reproduction of the song indicated by the song number. What is necessary is just to request | require transmission of the operation | movement data which shows the operation | movement performed on an image.
In this case, the generation unit 212 illustrated in FIG. 13 generates operation data instead of the spatial data in response to a request from the second request unit 115. Therefore, in this case, the transmission unit 213 illustrated in FIG. 13 transmits the operation data generated by the generation unit 212 to the terminal 1.
In this case, the display control unit 116 illustrated in FIG. 13 receives the operation data transmitted from the main device 2 in response to the request from the second request unit 115, the human image data read from the human image DB 123, and the third Based on the viewpoint specified by the specifying unit 113, an image when the above-described person image in the xyz coordinate space is viewed from the viewpoint is displayed.

(Modification 4)
The karaoke apparatus 3 may include a plurality of terminals 1. In this case, the terminals 1 other than the terminal 1 that has specified the song may display a different person image from the terminal 1 that has specified the song. FIG. 18 is a diagram illustrating a configuration for performing calculation in the karaoke apparatus 3 of the modification 4 and the timing thereof. The steps common to the flow of operation shown in FIG. 18 and the flow of operation shown in FIG. 15 are denoted by common reference numerals. Description of common steps may be omitted.

  The karaoke device 3 includes terminals 1 a and 1 b and a main device 2. In the example shown in FIG. 18, the terminal 1a accepts the user's operation and specifies the song, while the terminal 1b does not specify the song but specifies the person image number (step S301) and includes this person image number. An instruction signal is transmitted to the main device 2 (step S302). The control unit 21 of the main device 2 performs a song using the instruction signal (hereinafter referred to as instruction signal a) acquired from the terminal 1a in step S103 (step S104). The control unit 21 performs the first calculation with reference to the instruction signal a and the instruction signal acquired from the terminal 1b in step S302 (hereinafter referred to as instruction signal b) (step S105). In the first calculation, since the person image number indicated by the instruction signal a and the person image number indicated by the instruction signal b may be different, the control unit 21 generates individual spatial data, and the terminal 1a , 1b (steps S108 and S305).

  The sensor 133 of the terminal 1b detects the attitude of the casing of the terminal 1b, and sends a signal corresponding to this attitude to the control unit 11 (step S303). The control unit 11 of the terminal 1b processes the viewpoint data in the xyz coordinate space according to the posture of the casing detected by the sensor 133 of the terminal 1b, and specifies the current viewpoint (step S304).

  In addition, the control unit 11 of the terminal 1b that has acquired the spatial data from the main device 2 uses the spatial data, the viewpoint data processed in step S304, and the light source data fixed in the xyz coordinate space. Calculation is performed (step S306). And the control part 11 of the terminal 1b displays the image according to the image data produced | generated by 2nd calculation on the display screen 141 of the display part 14 of the terminal 1b (step S307).

  As described above, the karaoke apparatus 3 expresses a state in which individual human images operate on the terminal 1a used for music selection and the terminal 1b not used for music selection separately from the image provided to the main body apparatus 2. An image is provided to each terminal 1. Therefore, the plurality of terminals 1 individually display on the respective display units 14 the manner in which the person image associated with each terminal 1 dances.

  It should be noted that the person image number specifying process performed by the control unit 11 of the terminal 1b in step S301 may be performed after the main device 2 receives the instruction signal a including the song number and the person image number from the terminal 1a. Good. In this case, the music number may be read from the instruction signal a received by the main device 2 and sent to the terminal 1b. The control unit 11 of the terminal 1b refers to the song number sent from the main body device 2 from the correspondence table 122 stored in its own storage unit 12, and identifies the person image number associated therewith. Thus, step S301 may be realized.

(Modification 5)
In the above-described embodiment, the light source data is fixed in the xyz coordinate space. However, the light source data may be configured to be specified according to an operation by the terminal 1. FIG. 19 is a diagram illustrating a configuration for specifying a light source and its timing in the karaoke apparatus 3 of the fifth modification. The steps common to the operation flow shown in FIG. 19 and the operation flow shown in FIG. 15 are denoted by the same reference numerals. Description of common steps may be omitted.

  The karaoke device 3 includes terminals 1 a and 1 c and a main device 2. In the example shown in FIG. 19, the terminal 1a accepts the user's operation and specifies the music, while the terminal 1c does not specify the music, but the sensor 133 of the terminal 1c detects the attitude of the casing of the terminal 1c. Then, a signal corresponding to this posture is sent to the control unit 11 (step S401). And the control part 11 of the terminal 1c processes the light source data in xyz coordinate space according to the attitude | position of the housing | casing which the sensor 133 of the terminal 1c detected, and specifies the coordinate of the present light source (step S402).

  The control unit 11 of the terminal 1c that has identified the coordinates of the light source generates light source data that is data including the coordinates, and transmits the light source data to the main device 2 using the short-range communication unit 15 (step S403). The main body device 2 transfers this light source data to the terminal 1a as indicated by a broken line.

  The control unit 11 of the terminal 1a that acquired the spatial data from the main device 2 uses the spatial data, the viewpoint data processed in step S107, and the light source data transferred from the main device 2 in step S403. Calculation is performed (step S109). And the control part 11 of the terminal 1a displays the image according to the image data produced | generated by 2nd calculation on the display screen 141 of the display part 14 of the terminal 1a (step S110). The light source data was generated by the terminal 1c that was not used for music selection, but may be generated by the terminal 1a used for music selection.

  As described above, the karaoke apparatus 3 determines the position of the light source using the terminal 1. Thereby, in addition to being able to determine the angle for photographing the person image by operating the terminal 1, the user can determine the light applied to the person image. In other words, the user can enjoy the enjoyment of adjusting the light that illuminates the person image as if the lighting staff, in addition to the enjoyment of taking a picture of a human figure dancing like a photographer. .

(Modification 6)
The configuration for performing the first calculation and the second calculation described above may be determined according to each load. For example, when the first calculation is performed by the control unit 21 of the main body device 2, the control unit 21 causes the terminal 1 to perform the first calculation when a numerical value representing the CPU load exceeds a predetermined threshold. May be sent to the terminal 1. In addition, the terminal 1 and the main device 2 monitor the load of the CPU provided in each, and when the numerical value representing this load falls below a predetermined threshold value, there is a surplus in computing capacity over other configurations. You may make it send the notification that there exists. In this case, the terminal 1 or the main body device 2 that has sent the above-described notification may perform a calculation performed by another configuration.

(Modification 7)
In the above-described embodiment, the human image represents the human body as parts corresponding to 15 parts, but the number of parts is not limited to this. For example, finger parts may be provided for the right hand part and the left hand part, respectively, or a neck part may be provided between the head and chest parts.

(Modification 8)
In the above-described embodiment, the human figure represents the human body as a polygon part for each part, and is configured by combining these parts. However, the human figure is a representation of the human body. Not limited. The human figure may be a representation of a non-human creature such as a cat, monkey, or insect, or a fictional creature such as a sphinx or centaur, or an inanimate figure such as a robot. It may be. For example, objects such as clothes and accessories such as mufflers and sleeves may be included in the person image. In this case, these objects may be configured to move in accordance with a part of the human image whose motion is described in the motion data. For example, the movement of the muffler may be obtained by physical calculation in consideration of the movement of the neck portion around which the muffler is wound, the mass of the muffler, the surrounding pressure, and the like. In short, the person image may be any image as long as the image operates along with the reproduction of the music.

(Modification 9)
In the above-described embodiment, the song DB 221 stores song numbers and song data in association with each other, and one piece of motion data is described in the song data, but a plurality of pieces of motion data are described in the song data. It may be. In this case, an operation number is described as identification information for identifying the operation data in each header portion of the plurality of operation data, and the terminal 1 includes a song number and a person image in the instruction signal transmitted to the main device 2. In addition to the number, an operation number for specifying any one operation data may be included. The main body device 2 that has received the instruction signal including the operation number specifies one operation data from a plurality of operation data based on the operation number, and performs the above-described first calculation or the specified operation data. What is necessary is just to send to the terminal 1.

  Even if it is one song, when a plurality of singers sing a song of the song, or when a plurality of dancers dance to the song, a plurality of different actions are generally assigned to each person. According to this configuration, a plurality of motion data are described in the music data, and the user can select any one of them. For example, the user dances with a plurality of dancers with different choreographies. You can learn the choreography of the dancer you want in the song.

  In addition, as a mode in which the terminal 1 includes the operation number in the instruction signal, after the terminal 1 specifies a song according to the user's operation, the operation number may be specified by further accepting the user's operation. In addition, in the correspondence table 122, the song number, the person image number, and the action number described above are described in association with each other, and the person image number and the action number are specified in conjunction with the song number being specified. It may be. If the person image number and the action number have a one-to-one correspondence, the person image number may be substituted for the action number. For example, a person image number may be described in each header portion of a plurality of operation data.

(Modification 10)
In the above-described embodiment, the correspondence table 122 associates one person image number with one song number, but may associate a plurality of person image numbers with one song number. For example, as shown by a broken line in FIG. 7A, the correspondence table 122 may associate a song number, a person image number (first person), and “person image number (second person)”. In this case, motion data indicating the motion of the second person image (hereinafter referred to as motion data B) is described in addition to motion data indicating the motion of the first person image (hereinafter referred to as motion data A). If so, the control unit 11 of the terminal 1 and the control unit 21 of the main device 2 perform the operation indicated by the operation data A on the person image indicated by the first person image number, and the second person image number. The spatial data may be generated by performing the first calculation so that the action indicated by the action data B is performed on the person image indicated by.

  Specifically, for example, when the user selects the song indicated by the song number “P01”, the control unit 11 is associated with the song number “P01” in the correspondence table 122 illustrated in FIG. The first person image number “M01” and the second person image number “M11” are specified.

  As a result, the control unit 11 of the terminal 1 adds the first person image number “M01” and the second person image number “M11” to the song number “P01” as shown in FIG. 7B. Is transmitted to the main unit 2 using the short-range communication unit 15 as an instruction signal indicating an instruction based on the user's music selection. As a result, the person image data of the first person image identified by the person image number “M01” and the person image data of the second person image identified by the person image number “M11” are respectively the operation data A. The first calculation is performed in association with the motion data B to generate spatial data. When image data is generated by performing the second calculation using the generated spatial data, the above-described light source data, and viewpoint data, the control unit 11 of the terminal 1a displays an image corresponding to the image data. May be displayed on the display screen 141 of the display unit 14. As a result, the display screen 141 displays a state in which a plurality of (in this case, two) person images individually operate as the music specified by the music number is reproduced.

  That is, if there is a song (such as a duet song or a group song) for which a plurality of person images are to be displayed on one screen in a song in which a plurality of motion data is described in the song data, A plurality of person image numbers may be stored in association with the song number. Thereby, the user can display on the display screen 141 of the terminal 1 a video in which a plurality of human images dance with different motions as the above-described music is reproduced.

  When a plurality of person image numbers and a plurality of motion data are associated with one music number in this way, the order assigned to each of the person image numbers and the operation data (in this case) The person image number and the operation data may be associated with each other based on the first person and the second person). In the correspondence table 122, three or more person image numbers may be associated with one music number.

  Here, m and n are natural numbers of 1 or more, and the number m of the person image numbers described in association with one music number in the correspondence table 122 is the music data specified by the music number. When the number of motion data included is larger than n (that is, m> n), any one of the motion data from the first to the m-th person is included in the human image indicated by the (n + 1) th to m-th person image numbers. May be performed, or these may be assigned in order. In this case, the person images indicated by the person image numbers from the (n + 1) -th person to the m-th person may not be used.

  The number m of person image numbers described in correspondence with a certain music number in the correspondence table 122 is smaller than the number n of motion data included in the music data specified by the music number (that is, , M <n), the (m + 1) -th to n-th person motion data may be assigned a person image indicated by any one of the first to m-th person image numbers, or these may be assigned in order. Alternatively, a predetermined person image may be assigned. In this case, the operation data from the (m + 1) th person to the nth person may not be used.

  For example, in the correspondence table 122 shown in FIG. 7A, the song number “P02” is associated with the first person image number “M02”, but the second person image number is not set. . In this case, if each of the first and second motion data is described in the music data identified by the music number “P02”, the person image number associated with the second motion data is not set. It turns out that. Even in this case, if the above-described rule is stored in the storage unit 12 or the storage unit 22, the control unit 11 or the control unit 21 newly sets an unset person image number according to the rule. It can be set or restricted so that operation data corresponding to an unset person image number is not used.

(Modification 11)
In the above-described embodiment, the song data includes performance data, background data, lyrics data, and motion data, but may include presentation data indicating information to be presented to the user as the song progresses. Good. This presentation data may be, for example, information that notifies the operation indicated in the operation data several seconds in advance. In this case, since the user is warned immediately before the choreography becomes complicated, the choreography can be learned efficiently. Further, the presentation data may provide advice on the position of the viewpoint in the virtual space. Specifically, the presentation data may be control data for causing the display unit 14 of the terminal 1 to display an arrow or the like indicating that it is better to move the viewpoint to the right when a certain timing is reached. .

(Modification 12)
In the above-described embodiment, the operation unit 13 of the terminal 1 includes the sensor 133 including an acceleration sensor, an angular velocity sensor, and the like. However, the movement state and posture of the housing are detected, and signals corresponding to these are output. The sensor 133 to perform is not limited to this. For example, a configuration may be included in which a motion state or posture is detected by a camera that captures the user's body and a processor that analyzes an image captured by the camera.

(Modification 13)
In the above-described embodiment, the terminal 1 used for music selection displays an image representing a behavior of a person image during the reproduction of the music on the display unit 14. The operation may be accepted and music selection may be performed. Specifically, the terminal 1 displays on the display screen 141 of the display unit 14 an image representing how the person image operates, and displays a “switch button” on a part of the display screen 141. When a user's finger or the like touches the position where the “switch button” is displayed, the touch panel 132 detects the position touched by the user and sends an instruction to the control unit 11 to switch the screen. Receiving this instruction, the control unit 11 displays a music selection menu as shown in FIG. 4 on the display screen 141. As a result, the terminal 1 displays a person image that operates in accordance with the reproduction of the song, thereby producing a reproduction of the song and realizing a function of selecting a song while the display is being performed. Can do.
The control unit 11 may display a music selection menu on the display screen 141 so as to be superimposed on the manner in which the person image operates, or divide the display area of the display screen 141 and display the person image on one of them. May be displayed, and the music selection menu may be displayed on the other side.

(Modification 14)
The program in the above-described embodiment is provided in a state stored in a computer-readable recording medium such as a magnetic recording medium (magnetic tape, magnetic disk, etc.), an optical recording medium (optical disk, etc.), a magneto-optical recording medium, or a semiconductor memory. obtain. Further, the terminal 1 and the main device 2 may download the program via a network.

DESCRIPTION OF SYMBOLS 1 (1a, 1b, 1c) ... Terminal, 11 ... Control part, 111 ... 1st specific part, 112 ... 2nd specific part, 113 ... 3rd specific part, 114 ... 1st request part, 115 ... 2nd request part , 116 ... display control unit, 12 ... storage unit, 121 ... table of contents DB, 122 ... correspondence table, 123 ... person image DB, 13 ... operation unit, 131 ... operator, 132 ... touch panel, 133 ... sensor, 14 ... display unit , 141 ... display screen, 15 ... short-range communication unit, 2 ... main unit, 21 ... control unit, 211 ... reproduction unit, 212 ... generation unit, 213 ... transmission unit, 22 ... storage unit, 23 ... operation unit, 231 ... Operator, 232 ... Microphone, 24 ... Display unit, 241 ... Display screen, 25 ... Short range communication unit, 26 ... Communication unit, 27 ... Sound emitting unit, 3 ... Karaoke apparatus, 4 ... Network, 5 ... Server, 9 ... Karaoke system, CH ... parts, JA ... angle, P ... light source, PO ... the center of gravity, PV ... viewpoint coordinates, VA ... the angle of view

Claims (3)

A terminal,
A main unit and
The terminal
Storage means for storing song identification information for identifying a song and image identification information for identifying an image operating in a virtual space in association with reproduction of the song;
First identifying means for identifying song identification information of a song designated by the user;
Second specifying means for specifying image identification information corresponding to the music identification information specified by the first specifying means based on the storage content of the storage means;
Third specifying means for specifying a viewpoint in the virtual space according to a user operation;
First request means for transmitting the song identification information identified by the first identification means to the main unit and requesting reproduction of the song indicated by the song identification information;
The image identification information specified by the second specifying means is transmitted to the main device, and the image indicated by the image identification information is operated in the virtual space along with the reproduction of the music. Second request means for requesting transmission of spatial data indicating the position and orientation in the virtual space;
When viewing the image in the virtual space from the viewpoint based on the spatial data transmitted from the main body device in response to the request from the second request means and the viewpoint specified by the third specifying means Display means for displaying the image of
The main unit is
Playback means for playing back the song indicated by the song identification information in response to a request from the first request means;
Generating means for generating the spatial data in response to a request by the second request means;
Transmitting means for transmitting the spatial data generated by the generating means to the terminal. A terminal,
A main unit and
The terminal
Storage means for storing song identification information for identifying a song and image identification information for identifying an image operating in a virtual space in association with reproduction of the song;
First identifying means for identifying song identification information of a song designated by the user;
Second specifying means for specifying image identification information corresponding to the music identification information specified by the first specifying means based on the storage content of the storage means;
Third specifying means for specifying a viewpoint in the virtual space according to a user operation;
First request means for transmitting the song identification information identified by the first identification means to the main unit and requesting reproduction of the song indicated by the song identification information;
The music identification information and the image identification information specified by the second specifying means are transmitted to the main device, and the image identification information indicates in the virtual space as the music indicated by the music identification information is reproduced. Second request means for requesting transmission of operation data indicating an operation to be performed on the image and transmission of shape data indicating the shape of the image indicated by the image identification information;
Based on the motion data and the shape data transmitted from the main body device in response to a request by the second request means, and the viewpoint specified by the third specifying means, the viewpoint in the virtual space from the viewpoint Display means for displaying the image when the image is viewed, and
The main unit is
Shape data storage means for storing shape data indicating the shape of the image identified by the image identification information in association with each image identification information;
Playback means for playing back the song indicated by the song identification information in response to a request from the first request means;
Generating means for generating the motion data and the shape data in response to a request by the second request means;
A karaoke apparatus comprising: transmission means for transmitting the motion data and the shape data generated by the generation means to the terminal. A plurality of the terminals,
The transmission means of the main unit is
The karaoke apparatus according to claim 1 or 2, wherein the data requested by the second request unit from each of the plurality of terminals is transmitted to each terminal.

Images