JP7045022B2 - Karaoke system, VR content device and VR content program - Google Patents

Description

Patent Law Article 30 Paragraph 2 application Article 30 Paragraph 2 application, announced at Tokyo Game Show 2017 held at Makuhari Messe on September 21-24, 2017

The present invention is a karaoke system that allows you to enjoy various competitions and singing, a VR content device that allows you to enjoy various competitions, and VR content that realizes VR content functions by being implemented in various information processing devices. Regarding the program.

Traditionally, karaoke is performed in which various people gather, such as banquets and drinking parties, to sing along with the accompaniment. A karaoke device for performing karaoke usually has a basic function of displaying lyrics in synchronization with a performance. In addition to these basic functions, karaoke devices with singing scoring functions and various game functions using performance information have been proposed to expand the enjoyment of users.

In Patent Document 1, a music performance is performed using the performance information included in the music data, and a dance game is played based on the pattern data included in the music data and the music progress information output in synchronization with the performance. A karaoke system that can be enjoyed is disclosed.

In Patent Document 2, the first voice data and the second voice data input from the two microphones are compared with the reference data, and the first phrase point and the second phrase point are generated for each phrase break, and these phrase points are generated. A karaoke battle game that generates a fighting animation based on is disclosed.

Japanese Patent No. 3614049 Japanese Patent No. 3573425

Karaoke is an entertainment that was born in Japan and spread all over the world, but in recent years, the number of users has been sluggish due to the declining birthrate and aging population, the increasing preference for other media such as the Internet, and so on. In the karaoke battle game disclosed in Patent Document 2, the superiority or inferiority of singing is competed, and it is considered that a person who is not good at singing becomes difficult to win the game and cannot enjoy it.

The present invention has been made in view of such a situation, and one object of the present invention is to allow the user to enjoy a game played in conjunction with a performance regardless of the superiority or inferiority of singing.

Therefore, the karaoke system according to the present invention adopts the following configuration.
It is a karaoke system that allows the control unit to execute performance processing and game processing that perform performances based on performance information.
The game process is a process in which various displays are performed based on the user's operation by the operation means, and the result is determined without using the performance information or the information output in synchronization with the performance of the performance information. Started in synchronization with the start of the performance process,
If the game processing ends during the performance processing, the performance processing is interrupted and the performance processing is interrupted.
When the performance process is completed without the game process being completed, the game process is terminated and the result of the game process is notified.

Further, in the karaoke system according to the present invention.
When the performance process is completed without the game process being completed, the event process is executed before the result is notified.

Further, in the karaoke system according to the present invention.
When the performance volume is low or there are no lyrics to be sung just before the end of the performance process, a notification is given to indicate that the performance or the game is continuing.

Further, in the karaoke system according to the present invention.
The game processing is characterized in that the opponent arranged in the virtual space is displayed at a position corresponding to the movement of the headset on the display unit arranged on the headset.

Further, in the karaoke system according to the present invention.
The performance process is characterized in that it is started when the operating means used in the game process makes a predetermined movement.

Further, in the karaoke system according to the present invention.
The game processing is characterized in that it is executed using an audio signal input from a microphone.

The VR content device according to the present invention is
A VR content device that causes the control unit to execute game processing in cooperation with a karaoke device that executes performance processing based on performance information.
The game process is a process in which various displays are performed based on the user's operation by the operation means, and the result is determined without using the performance information or the information output in synchronization with the performance of the performance information. Started in synchronization with the start of the performance process,
If the game processing ends during the performance processing, the performance processing is interrupted and the performance processing is interrupted.
When the performance process is completed without the game process being completed, the game process is terminated and the result of the game process is notified.

The VR content program according to the present invention is
A game program that can be executed by an information processing device connected to a karaoke device that performs performance processing based on performance information.
Enables game processing linked to karaoke equipment,
The game process is a process in which various displays are performed based on the user's operation by the operation means, and the result is determined without using the performance information or the information output in synchronization with the performance of the performance information. Started in synchronization with the start of the performance process,
If the game processing ends during the performance processing, the performance processing is interrupted and the performance processing is interrupted.
When the performance process is completed without the game process being completed, the game process is terminated and the result of the game process is notified.

The karaoke system, the VR content device, and the VR content program according to the present invention determine the result without using the performance information or the information output in synchronization with the performance of the performance information, and therefore the superiority or inferiority of the singing. It is possible to enjoy the game without worrying about. Further, since the game processing is performed in synchronization with the performance processing, the user can enjoy the game together with the singing, and it is possible to add new value to the karaoke.

The figure which shows the karaoke system of this embodiment A block diagram showing the configuration of the game device of the present embodiment A perspective view showing the appearance of the HMD (head-mounted display) of the present embodiment. Block diagram showing the configuration of the karaoke device of this embodiment A flow chart showing processing on the karaoke device side of the present embodiment The figure which shows the data structure of the music information used in this embodiment A flow chart showing processing on the game device side of this embodiment Schematic diagram for explaining the visual field image in this embodiment Schematic diagram for explaining the visual field image in this embodiment

FIG. 1 is a diagram showing a karaoke system of the present embodiment. The karaoke system of the present embodiment includes a game device 1 (corresponding to the "device for VR contents" of the present invention) and a karaoke device 4. As peripheral devices, an HMD 3 (head-mounted display), headphones 32, a microphone 33, a tracker 31R, 31L, a base station 34a, 34b, a monitor 21, a mixer 23, an amplifier 24, and a speaker 25 are provided. In this embodiment, the HMD3, the trackers 31R, and 31L constitute an operating means for detecting the user's operation.

The HMD3 is attached to the user's head, and it is possible to project an image on each of the user's eyes by two display units 35R and 35L provided inside, so that the user can experience the virtual space. Further, the HMD3 can detect its position, direction and the like by infrared rays emitted from the base stations 34a and 34b. Further, the HMD 3 is provided with a microphone 33 for picking up the voice of the user and headphones 32 for providing the voice to the user. The trackers 31R and 31L are devices that the user grips or wears on the user's hand, have a plurality of light receiving units (not shown), and are irradiated by infrared rays emitted from the base stations 34a and 34b. It is possible to track the position, direction, etc.

The base stations 34a and 34b are installed at appropriate places in the room where the karaoke system is installed, and can detect the position and direction of the head of the user wearing the HMD3 and the positions and directions of the trackers 31R and 31L. It is possible. The base stations 34a and 34b of the present embodiment are connected to each other by a cable, irradiate infrared rays in synchronization with each other, and receive infrared rays by a light receiving unit provided on the HMD3, tracker 31R, and 31L side to receive infrared rays. The positions and directions of the trackers 31R and 31L, that is, the user's actions are to be detected. The detected position and movement of the user are reflected in the movement of the character as the user's alter ego in the virtual space. In addition, for the position and detection of the user's operation, various forms can be adopted instead of using such base stations 34a, 34b, HMD3 and trackers 31R, 31L.

The game sound output from the game device 1, the performance sound output from the karaoke device 4, and the singing sound are mixed by the mixer 23 and output to the headphone 32 and the amplifier 24 on the user side. The signal input to the amplifier 24 is emitted and output from the speaker 25. Further, in the present embodiment, the video output from the game device 1 is displayed not only on the HMD 3 but also on the monitor 21. The monitor 21 and the speaker 25 are provided to provide various images and sounds to the user before wearing the HMD 3 or to the attendants.

FIG. 2 is a block diagram showing the configuration of the game device 1 of the present embodiment. The game device 1 includes a CPU 10 for controlling each configuration in an integrated manner, and a memory 11 for temporarily storing information necessary for executing various programs. The CPU 10 and the memory 11 form a control unit in the game device 1. Further, the game device 1 has various programs such as a game program (corresponding to the "VR content program" of the present invention) and a hard disk 17 as a storage unit for storing various information used in the various programs.

Further, the game device 1 of the present embodiment has an acoustic control unit 15 that inputs and outputs various sounds. The sound control unit 15 captures the user's singing sound input from the microphone 33 when the game program is executed, and outputs the singing sound and various sound effects output in association with the game processing to the mixer 23. do.

Further, the game device 1 includes image forming means for displaying various images on the monitor 22. The image forming means includes an image forming unit 13 that forms a game image, a video RAM 12 that temporarily stores the formed game image, and an image control unit 14. The image control unit 14 synthesizes the formed game image and the lyrics image output from the karaoke device 4, and displays and outputs them to the right-eye and left-eye display units 35R and 35L arranged in the monitor 21 and HMD3. It is possible to do. The left and right display units 35R and 35L can display and output images having parallax so that the user can see them stereoscopically. Further, in the HMD 3 of the present embodiment, a camera 39 for photographing the front of the user is arranged. The image control unit 14 can also display the image taken by the camera 39 on the HMD 3.

The game device 1 includes an interface 18 by serial communication for inputting / outputting various information to and from the outside. The interface 18 of the present embodiment is connected to peripheral devices such as the HMD3, trackers 31R, and 31L, and the karaoke device 4. In this embodiment, these peripheral devices and the game device 1 are connected by wire or wirelessly.

As described above, the base stations 34a and 34b are means for detecting the positions and directions of the HMD3, the trackers 31R, and 31L. In the present embodiment, the base stations 34a and 34b are arranged at appropriate positions in the room and irradiate infrared rays in synchronization with each other. do. The HMD3, trackers 31R, and 31L receive infrared rays by using a plurality of light receiving units provided therein. The game device 1 detects the position and direction of the HMD 3 and the tracker 31R based on the light receiving timing of the light receiving unit.

Further, the interface 18 is also connected to the karaoke device 4, and the game device 1 can detect the state of the karaoke device 4, for example, whether or not it is playing (playing state, non-playing state). It has become. In addition, as the state of the karaoke device 4, it may be possible to determine the performance section type of the music being played, for example, an intro, an A melody, a B melody, an interlude, a sequel, and the like. Further, the game device 1 can transmit various commands, for example, a performance start command, a performance stop command, or the like to the karaoke device 4 via the interface 18.

FIG. 3 is a perspective view showing an HMD3 (head-mounted display) used in the present embodiment. The figure on the left is a perspective view of the user wearing the HMD3 from the rear, and the figure on the right is a perspective view of the user wearing the HMD3 from the front. The HMD 3 is configured to have an HMD housing 37 accommodating two display units 35R and 35L for the right eye and a left eye, and a head band 36 for wearing on the user's head. In the present embodiment, the headset is composed of the HMD housing 37 and the headband 36. On the front surface of the HMD housing 37, four light receiving portions 38a to 38d are provided for arranging detection. The number of light receiving units 38a to 38d is not limited to this, and the number may be increased in order to improve the detection accuracy of the position and direction.

Further, the HMD housing 37 is provided with a camera 39 for photographing the front of the user wearing the HMD 3. Using this camera 39, it is also possible to synthesize the surrounding image of the user and the images of the display units 35R and 35L so that the user can experience so-called AR (augmented reality).

Headphones 32R and 32L are attached to the headband 36 so that the user can hear the sound. Further, a microphone 33 for capturing a user's singing voice via an arm is attached to the headphone 32L on the left side. With such a configuration, the user can listen to the performance sound and the like with the headphones 32R and 32L regardless of the direction, and can sing without holding the microphone 33 by hand. ing. The microphone 33 may be fixed to the HMD housing 37 instead of being fixed to the headphones 32L. Alternatively, the HMD3 and the headphones 32R and 32L may be separately worn by the user.

FIG. 4 is a block diagram showing the configuration of the karaoke device 4 of the present embodiment. The karaoke device 4 temporarily stores the CPU 40 for controlling each configuration in a centralized manner and information necessary for executing various programs. A memory 41 for the purpose is provided. The CPU 40 and the memory 41 constitute a control unit in the karaoke device 4. Further, the karaoke device 4 has various programs such as a performance program and a hard disk 46 as a storage unit for storing various information used in the various programs.

Further, the karaoke device 4 of the present embodiment has an acoustic control unit 45 for inputting / outputting various voices. The acoustic control unit 45 outputs the performance sound output along with the performance to the mixer 23 when the performance program is executed. Further, when the karaoke device 4 is used alone, it is possible to mix the singing voice input to the first microphone 26a and the second microphone 26b with the performance sound and output it to the outside.

Further, the karaoke device 4 includes a video forming means for externally outputting the video. The image forming means includes an image forming unit 43, a video RAM 42 for temporarily storing the formed game image, and an image control unit 44. The karaoke device 4 of the present embodiment outputs a lyric image with a blue background as a background when playing in conjunction with the game device 1. The game device 1 superimposes the game image on the background portion (blue background portion) of the lyrics image and displays and outputs it to the display units 35R and 35L of the HMD3. At the time of normal performance that is not linked with the game device 1, the background image accompanying or corresponding to the music is combined with the lyrics image and output.

The karaoke device 4 includes an interface 47 by serial communication in order to input / output various information to and from the outside. The interface 47 of the present embodiment is wiredly connected to the interface 18 of the game device 1. The game device 1 can detect various states of the karaoke device 4 via the interface 47. Further, the game device 1 can transmit various commands to the karaoke device 4 via the interface 47.

The karaoke device 4 includes a LAN communication unit 48 as a communication means for connecting to the LAN 26. The LAN 26 is connected to the router 27 and can be connected to the Internet. In the example of FIG. 4, the karaoke device 4 can communicate with the server device 5 that manages various information via the LAN communication unit 48 and the router 27. The karaoke device 4 can receive music information stored in the storage unit 51 of the server device 5 connected by communication, or various programs.

The karaoke system of the present embodiment and its peripheral devices have been described above, but the karaoke system of the present embodiment is a new game for the user by operating the game device 1 and the karaoke device 4 in conjunction with each other. It is possible to provide.

The processing of the interlocking of the game device 1 and the karaoke device 4 will be described. FIG. 5 is a flow chart showing processing on the karaoke device 4 side of the present embodiment. Further, FIG. 6 is a diagram showing a data structure of music information used in the karaoke device 4 of the present embodiment.

As shown in FIG. 5, first, the user selects a song to be sung by using an input means (not shown) of the karaoke device 4 (S101). When the music is selected, the karaoke device 4 reads the music information corresponding to the selected music from the hard disk 46 and enters the playback standby state (S102). Here, the reproduction standby state is a state in which the selected music information can be immediately reproduced by receiving the reproduction start command.

The karaoke device 4 of the present embodiment cancels the playback standby state and starts playing music information by receiving a performance start command from the game device 1 connected via the interface 18 (S103: Yes). (S105). At that time, the karaoke device 4 transitions from the non-playing state to the playing state, and this state can be detected via the interface 47 of the karaoke device 4. The game device 1 can detect the state of the karaoke device 4 via the interface 18.

Here, the data structure of the music information used in the karaoke device 4 of the present embodiment will be described. FIG. 6 is a diagram showing a data structure of music information used in the karaoke device 4. The music information is composed of meta information given for performing music selection and the like, and actual information for performing performance and displaying lyrics. The meta information includes a music ID, a music name, a singer name, a lyricist name, a composer name, a section identification information, and the like for managing the music. The section identification information is information indicating a performance section type corresponding to the progress of the music, such as a prelude, an A melody, a B melody, and an interlude. By using this section identification information, it is possible to determine the type of performance section currently being played during music playback.

The actual information includes performance information, lyrics information, and background video information. By causing the acoustic control unit 15 to play the performance information when playing the music, it is possible to acoustically output the performance sound as a singing accompaniment from the headphones 32R and 32L. Further, when the music is reproduced, the lyrics information is reproduced by the image forming unit 43, and the image is output to the HMD 3, so that the user can assist the singing.

The background video information is video information used during normal reproduction of music information, and is not used when linked with the game device 1 of the present embodiment. At the time of normal reproduction, it is possible to provide the user with an image having an atmosphere suitable for the music by reproducing the background image information, synthesizing it with the reproduced lyrics image, and outputting it.

During normal playback in the karaoke device 4, the music is played to the end unless there is an operation to stop the performance by the user. When the karaoke device 4 in the present embodiment receives the performance stop command output when the game ends in the game device 1, the karaoke device 4 interrupts the performance even during the music reproduction (S108). At that time, the state of the karaoke device 4 is changed from the playing state to the non-playing state (S109). Further, when the performance stop command is not received from the game device 1 during the music performance (S107: Yes), the performance state is changed to the non-performance state (S109) at the end of the performance.

FIG. 7 is a flow chart showing processing on the game device 1 side that operates in conjunction with the karaoke device 4. In the game device 1, the game program is executed when the game device 1 is linked with the karaoke device 4. The game program of the present embodiment is a game in which boxing can be enjoyed in a virtual space, and the user operates the HMD3, the tracker 31R, and 31L while visually recognizing the image displayed on the HMD3 to interact with the opponent. It is possible to experience a boxing match.

Further, during the game, the karaoke device 4 plays a song selected by the user, so that the singing at the time of the game can be enjoyed. First, the game device 1 from which the game program has been started is in a standby state (S201). The user who plays the game wears the HMD3, the tracker 31R, and the 31L after selecting a song for the karaoke device 4. Here, the user wearing the HMD3, the trackers 31R, and 31L is obstructed by the HMD3 in the surrounding field of view. Therefore, in the present embodiment, the game is started by the movement of the trackers 31R and 31L as the operating means satisfying the predetermined conditions, for example, the ready user performs an operation such as raising both hands up. (S203). Here, the operation of raising both hands can be detected on condition that both the trackers 31R and 31L are higher than the HMD3. It should be noted that the start of the game is not limited to such a form, but it is conceivable to adopt various forms such as moving the HMD3 as an operation means by shaking the head vertically. Alternatively, it may be started on condition that the user is facing a predetermined direction (for example, forward) within a predetermined range. Such conditions can be detected by the position and direction of the HMD3 and the positions and directions of the trackers 31R and 31L.

FIG. 8 is a schematic diagram for explaining the visual field image 6 displayed on the HMD 3 during the game. The visual field image 6 includes the opponent 62 and the opponent grabs 63R and 63L, which are images whose appearance changes depending on the position and direction of the HMD3. Further, in the visual field image 6, the own grabs 64R and 64L, which are the images of the grabs that he / she is wearing, are displayed. These are also images whose appearance changes depending on the position and direction of the HMD3 and the positions and directions of the trackers 31R and 31L. By displaying the opponent 62 and various images of himself / herself in the virtual space in this way, it is possible to experience boxing with the opponent 62.

By the way, in this game in which the HMD3 is attached, it is preferable to adapt the size of the opponent 62 to the user. For example, if the size of the opponent 62 is fixed, a short user may not reach the head of the opponent 62. Therefore, in the present embodiment, the size change process for changing the size of the opponent 62 in the virtual space is performed based on the position of the HMD3 when the trackers 31R and 31L, which are the signals for starting the game, are moving. With such a form, it becomes possible to make an opponent 62 having a size according to the height of the user estimated by the position of the HMD 3, and it becomes possible to play the game in an appropriate state. The height of the user may be estimated not at the position of the HMD3 but at the positions of the trackers 31R and 31L.

In addition to this, the visual field image 6 displays lyrics 61, a punching force meter 65, calories burned 66, a self-down number 67, and an opponent down number 68. These pieces of information are always displayed at predetermined positions in the visual field image 6, and it is possible to provide various information about the game to the user. Here, the calorie consumption 66 is the calorie consumption of the user calculated based on the movements of the HMD3 and the trackers 31R and 31L during the game period. The user can play the game while referring to the displayed calorie consumption 66.

Further, the game device 1 outputs a performance start command to the karaoke device 4 substantially at the same time as the game start (S203) (S204). As described above, the karaoke device 4 starts playing the music information in the playback standby state by receiving the performance start command. At that time, the lyrics video output from the karaoke device 4 is combined with the game video formed by the game device 1 and displayed at a predetermined position in the visual field video 6 as shown in FIG. The user can sing and play a game while watching the lyrics 61 displayed on the visual field image 6.

During the game, the opponent 62 attacks by paying out the opponent grabs 63R and 63L in the virtual space. In addition, the user moves his / her hand in the real space to change the position and direction of the trackers 31R and 31L, operates the own grabs 64R and 64L displayed in the virtual space, and attacks the opponent 62. Is possible. Whether or not the attack from the opponent 62 is a hit is determined by the positional relationship between the player's position in the virtual space and the opponent grabs 63R and 63L of the opponent 62. By moving in the real space, the user can move his / her position in the virtual space and avoid the attack of the other party.

During the game period, if your attack hits the opponent 62 (S217: Yes), the stamina amount of the opponent 62 is reduced based on the hit (S218). In the game of the present embodiment, the stamina amount is set for each of the player and the opponent 62, and when this amount becomes a predetermined amount (0 in the present embodiment), the game is down.

In the present embodiment, the amount of stamina is reduced by the amount of damage associated with the attack, but the amount of damage is calculated using the singing voice input to the microphone 33 (S218). Specifically, the amount of damage is calculated based on the volume of the singing voice input to the microphone 33 at the time of hitting the opponent 62. In this embodiment, when the attack on the opponent 62 is hit, the larger the singing voice is, the larger the damage amount is. Further, as shown in the visual field image 6 of FIG. 8, the volume of the singing voice input to the microphone 33 is always displayed in real time as the punch force meter 65. As the volume of the singing voice becomes louder, the bar displayed on the punch force meter 65 is displayed with a large swing to the right. The user can inflict a large amount of damage on the opponent 62 by launching an attack on the opponent 62 while referring to the punch force meter 65.

In the present embodiment, the volume of the singing voice at the time when the attack hits the opponent 62 is used as the damage amount, but in addition to this, the volume of the singing voice may be integrated to calculate the damage amount. For example, after attacking the opponent 62, it is conceivable to accumulate the volume of the singing voice until the next attack and calculate the damage amount based on the accumulated amount.

By reducing the calculated damage amount from the stamina amount of the opponent 62 (S219), it is determined whether or not the opponent 62 goes down. If the amount of stamina is 0 when the amount of damage is reduced from the amount of stamina, 1 is added to the opponent's down number B (S221). As shown in the visual field image 6 of FIG. 8, the opponent down number B is displayed as the opponent down number 68. Then, the opponent down process (S222) that displays how the opponent 62 goes down is executed. In the present embodiment, the upper limit of the opponent down number B is not set, and the opponent 62 returns to the battle state as many times as possible during the game period after the opponent down process (S222) is completed. Therefore, in the present embodiment, it is possible to determine the superiority or inferiority of the game by competing for the opponent down number B within the playing period of the music.

On the other hand, if the attack of the opponent 62 hits oneself during the game period (S209: Yes), the amount of one's stamina is reduced by a predetermined amount based on the hit (S210). By reducing the predetermined amount from the amount of stamina of oneself (S210), it is determined whether or not oneself goes down. When the predetermined amount is reduced from the stamina amount and the stamina amount is 0 (S211: Yes), 1 is added to the own down number A (S212). As shown in the visual field image 6 of FIG. 8, the self-down number A is displayed as the self-down number 67. Then, the self-down process (S213) regarding the own down is executed.

In the self-down processing (S213) of the present embodiment, the battle state is restored after a predetermined time has elapsed. Since the battle time is shortened by performing the self-down process (S213), in this game in which the opponent's down number B is acquired within the game period, the own down is in a disadvantageous state. In the period until returning to the battle state, the time for returning to the battle state may be shortened based on the volume of the singing voice input to the microphone 33.

In the present embodiment, when the number of own downs A is 3 or more (S214: Yes), the game is terminated. On the other hand, when the number of own downs A is less than 3 (S214: No), the game continues. At the end of the game, a losing process (S215) such as displaying the state of losing or notifying the result of losing is performed, and a performance stop command is output to the karaoke device 4 (S216). The karaoke device 4 that has received the performance stop command interrupts the performance (S108). As described above, in the present embodiment, the game and the karaoke (singing) are linked by stopping the performance of the karaoke device 4 when the game of the game device 1 ends.

By the way, during the game period, the game device 1 monitors the state of the karaoke device 4, but when it is detected that the karaoke state has changed from the playing state to the non-playing state during the game period (S205: Yes). The game is to be terminated. That is, when the music played by the karaoke device 4 ends, the game ends with the game device 1. At that time, in the present embodiment, if the game is not completed without being taken 3 down by the opponent 62, an event in which the user can participate by using an operation means such as HMD3, tracker 31R, 31L is executed. It is supposed to be. In this event, the opponent 62 does not attack, and it is determined whether or not his / her attack hits within a predetermined time (10 seconds in this embodiment) (S208).

If it is determined that the opponent 62 has been hit during the execution of the event (S206: Yes), or if the predetermined time has elapsed (S207: Yes), the event is terminated and the winning process (S208) is executed. do. The hit to the opponent 62 in the event may or may not be added to the result of the game.

In the winning process (S208), the user is notified of the result of the game. In the present embodiment, the user is notified that the opponent 62 has won, and the evaluation based on the opponent down number B is notified. As an evaluation, for example, if the opponent's down number is 4 or more and less than 5, a bronze medal is displayed, if it is 6 or more and less than 9, a silver medal is displayed, and if it is 10 or more, a gold medal is displayed.

By the way, when playing a game with the HMD3 attached as in the present embodiment, it is conceivable that the user moves to an unexpected place. For example, in a narrow space such as a karaoke box, it is possible to hit a wall, furniture, or a karaoke system. Assuming such a situation, the game device 1 of the present embodiment is photographed by a camera 39 provided in the HMD 3 on the display units 35R and 35L of the HMD 3 when the user deviates from a predetermined area in the real space. By displaying the image in the real space, that is, the image in front of the user, the user is notified of the state of the real space and returned to a predetermined area.

FIG. 9 is a schematic diagram for explaining the visual field image 6 when the user deviates from the predetermined area. In the center of the visual field image 6, a camera image display field 69 for displaying a camera image taken by the camera 39 provided in the HMD 3 is displayed. Since FIG. 9 is a schematic diagram, the camera image is not described in the camera image display column 69. Further, in the camera image display field 69, the message "Please return to the position where the camera image disappears" is superimposed on the camera image and displayed. The user can return to an appropriate position for playing a game while checking the state of the real space with the camera image displayed in the camera image display field 69.

Although one embodiment of the present invention has been described above, the present invention is not limited to this embodiment, and various modifications can be adopted. Various modifications will be described below.

(First modification)
In the above-described embodiment, the game is in a virtual space using HMD3, trackers 31R, 31L, etc. as operating means, but it is not always necessary to use such a game. For example, it may be a game in which an operation is performed by a hand-held controller as an operation means while watching an image displayed on a monitor such as a liquid crystal display device.

(Second modification)
In the above-described embodiment, a boxing type game is used, but the type of game can be applied to various games in which an opponent exists. For example, it may be a martial art other than boxing, or it may be a game in which a weapon such as a sword is used to fight an opponent such as a monster. Alternatively, it may be applied to a car race and also applied to a game in which a ranking with an opponent is competed.

(Third modification example)
In the above-described embodiment, the game device 1 monitors the playing state of the karaoke device 2 by serial communication connected by the interface 18, but if the playing state of the karaoke device 2 is only such a state signal. Instead, the determination may be made based on the volume of the performance sound output from the karaoke device 2.

(Fourth modification)
In the above-described embodiment, the amount of damage in the game is calculated based on the volume of the singing voice input to the microphone 33. According to such an embodiment, it is possible to enjoy the game without worrying about the superiority or inferiority of the singing. The damage amount may be calculated by using not only the volume of the singing voice but also other feature amounts of the singing voice. For example, it is conceivable to use the pitch of the singing voice and increase the amount of damage as the pitch increases. By using the feature amount of the singing voice in this way and playing the game without using the performance information in the song information or the information output in synchronization with the performance information such as the singing tune, the user can sing. You can sing and enjoy the game with all your might, without worrying about the superiority or inferiority.

(Fifth modification)
In the above-described embodiment, the performance sound may be reduced or the lyrics may not be displayed during the performance of the music in the karaoke device 4, especially in the after-play portion. In such a case, it is conceivable that the user mistakes the end of the performance for the end of the game and removes the HMD3. In particular, in the above-described embodiment, since the event is prepared at the end of the game, if the MMD3 is removed, the event cannot be participated. In consideration of such a situation, the game device 1 informs the user that the performance or the game is continuing when the performance sound becomes less than a predetermined value or the lyrics are no longer displayed. It is preferable to give a notice to indicate. For such a notification, it is conceivable to display a message on the HMD 3 or to read the message aloud from the headphones 32. Further, in order to give such a notification in an appropriate situation, the game device 1 monitors the playing state (playing section type) of the karaoke device, and the section where the playing state is the lyric, or a predetermined time before the end of the performance ( For example, only in the section that is 30 seconds ago), the above-mentioned performance sound or the lyrics may be used for determination and notification may be given.

In the present embodiment, the karaoke system including the game device 1 (corresponding to the "device for VR contents" of the present invention) and the karaoke device 2 has been described as an example, but the present invention is as described above. Not limited to the karaoke system, it may be composed of one information processing device. Further, a game program (corresponding to the "VR content program" of the present invention) capable of realizing the function executed by the game device 1 of the present invention also belongs to the category of the present invention.

1: Game device 35R, 35L: Display unit 2: Karaoke device 36: Headband 3: HMD 37: HMD housing 4: Karaoke device 38a to 38d: Light receiving unit 5: Server device 39: Camera 6: Field image 40: CPU
10: CPU 41: Memory 11: Memory 42: Video RAM
12: Video RAM 43: Video forming unit 13: Image forming unit 44: Video control unit 14: Video control unit 45: Sound control unit 15: Sound control unit 46: Hard disk 17: Hard disk 47: Interface 18: Interface 48: LAN communication unit 21: Monitor 51: Storage unit 23: Mixer 61: Lyrics 24: Speaker 62: Opponent 25: Amplifier 63R, 63L: Opponent grab 26a, 26b: Microphone 64R, 64L: Own grab 27: Router 65: Punching power Meter 31R, 31L: Tracker 66: Calories burned 31R: Tracker 67: Own down number 32 (32R, 32L): Headphone 68: Opponent down number 33: Microphone 69: Camera image display column 34a, 34b: Base station

Claims (8)

It is a karaoke system that allows the control unit to execute performance processing and game processing that perform performances based on performance information.
The game process is a process in which various displays are performed based on the user's operation by the operation means, and the result is determined without using the performance information or the information output in synchronization with the performance of the performance information. Started in synchronization with the start of the performance process,
If the game processing ends during the performance processing, the performance processing is interrupted and the performance processing is interrupted.
A karaoke system characterized in that when the performance process is completed without the game process being completed, the game process is terminated and the result of the game process is notified. The karaoke system according to claim 1, wherein when the performance process is completed without the game process being completed, the event process is executed before the result is notified. Claim 1 or claim 2 is characterized in that when the performance volume is low or there are no lyrics to be sung, a notification is given to indicate that the performance or the game is continuing just before the end of the performance process. The karaoke system described in. The game processing is any one of claims 1 to 3, wherein the opponent arranged in the virtual space is displayed at a position corresponding to the movement of the headset on the display unit arranged on the headset. The karaoke system described in item 1. The karaoke system according to claim 4, wherein the performance process is started when the operating means used in the game process makes a predetermined movement. The karaoke system according to any one of claims 1 to 5, wherein the game processing is executed using an audio signal input from a microphone. A VR content device that causes the control unit to execute game processing in cooperation with a karaoke device that executes performance processing based on performance information.
The game process is a process in which various displays are performed based on the user's operation by the operation means, and the result is determined without using the performance information or the information output in synchronization with the performance of the performance information. Started in synchronization with the start of the performance process,
If the game processing ends during the performance processing, the performance processing is interrupted and the performance processing is interrupted.
A device for VR content characterized in that when the performance process is completed without the game process being completed, the game process is terminated and the result of the game process is notified. A program for VR content that can be executed by an information processing device connected to a karaoke device that executes a performance process that performs a performance based on performance information.
Enables game processing linked to karaoke equipment,
The game process is a process in which various displays are performed based on the user's operation by the operation means, and the result is determined without using the performance information or the information output in synchronization with the performance of the performance information. Started in synchronization with the start of the performance process,
If the game processing ends during the performance processing, the performance processing is interrupted and the performance processing is interrupted.
A program for VR content characterized in that when the performance process is completed without the game process being completed, the game process is terminated and the result of the game process is notified.

Images