JP2014023745A - Dance teaching device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a dance teaching device which accurately displays/evaluates the total body exercise of one or a plurality of learners.SOLUTION: A dance teaching device includes memory means for memorizing music data, character data, and the action data thereof, sensor means for recognizing the motion of each portion of the body of a learner, display means capable of displaying both a model dance by a character and a dance of the learner, acoustic means for presenting music as sound, communication means for receiving each of the data from a server through a communication channel and transmitting, if necessary, the dance of the learner, input-output means, and control means therefor.

Description

The present invention relates to a dance learning apparatus that can be demonstrated by one or more learners.

  Since 2012, dance will be incorporated into the compulsory classes for junior high school health and physical education, and demand for dance practice is increasing. In particular, there is a need for means to practice while having fun.

  Conventionally, dance games have been put to practical use in arcade game machines and home game machines as games that can be practiced while enjoying dance. In such a dance game, a sign such as an arrow that moves on the screen is displayed according to the song, and at the moment when it overlaps with a mark indicating a step, you can score by stepping on or pressing the corresponding floor button. is there. Various steps are combined in the song, and if the learner does not correctly follow the steps according to the song, the game is over or the points are deducted. In addition, recently, there are some that support whole body movements including upper body movements.

  However, in recent years, there are many collective dances by a plurality of artists, and when a plurality of learners practice a dance at the same time, the above game cannot sufficiently cope. Also, in a home video game machine, it is usual to sell several songs together in a package, and when trying to practice a new song, new software or hardware must be purchased.

  Accordingly, there is a need for a dance learning device that accurately displays and evaluates the learner's whole body movement. There is also a need for a dance learning device that accurately displays and evaluates each person's whole body movement even when a plurality of learners perform at the same time. Furthermore, it is preferable that new songs can be practiced one after another without the need to purchase dedicated hardware and software. The present invention aims to solve the above problems.

The present inventors have studied the means for solving the above problems and have completed the following invention.
[1] Music means, character data and storage means for storing motion data, sensor means for recognizing movement of each part of the learner's body, display means capable of displaying both the model dance by the character and the learner's dance, music A dance learning apparatus comprising: a sound means, a communication means for receiving each data from a server via a communication line, and transmitting a learner's dance as necessary; an input / output means; and a control means for these. .
[2] The sensor means for recognizing the learner has a function for detecting and recognizing the movement of each part of the learner's body and a depth sensor, and this enables the movement of each part of the learner's body to be recognized in three dimensions The dance learning apparatus according to [1].
[3] The dance learning apparatus according to [1] or [2], further including a photographing unit that photographs the learner's dance.
[4] The dance according to [3], further including a floor and a wall or a column surrounding the floor as a dance space for the learner, and a plurality of photographing means for photographing the learner's dance from a plurality of places as the photographing means. Teaching device.
[5] The learner or the character displayed on the display means is a learner or character by three-dimensionally reconstructing the movement of each part of the learner's body recognized by the sensor means, whether it is an image taken by the photographing means The dance learning device according to any one of [1] to [4], wherein the dance learning device is an image displayed as a movement of the movement.
[6] Any one of [1] to [5] above, wherein a score is given by comparing the motion of each part of the learner's body recognized by the image captured by the image capturing unit or the sensor unit with the movement data of the model dance. The dance training device described in 1.
[7] A user image is an image captured by the image capturing means or displayed as a learner or character motion by three-dimensionally reconstructing the movement of each part of the learner's body recognized by the sensor means. The dance training device according to any one of [1] to [6], which can be registered in the server as a model video or a contest participation video.
[8] The dance learning device according to [7], wherein a member user can add a comment to the video registered in the server as the user video, the model video, or the contest participation video.

  It is possible to provide a dance practice means to practice while enjoying the dance.

  In order to achieve the above object, the following invention is disclosed.

  The dance learning apparatus of the present invention includes a storage unit, a sensor unit, a display unit, an acoustic unit, a communication unit, an input / output unit, and a control unit thereof, and is configured as follows.

  The storage means stores music data, character data, and motion data.

  The music data is typically a data string in which the waveform of a sound can be handled by a computer, such as PCM (Pulse Code Modulation) format, MP3 (MPEG Audio Layer-3) format, or Ogg Vorbis format. However, when the acoustic apparatus of the present invention includes an electronic musical instrument sound source, a command sequence for issuing a performance instruction to the electronic musical instrument, such as a MIDI (Musical Instrument Digital Interface) format, may be used as musical composition data.

  Character data is data of a character that performs a dance performance on the display means or is displayed in place of the learner. The character is an image drawn in CG and having a skeleton similar to the human body, such as a figure imitating a real man or woman, animals, robots, movies and novels, characters in anime and manga, vocaloids, and the like.

  The motion data is data in which the movement of each part of the body is recorded according to the progress of the music for each music. For example, three-dimensional coordinates indicating the main parts of the skeleton such as main joints and angle data of limbs, heads, fingertips, etc. extending from these to the body end are included.

  In addition to the above, the storage means may store various instructions, comment data to be described later, data for identifying the learner, and data for operating the apparatus such as billing information. Here, the instruction is, for example, a tutorial regarding the operation of the apparatus, instruction information for guiding a learner's dance operation corresponding to each piece of music data, or the like.

  The sensor means recognizes the movement of each part of the learner's body and includes, for example, an infrared sensor, an ultrasonic sensor, a visible light sensor, and the like. When sensors are provided on the floor, pressure sensors are included. The sensor means here may include means for analyzing the information obtained from the sensor as described above regardless of its name, but may be configured to analyze the data obtained by the sensor means by the control means described later. Good.

  The sensor means first recognizes the learner. This is done by the learner grasping the body image by erecting or moving in front of the sensor means. The body image is grasped by using a conventional image recognition method. For example, the human body model is constructed by acquiring coordinates of the head, both shoulders, elbow joint, wrist joint, finger joint, waist, knee joint, ankle joint, and the like. Secondly, the sensor means grasps the movement of the learner as the movement of each part, and detects and identifies the movement of each part of the learner's body during the dance operation (hereinafter also referred to as “tracking”). At this time, it is desirable to estimate the distance between the sensor means and the learner using a depth sensor. The configuration of the depth sensor is not particularly limited, but by irradiating an infrared point pattern and photographing it, the shape change of the target is acquired from the pattern change, including the depth (depth of each point) A method of acquiring a “distance image” that is an image is mentioned. The movement of the learner during the dance is determined by using each of the three-dimensional parts so that kinematic constraints and temporal consistency are maintained using the detection data obtained by the sensor at regular intervals. Are extracted, a plurality of joint connection hypothesis models are constructed, and finally the motion of an actual human skeleton is estimated. Examples of such sensor means include Kinect, but other or future developed sensor means are also included as long as the movement of the learner can be tracked over time and three-dimensionally.

  The sensor means can recognize multiple learners and track their movements by constructing a skeleton model for each learner. Individual tracking is difficult. Therefore, the learner's dance space includes a floor and a wall or a column surrounding the floor, and the photographing means includes a plurality of photographing means for photographing the learner's dance from a plurality of places, thereby adding auxiliary data other than the depth sensor. It is preferable. As described above, a pressure sensor may be provided on the floor. The floor can also be matte and movable.

  In addition, the sensor unit itself may be an imaging unit, but an imaging unit may be provided separately from the sensor unit. An example of such photographing means is a digital video camera provided with a CCD or the like.

  The display means is a display device that can display both the model dance by the character and the learner's dance, and examples include a liquid crystal display demonstration, a plasma display, and other flat displays. Character model dances and learner dances can be displayed in any form. Typically, both are displayed side by side on the left and right or top and bottom, but either one may be displayed in a wipe or a plurality may be displayed. For the model dance, one or more movements to come next may be displayed in advance.

  The display may be 3D video or AR video.

  The acoustic means is a device that presents music as sound, and is typically a speaker or headphones. When the music is stored as MIDI data, an electronic musical instrument sound source can be used as the acoustic means.

  The communication means is for downloading the above-mentioned various data from the server or uploading a learner's dance video to the server, and usually communicates through the Internet. In addition to these, it is also used for communication such as personal recognition and billing information. In the present invention, the server refers to one or more computers that are capable of storing and transmitting / receiving data installed on a communicable network such as the Internet, and does not indicate a specific computer.

  The input / output means is an input / output interface other than the above-described sensor means, photographing means, display means, and communication line for the apparatus of the present invention, and includes, for example, output to a printer and input from a keyboard. Preferably, it is an input / output interface for accepting a remote control operation from a portable terminal such as a dedicated remote controller, a mobile phone, a smartphone, or a tablet. Such a remote control operation is preferably realized by an intuitive icon button or the like in the case of a dedicated remote control, and by an application or the like having an intuitive icon in the case of a mobile terminal.

  The control means controls the above-described series of devices, and is typically a computer integrated with the storage means communication means and the input / output means. However, the combination of the control means and other means is not particularly limited.

The operation of the apparatus of the present invention by the control means will be described below.
First, the apparatus of the present invention can operate in a member mode and a guest mode. The member mode is a demonstration by a pre-registered learner, and the guest mode is a mode for a learner who performs temporarily.

  The control means basically accepts the member mode and guest mode by remote control operation or the like. In member mode, request login data such as membership number and password. When login data is received, authentication is performed (accessing to an authentication server as necessary), and thereafter, operations are performed in association with a specific member. In guest mode, there is no login.

  In any case, when a signal for starting the practice means (generally by the above-described remote control operation) is received, reproducible music data is retrieved from the storage means and output to the display means. The learner selects a desired one from them, or performs a search operation (also generally by the above-described remote control operation). If there is no desired dance song as a result of the search, the server connects to the server through the communication means, searches the server for a dance song, downloads data if there is data, and displays that information if there is not. indicate.

  In the present invention, the instruction information (model dance) for guiding the learner's dance movement and the learner's own video can be displayed by a specific character, but the learner selects such a character at the setting stage. To do. The model dance and the virtual learner character may be the same or different. Here, the virtual learner is reconstructed with the character information generated by the control means on the display means by the character selected by the learner (each learner if there are multiple learners). Say learner. The virtual learner can select the learner himself and includes various characters as described above. Furthermore, the learner himself can decide on changes in gender, height, clothing, and accessories. These pieces of information are stored in the storage means in the member mode or can be received via the server, and based on the information, a certain character can always be reproduced regardless of the dance practice means to be used. Similarly, as for the demonstration screen, the learner can select various stages such as on stage, in the city, on the water, and in the desert.

  The dance learning apparatus of the present invention has a mode for receiving a dance of a plurality of learners and a mode for receiving a dance of one learner. Also, the type of model dance (when there are a plurality of model dance data for one music piece), the difficulty level of dance evaluation, the speed of music, and other settings are made.

  Thus, when the learner's desired dance music is found and the above settings are completed, this is reproduced, and screen data (character) corresponding to the music data and with instruction information for guiding the learner's dance operation is provided. Is output to the display means. That is, the control means sequentially reads the acoustic data from the storage means (typically from the beginning), expands the compressed acoustic data if necessary, or interprets it as a command, obtains waveform data, This is output from acoustic means such as a speaker. The sound waveform reproduced here is a dance song given to the learner in this practice means.

  When the mode for receiving the dance of one learner is selected, the learner's real image by the photographing means and the virtual learner's dance data generated by the control means are output to the display means. At this time, it may be displayed as a dance operation of a plurality of learners (dance mates). When the multiple learner mode is selected, the screen display includes a plurality of virtual learners (multiple learners reconstructed by characters) information corresponding to the plurality of learners. But a control means may select the character of one learner from several learners, and may output information only about the said character for every time.

  While the learner is dancing, the sensor means tracks the movement of the learner and displays it on the display means as a virtual learner image. Further, when there is another photographing means, the learner's movement may be displayed as a real image.

  The control means compares the model dance of the character and the dance of the learner for each point. The comparison point is, for example, a movement that becomes a dance point among the movements of the step and the upper body. These comparison points are added if they match, or deducted if they do not match. Furthermore, a special point may be provided to add points for a break in movement or a complicated series of movements. It is desirable to display these added points and deducted points in real time during the demonstration with characters such as “Excellent!”, “Good!” And “Bad!” At the corresponding timing. Further, an evaluation sound such as a chime may be generated in accordance with these display timings.

  Finally, the dance data created based on the music data and the action data is compared with the learner's dance data, and the learner's dance action evaluation data is created, and the evaluation data (score) is displayed. The comment is output to the means, and a comment corresponding to the evaluation data is output to the display means. In a plurality of learner modes, comments and scores are displayed for each learner. When dances of a plurality of learners occur at the same timing and in the same direction during a predetermined period, the overall evaluation may be performed as the synchronization rate. Similarly, for a song that is originally danced by a plurality of dancers, the overall evaluation as a dance team may be performed by comparing with the movements of each dancer.

  In addition, after the dance is finished, the control means re-develops the learner or model dance video based on the learner's operation, or displays it from a different angle, panning, zooming, etc. Video editing such as up can also be performed.

  The dance learning apparatus has a mode selection function according to the level, and the control unit may change the determination data that predefines the operation of a predetermined portion of the music data in accordance with the selection. The learner's dance result (score) may be stored in the storage means or the server, and the comment may be output to the display means based on the history information of the learner's dance operation for a predetermined period. The ranking may be displayed based on the registered data or the regional or national data registered in the server.

  In the dance learning apparatus of the present invention, the actual video of the learner's dance and the dance video of the virtual learner (the learner reconstructed by the character) can be registered in the server, preferably as a user video, a model video or a contest participation video. is there. For example, the user video can be displayed on a moving image site or the like, or can be mutually viewed in a predetermined circle. The model video is basically the same, except that the data can be registered as a model dance of the dance practice means itself. By configuring in this way, the learner not only performs a predetermined dance performance according to the music data, but also the learner himself dances freely according to the music, You can propose your own dance (choreography). In addition, the contest participation video is for competing for superiority or inferiority for certain task dances and free dances. Or you may comprise so that the dance practice means connected with the internet may play against other learners and teams connected to the internet. With this configuration, more interactive dance practice means are possible.

  In addition, for these videos, a certain member may be able to post cheers, applause, comments, etc. on the performance of other members. Therefore, the present invention also provides an SNS centered on dance images.

  In addition, the dance learning apparatus of this invention may be comprised as a home dance learning apparatus, and may be comprised as an arcade practice means apparatus. Depending on these instrument configurations, the apparatus of the present invention can include conventional variations and modifications. For example, in the case of a home dance training device, when the learner turns on the power of the home practice means, the present invention receives the reading of the recording medium (for example, DVD) loaded therein, and the control means starts processing. In the arcade practice means device, when the learner inserts a predetermined number of coins from the coin slot and the coin confirmation device determines that these coins are authentic, the coin confirmation device practices the control means. A means start signal is output, and in response to this, the control means starts processing.

The dance learning apparatus of the present invention can be installed at home, at a practice means center, or in a dance class, and can be widely applied.

The beginning of the form

Claims (8)

Storage means for storing song data, character data and its movement data, sensor means for recognizing movements of each part of the learner's body, display means for displaying both model dance and learner's dance, and music as sound A dance learning apparatus comprising: communication means for presenting the data received from the server via an acoustic means and a communication line, an input / output means for transmitting the learner's dance as necessary, and a control means for these. The sensor means for recognizing the learner has a function of detecting and recognizing movements of each part of the learner's body and a depth sensor, whereby the movement of each part of the learner's body can be recognized three-dimensionally. The dance learning apparatus according to 1. The dance learning apparatus according to claim 1, further comprising a photographing unit that photographs the learner's dance. The dance learning apparatus according to claim 3, further comprising a floor and a wall or a column surrounding the floor as a learner's dance space, and a plurality of photographing means for photographing the learner's dance from a plurality of locations as the photographing means. The learner displayed on the display means is a picture taken by the photographing means or the movement of each part of the learner's body recognized by the sensor means is three-dimensionally reconstructed as a learner or character movement. The dance learning apparatus according to claim 1, wherein the dance learning apparatus is a displayed image. The dance learning apparatus according to claim 1, wherein a score is given by comparing the motion of each part of the learner's body recognized by the image captured by the imaging unit or the movement of the model with the movement data of the model dance. . A user image, a model image, which is a video taken by the photographing means or three-dimensionally reconstructed the movement of each part of the learner's body recognized by the sensor means and displayed as the movement of the learner or character Or the dance training apparatus in any one of Claims 1-6 which can be registered into a server as a contest participation video. The dance learning device according to claim 7, wherein a member user can make a comment on the video registered in the server as the user video, the model video, or the contest participation video.