JP2017181858A - Evaluation control device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To comparatively evaluate each of singing users by using the positions of bodies of the singing users who cannot learn the singing state of each other than singing voice and the position of a body of a comparison object person.SOLUTION: An evaluation control device determines that a first user is detected S10 when positional information on a user having first posture is received from a state detection device. Movement of body parts indicated by first bone information indicates movement of body parts of the user or movement of body parts of an avatar. Movement of body parts indicated by second bone information indicates movement of body parts of a comparison object person. The evaluation control device calculates sum and difference of an evaluation value evaluating movement of body parts indicated by the calculated first bone information and an evaluation value evaluating movement of body parts indicated by the second bone information S32, and outputs the calculated results on a display.SELECTED DRAWING: Figure 3

Description

  The present invention relates to an evaluation control apparatus that detects a user's movement and evaluates the compatibility of the user's movement by comparing the movement with a predetermined comparison person's movement.

  Conventionally, the singing voice of multiple ugly hands is obtained, and the content of the judgment of the hunting hand is judged from the difference in pitch and rhythm of the craving hand obtained from the voice and the difference in volume. A karaoke device for display has been devised.

JP 2007-156330 A

  When the technique described in Patent Document 1 is used, the scolder can know the compatibility of the singing voice among a plurality of scoops after the singing is finished. It can be considered that the compatibility of a plurality of ugly hands can be determined not only by the singing voice compatibility but also by the relative position of the body when singing the ugly hand. Since each of the ugly hands concentrates on the singing during the singing, the relative positions at the time of singing cannot be recognized, and the state during the singing other than the singing voice cannot be known.

  The present invention has been made to solve the above-described problems, and relative evaluation of a user who is singing using the position of the body of the user who is singing and the position of the body of the person to be compared is performed. The purpose is to do.

  The evaluation control apparatus according to claim 1, wherein an identification unit that identifies a user in a predetermined space as an observation target, and a specific body of the observation target that is identified by the identification unit during reproduction of a predetermined music piece The position acquisition information is obtained by associating the position information of the part with the time information indicating the reproduction time of the music and the time information from the reproduction start time to the reproduction end time of the music. Acquired by the observation acquisition means, the comparison acquisition means for acquiring the position information of the specific body part of a predetermined comparison target person different from the observation target person from which the position information was acquired by the observation acquisition means. In addition, the position information of the observation subject and the position information of the comparison subject read by the comparison and reading unit are started to reproduce the music corresponding to the respective time information. The relative distance of the specific body part from the comparison target person and the direction approaching the comparison target person are positive for the observation target person at the same timing from the end to the reproduction end time. A determination unit configured to determine at least one of a direction and an inclination of the specific body part; and a result of evaluating the movement of the observation target during reproduction of the music in accordance with a determination result by the determination unit. And an output control means.

  The evaluation control apparatus according to claim 2, wherein the position of the observation target acquired by the detection means that detects the playback start time and the playback end time of the music played in the predetermined space, and the observation acquisition means Storage means for storing information in a storage unit in association with time information indicating the time when the position information was acquired, and the observation acquisition means is a music reproduction start time detected by the detection means The position information of the person to be observed associated with the time information corresponding to the reproduction end time is read out from the storage unit, and the comparison and acquisition unit is configured to obtain the reproduction start time of the music piece. Position information associated with the time information from the reproduction end time to the reproduction end time, and a predetermined comparison object different from the observation target person from which the position information was read by the reading means The position information of the specific body part is read out from the storage unit and acquired, and the observation indicated by the position information from the reproduction start time to the reproduction end time of the music acquired by the observation acquisition means First determination means for determining the movement transition of the target person, and determination of the movement transition of the comparison target indicated by the position information from the reproduction start time to the reproduction end time of the music acquired by the comparison acquisition means. Second determination means, wherein the determination means includes the movement transition of the observation target determined by the first determination means and the movement transition of the comparison target determined by the second determination means. The time information is compared at a predetermined timing from the playback start time to the playback end time of the music corresponding to the time information. A distance transition of the relative distance of the specific body part with the comparison target person, and a direction transition of at least one of the direction and inclination of the specific body part with the direction approaching the comparison target person being positive. The output control unit determines a result of evaluating the movement of the observation target person from the reproduction start time to the reproduction end time of the music according to the determination result of the distance transition and the direction transition by the determination unit. It is characterized by outputting.

  The evaluation control apparatus according to claim 3, wherein the reflected person image reflects the position of the specific body part of the observation subject according to the position information of the specific body part of the observation subject acquired by the observation acquisition unit. Is displayed on a predetermined display unit, and the determination unit uses the position information of the reflected human image displayed on the display unit as the position information of the observation subject. It compares with the said positional information on a subject, It is characterized by the above-mentioned.

  The evaluation control apparatus according to claim 4, wherein the comparison and acquisition unit uses the virtual person image displayed on the display unit provided in the space as the comparison target person and the virtual person on the display unit in the space. Position information of the specific body part of the image is acquired, and the determination means includes the position information of the observation target person from a predetermined reference position and the display unit on the display unit from the predetermined reference position. The position information of the virtual person image is compared to determine the relative distance and at least one of the direction and the inclination.

  The evaluation control apparatus according to claim 5, wherein the determination unit is displayed on the position information of the reflected person image displayed on the display unit and the display unit acquired by the comparison acquisition unit. Comparing the position information of the virtual person image and determining the relative distance of the reflected person image on the display unit from the virtual person image and at least one of the orientation and the inclination. Features.

The evaluation control apparatus according to claim 6, wherein the identification unit identifies the first user in the predetermined space as the observation target and the second user in the predetermined space as the comparison target person. The observation acquisition unit acquires position information of a specific body part of the first user identified as the observation target by the identification unit, and the comparison acquisition unit acquires the comparison by the identification unit. The position information of the specific body part of the second user identified as the target person is acquired, and the determination unit obtains the position information of the first user and the position information of the second user. The first user at the same timing from the reproduction start time to the reproduction end time of the music, and the relative distance of the specific body part with the second user at the compared timing, At least one of the direction and inclination of the specific body part with the direction approaching the user as positive is determined as the determination result of the first user, and the first user is compared with the first The relative distance of the specific body part to the user and at least one of the direction and inclination of the specific body part with the direction approaching the first user as positive are determined as the determination result of the second user. The output control means outputs the result of evaluating the relative relationship between the observation subject and the comparison subject from the reproduction start time to the reproduction end time of the music.

  The evaluation control program according to claim 7 is a program executed by a computer, wherein an identification step of identifying a user in a predetermined space as an observation target, and the observation target identified by the identification step An observation acquisition step for acquiring position information of a specific body part is associated with the time information from the reproduction start time to the reproduction end time of the music, and the position information is obtained by the observation acquisition step. A comparison acquisition step of acquiring position information of the specific body part of a predetermined comparison target person different from the acquired observation target person, and the position information of the observation target person acquired by the observation acquisition step; The position information of the person to be compared read by the comparison reading means is the music corresponding to the time information. For the observation target person at the same timing from the reproduction start time to the reproduction end time, the relative distance of the specific body part from the comparison target person and the direction approaching the comparison target person. A determination step of determining at least one of the direction and inclination of the specific body part to be positive, and depending on a determination result by the determination step, the observation target person from the reproduction start time to the reproduction end time of the music piece And an output control step for outputting a result of evaluating the movement.

  According to the evaluation control apparatus of claim 1, the determination unit reproduces the position information of the observation target acquired by the observation acquisition unit and the position information of the comparison target acquired by the comparison acquisition unit and the reproduction of the music from the start of reproduction. By comparing until the end and determining the observation target person with respect to the comparison target person, it is possible to evaluate the movement of the observation target person relative to the comparison target person during reproduction of the music.

  According to the evaluation control apparatus of claim 2, the movement transition of the observation target person during reproduction of the music determined by the first determination means is compared with the comparison target during reproduction of the music determined by the second determination means by the determination means. The movement transition of the observation target relative to the movement transition of the comparison target during the reproduction of the music is determined by comparing the movement transition of the observation target with respect to the movement transition of the comparison target person. Can be evaluated.

  According to the evaluation control device according to claim 3, by using the position information of the reflected person image displayed by the display control means as the position information of the observation target person, by comparing the position information of the comparison target person, For the comparison target person, it is possible to evaluate the movement of the observation target person using the position information of the reflected person image instead of the position information of the observation target person.

  According to the evaluation control apparatus of the fourth aspect, the comparison target person is a virtual person by comparing the position information of the observation target person with the position information of the virtual person image displayed on the display unit as the comparison target person. Even in some cases, it is possible to evaluate the movement of the observation subject relative to the comparison subject.

  According to the evaluation control device of claim 5, the position information of the reflected person image and the position information of the virtual person image are compared and determined, and the observation is performed by the person image displayed on the display unit. It is possible to evaluate the movement of the subject.

  According to the evaluation control apparatus according to claim 6, the first user and the second user are respectively observed by comparing and determining the position information of the first user and the second user in the same space. It is possible to evaluate the relative relationship of movement between the target user and the comparison target user.

  According to the program according to claim 7, from the determination step, the position information of the person to be observed acquired by the observation acquisition step, the position information of the person to be compared acquired by the comparison acquisition step, and the reproduction start to the reproduction end of the music The movement of the observation target with respect to the comparison target during the reproduction of the music can be evaluated by comparing the observation target and determining the observation target with respect to the comparison target.

It is a schematic diagram which shows the usage example of the evaluation control apparatus 1 of this embodiment. 2 is an explanatory diagram showing an outline of an electrical configuration of an evaluation control device 1. FIG. 3 is a flowchart showing a processing procedure of evaluation control processing executed by a CPU 11 of the evaluation control device 1. It is a figure which shows the process sequence of the sub flowchart of an evaluation control process. It is a figure which shows the criterion of the determination process performed in an evaluation control process. It is a figure explaining the structure of the memory content when the determination result by a determination process is memorize | stored.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1A to FIG. 1D are schematic diagrams illustrating a usage example of an evaluation processing apparatus 1 which is an embodiment of the evaluation processing apparatus of the present invention. The evaluation processing apparatus 1 is installed and used in a room such as a karaoke room, for example. As shown in FIGS. 1 (A) to 1 (D), when the evaluation processing device 1 is used, a state detection device 2, a karaoke device 3, a display 5, and a speaker 6 are provided together with the evaluation processing device 1. The evaluation processing device 1 is electrically connected to the state detection device 2, the karaoke device 3, and the display 5. The speaker 6 is electrically connected to the karaoke apparatus 3. In this embodiment, the display 5 shows an example provided on one wall in the room, but the configuration of the display 5 is not limited to this. Instead of the display 5, a projector may be provided, and an image may be projected onto a projection screen installed on the wall or by the projector. The speaker 6 may not be provided.

  The evaluation processing device 1 is used by at least one user. FIG. 1A is a diagram illustrating a first example in which the evaluation processing device 1 is used by two users, a user U1 and a user U2. FIG. 1B is a diagram illustrating a second example in which the evaluation processing device 1 is used by two users, the user U1 and the user U2. In the second example, the positions of the body parts of the user U1 and the user U2 are detected by the state detection device 2, and an avatar A1 that reflects the position of the body part of the user U1 and an avatar A2 that reflects the position of the body part of the user U2 Shows an example displayed on the display 5.

  FIGS. 1C and 1D are a third example and a fourth example in which the evaluation processing apparatus 1 is used by one user U1. In the third example and the fourth example, there is no user U2 who was in the first example and the second example, or the position of the body part of the user U2 detected in the first example and the second example is not detected. In the third example shown in FIG. 1C and the fourth example shown in FIG. 1D, the display 5 instead of the avatar A2 reflecting the position of the body part of the user U2 shown in the second example, A virtual person V2 is displayed. The movement of the body part of the virtual person V2 is stored in advance in the evaluation processing device 1 as described later. In the fourth example shown in FIG. 1D, the position of the body part of the user U1 is detected by the state detection device 2, and the avatar A1 reflecting the position of the body part of the user U1 is displayed on the display 5. For example, when a karaoke piece is reproduced by the karaoke device 3, the user U <b> 1 and the user U <b> 2 sing according to the performance sound of the karaoke piece outputted from the speaker 6.

[Electrical Configuration of Evaluation Processing Apparatus 1]
FIG. 2 is a block diagram showing an outline of an electrical configuration of the evaluation processing apparatus 1 and an electrical connection with an external device electrically connected to the evaluation processing apparatus 1. Examples of external devices that are electrically connected to the evaluation processing device 1 include a state detection device 2, a karaoke device 3, and a display 5. The evaluation processing apparatus 1 includes a CPU 11, an information receiving unit 12, a ROM 13, and a RAM 14. In the CPU 11, the information receiving unit 12, the ROM 13, the RAM 14, the display 5, and the karaoke device 3 are electrically connected. The information receiving unit 12 is electrically connected to the state detection device 2 of the external device. The state detection device 2 detects the position of the user with a sensor such as an infrared sensor or a depth sensor, for example. The state detection device 2 stores a variety of basic bone information in which the positional relationship between a plurality of body parts is determined in advance, and identifies the body part corresponding to the position detected by the sensor by referring to the basic bone information. It is possible. The state detection device 2 associates coordinate information including x coordinates, y coordinates, and z coordinates as position information indicating the position of the user's body part with the body part indicated by the coordinate information, and receives the information receiving unit 12. Notify In the present embodiment, the head, neck, right shoulder, and left shoulder are used as examples of body parts. Instead of the neck, the center of both shoulders may be defined as the neck. In the present embodiment, the x coordinate, the y coordinate, and the z coordinate are coordinates indicating each direction shown in FIG. Furthermore, in the present embodiment, when the evaluation processing device 1 has a function capable of displaying a user image on the display 5, the state detection device 2 desirably has a camera function. At this time, the state detection device 2 sequentially transmits captured images by the camera function to the evaluation processing device 1. The information receiving unit 12 receives the position information of the user's body part detected by the state detection device 2 from the state detection device 2.

  In addition to the evaluation processing device 1, an input device 4 and a speaker 6 are electrically connected to the karaoke device 3. The input device 4 receives, for example, designation of karaoke music to be played back by the karaoke device 3 from the user. The karaoke device 3 reproduces the karaoke music specified by the input device 4 and outputs a performance sound from the speaker 6. The karaoke device 3 notifies the evaluation processing device 1 of the start of reproduction and the end of reproduction when starting reproduction of karaoke music and ending reproduction of karaoke music. The input device 4 may be configured integrally with the karaoke device 3. On the display 5, lyrics of karaoke music reproduced by the karaoke device 3 and results of processing performed by the evaluation processing device 1 are output and displayed.

  The ROM 13 of the evaluation processing apparatus 1 includes an evaluation control program storage area 131, a determination condition storage area 132, a user reflection avatar information storage area 133, and a virtual person information storage area 134. The evaluation control program storage area 131 stores an evaluation control program executed by the CPU 11. The evaluation control device 1 executes an evaluation control process to be described later by executing an evaluation control program by the CPU 11. The determination condition storage area 132 stores determination conditions used for the determination process in the evaluation control process. Details of the determination condition will be described in detail in an evaluation control process described later. The determination process stored in the determination condition storage area 132 is used for a first bone determination process and a second bone determination process described later.

  The user reflection avatar information storage area 133 stores user reflection avatar information. The user reflected avatar information is information for displaying on the display 5 an avatar image reflecting the position of the body part of the user U1, as shown in an avatar A1 in FIGS. 1B and 1D. The user reflected avatar information includes, for example, image data imitating a character or an animal and divided into parts corresponding to human body parts as part images. In the evaluation control process executed by the CPU 11, the evaluation control device 1 reproduces the body of the user U1 by combining the part images of the user reflected avatar information and detects the body part of the user U1 detected by the state detection device 2. An avatar image created by reflecting the position on the position of the part image that reproduces the body of the user U1 is displayed on the display 5.

  Virtual person information is stored in the virtual person information storage area 134. The virtual person information is information for displaying a virtual person image as shown by the virtual person V2 in FIGS. 1C and 1D on the display 5. A virtual person image is an image showing a virtual person that does not reflect a real user. In the virtual person image, the position information of the body part of the virtual person and the display timing are stored in association with each other. The display timing is, for example, information indicating an elapsed time from the start of display with the virtual person image as the virtual person V2. Thereby, the transition of the position of the body part of virtual person V2 which is a virtual person can be specified. CPU11 displays the virtual person image showing the transition of the position of a body part on the display 5 as the virtual person V2 along with the transition of time information. In the virtual person information storage area 143, a plurality of types of virtual person information may be stored.

  The RAM 14 includes a first bone information storage area 141, a second bone information storage area 142, a first determination information storage area 143, and a second determination information storage area 144. In the first bone information storage area 141 and the second bone information storage area 142, position information indicating the position of the user's body part acquired from the state detector 2 and time information indicating the acquired time are stored. Corresponding and stored as bone information. Position information indicating the position of the user's body part is periodically acquired from the state detection unit 2 a plurality of times. The acquired position information of a plurality of body parts is stored as bone information in the first bone information storage area 142 and the second bone information storage area 142 in association with time information.

  In the first bone information storage area 141, when the CPU 11 executes the evaluation control process, for example, the position information indicating the position of the body part of the user U1 shown in FIG. 1A and the position are acquired. Are associated with each other and stored as first bone information. Alternatively, as shown in FIGS. 1B and 1D, when the avatar A1 reflecting the position of the body part of the user U1 is displayed on the display 5, the first bone information accumulation storage area 141 has an avatar. On the display 5 on which the part image of A1 is displayed, information indicating the display position of the body part included in the part image is associated with time information when the part image is displayed on the display 5, It may be stored as 1-bone information.

  When the CPU 11 executes the evaluation control process in the second bone information storage area 142, for example, the position information indicating the position of the body part of the user U2 shown in FIG. 1A and the position are acquired. Are associated with each other and stored as second bone information. Alternatively, as shown in FIG. 1B, when the avatar A2 reflecting the position of the body part of the user U2 is displayed on the display 5, the part image of the avatar A2 is displayed in the second bone information storage area 142. Information indicating the display position of the body part included in the part image is associated with time information when the part image is displayed on the display 5 and stored as second bone information. May be. Alternatively, as shown in FIG. 1C and FIG. 1D, when the virtual person V2 is displayed on the display 5, the virtual corresponding to the virtual person image is stored in the second bone information storage area 142. Information indicating the display position on the display 5 on which the virtual person image indicating the virtual person V2 is displayed, and the virtual person image is displayed on the display 5, which is a display position determined from the position information of the body part of the person. Time information may be associated with each other and stored as second bone information.

  In the first determination information storage area 143 and the second determination information storage area 144, determination results determined by the first bone determination process and the second bone determination process of the evaluation control process executed by the CPU 11 are stored. Details of the determination result will be described later. In the first determination information storage area 143, the determination result determined by the first bone determination process is stored. In the second determination information storage area 144, the determination result determined by the second bone determination process is stored.

[Description of Operation of Evaluation Control Device 1]
The operation and action of the evaluation control apparatus 1 of the present embodiment will be described with reference to FIGS. 3 and 4A to 4C are flowcharts showing the processing procedure of the operation of the evaluation control device 1. FIG. FIG. 3 is a flowchart showing the processing procedure of the evaluation control process of the evaluation control program executed by the CPU 11 of the evaluation control device 1. 4 (A), 4 (B), and 4 (C) are sub views showing the processing procedures of the user detection process, the first bone determination process, and the second bone determination process, respectively, included in the evaluation control program in the evaluation control process. It is a flowchart. FIG. 5A to FIG. 5C are explanatory diagrams for explaining determination conditions used in the first bone determination process shown in FIG. 4B and the second bone determination process shown in FIG. 4C. . Each determination condition shown in FIGS. 5A to 5C is stored in the determination condition storage area 132 of the ROM 13. FIGS. 6A to 6B are diagrams illustrating a determination result table in which the determination results determined by the first bone determination process and the second bone determination process are stored. FIG. 6C is a diagram illustrating a calculation result table in which the ratio calculation is performed from the determination result for the first bone information or the second bone information in the evaluation control process. A determination result table storing the determination result determined by the first bone determination process and a calculation result table corresponding to the first bone information are stored in the first determination information storage area 143 of the RAM 14. The determination result table storing the determination result determined by the second bone determination process and the calculation result table corresponding to the second bone information are stored in the second determination information storage area 144 of the RAM 14.

  The procedure of the evaluation control process will be described with reference to FIG. The evaluation control process shown in FIG. 3 is started when the evaluation control program is executed by the CPU 11 when the power supply of the evaluation control apparatus 1 is turned on.

  When the evaluation control apparatus 1 is powered on, step S10 is executed. In step S10, the user detection process in steps S102 to S124 shown in FIG. 4A is executed. In step S102 of the user detection process shown in FIG. 4A, it is determined whether or not the first user has been detected. In step S102, for example, when the position information of the user who has performed the predetermined first preference is received from the state detection device 2, it is determined that the first user has been detected. When it is determined that the first user has been detected (S102: YES), after the determination is made, the determined user is identified as the user U1, and the user U1 is continuously detected based on the position information of the body part. Is called. When it is not determined that the first user has been detected (S102: NO), the process of step S102 is repeatedly executed.

  In step S104, it is determined whether or not an image indicating the user U1 has been set to be displayed on the display 5. In step S104, for example, the user U1 specified in step S102 receives the user position information indicating that the user U1 has performed the predetermined second preference, or information indicating an instruction to display an image by the user U1, thereby receiving the user U1. It is determined that the setting for displaying the image indicating is on the display 5 has been made. Alternatively, it may be determined by acquiring setting information set in advance by any user using the evaluation control device 1 or setting information set according to the presence or absence of the camera function of the evaluation control device 1. Good. When it is determined that the image indicating the user U1 is set to be displayed on the display 5 (S104: YES), the process of step S106 is executed. When it is not determined that the image indicating the user U1 is set to be displayed on the display 5 (S104: NO), the process of step S112 is executed.

  In step S106, it is determined whether or not an avatar image reflecting the position of the body part of the user U1 is set to be displayed on the display 5. In step S106, for example, the user U1 specified in step S102 receives the user position information indicating that the user U1 has performed the predetermined third appearance, or information indicating an instruction to display an avatar image by the user U1, thereby receiving the user. It is determined that the avatar image reflecting the position of the body part of U1 is set to be displayed on the display 5. Alternatively, it may be determined by acquiring setting information set in advance by any user using the evaluation control device 1 or setting information set according to the function of the evaluation control device 1. When it is determined that the avatar image reflecting the position of the body part of the user U1 is set to be displayed on the display 5 (S106: YES), the process of step S110 is executed. When it is not determined that the avatar image that reflects the position of the body part of the user U1 is displayed on the display 5 (S106: NO), the process of step S108 is executed.

  In step S108, a process of displaying the image of the user U1 captured by the camera function of the state detection device 2 on the display 5 as the image of the first user is performed. By executing step S108, the image of the first user displayed on the display 5 is updated each time a captured image is received from the state detection device 2. In step S110, the part image corresponding to each body part of the position information of the user U1 received from the state detection device 2 is read from the user reflection avatar information storage area 133 of the ROM 13, and each position information received from the state detection device 2 is read. An avatar image created by reflecting the coordinate information associated with the body part is displayed on the display 5.

  In step S112, it is determined whether or not a setting for displaying a virtual person image as shown by the virtual person V2 in FIGS. 1C and 1D on the display 5 is made. In step S112, for example, by receiving information indicating an instruction to display the virtual person image on the display 5 by the user U1, it is determined that the setting for displaying the virtual person image on the display 5 has been made. Alternatively, it may be determined by acquiring setting information set in advance by any user using the evaluation control device 1 or setting information set according to the function of the evaluation control device 1. When it is determined that the virtual person image is set to be displayed on the display 5 (step S112: YES), the process of step S114 is executed. When it is not determined that the virtual person image is set to be displayed on the display 5 (step S112: NO), the process of step S116 is executed. In step S114, the virtual person image stored in the virtual person information storage area 134 of the ROM 13 is displayed on the display 5 according to the associated time information.

  In step S116, it is determined whether or not a second user has been detected. In step S116, when it is determined that the first user is detected and the detection of the user U1 detected as the first user is continued, the position information of the user who has a predetermined first preference different from the user U1. Is newly received from the state detection device 2, it is determined that the second user has been detected. When it is determined that the second user has been detected (S116: YES), after the determination, the determined new user is identified as the user U2, and the detection of the user U2 based on the position information of the body part is continued. Done. When it is not determined that the second user has been detected (S102: NO), the process of step S116 is repeatedly executed.

  In step S118, it is determined whether or not an image indicating the user U2 has been set to be displayed on the display 5. In step S118, for example, by receiving the user's location information indicating that the user U2 specified in step S116 has performed the predetermined second preference, or information indicating an instruction to display an image by the user U2, the user U2 It is determined that the setting for displaying the image indicating is on the display 5 has been made. Alternatively, it may be determined by acquiring setting information set in advance by any user using the evaluation control device 1 or setting information set according to the presence or absence of the camera function of the evaluation control device 1. Good. When it is determined that the image indicating the user U2 is set to be displayed on the display 5 (S118: YES), the process of step S118 is executed. When it is not determined that an image indicating the user U1 is set to be displayed on the display 5 (S118: NO), the user detection process is terminated.

  In step S120, it is determined whether or not the avatar image reflecting the position of the body part of the user U2 is set to be displayed on the display 5. In step S120, for example, the user U2 specified in step S116 receives the user's position information indicating that the user has performed the predetermined third appearance, or information indicating an instruction to display an avatar image by the user U2, thereby receiving the user. It is determined that the avatar image reflecting the position of the body part of U2 is set to be displayed on the display 5. Alternatively, it may be determined by acquiring setting information set in advance by any user using the evaluation control device 1 or setting information set according to the function of the evaluation control device 1. When it is determined that the avatar image reflecting the position of the body part of the user U2 is set to be displayed on the display 5 (S120: YES), the process of step S124 is executed. When it is not determined that the avatar image reflecting the position of the body part of the user U2 is set on the display 5 (S120: NO), the process of step S122 is executed.

  In step S122, the process of displaying the image of the user U2 captured by the camera function of the state detection device 2 on the display 5 as the image of the second user is performed. By executing step S122, the image of the second user displayed on the display 5 is updated every time a captured image is received from the state detection device 2. In step S124, according to the position information indicating the position of the user's body part transmitted from the state detection device 2, the position of the body part of the user U2 is stored in the position of the part image stored in the user reflected avatar information storage area 133 of the ROM 13. The avatar image created by reflecting the above is displayed on the display 5.

  When it is not determined in step S118 that the image indicating the user U1 is displayed on the display 5, the image of the user U2 is displayed on the display 5 in step S122, or the position of the body part of the user U2 is determined in step S124. When the avatar image created by reflecting the position of the part image is displayed on the display 5, the user detection process is terminated. When the user detection process is finished, the process of step S12 of the evaluation control process shown in FIG. 3 is executed.

  In step S12, it is determined whether or not the karaoke device 3 has started playing karaoke music. In step S12, when it is determined that the start of reproduction of karaoke music has been notified from the karaoke apparatus 3, it is determined that reproduction of karaoke music has been started (S12: YES), and the process of step S14 is executed. In step S12, when it is not determined that the karaoke device 3 has started the reproduction of karaoke music, it is not determined that the reproduction of karaoke music has been started (S12: NO), and the process of step S12 is repeatedly executed.

  In step S14, the position information of the user's body part detected by the state detection device 2 after it is determined in step S12 that the reproduction of the karaoke music has been started is stored in association with the acquired time information. Specifically, for the user U1 detected by the user detection process and identified as the first user, the position information received from the state detection device 2 after it is determined in step S12 that the reproduction of karaoke music has started. Coordinate information (x coordinate, y coordinate, z coordinate) indicating the position of each body part and any body part of a certain head, neck, right shoulder, left shoulder, and time information when the position information is acquired, The first bone information is stored in the first bone information storage area 141 as 1 bone information. Alternatively, when the avatar image shown in the avatar A1 created by reflecting the position of the body part of the user U1 is displayed on the display 5 like the avatar A1 in FIGS. Instead of the coordinate information of the position information received from the apparatus 2, the part image corresponding to each body part included in the avatar image created by the process of step S108 is displayed on the display indicating the position where the part image is displayed on the display 5. The coordinate information may be stored in the first bone information storage unit 141 as the first bone information in association with the time information. At this time, the time information is time information when the position information received from the state detection device 2 is acquired or time information when the created avatar image is displayed on the display 5.

  Furthermore, in step S14, for the user U2 identified as the second user by the user detection process, the head which is the positional information received from the state detection device 2 after it is determined in step S12 that the reproduction of karaoke music has been started. , The body part of any one of the neck, right shoulder, and left shoulder, and coordinate information (x coordinate, y coordinate, z coordinate) indicating the position of each body part, and time information when the position information is acquired, the second bone The information is stored in the second bone information storage area 142 as information. Alternatively, similarly to the first bone information, the avatar image shown in the avatar A1 created by reflecting the position of the body part of the user U1 is displayed on the display 5 like the avatar A1 in FIGS. 1B and 1D. If the part image corresponding to each body part included in the created avatar image is displayed on the display 5, the coordinate information on the display indicating the position displayed on the display 5 is associated with the time information, You may memorize | store in the 2nd bone information storage storage means 142 as bone information.

  As another example of the second bone information in step S14, the virtual person V2 shown in FIGS. 1C and 1D when the user detection process is set to display a virtual person image. Since it is determined in step S12 that the reproduction of the karaoke music has been started, the virtual person image indicating the virtual person V2 is stored in the virtual person information storage area 134 in association with the virtual person image when displayed on the display 5. The position information of the body part of the virtual person being displayed and the time information when the virtual person is displayed may be stored in the second bone information storage area 142 as second bone information. The time information when displayed may be calculated from the display timing stored in the virtual person information storage area 134 in association with the position information of the physical information.

  In step S16, it is determined whether or not the karaoke apparatus 3 has finished reproducing the karaoke music that has been reproduced in step S12. In step S16, when it is determined that the end of reproduction of the karaoke music has been notified from the karaoke apparatus 3, it is determined that the reproduction of the karaoke music has been completed (S16: YES), and the process of step S18 is executed. In step S18, when it is not determined that the karaoke song 3 has been notified of the end of reproduction of karaoke music, it is not determined that the reproduction of karaoke music has been completed (S16: NO), and the process of step S14 is repeatedly executed. In step S14, the first bone information and the second bone information are stored in the first bone information accumulation storage area 141 and the second bone information accumulation storage area 142, respectively, until it is determined in step S16 that the reproduction of the karaoke music piece is finished. It is memorized repeatedly.

  In step S18, a first bone determination process is performed. In step S20 following the first bone determination process in step S18, a second bone determination process is performed. In the first bone determination process, the processes in steps S182 to S194 shown in FIG. 4B are performed. In the first bone determination process, the first bone information from the start of playback to the end of playback of karaoke music stored in the first bone information storage area 141 is repeatedly read and determined. In the second bone determination process, the processes in steps S202 to S214 shown in FIG. 4C are performed. In the second bone determination process, the second bone information from the start of playback to the end of playback of the karaoke music stored in the second bone information storage area 142 is repeatedly read and determined. The processes of steps S182 to S186 of the first bone determination process and the processes of steps S202 to S206 of the second bone determination process are the same, and the processes of steps S188 to S194 of the first bone determination process are the same. Since the processing contents correspond to the processing of step S208 to step S214 of the second bone determination process, details of the first bone determination process and the second bone determination process will be described below in parallel. .

  In steps S182 and S202, the position information of the first bone information associated with earlier time information among the first bone information stored in the first bone information storage area 141 is read. In step S184, in step S204, time information that is the same as or close to the time information of the first bone information read in steps S182 and S202 among the second bone information stored in the second bone information storage area 141 corresponds. The position information of the attached second bone information is read out. In step S182 and step S202, position information about a plurality of body parts associated with the same time information is read for each piece of time information as first bone information and in step S184 and step S204 as second bone information. It is. For example, coordinate information indicating the positions of the head, neck, left shoulder, and right shoulder associated with the same time information is read as first bone information and second bone information.

  In the processing from step S186 to step S190 and the processing from step S206 to step S210, the determination processing is performed for the same first bone information and second bone information read in steps S182 and S184, and steps S202 and S204, respectively. Is done. In steps S186 to S190 and steps S206 to S210, the coordinate information of the first bone information read in steps S182 and S202 and the coordinates of the second bone information read in steps S184 and S208. A determination process is performed by comparing the information.

  In step S186 and step S206, the distance between the first bone information and the second bone information is determined according to the determination condition shown in FIG. FIG. 5A is a diagram illustrating the distance determination condition stored in the determination condition storage area 132 of the ROM 13 and a distance calculation formula corresponding to the condition. In steps S186 and S206, the coordinate information of the first bone information read in steps S182 and S202 is compared with the coordinate information of the second bone information read in steps S184 and S204, and FIG. The distance is determined according to the determination condition shown in (A).

  More specifically, in steps S186 and S206, the first bone information is A, the right shoulder x coordinate of the first bone information is the right shoulder Ax, the left shoulder x coordinate is the left shoulder Ax, and the right shoulder z The coordinates are the right shoulder Az, and the z coordinate of the left shoulder is the left shoulder Az. The second bone information is B, the right shoulder x coordinate of the second bone information is the right shoulder Bx, the left shoulder x coordinate is the left shoulder Bx, the right shoulder z coordinate is the right shoulder Bz, and the left shoulder z coordinate is the left shoulder Bz. And

  In step S186 and step S206, as shown in the distance condition of FIG. 5A, the left shoulder Ax and the left shoulder Bx are compared. If left shoulder Ax <left shoulder Bx, (left shoulder Bx, left shoulder Bz) to (right shoulder Ax) , Right shoulder Az) is determined as the determination distance. If left shoulder Ax> left shoulder Bx, a linear distance of a difference obtained by subtracting (right shoulder Bx, right shoulder Bz) from (left shoulder Ax, left shoulder Az) is determined as a determination distance.

  In step S188, the orientation of the first bone information with respect to the second bone information is determined. In step S208, the orientation of the second bone information with respect to the first bone information is determined. The determinations in step S188 and step S208 are performed using the determination conditions shown in FIG. FIG. 5B is a diagram showing the orientation determination conditions stored in the determination condition storage area 132 of the ROM 13. In steps S188 and S208, the coordinate information of the first bone information read in steps S182 and S202 is compared with the coordinate information of the second bone information read in steps S184 and S204, and FIG. The direction is determined according to the determination condition shown in (B).

  More specifically, in steps S188 and S208, the first bone information is A, the left shoulder x coordinate of the first bone information is the left shoulder Ax, the right shoulder z coordinate is the right shoulder Az, and the left shoulder z coordinate. Is the left shoulder Az. The second bone information is B, the x coordinate of the left shoulder of the second bone information is the left shoulder Bx, the z coordinate of the right shoulder is the right shoulder Bz, and the z coordinate of the left shoulder is the left shoulder Bz. In step S188 and step S208, as shown in FIG. 5B, as the orientation condition 1, the left shoulder Ax and the left shoulder Bx are compared, and it is determined whether the left shoulder Ax <the left shoulder Bx or the left shoulder Ax> the left shoulder Bx. Is done.

  In step S188, when it is determined in the orientation condition 1 that the left shoulder Ax <the left shoulder Bx, as the orientation condition 2, whether the difference obtained by subtracting the left shoulder Az from the right shoulder Az is equal to or greater than a predetermined distance, or the left shoulder It is determined whether or not the difference obtained by subtracting the right shoulder Az from Az is equal to or greater than a predetermined distance. When it is determined in the orientation condition 1 that the left shoulder Ax> the left shoulder Bx, as the orientation condition 2, whether the difference obtained by subtracting the right shoulder Az from the left shoulder Az is equal to or greater than a predetermined distance, or the right shoulder Az to the left shoulder It is determined whether or not the difference obtained by subtracting Az is a predetermined distance or more.

  For example, in step S188, as shown in FIG. 5B, when left shoulder Ax <left shoulder Bx (orientation condition 1) and right shoulder Az−left shoulder Az> 5 cm (orientation condition 2), or left shoulder Ax> left shoulder Bx When (orientation condition 1) and left shoulder Az−right shoulder Az> 5 cm (orientation condition 2) are satisfied, it is determined that the first bone information is facing the other party. When left shoulder Ax <left shoulder Bx (orientation condition 1) and left shoulder Az−right shoulder Az> 5 cm (orientation condition 2), or left shoulder Ax> left shoulder Bx (orientation condition 1) and right shoulder Az−left shoulder Az> 5 cm (orientation) When the condition 2) is satisfied, it is determined that the first bone information is in the direction opposite to the partner direction.

  In step S208, when it is determined in the orientation condition 1 that the left shoulder Ax <the left shoulder Bx, as the orientation condition 2, whether the difference obtained by subtracting the right shoulder Bz from the left shoulder Bz is equal to or greater than a predetermined distance, It is determined whether or not the difference obtained by subtracting the left shoulder Bz from the shoulder Bz is greater than or equal to a predetermined distance. When it is determined in the orientation condition 1 that the left shoulder Ax> the left shoulder Bx, as the orientation condition 2, whether the difference obtained by subtracting the left shoulder Bz from the right shoulder Bz is equal to or greater than a predetermined distance, or the left shoulder Bz to the right shoulder It is determined whether or not the difference obtained by subtracting Bz is greater than or equal to a predetermined distance.

For example, in step S208, as shown in FIG. 5B, when left shoulder Ax <left shoulder Bx (orientation condition 1) and left shoulder Bz−right shoulder Bz> 5 cm (orientation condition 2), or left shoulder Ax> left shoulder Bx When (orientation condition 1) and right shoulder Bz−left shoulder Bz> 5 cm (orientation condition 2) are satisfied, it is determined that the second bone information is facing the other party. When left shoulder Ax <left shoulder Bx (orientation condition 1) and right shoulder Bz−left shoulder Bz> 5 cm (orientation condition 2) or left shoulder Ax> left shoulder Bx (orientation condition 1) and left shoulder Bz−right shoulder Bz> 5 cm (orientation) When the condition 2) is satisfied, it is determined that the second bone information is in the direction opposite to the partner direction. In step S188 and step 208, if the left shoulder Ax, Az and right shoulder Az, left shoulder Bx, Bz and right shoulder Bz do not match either the orientation condition 1 or the orientation condition 2, the first bone information or the second bone The information is determined to be front-facing.

  In step S190, the angle of the first bone information with respect to the second bone information is determined. In step S210, the angle of the second bone information with respect to the first bone information is determined. The determinations in step S190 and step S210 are performed using the determination conditions shown in FIG. FIG. 5C is a diagram showing the angle determination conditions stored in the determination condition storage area 132 of the ROM 13. In steps S190 and S210, the coordinate information of the first bone information read in steps S182 and S202 is compared with the coordinate information of the second bone information read in steps S184 and S204, and FIG. The inclination is determined according to the angle determination condition shown in (C).

  More specifically, in steps S190 and S210, the first bone information is A, the left shoulder x coordinate of the first bone information is the left shoulder Ax, the head x coordinate is the head Ax, and the neck x coordinate is The neck is Ax. The second bone information is B, the left shoulder x coordinate of the second bone information is the left shoulder Bx, the head x coordinate is the head Bx, and the neck x coordinate is the neck Bx.

  In step S190 and step S210, as shown in FIG. 5C, as the angle condition 1, the left shoulder Ax and the left shoulder Bx are compared to determine whether the left shoulder Ax <the left shoulder Bx or the left shoulder Ax> the left shoulder Bx. Is done. In step S190, when it is determined in the angle condition 1 that the left shoulder Ax <the left shoulder Bx, as the angle condition 2, whether the difference obtained by subtracting the neck Ax from the head Ax is equal to or greater than a predetermined distance, or the neck Ax It is determined whether or not the difference obtained by subtracting the head Ax from the predetermined distance is greater than or equal to a predetermined distance. When it is determined in the angle condition 1 that the left shoulder Ax> the left shoulder Bx, as the angle condition 2, whether the difference obtained by subtracting the head Ax from the neck Ax is equal to or greater than a predetermined distance, or the neck Ax from the head Ax It is determined whether or not the subtracted difference is a predetermined distance or more.

  For example, in step S190, as shown in FIG. 5C, when left shoulder Ax <left shoulder Bx (angle condition 1) and head Ax−neck Ax> 2 cm (angle condition 2), or left shoulder Ax> left shoulder Bx ( When the angle condition 1) and the neck Ax−the head Ax> 2 cm (angle condition 2) are satisfied, it is determined that the head of the first bone information is an opponent inclination in which the head is inclined in the opponent direction. When left shoulder Ax <left shoulder Bx (angle condition 1) and neck Ax−head Ax> 2 cm (angle condition 2), or left shoulder Ax> left shoulder Bx (angle condition 1) and head Ax−neck Ax> 5 cm (angle condition 2) ), It is determined that the head of the first bone information has an opposite inclination that is inclined in a direction opposite to the opponent direction.

  In step S210, when it is determined that the left shoulder Ax <the left shoulder Bx in the angle condition 1, whether the difference obtained by subtracting the head Bx from the neck Bx is equal to or greater than a predetermined distance as the angle condition 2, or the head Bx It is determined whether or not the difference obtained by subtracting the neck Bx from the distance is equal to or greater than a predetermined distance. When it is determined that the left shoulder Ax> the left shoulder Bx in the angle condition 1, whether or not the difference obtained by subtracting the neck Bx from the head Bx is equal to or greater than a predetermined distance, or the difference obtained by subtracting the head Bx from the neck Bx is predetermined. It is determined whether the distance is equal to or greater than the distance.

  For example, in step S210, as shown in FIG. 5C, when left shoulder Ax <left shoulder Bx (angle condition 1) and head Bx−neck Ax> 2 cm (angle condition 2), or left shoulder Ax> left shoulder Bx ( When the orientation condition 1) and the neck Ax−head Ax> 2 cm (angle condition 2) are satisfied, it is determined that the head of the second bone information is the other party inclination that is inclined in the other party direction. When satisfying left shoulder Ax <left shoulder Bx (orientation condition 1) and neck Ax−head Ax> 2 cm (orientation condition 2), or left shoulder Ax> left shoulder Bx (orientation condition 1) and head Ax−neck Ax> 2 cm (orientation condition 2) ), It is determined that the head of the second bone information has an opposite inclination that is inclined in a direction opposite to the opponent direction. In step S190 and step 210, if the left shoulder Ax, head Ax, neck Ax, left shoulder Bx, head Bx, and neck Bx do not match either the tilt condition 1 or the tilt condition 2, the head of the first bone information or The head of the second bone information is determined to be facing the front.

  In steps S192 and S202, the determination results of the first bone information in steps S186 to S190 and the determination results of the second bone information in steps S206 to S210 are the determinations shown in FIGS. 6 (A) and 6 (B). The result table is stored in the first determination information storage area 143 and the second determination information storage area 144 of the RAM 14, respectively.

  Specifically, in step S192, whether the first bone information determined by the processing in steps S186 to S190 corresponds to the close distance, the individual distance, or the social distance from the distance determined in step S186. It is determined, and from the result determined in step S188, it is determined whether it corresponds to the opponent direction, the opposite direction, or the front direction. For each distance, thresholds are set such that the distance between the first bone information and the second bone information is long in the order of close distance, individual distance, and social distance. For example, when the distance of the first bone information determined in step S186 is less than 45 cm, it is determined that it corresponds to the close distance, and when it is 45 cm or more and less than 120 cm, it is determined that it corresponds to the individual distance, and when it is 120 cm or more It is judged that it corresponds to the distance. In step S192, the first bone information determined by the processing in steps S186 to S190 is the combination of items IE, PE, SE, IA, PA, SA, IS, PS shown in FIG. , SS, the combination item is counted as +1, and the determination result table A shown in FIG. 6A of the first determination information storage area 143 is updated and stored.

  Furthermore, in step S192, it is determined from the result determined in step S190 whether the head of the first bone information corresponds to the opponent's inclination, the opposite inclination, or the front. In step S192, it is determined whether the head of the first bone information corresponds to the tilt item SLE, SLA, or SLS shown in FIG. 6B in the tilt, and the corresponding tilt item is +1. The determination result table B shown in FIG. 6B in the first determination information storage area 143 is updated and stored.

  Similarly, in step S212, it is determined whether the second bone information determined by the processing in steps S206 to S210 corresponds to the close distance, the individual distance, or the social distance from the distance determined in step S206. And it is judged by step S208 whether it corresponds to a partner direction, an opposite direction, or a front direction. For each distance, thresholds are set such that the distance between the first bone information and the second bone information is long in the order of close distance, individual distance, and social distance. For example, when the distance of the second bone information determined in step S206 is less than 45 cm, it is determined that it corresponds to the close distance, and when it is 45 cm or more and less than 120 cm, it is determined that it corresponds to the individual distance. It is judged that it corresponds to the distance. In step S212, the second bone information determined by the processing in steps S206 to S210 is the combination items IE, PE, SE, IA, PA, SA, IS, PS shown in FIG. , SS, which combination item is determined, the corresponding combination item is counted as +1, and the determination result table A shown in FIG. 6A in the second determination information storage area 144 is updated and stored.

  Further, in step S212, based on the result determined in step S210, it is determined whether the head of the second bone information corresponds to the opponent inclination, the opposite inclination, or the front. In step S212, it is determined whether the head of the second bone information corresponds to the tilt item SLE, SLA, or SLS shown in FIG. 6B in the tilt, and the corresponding tilt item is +1. The determination result table B shown in FIG. 6B in the second determination information storage area 144 is updated and stored.

  In step S194 and step S214, it is determined whether or not the determination is completed for all the first bone information and all the second bone information accumulated in step S14. If it is not determined in step S194 that all of the first bone information has been determined (step S194: NO), the processes in steps S182 to S192 are performed until it is determined that all of the first bone information has been determined. Is done. When it is not determined in step S214 that all the second bone information has been determined (step S214: NO), the processes in steps S202 to S212 are performed until it is determined that all the second bone information has been determined. Is done. When it is determined that all the first bone information has been determined (step S194: YES) and all the second bone information has been determined (step S214: YES), the first bone determination processing is performed. And the 2nd bone determination process is complete | finished and the process of step S22 of an evaluation control process is performed.

  In step S22 following step S22 and step S22, for the first bone information determined by the first bone determination process in step S18, the ratio of the first bone information corresponding to each of the opposite direction, the opposite direction, and the front is calculated. Is done. In step S26 following step S26 and step S26, for the second bone information determined by the second bone determination process in step S20, the ratio of the second bone information corresponding to the opponent direction, the opposite direction, and the front direction is calculated. Is done.

  In step S22, the counter inclination number SLA and the counter inclination number SLA of the determination result table B shown in FIG. 6B, which are determined by the first bone determination process of step S18 and stored in the first determination information storage area 143 of the RAM 14, are stored. From the front-facing number of times SLS, the opponent inclination ratio SRE, the front ratio SRS, and the opponent inclination ratio SRA are respectively calculated as the ratio by inclination, which is the ratio of the frequency of each inclination item, with SLE + SLA + SLS as a population. The calculated ratios by inclination of the first bone information are stored in the calculation result table C shown in FIG. 6C of the first determination information storage area 143.

  Similarly, in step S26, from the determination result table B shown in FIG. 6B, which is determined by the second bone determination process of step S20 and stored in the second determination information storage area 144 of the RAM 14, as a ratio by inclination, The opponent inclination ratio SRE, the front ratio SRS, and the opponent inclination ratio SRA are calculated. The calculated ratios by inclination of the second bone information are stored in the calculation result table C shown in FIG. 6C in the second determination information storage area 144.

  In step S24, according to the combination of the distance and orientation of the determination result table A shown in FIG. 6A, which is determined by the first bone determination process of step S18 and stored in the first determination information storage area 143 of the RAM 14. From the respective counter numbers, the opposite direction, and the front direction count of each item shown, the opposite direction ratio ER, the front direction ratio SR, and the reverse direction ratio AR as the ratio by direction that is the ratio of the frequency of each direction item. Are calculated respectively. For each orientation ratio, the sum of IE + IA + IS + PE + PA + PS + SE + SA + SS, which is each item according to the combination of distance and orientation, is used as a mother set, and the opponent orientation ratio ER is calculated from the sum of IE + PE + SE, and the opposite direction ratio AR is calculated from the sum of IA + PA + SA. , The front-facing ratio SR is calculated from the sum of IS + PS + SS. The calculated ratio for each direction of the first bone information is stored in the calculation result table shown in FIG. 6C of the first determination information storage area 143.

  In step S28, according to the combination of the distance and orientation of the determination result table A shown in FIG. 6A, which is determined by the second bone determination process of step S20 and stored in the second determination information storage area 144 of the RAM 14. From the respective counter numbers, the opposite direction, and the front direction count of each item shown, the opposite direction ratio ER, the front direction ratio SR, and the reverse direction ratio AR as the ratio by direction that is the ratio of the frequency of each direction item. Are calculated respectively. The calculated ratio for each direction of the second bone information is stored in the calculation result table shown in FIG. 6C of the second determination information storage area 144.

  In step S30, the movement of the body part indicated by the first bone information and the movement of the body part indicated by the second bone information are evaluated. When the evaluation control apparatus 1 of the present embodiment is used in the first example of use, the movement of the body part indicated by the first bone information indicates the movement of the position of the body part of the user U1, and the second bone information The movement of the body part indicated by indicates the movement of the position of the body part of the user U2. The evaluation value of the movement of the user U1 is calculated using the determination result table A shown in FIG. 6A and the calculation result table C shown in FIG. 6C stored in the first determination information storage area 143. The evaluation value of the movement of the user U2 is calculated using the determination result table A shown in FIG. 6A and the calculation result table C shown in FIG. 6C stored in the second determination information storage area 143. The equation for calculating the evaluation value V is defined as shown in (1) below, for example.

  V = 50 * (IE * SRE * ER + IA * SRA * AR + IS * SRS * SR) + 25 * (PE * SRE * ER + PA * SRA * AR + PS * SRS * SR) + 25 * (SE * SRE * ER + SA * SRA * AR + SS * SRS * SR) (1)

  Note that the first bone information and the second bone information evaluated in step S30 differ depending on the usage example of the evaluation processing apparatus 1 of the present embodiment. For example, when the evaluation control device 1 of the present embodiment is used in the second example of use, the movement of the body part indicated by the first bone information is the movement of the body part of the user U1 or the body part of the avatar A1. The movement of the body part indicated by the second bone information indicates the movement of the position of the body part of the user U2 or the movement of the body part of the avatar A2.

  When the evaluation control apparatus 1 of the present embodiment is used in the second example of use, the movement of the body part indicated by the first bone information is the movement of the body part of the user U1 or the movement of the body part of the avatar A1. The movement of the body part indicated by the second bone information indicates the movement of the body part of the user U2 or the movement of the body part of the avatar A2. When the evaluation control device 1 according to the present embodiment is used in the usage example of the second example, the movement of the body part indicated by the first bone information indicates the movement of the body part of the user U1, and is indicated by the second bone information. The movement of the body part indicated indicates the movement of the body part of the virtual person V2. When the evaluation control apparatus 1 of the present embodiment is used in the second example of use, the movement of the body part indicated by the first bone information is the movement of the body part of the user U1 or the movement of the body part of the avatar A1. The movement of the body part indicated by the second bone information indicates the movement of the body part of the virtual person V2.

  In step S32, the sum of the evaluation value V1 that evaluates the movement of the body part indicated by the first bone information calculated in step S30 and the evaluation value V2 that evaluates the movement of the body part indicated by the second bone information and Control for calculating the difference and outputting the calculated result to the display 5 is performed. Thereby, at least the user U1 can know the compatibility between himself / herself and another user or a comparison target person such as an avatar from the start of playback to the end of playback of karaoke music.

  In step S34, it is determined whether an instruction to end the evaluation control apparatus 1 has been issued. If it is determined that an instruction to end the evaluation processing apparatus 1 has been issued (step S34: YES), the evaluation control process is ended. If it is not determined that the end instruction of the evaluation processing apparatus 1 has been issued (step S34: NO), the process of step S10 is executed again.

  In the above embodiment, the display 5 is installed in the same space as the space where the user U1 is located. The place where the display 5 is installed is not limited to this. For example, an image showing the movement of the body part of the user U1 may be displayed on a display device provided in a remote space away from the space where the user U1 is present.

DESCRIPTION OF SYMBOLS 1 Evaluation processing apparatus 2 State detection apparatus 3 Karaoke apparatus 5 Display 11 CPU
13 ROM
131 Evaluation control program storage area 132 Judgment condition storage area 14 RAM
131 First bone information storage area 132 Second bone information storage area 133 Second determination information storage area 134 Second determination information storage area

Claims (7)

Identification means for identifying a user in a predetermined space as an observation target;
Observation acquisition means for acquiring position information of a specific body part of the observation target person identified by the identification means during reproduction of a predetermined music in association with time information indicating the reproduction time of the music;
Predetermined comparison target person that is position information associated with the time information from the reproduction start time to the reproduction end time of the music, and is different from the observation target person from which the position information was acquired by the observation acquisition means Comparison acquisition means for acquiring position information of the specific body part of
Reproduction of the music corresponding to the time information corresponding to the position information of the observation target acquired by the observation acquisition means and the position information of the comparison target read by the comparison reading means Comparison is made at the same timing from the start time to the reproduction end time, and the relative distance of the specific body part to the comparison target person and the direction approaching the comparison target person are positive for the observation target person at the compared timing. Determination means for determining at least one of the direction and inclination of the specific body part to be performed;
Output control means for outputting a result of evaluating the movement of the observation target person during reproduction of the music according to the determination result by the determination means;
An evaluation control device comprising: Detecting means for detecting a reproduction start time and a reproduction end time of the music reproduced in the predetermined space;
Storage means for storing the position information of the observation target acquired by the observation acquisition means in a storage unit in association with time information indicating the time when the position information was acquired;
The observation acquisition means reads from the storage unit the position information of the observation subject associated with the time information corresponding to the music from the reproduction start time to the reproduction end time detected by the detection means. To get and
The comparison acquisition means is position information in which the time information from the reproduction start time to the reproduction end time of the music is associated, and the observation subject from which the position information is read by the reading means The position information of the specific body part of a different predetermined comparison target person is read out from the storage unit and acquired.
First determination means for determining a movement transition of the observation target indicated by the position information from a reproduction start time to a reproduction end time of the music acquired by the observation acquisition means;
Second determination means for determining movement transition of the comparison target indicated by the position information from the reproduction start time to the reproduction end time of the music acquired by the comparison acquisition means,
The determination means corresponds to the piece of music in which the transition transition of the observation target determined by the first determination means and the transition transition of the comparison target determined by the second determination means correspond to the time information. For each predetermined timing from the reproduction start time to the reproduction end time, and for the observation target person at the compared timing, the distance transition of the relative distance of the specific body part with the comparison target person, and the comparison target person Determining a direction transition of at least one of the direction and inclination of the specific body part with the direction approaching the positive being positive,
The output control means outputs a result of evaluating the movement of the observation subject from the reproduction start time to the reproduction end time of the music according to the determination result of the distance transition and the direction transition by the determination means. The evaluation control apparatus according to claim 1. Display that displays a reflected human image reflecting the position of the specific body part of the observation subject on a predetermined display unit according to the position information of the specific body part of the observation subject acquired by the observation acquisition means With control means,
The determination means compares position information of the reflected human image displayed on the display unit with the position information of the comparison target person as the position information of the observation target person. 3. The evaluation control device according to 1 or 2. The comparison acquisition means uses the virtual person image displayed on the display unit provided in the space as the comparison target person, and the position information of the specific body part of the virtual person image on the display unit in the space. To get and
The determination means compares the position information of the observation target person from a predetermined reference position with the position information of the virtual person image on the display unit from the predetermined reference position, and the relative distance, The evaluation control apparatus according to claim 1, wherein at least one of the direction and the inclination is determined. The determination unit includes the position information of the reflected person image displayed on the display unit, and the position information of the virtual person image displayed on the display unit acquired by the comparison acquisition unit. 5. The evaluation control according to claim 4, wherein the evaluation is performed by comparing the reflected human image on the display unit with the relative distance from the virtual human image and at least one of the orientation and the inclination. apparatus. The identifying means identifies a first user in a predetermined space as the observation target, identifies a second user in the predetermined space as a comparison target,
The observation acquisition means acquires position information of a specific body part of the first user identified as the observation subject by the identification means,
The comparison acquisition means acquires position information of a specific body part of the second user identified as the comparison target person by the identification means,
The determination means compares the position information of the first user with the position information of the second user at the same timing from the reproduction start time to the reproduction end time of the music, and at the timing compared. For the first user, at least one of the relative distance of the specific body part with the second user and the direction and inclination of the specific body part with the direction approaching the second user as positive, As the determination result of the first user, the relative distance of the specific body part to the first user and the direction approaching the first user are corrected for the second user at the compared timing. Determining at least one of the direction and inclination of the specific body part as a determination result of the second user,
The said output control means outputs the result of having evaluated the relative relationship of the movement of the said observation subject and the said comparison subject from the reproduction start time to the reproduction end time of the said music. The evaluation control apparatus according to any one of the above. A program executed by a computer,
An identification step for identifying a user in a predetermined space as an observation target;
An observation acquisition step of acquiring position information of a specific body part of the observation subject identified by the identification step;
Predetermined comparison target person that is position information associated with the time information from the reproduction start time to the reproduction end time of the music, and is different from the observation target person from which the position information was acquired by the observation acquisition step A comparative acquisition step of acquiring position information of the specific body part of
The reproduction of the music corresponding to the time information corresponding to the position information of the observation target acquired by the observation acquisition step and the position information of the comparison target read by the comparison reading unit Comparison is made at the same timing from the start time to the reproduction end time, and the relative distance of the specific body part to the comparison target person and the direction approaching the comparison target person are positive for the observation target person at the compared timing A determination step of determining at least one of a direction and an inclination of the specific body part to be performed;
According to the determination result by the determination step, an output control step for outputting a result of evaluating the movement of the observation subject from the reproduction start time to the reproduction end time of the music;
An evaluation control program comprising: