WO2024053178A1

WO2024053178A1 - Dance practice system, dance practice method, and program

Info

Publication number: WO2024053178A1
Application number: PCT/JP2023/019957
Authority: WO
Inventors: 力五反田; 毅齋藤
Original assignee: パナソニックＩｐマネジメント株式会社
Priority date: 2022-09-07
Filing date: 2023-05-29
Publication date: 2024-03-14

Abstract

This dance practice system (100) comprises a first input interface (11), a second input interface (12), a processor (13), and an output interface (14). The processor (13) generates guide image data that provides instructions to a user such that the movement of the user becomes similar to that of a performer. The processor (13) determines to which frame, among a plurality of frames of model image data, a specific frame acquired at a specific time point from among a plurality of frames of user image data corresponds. When the specific frame is delayed with respect to the determined frame, the processor (13) performs control so as to expedite the timing at which the guide image data is to be displayed on a display (31) further when the time difference between the determined frame and the specific frame is greater.

Description

Dance practice system, dance practice method, and program

The present disclosure relates to a dance practice system and the like for supporting dance practice by a user.

For example, Patent Document 1 discloses a video generation device that presents an ideal dance video to a dancing target as a virtual video of the target.

International Publication No. 2021/039857

The video generation device disclosed in Patent Document 1 can present an ideal dance video to the user as a virtual video of the user himself/herself, but it is possible to present the ideal dance video to the user as a virtual video of the user himself/herself. However, this does not mean that the user can dance exactly as shown in the video. In other words, the video generation device disclosed in Patent Document 1 has a problem in that when a user practices dancing, an improvement in dance proficiency cannot be expected.

The present disclosure provides a dance practice system and the like that allows users to easily improve their dancing proficiency when practicing dancing.

A dance practice system according to one aspect of the present disclosure includes a first input interface, a second input interface, a signal processing circuit, and an output interface. The first input interface acquires user image data including a plurality of frames representing user actions based on image data generated by imaging with a camera. The second input interface acquires model image data including a plurality of frames showing the movements of a performer serving as a dance model. The signal processing circuit generates guide image data that instructs the user so that the user's movements approximate those of the performer. The output interface displays the user image data, the model image data, and the guide image data on a display. The signal processing circuit determines which frame among the plurality of frames of the model image data a specific frame acquired at a predetermined time point among the plurality of frames of the user image data corresponds to. . When the specific frame is delayed with respect to the determined frame among the plurality of frames of the model image data, the signal processing circuit determines that the larger the time difference between the determined frame and the specific frame, Control is performed to advance the timing at which the guide image data is displayed on the display.

In a dance practice method according to one aspect of the present disclosure, user image data including a plurality of frames showing user movements is obtained based on image data captured and generated by a camera. In the dance practice method, model image data including a plurality of frames showing the movements of a performer serving as a dance model is acquired. In the dance practice method, guide image data is generated that instructs the user so that the user's movements approximate those of the performer. In the dance practice method, the user image data, the model image data, and the guide image data are displayed on a display. In the process of generating the guide image data, a specific frame acquired at a predetermined time point among the plurality of frames of the user image data corresponds to which frame among the plurality of frames of the model image data. to decide. In the process of generating the guide image data, if the specific frame lags behind the determined frame among the plurality of frames of the model image data, the time difference between the determined frame and the specific frame. The larger the value, the earlier the timing at which the guide image data is displayed on the display is controlled.

A program according to one aspect of the present disclosure causes one or more processors to execute the dance practice method.

According to the dance practice system and the like in the present disclosure, there is an advantage that when a user practices dance, it is easy to improve the user's dance proficiency.

FIG. 1 is a block diagram showing an example of the overall configuration including a dance practice system according to an embodiment. FIG. 2 is a schematic diagram showing an example of use of the dance practice system according to the embodiment. FIG. 3 is a schematic diagram showing an example of guide image data. FIG. 4 is a flowchart illustrating a processing example of the analysis unit according to the embodiment. FIG. 5 is an explanatory diagram of the operation in the analysis section according to the embodiment. FIG. 6 is a flowchart illustrating an example of processing by the correction unit according to the embodiment. FIG. 7 is an explanatory diagram of the operation in the correction section according to the embodiment. FIG. 8 is a flowchart illustrating an example of processing by the generation unit according to the embodiment. FIG. 9 is an explanatory diagram of a first specific example of the dance practice system according to the embodiment. FIG. 10 is an explanatory diagram of a second specific example of the dance practice system according to the embodiment.

Hereinafter, embodiments will be specifically described with reference to the drawings. Note that the embodiments described below are all inclusive or specific examples. The numerical values, shapes, materials, components, arrangement positions and connection forms of the components, steps, order of steps, etc. shown in the following embodiments are examples, and do not limit the present disclosure. Further, among the constituent elements in the following embodiments, constituent elements that are not described in the independent claims will be described as arbitrary constituent elements.

Note that each figure is a schematic diagram and is not necessarily strictly illustrated. Further, in each figure, substantially the same configurations are denoted by the same reference numerals, and overlapping explanations may be omitted or simplified.

(Embodiment)
[1. composition]
[1-1. overall structure]
First, the overall configuration including a dance practice system according to an embodiment will be described using FIG. 1. FIG. 1 is a block diagram showing an example of the overall configuration including a dance practice system according to an embodiment. The dance practice system 100 is a system for supporting the user U1 (see FIG. 2) to imitate the movements of a performer who is a model for the dance when the user U1 (see FIG. 2) practices dancing. Here, dance refers to a series of movements performed to accompaniment, for example. Note that dance also includes non-musical dance in which body movements themselves function as accompaniment. In the embodiment, the dance practice system 100 is installed in a computer 3 such as a personal computer. In other words, in the embodiment, the dance practice system 100 includes the computer 3. In addition to the computer 3, the dance practice system 100 may further include at least one of a camera 2, a display 31, and a server 4, which will be described later.

FIG. 2 is a schematic diagram showing an example of how the dance practice system 100 according to the embodiment is used. As shown in FIG. 2, the computer 3 displays model image data P1, user image data P2, and guide image data P3 on a display 31 provided separately from the computer 3. A camera 2 is attached to the upper part of the bezel of the display 31 to take an image of the user U1 directly facing the display 31.

The camera 2 has an image sensor such as a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor, and captures an image of the user U1 directly facing the display 31. The camera 2 generates uncorrected image data by capturing an image of the user U1, and the generated uncorrected image data is used as a first input interface (hereinafter referred to as I/F (Interface)) 11 of a dance practice system 100, which will be described later. Output to. User image data P2 is generated by correcting this uncorrected image data by the processor 13 of the dance practice system 100, which will be described later. Further, in the embodiment, the camera 2 is attached to the display 31 in advance, but it does not need to be attached in advance and may be attached afterwards.

The model image data P1 is image data showing the movements of a performer serving as a dance model, and is moving image data including a plurality of frames. In the embodiment, the model image data P1 is moving image data about one performer performing the above-mentioned model motion. Furthermore, in the embodiment, the model image data P1 is video content that is created in advance and includes a scene in which a performer performs the above-mentioned model motion. Here, in the embodiment, the model image data P1 is moving image data obtained by capturing an image of the performer performing a horizontally reversed action of the above-mentioned model action from the front of the performer. . For example, the motion of raising the right hand in the above-mentioned model motion becomes the motion of raising the left hand in the model image data P1 displayed on the display 31.

The user U1 looks at the model image data P1 displayed on the display 31 and practices dancing by imitating the performer's movements (that is, performing movements that are mirror images of the above-mentioned model movements). do. For example, if the performer in the model image data P1 displayed on the display 31 is raising his left hand, that is, the performer is raising his right hand as seen from the user U1, the user U1 can imitate that movement and make his own. Raise your right hand. Here, as will be described later, the user image data P2 is obtained by capturing an image of the user U1 directly facing the display 31 with the camera 2, and displaying the captured uncorrected image data on the display 31 by inverting the left and right sides of the image data. Therefore, the performer in the model image data P1 and the user U1 in the user image data P2 are displayed on the display 31 as if they were performing the same action. For example, if the user U1 raises his right hand in imitation of the performer raising his right hand displayed on the display 31, in the user image data P2 displayed on the display 31, the user U1 also raises his right hand. I will raise my hand.

In the embodiment, the model image data P1 is distributed from a server 4, which will be described later, via a network N1 such as the Internet. In the embodiment, the processor 13 of the dance practice system 100 obtains model image data P1 from the server 4 in advance, and analyzes the obtained model image data P1. Then, the processor 13 of the dance practice system 100 stores information including the acquired model image data P1 and the analysis result of the model image data P1 in the memory 15. Further, in the embodiment, the dance practice system 100 displays the model image data P1 stored in the memory 15 on the display 31 when the user practices dancing.

In addition, when the video data is obtained by capturing the image of the performer performing the above-mentioned model movements from the front, the processor 13 of the dance practice system 100 may By performing a process of inverting the left and right sides of the acquired moving image data, the inverted moving image data may be obtained as the model image data P1.

In the embodiment, the model image data P1 is displayed on the display 31 at a frame rate of 30 fps (frames per second) or 60 fps. Note that the frame rate may be a value other than 30 fps and 60 fps. Further, in the embodiment, the model image data P1 is displayed on the lower right corner of the display 31 in a size smaller than the size of the user image data P2. Hereinafter, the area in which the model image data P1 is displayed on the display 31 will also be referred to as the "sub-screen 312."

The user image data P2 is image data showing the actions of the user U1 based on image data generated by capturing an image of the user U1 with the camera 2, and is moving image data composed of a plurality of frames. In the embodiment, user image data P2 is displayed on display 31 at the same frame rate as model image data P1.

Furthermore, in the embodiment, the user image data P2 is displayed on the display 31 as a mirror image of the user U1 directly facing the display 31. Specifically, the user image data P2 is obtained by processing the left and right sides of the image data by the processor 13 of the dance practice system 100 with respect to the uncorrected image data input from the camera 2 to the first input I/F 11 of the dance practice system 100. Generated by performing inversion processing.

Furthermore, in the embodiment, the user image data P2 is displayed at the center of the display 31 in a size larger than the size of the sub-screen 312. Hereinafter, the area on the display 31 where the user image data P2 and the guide image data P3 are displayed will also be referred to as the "main screen 311."

The guide image data P3 is image data that instructs the user U1 so that the user U1's movements approximate those of the performer, and is moving image data that includes a plurality of frames. The guide image data P3 includes an arrow indicating the next action that the user U1 should perform, as shown in FIG. 2, for example. In the example shown in FIG. 2, the user U1 moves each of the right hand, right foot, left hand, and left foot according to the arrows located near each of the right hand, right foot, left hand, and left foot on the display 31. In this way, even if the user U1 does not remember the performer's movements, by performing the movements while looking at the guide image data P3, the user U1 can understand the next movement to be performed, so that the user U1 can perform his/her own movements as a dancer. It is possible to approach the behavior that is a model.

In the embodiment, the frame rate when the guide image data P3 is displayed on the display 31 is the same as the frame rate when the user image data P2 is displayed on the display 31. Further, the guide image data P3 is updated frame by frame and displayed on the display 31, similarly to the user image data P2. Further, in the embodiment, the guide image data P3 is displayed on the display 31 in a form superimposed on the user image data P2 on the main screen 311. Note that the guide image data P3 does not need to be updated every frame, and may be updated every several frames, for example.

FIG. 3 is a schematic diagram showing an example of guide image data P3. In FIG. 3, only the main screen 311 of the main screen 311 and the sub-screen 312 displayed on the display 31 is illustrated. As shown in FIG. 3, the guide image data P3 is displayed on the main screen 311 so as to be superimposed on the user image data P2. In the example shown in FIG. 3, the guide image data P3 includes image data P31 corresponding to the left hand of the user U1 and image data P32 corresponding to the right hand of the user U1. The image data P31 includes a destination point P312 indicating the next position to be reached by the user's U1's left hand, and an arrow P311 indicating the direction and amount of movement of the user's U1's left hand from the current position to the destination point. The image data P32 also includes a destination point P322 that indicates the next position that the right hand of the user U1 should reach, and an arrow P321 that indicates the direction and amount of movement of the right hand of the user U1 from the current position to the destination point. include. That is, in the embodiment, the guide image data P3 indicates a destination point indicating the next position that the user U1 should reach on the display 31, and a direction and movement amount from the current position of the user U1 to the destination point on the display 31. An arrow is included for each part of the user U1. Note that the guide image data P3 may include reaching points and arrows for the entire body of the user U1 instead of for each part of the user U1.

In the embodiment, the destination point and arrow in the guide image data P3 are displayed on the display 31 so that the larger the amount of movement of each part of the user U1 from the current position to the destination point is, the darker the color is, and the smaller the amount of movement is, the lighter the color is. Ru. Further, in the embodiment, the arrow in the guide image data P3 is displayed on the display 31 in a larger size as the amount of movement is larger, and in a smaller size as the amount of movement is smaller. The "arrow size" here includes the length of the arrow. That is, the arrow in the guide image data P3 is displayed on the display 31 so that the larger the amount of movement is, the longer the arrow is, and the smaller the amount of movement is, the shorter the arrow is. That is, the guide image data P3 changes at least one of its size and shading depending on the difference between the user U1's motion and the performer's motion.

The user U1 can practice dancing by imitating the dance model movements while looking at the model image data P1 displayed on the display 31. By further viewing the user image data P2 showing his own movements and the guide image data P3, the user U1 can easily bring his own movements closer to those of the performer, and can practice dancing more efficiently. It is.

[1-2. Dance practice system configuration]
Next, the configuration of the dance practice system 100 will be specifically explained. As shown in FIG. 1, the dance practice system 100 includes a first input I/F 11, a second input I/F 12, a processor 13, an output I/F 14, and a memory 15.

The first input I/F 11 is, for example, a wired communication interface, and is connected to the camera 2 using a cable such as a USB (Universal Serial Bus) cable. The first input I/F 11 acquires uncorrected image data from the camera 2 by performing wired communication with the camera 2 . Note that the first input I/F 11 may be a wireless communication interface. In this case, the first input I/F 11 communicates with the camera 2 via a network such as a LAN (Local Area Network) based on a wireless communication standard such as Wi-Fi (registered trademark). Obtain uncorrected image data from.

In the embodiment, the processor 13 generates the user image data P2 by performing a process of inverting the left and right sides of the uncorrected image data acquired by the first input I/F 11. . That is, the first input I/F 11 indirectly acquires the user image data P2 by acquiring the uncorrected image data.

The second input I/F 12 is, for example, a wireless communication interface, and is based on a wireless communication standard such as Wi-Fi (registered trademark), and receives data from the server 4 that provides a video distribution service via a network N1 such as the Internet. Acquire main image data P1. Video distribution services include, for example, YouTube (registered trademark), Instagram (registered trademark), TikTok (registered trademark), and the like.

Note that the second input I/F 12 may obtain the model image data P1 stored in an information terminal such as a smartphone owned by the user U1, or may obtain the model image data P1 stored in an information terminal such as a smartphone owned by the user U1, or a hard disk drive (HDD) owned by the user U1. The model image data P1 stored in an external storage device such as the above may be acquired. When acquiring the model image data P1 from an information terminal or external storage device, the second input I/F 12 performs wired communication with the information terminal or external storage device using a cable such as a USB cable. The model image data P1 may also be acquired.

The processor 13 is, for example, a CPU (Central Processing Unit), and processes the user image data P2 acquired through the first input I/F 11 and the model image data P1 acquired through the second input I/F 12. Information processing, such as processing to generate guide image data P3, is performed based on the guide image data P3. The above information processing is realized by the processor 13 executing a computer program stored in the memory 15. Processor 13 is an example of a signal processing circuit of dance practice system 100.

In the embodiment, the processor 13 executes the computer program stored in the memory 15 to perform the analysis section 131, the correction section 132, the generation section 133, the superimposition section 134, and the synthesis section 135. Function.

The analysis unit 131 analyzes the model image data P1 acquired by the second input I/F 12. In the embodiment, analysis of the model image data P1 by the analysis unit 131 is performed before displaying the model image data P1 on the display 31. That is, in the embodiment, the display of the model image data P1 on the display 31 is executed after the analysis of the model image data P1 by the analysis unit 131 is completed. Hereinafter, specific processing by the analysis unit 131 will be explained using FIGS. 4 and 5.

FIG. 4 is a flowchart illustrating a processing example of the analysis unit 131 according to the embodiment. First, the analysis unit 131 divides the model image data P1 acquired by the second input I/F 12 into frames (S11). A frame number is assigned to each divided frame, with the frame at the start of the model image data P1 being the first frame. Hereinafter, assuming that the model image data P1 obtained in advance is N frames (N is a natural number), the nth frame to be processed (n is a natural number, 1≦n≦N) will be referred to as "frame n". .

Next, the analysis unit 131 calculates coordinate data L1(n) of each part of the performer by analyzing each part of the performer for each frame (S12). The "coordinate data" here is data representing coordinates on the XY plane (that is, coordinates on a two-dimensional orthogonal coordinate system) in the image data of the frame. The coordinate data L1(n) of each part of the performer calculated by the analysis unit 131 is stored in the memory 15 for each frame (S13). That is, the memory 15 stores coordinate data L1(1), L1(2), . . . , L1(N).

In step S12, the analysis unit 131 calculates the coordinate data L1(n) of each part of the performer using an appropriate algorithm that detects the skeletal coordinates of a human image included in image data such as Kinect (registered trademark). calculate. In the embodiment, the analysis unit 131 calculates coordinate data of each of the performer's head, neck, right shoulder, left shoulder, right hand, left hand, right foot, and left foot in each frame, and converts these coordinate data into coordinate data. Let it be L1(n). Note that each part of the performer listed above is an example, and the analysis unit 131 may calculate coordinate data of other parts of the performer. Furthermore, the number of types of parts of the performer that the analysis unit 131 analyzes may be greater or less than the number of parts listed above.

Next, for each frame, the analysis unit 131 generates difference data D1 between the coordinate data L1(n) of each part of the performer in the frame and the coordinate data L1(n+1) of each part of the performer in the next frame. (n) is calculated (S14). The difference data D1(n) is calculated by subtracting the coordinate data L1(n) from the coordinate data L1(n+1). The difference data D1(n) calculated by the analysis unit 131 is stored in the memory 15 for each frame (S15). That is, the memory 15 stores difference data D1(1), D1(2), . . . , D1(N-1).

FIG. 5 is an explanatory diagram of the operation in the analysis unit 131 according to the embodiment. (a) of FIG. 5 shows "frame 1" of the model image data P1, and (b) of FIG. 5 shows the parts of the performer obtained by analyzing "frame 1" of the model image data P1. Coordinate data L1(1) is shown. (c) of FIG. 5 shows "frame 2" of the model image data P1, and (d) of FIG. 5 shows the parts of the performer obtained by analyzing "frame 2" of the model image data P1. Coordinate data L1(2) is shown. (e) of FIG. 5 shows "frame 3" of the model image data P1, and (f) of FIG. 5 shows each part of the performer obtained by analyzing "frame 3" of the model image data P1. coordinate data L1(3) is shown. In each of FIGS. 5(b), (d), and (f), "part" indicates the part of the performer, "XY coordinates" indicates the coordinate data of each part of the performer in the frame, and "XY difference" The difference between the coordinate data of each part of the performer in a frame and the coordinate data of each part of the performer in the next frame is shown. Note that in FIG. 5, the performers are represented only by their skeletons. In subsequent drawings, the performers are similarly illustrated only with their skeletons.

In FIG. 5, for example, the difference data (a12, b12) of the performer's head in "frame 2" of the model image data P1 is the coordinate data (a12, b12) of the performer's head in "frame 2" of the model image data P1 ( x12, y12) and the coordinate data (x13, y13) of the performer's head in "frame 3" of the model image data P1. In other words, the difference data D1(2) is derived from the coordinate data L1(3) of each part of the performer in "Frame 3" of the model image data P1, and the coordinate data L1(3) of each part of the performer in "Frame 2" of the model image data P1. It is calculated by subtracting the coordinate data L1(2).

Furthermore, the analysis unit 131 stores the model image data P1 in the memory 15 (S16). Step S16 may be executed in parallel with steps S11 to S15, or may be executed before step S11.

The correction unit 132 generates a guide image in the generation unit 133 based on the user image data P2 acquired through the first input I/F 11 and the model image data P1 acquired through the second input I/F 12. A guide correction value used when correcting data P3 is calculated. Hereinafter, specific processing by the correction unit 132 will be explained using FIG. 6.

FIG. 6 is a flowchart showing an example of processing by the correction unit 132 according to the embodiment. First, the correction unit 132 calculates coordinate data of each part of the user U1 in a specific frame acquired at a predetermined time point of the user image data P2 acquired via the first input I/F 11 (S21). The coordinate data of each part of the user U1 calculated by the correction unit 132 is stored in the memory 15 in association with a specific frame.

Here, the specific frame may be, for example, the latest frame of the user image data P2, or may be any frame. Note that it is preferable to use the latest frame of the user image data P2 as the specific frame because it is possible to calculate the guide correction value according to the latest state of the user U1. In the following description, it is assumed that the specific frame of the user image data P2 is "frame n". Therefore, the coordinate data L2(n) of each part of the user U1 in the specific frame is stored in the memory 15 in association with the specific frame.

In step S21, similarly to the analysis unit 131, the correction unit 132 uses an appropriate algorithm for detecting the skeletal coordinates of a human image included in image data such as Kinect (registered trademark) to calculate each of user U1's points in a specific frame. Coordinate data L2(n) of the part is calculated. The parts of the user U1 that are to be calculated by the correction unit 132 are the same as the parts of the performer that are to be calculated by the analysis unit 131. Therefore, here, the correction unit 132 calculates coordinate data of each of the head, neck, right shoulder, left shoulder, right hand, left hand, right foot, and left foot of the user U1 in the specific frame.

Next, the correction unit 132 determines a corresponding frame corresponding to the specific frame by analyzing which frame of the plurality of frames of the model image data P1 the specific frame corresponds to (S22). The corresponding frame is a frame in which the movement of the performer is closest to the movement of user U1 in the specific frame.

Specifically, the correction unit 132 uses the coordinate data L2(n) of each part of the user U1 in a specific frame (here, "frame n" of the user image data P2) and the model stored in the memory 15. Difference data D12 (n+k) between coordinate data L1 (n+k) (-α≦k≦α, k is an integer) of each part of the performer in each of 2×α+1 (α is a natural number) frames in the image data P1 is calculated. calculate.

As a specific example, when α=2, the correction unit 132 calculates coordinate data L2(n) of each part of the user U1 in "frame n" of the user image data P2 and "frame (n)" in the model image data P1. Coordinate data L1(n-2), L1(n-1),..., L1( D12(n-2), D12(n-1), . . . , D12(n+2) are calculated.

Then, the correction unit 132 determines the frame for which the calculated difference data D12(n+k) is the minimum as the corresponding frame corresponding to the specific frame. Here, the fact that the calculated difference data becomes the minimum corresponds to, for example, that the integrated value of the difference data of each part becomes the minimum. The difference data of each part here is the length of a vector indicated by the difference data, or a value corresponding to the length. For example, when the vector indicated by the difference data of each part is (dx1, dy1), the value of the difference data is dx12+dy12.

FIG. 7 is an explanatory diagram of the operation in the correction unit 132 according to the embodiment. (a) of FIG. 7 shows a specific frame (here, "frame n") of the user image data P2, and (b) of FIG. 7 shows the coordinates of each part of the user U1 obtained by analyzing the specific frame. Data L2(n) is shown. (c), (e), (g), (i), and (k) in FIG. 7 are "frame (n-2)" (k=-2) and "frame (n −1)” (k=−1), “frame n” (k=0), “frame (n+1)” (k=1), and “frame (n+2)” (k=2).

In FIG. 7(b), "part" indicates the part of the user U1, and "XY coordinates" indicates the coordinates of each part of the user U1 in the frame.

In the example shown in FIG. 7, coordinate data L2(n) of each part of the user U1 in a specific frame and coordinate data L2(n) of each part of the performer in "frame (n-1)" (k=-1) of the model image data P1 are shown. The difference from the coordinate data L1(n-1) is the minimum. Therefore, the correction unit 132 determines "frame (n-1)" (k=-1) of the model image data P1 as the corresponding frame.

Returning to FIG. 6, the correction unit 132 calculates the guide correction value β1 (S23). The guide correction value β1 calculated by the correction unit 132 is stored in the memory 15. Specifically, the correction unit 132 calculates the guide correction value β1 based on the formula “β1=−k1”. Here, "k1" is the value of "k" when the above-mentioned difference data D12 (n+k) is the minimum. In other words, "k1" is calculated by subtracting the number assigned to the frame corresponding to the acquisition time of the specific frame in the model image data P1 from the number assigned to the corresponding frame in the model image data P1. .

Here, the guide correction value β1 indicates the time difference between the determined frame (corresponding frame) among the plurality of frames of the model image data P1 and the specific frame in the user image data P2. In other words, the guide correction value β1 indicates the degree to which the movement of the user U1 follows the movement of the performer in the model image data P1. For example, when the guide correction value β1 is zero, it indicates that the user U1 can move without delay with respect to the movement of the performer in the model image data P1. On the other hand, when the guide correction value β1 is a positive value (that is, "k1<0"), it indicates that the movement of the user U1 lags behind the movement of the performer in the model image data P1. In this case, the larger the absolute value of the guide correction value β1 is, the more the user U1's movement lags behind the performer's movement in the model image data P1. Further, when the guide correction value β1 is a negative value (that is, "k1>0"), it indicates that the movement of the user U1 precedes the movement of the performer in the model image data P1. In this case, the larger the absolute value of the guide correction value β1 is, the more the user U1's movement precedes the performer's movement in the model image data P1.

Note that the correction unit 132 limits the value of the guide correction value β1 so that the offset β, which will be described later, satisfies "β=β0+β1≧0". Here, "β0" represents the initial value of offset β. For example, when "β0+β1≧0", that is, "β0≧k1", the guide correction value β1 becomes "β1=k1". On the other hand, when "β0+β1<0", that is, "β0<k1", the guide correction value β1 becomes "β1=-β0". To give a specific example, when "β0=2" and "k1>2", the guide correction value β1 becomes "β1=-β0=-2".

The generation unit 133 generates guide image data P3 based on the guide correction value β1 calculated by the correction unit 132. Hereinafter, specific processing by the generation unit 133 will be explained using FIG. 8.

FIG. 8 is a flowchart illustrating an example of processing by the generation unit 133 according to the embodiment. First, the generation unit 133 calculates the offset β (S31). The offset β calculated by the generation unit 133 is stored in the memory 15. Here, the offset β is calculated by adding the guide correction value β1 calculated by analyzing a specific frame in the user image data P2 to the initial value β0 of the offset β, and the unit is the number of frames. In the embodiment, the initial value β0 of the offset β is two frames.

Next, the generation unit 133 generates guide image data P3 based on the calculated offset β (S32). Specifically, the generation unit 133 generates a “frame ( n+1)'' (hereinafter referred to as the ``previous frame''), and ``frame (n+1+β)'' (hereinafter referred to as the ``later frame''), which is a frame of the model image data P1 after an offset β from the relevant frame. decide. Next, the generation unit 133 reads, from the memory 15, difference data D1(n+1), . Then, the generation unit 133 generates guide image data P3 based on the difference D obtained by adding all the read difference data D1(n+1), . . . D1(n+β). The difference D is represented by "D=D1(n+1)+...+D1(n+β)".

In this case, the generation unit 133 generates an image of the arrival point in the guide image data P3 for each part of the user U1 based on the coordinate data L1 (n+1+β) of each part of the performer in "frame (n+1+β)" which is the subsequent frame. to be generated. Furthermore, based on the difference D, the generation unit 133 generates an arrow image in the guide image data P3 for each part of the user U1. In this manner, in the embodiment, the generation unit 133 generates a frame (later frame) that is after a predetermined time point by a time corresponding to the time difference (guide correction value β1), among the plurality of frames of the model image data P1. With reference to this, guide image data P3 is generated.

Then, the generation unit 133 outputs the generated guide image data P3 (S33). Here, the generation unit 133 outputs the generated guide image data P3 to the superimposition unit 134.

The superimposing unit 134 generates image data for each frame by superimposing the guide image data P3 generated by the generating unit 133 on the user image data P2 acquired via the first input I/F 11. The superimposing unit 134 outputs the superimposed image data to the combining unit 135.

The composition unit 135 generates image data for each frame by combining the image data superimposed by the superposition unit 134 and the model image data P1 read from the memory 15. Here, the combining unit 135 displays the image data (that is, user image data P2 and guide image data P3) superimposed by the superimposing unit 134 on the main screen 311 of the display 31, and the model image data P1 on the sub-screen 312 of the display 31. Combine these data so that they are displayed in The combining unit 135 outputs the combined image data to the output I/F 14.

The output I/F 14 causes the display 31 to display the image data synthesized by the synthesis unit 135. As a result, the user image data P2 and the guide image data P3 are displayed on the main screen 311 of the display 31, and the model image data P1 is displayed on the sub screen 312. That is, the output I/F 14 causes the display 31 to display the user image data P2, the model image data P1, and the guide image data P3.

The memory 15 is a storage device that stores various information necessary for the processor 13 to perform information processing, computer programs executed by the processor 13, and the like. The memory 15 also stores user image data P2 acquired via the first input I/F 11, model image data P1 acquired via the second input I/F 12, and information on each part of the performer calculated by the analysis unit 131. Coordinate data, difference data, guide image data P3 generated by the generation unit 133, etc. are stored. The memory 15 is realized, for example, by a semiconductor memory.

[2. Concrete example]
A specific example of the dance practice system 100 according to the embodiment will be described below with reference to FIGS. 9 and 10. FIG. 9 is an explanatory diagram of a first specific example of the dance practice system 100 according to the embodiment. FIG. 10 is an explanatory diagram of a second specific example of the dance practice system 100 according to the embodiment. In each of FIGS. 9 and 10, "frame 1", "frame 2", "frame 3", "frame 4", and "frame 4" of the model image data P1 displayed on the sub-screen 312 of the display 31 are shown in the upper row. "Frame 5" and "Frame 6" are shown side by side in the horizontal direction with only the sub-screen 312 cut out. In each of FIGS. 9 and 10, "frame 2" and "frame 3" of the user image data P2 displayed on the main screen 311 of the display 31 are shown in the horizontal direction by cutting out only the main screen 311. are shown side by side. Furthermore, in each of FIGS. 9 and 10, the "current frame" represents the frame displayed on the display 31, and the "next frame" represents the frame displayed on the display 31 after the current frame. represents a frame.

In the first specific example shown in FIG. 9, "Frame 2", which is the current frame (specific frame) of user image data P2, and "Frame 2", which is the current frame of model image data P1, match. It shows the situation. In other words, in the first specific example, the movement of the user U1 is able to follow the movement of the performer without delay. Therefore, in the first specific example, the correction unit 132 of the processor 13 determines "frame 2" of the model image data P1 as the corresponding frame. Then, the correction unit 132 calculates "k1=0". "k1" is the value of "k" when the difference data D12 (n+k) is the minimum. Accordingly, the correction unit 132 calculates the guide correction value β1 as “β1=−k1=0”.

Next, the generation unit 133 of the processor 13 calculates the offset β as “β=β0+β1=2”. Then, the generation unit 133 generates guide image data P3 based on the calculated offset β. Here, the generation unit 133 generates "Frame 3" of the model image data P1 corresponding to "Frame 3" which is the next frame of "Frame 2" of the user image data P2 displayed on the display 31 from the previous frame. decided on. Furthermore, the generation unit 133 determines “frame 5”, which is a frame after the previous frame of the model image data P1 by an offset β (here, “β=2”), as the subsequent frame.

Next, the generation unit 133 reads from the memory 15 the difference data D1(3) and D1(4) corresponding to each of all frames from the previous frame to the subsequent frame (excluding the subsequent frame). Then, the generation unit 133 generates guide image data P3 based on the difference D obtained by adding all the read difference data D1(3) and D1(4).

In the first specific example, when "frame 3" which is the next frame of user image data P2 is displayed on the display 31, image data P33 corresponding to the left hand of user U1 and an image corresponding to the right hand of user U1 are displayed. Guide image data P3 including data P34 and image data P35 corresponding to the right foot of user U1 is displayed in a superimposed manner. In the first specific example, the arrows P331, P341, and P351 in the image data P33, P34, and P35 are generated with reference to the difference D. Difference D is a difference corresponding to each of all frames (excluding the subsequent frame) from "Frame 3" of model image data P1, which is the previous frame, to "Frame 5" of model image data P1, which is the subsequent frame. Calculated with reference to data D1(3) and D1(4). Similarly, in the first specific example, each destination point P332, P342, P352 in each image data P33, P34, P35 is generated with reference to "frame 5" of model image data P1, which is a subsequent frame.

The user U1 moves his right hand, left hand, and right foot according to the guide image data P3 while looking at the guide image data P3 shown in FIG. This makes it easier to maintain a state in which the user U1's movements can follow the performer's movements without delay.

On the other hand, in the second specific example shown in FIG. 10, "frame 2" is the current frame of the user image data P2, and "frame 1" is the previous frame of the model image data P1. This shows a situation in which the two are in agreement. In other words, in the second specific example, the movement of the user U1 lags behind the movement of the performer by one frame. Therefore, in the second specific example, the correction unit 132 of the processor 13 determines "frame 1" of the model image data P1 as the corresponding frame. Then, the correction unit 132 calculates "k1=-1". "k1" is the value of "k" when the difference data D12 (n+k) is the minimum. Accordingly, the correction unit 132 calculates the guide correction value β1 as “β1=−k1=1”.

Next, the generation unit 133 of the processor 13 calculates the offset β as “β=β0+β1=3”. Then, the generation unit 133 generates guide image data P3 based on the calculated offset β. Here, the generation unit 133 generates "Frame 3" of the model image data P1 corresponding to "Frame 3" which is the next frame of "Frame 2" of the user image data P2 displayed on the display 31 from the previous frame. decided on. Furthermore, the generation unit 133 determines "frame 6", which is a frame after the previous frame of the model image data P1 by an offset β (here, "β=3"), as the subsequent frame.

Next, the generation unit 133 generates difference data D1(3), D1(4), and D1(5) corresponding to each of all frames from the previous frame to the subsequent frame (excluding the subsequent frame) from the memory 15. read out. Then, the generation unit 133 generates guide image data P3 based on the difference D obtained by adding all the read difference data D1(3), D1(4), and D1(5).

In the second specific example, when "frame 3", which is the next frame of user image data P2, is displayed on the display 31, image data P33 corresponding to the left hand of user U1 and an image corresponding to the right hand of user U1 are displayed. Guide image data P3 including data P34 and image data P35 corresponding to the right foot of user U1 is displayed in a superimposed manner. In the second specific example, the arrows P331, P341, and P351 in the image data P33, P34, and P35 are generated with reference to the difference D. Difference D is a difference corresponding to each of all frames (excluding the subsequent frame) from "Frame 3" of model image data P1, which is the previous frame, to "Frame 6" of model image data P1, which is the subsequent frame. Calculated with reference to data D1(3), D1(4), and D1(5). Similarly, in the second specific example, each destination point P332, P342, P352 in each image data P33, P34, P35 is generated with reference to "frame 6" of the model image data P1, which is the subsequent frame.

The user U1 moves his right hand, left hand, and right foot according to the guide image data P3 while looking at the guide image data P3 shown in FIG. This makes it easier for the user U1's movements to follow the performer's movements without being delayed, even if the user U1's movements are behind the performer's movements at the current moment.

As described above, in the dance practice system 100 according to the embodiment, the processor 13 selects a specific frame of the user image data P2 for the determined frame (corresponding frame) out of the plurality of frames of the model image data P1. is delayed, the larger the time difference (guide correction value β1) between the determined frame and the specific frame, the more the guide image data P3 is generated by referring to the model image data P1 that is earlier in time. That is, in the dance practice system 100 according to the embodiment, the processor 13 controls the display 31 to display the guide image data P3 earlier as the time difference becomes larger.

Therefore, in the dance practice system 100 according to the embodiment, even if the movement of the user U1 is delayed and cannot follow the movement of the performer, the user U1 can continue to watch the guide image data P3 optimized according to the delay. You can practice dancing. Therefore, the dance practice system 100 according to the embodiment has the advantage that when the user U1 practices dancing, it is easier to imitate the movements of the performer, and the dance proficiency level is easier to improve.

By the way, in the embodiment, the reason why the generation unit 133 of the processor 13 determines "frame (n+1)", which is the next frame of "frame n", as the previous frame is as follows. That is, when the user image data P2 is displayed on the display 31, the guide image data P3 is displayed in a superimposed manner, but the guide image data P3 covers the entire range from the previous frame to the subsequent frame in the model image data P1. Generated with reference to the amount of change (amount of movement). Here, if the previous frame is determined to be "frame n", before displaying "frame n" of the user image data P2 on the display 31, a process of generating guide image data P3 corresponding to "frame n" is performed. must be carried out. Therefore, in this case, a delay occurs from the time the camera 2 captures an image of the user U1 until the display 31 displays "frame n" of the user image data P2.

On the other hand, in the embodiment, when displaying "frame n" of the user image data P2 on the display 31, "frame (n-1)" which is the frame before "frame n" is displayed on the display 31. The guide image data P3 corresponding to "frame n" generated at the time of display is superimposed and displayed. Thereby, in the embodiment, it is possible to reduce the delay from the time when the user U1 is imaged by the camera 2 to the time when "frame n" of the user image data P2 is displayed on the display 31. Here, while displaying "frame n" of the user image data P2 on the display 31, "frame (n+1)" is displayed in order to generate guide image data P3 corresponding to the next frame "frame (n+1)". It is set to the previous frame. Then, guide image data P3 corresponding to "frame (n+1)" can be displayed on the display 31 at the timing when "frame (n+1)" of user image data P2 is displayed.

Note that the previous frame may be determined as "frame n" and the subsequent frame as "frame (n+β)", but if the above delay is taken into account, the previous frame may be determined as "frame (n+1)" and the subsequent frame as "frame (n+β)". It is preferable to determine "frame (n+β+1)".

[3. Other embodiments]
Although the embodiments have been described above, the present disclosure is not limited to the above embodiments.

In the above embodiment, the model image data P1 may be streamed from the server 4. In this case, each time the processor 13 of the dance practice system 100 acquires the model image data P1 from the server 4, it analyzes the model image data P1 that has been obtained, and displays the obtained model image data P1 on the display 31. You may also do this. In addition, in this case, the analysis unit 131 of the processor 13 calculates, for each frame, the coordinate data L1(n) of each part of the performer in the frame, and the coordinate data L1(n-) of each part of the performer in the previous frame. What is necessary is to calculate the difference data D1(n) with respect to 1).

In the above embodiment, the user image data P2 may be displayed on the display 31 as a CG (Computer Graphics) model imitating the user U1 instead of the user U1. In this case, the processor 13 of the dance practice system 100 analyzes the image data captured and generated by the camera 2, and generates data obtained by analyzing the posture of the user U1 (coordinates of each part of the user U1) in the image data. The CG model may be generated based on the acquired data.

In the embodiment described above, the processor 13 allows the user U1 to follow the movement of the performer, for example, when a predetermined condition is satisfied that the guide correction value β1 continues to be zero for a predetermined period of time (a predetermined frame) or more. You may conclude that it has become so. In this case, the processor 13 may cause the display 31 to display guide image data P3 that instructs the user U1 to make the user U1's movements closer to those of the performer. In other words, when the time difference is less than or equal to the threshold value (here, the guide correction value β1 is zero), the processor 13 transmits the guide image data P3 based on the performer's movement with high resolution via the output I/F 14. It may also be displayed on the display 31.

For example, assume that before a predetermined condition is met, the processor 13 has generated and outputted guide image data P3 that instructs the movement of the right hand of the user U1. In this case, after the predetermined conditions are met, the processor 13 may generate and output guide image data P3 that instructs not only the movement of the right hand of the user U1 but also the movement of the fingers of the right hand of the user U1. good.

In the above embodiment, the model image data P1 is image data showing the movements of one performer, but is not limited to this. For example, the model image data may be image data showing the actions of a plurality of performers. In this case, the processor 13 may extract one performer from among the plurality of performers using a suitable image analysis algorithm, and use image data representing the extracted one performer's movements as the model image data P1.

In the above embodiment, the analysis of the model image data P1 by the analysis unit 131 is completed before displaying the model image data P1 on the display 31, but the analysis is not limited to this. For example, the analysis of the model image data P1 by the analysis unit 131 may be performed in parallel with the process of displaying the model image data P1 on the display 31.

In the above embodiment, the destination point in the guide image data P3 is a star mark, but it is not limited to this, and may be a circle mark or another shape such as a shape imitating a human hand or foot. Further, the arrow in the guide image data P3 is not limited to this, and may have other shapes such as a triangular mark or a solid line or a broken line. Further, in the embodiment, both the shading and the size of the reaching point and the arrow in the guide image data P3 are changed according to the amount of movement, but only one of them may be changed. Further, the reaching point and the arrow in the guide image data P3 may have constant shading and size regardless of the amount of movement.

In the embodiments described above, the position of the sub-screen 312 is not limited to the lower right corner of the display 31, but may be located elsewhere on the display 31. Furthermore, the shape of the sub-screen 312 is not limited to a rectangular shape, but may be any other shape. Further, the size of the sub-screen 312 may be larger or smaller than the size shown in FIG. 2.

Furthermore, in the embodiment described above, the dance practice system 100 is installed in the computer 3, but the present invention is not limited to this. For example, the dance practice system 100 may be realized by a server that communicates with each of the camera 2 and the display 31 via a network N1 such as the Internet. The server may be the same as or different from the video distribution server 4 in the embodiment. Further, for example, the dance practice system 100 may be realized by a general-purpose information terminal such as a smartphone or a tablet terminal. In this case, by installing an application for the dance practice system 100 on the information terminal, it is possible to realize the dance practice system 100 on the information terminal. Further, for example, the dance practice system 100 may be configured as a television receiver including the computer 3 and the display 31.

Furthermore, in the above embodiment, the dance practice system 100 is realized by a single device, but it may be realized by a plurality of devices. When the dance practice system 100 is realized by a plurality of devices, the functional components included in the dance practice system 100 may be distributed to the plurality of devices in any manner. For example, the dance practice system 100 may be realized in a distributed manner over multiple servers. Further, for example, the dance practice system 100 may be realized in a distributed manner between a server and a computer 3.

Furthermore, the communication method between devices in the above embodiment is not particularly limited. In the above embodiment, when two devices communicate, a relay device (not shown) may be interposed between the two devices.

Furthermore, the order of processing described in the above embodiment is an example. The order of multiple processes may be changed, and multiple processes may be executed in parallel. Further, the processing executed by a specific processing unit may be executed by another processing unit. Furthermore, part of the digital signal processing described in the above embodiments may be realized by analog signal processing.

Furthermore, in the above embodiments, each component may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory.

Additionally, each component may be realized by hardware. For example, each component may be a circuit (or integrated circuit). These circuits may constitute one circuit as a whole, or may be separate circuits. Further, each of these circuits may be a general-purpose circuit or a dedicated circuit.

Furthermore, the general or specific aspects of the present disclosure may be implemented in a system, device, method, integrated circuit, computer program, or computer-readable recording medium such as a CD-ROM. Further, the present invention may be realized by any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium. For example, the present disclosure may be implemented as a dance practice method executed by a computer, or may be realized as a program for causing a computer to execute such a dance practice method. Further, the present disclosure may be realized as a computer-readable non-transitory recording medium on which such a program is recorded. Note that the program here includes an application program for causing a general-purpose information terminal to function as the dance practice system of the above embodiment.

Other embodiments may be obtained by making various modifications to each embodiment that a person skilled in the art would think of, or may be realized by arbitrarily combining the components and functions of each embodiment without departing from the spirit of the present disclosure. These forms are also included in the present disclosure.

(summary)
As described above, the dance practice system 100 according to the first aspect includes the first input I/F 11, the second input I/F 12, the processor 13, and the output I/F 14. Processor 13 is an example of a signal processing circuit. The first input I/F 11 acquires user image data P2 including a plurality of frames indicating the user U1's actions based on image data generated by capturing an image with the camera 2. The second input I/F 12 acquires model image data P1 including a plurality of frames showing the movements of a performer serving as a dance model. The processor 13 generates guide image data P3 that instructs the user U1 so that the user U1's motions approximate those of the performer. The output I/F 14 causes the display 31 to display the user image data P2, the model image data P1, and the guide image data P3. The processor 13 determines which frame among the plurality of frames of the model image data P1 the specific frame acquired at a predetermined time point among the plurality of frames of the user image data P2 corresponds to. When the specific frame is delayed with respect to the determined frame among the plurality of frames of the model image data P1, the processor 13 displays the guide image data P3 on the display 31 as the time difference between the determined frame and the specific frame is larger. Controls the timing to display earlier.

According to this, there is an advantage that when the user U1 practices dancing, it is easy to improve the dance proficiency level.

Furthermore, in the dance practice system 100 according to the second aspect, in the first aspect, the processor 13 selects a frame that is after the predetermined time point by a time corresponding to the time difference, among the plurality of frames of the model image data P1. With reference to this, guide image data P3 is generated.

Further, in the dance practice system 100 according to the third aspect, in the first or second aspect, the processor 13 outputs the model image data P1 to the display 31 via the I/F 14 at a timing that the user U1 wants to perform the operation. The guide image data P3 is displayed on the display 31 via the output I/F 14 at a timing earlier than the timing.

According to this, there is an advantage that by viewing the guide image data P3, the user U1 can easily make his own movements follow the movements of the performer.

Further, in the dance practice system 100 according to the fourth aspect, in any one of the first to third aspects, the guide image data P3 is adjusted in size and size according to the difference between the user U1's motion and the performer's motion. Change at least one of the shades.

According to this, there is an advantage that by viewing the guide image data P3, the user U1 can easily grasp visually how much movement is required to perform the next action.

Furthermore, in the dance practice system 100 according to the fifth aspect, in any one of the first to fourth aspects, when the time difference is less than or equal to the threshold, the processor 13 performs a guide based on the performer's movement with high resolution. The image data P3 is displayed on the display 31 via the output I/F 14.

According to this, when the user U1 is able to follow the performer's movements, the user can be guided to imitate the performer's movements in more detail, which has the advantage of further improving dance proficiency. be.

Further, in the dance practice system 100 according to the sixth aspect, in any one of the first to fifth aspects, the guide image data P3 is a destination point P312 indicating the next position that the user U1 should reach on the display 31. to P352, and arrows P311 to P351 indicating the direction and amount of movement from the current position of the user U1 to the destination points P312 to P352 on the display 31.

According to this, there is an advantage that by viewing the guide image data P3, the user U1 can easily understand visually how to perform the next operation.

Further, in the dance practice method according to the seventh aspect, user image data P2 including a plurality of frames showing the movements of the user U1 based on image data captured and generated by the camera 2 is obtained, and a dance example is obtained. The model image data P1 including a plurality of frames showing the movements of a performer is obtained, guide image data P3 is generated that instructs the user U1 to make the movements of the user U1 approximate those of the performer (S32), and the user The image data P2, the model image data P1, and the guide image data P3 are displayed on the display 31. In the process of generating the guide image data P3, it is determined which frame among the plurality of frames of the model image data P1 a specific frame acquired at a predetermined time point among the plurality of frames of the user image data P2 corresponds to. Determine (S21, S22). In the above process, if the specific frame is delayed with respect to the determined frame (corresponding frame) among the plurality of frames of the model image data P1, the larger the time difference between the determined frame and the specific frame, the more the display 31 to display the guide image data P3 earlier (S23, S31, S32).

Furthermore, the program according to the eighth aspect causes one or more processors to execute the dance practice method according to the seventh aspect.

The present disclosure can be used in a system for supporting a user's dance practice.

100 Dance practice system 11 1st input I/F
12 2nd input I/F
13 Processor (signal processing circuit)
131 Analysis section 132 Correction section 133 Generation section 134 Superposition section 135 Synthesis section 14 Output I/F
15 Memory 2 Camera 3 Computer 31 Display 311 Main screen 312 Sub-screen 4 Server N1 Network P1 Model image data P2 User image data P3 Guide image data P31, P32, P33, P34, P35 Image data P311, P321, P331, P341, P351 Arrow P312, P322, P332, P342, P352 Achievement point U1 User

Claims

a first input interface that acquires user image data including a plurality of frames indicating user actions based on image data captured and generated by a camera;
a second input interface for acquiring model image data including a plurality of frames showing movements of a performer serving as a dance model;
a signal processing circuit that generates guide image data that instructs the user so that the user's movements approach those of the performer;
an output interface for displaying the user image data, the model image data, and the guide image data on a display,
The signal processing circuit includes:
determining which frame of the plurality of frames of the model image data a specific frame acquired at a predetermined time point among the plurality of frames of the user image data corresponds to;
When the specific frame lags behind the determined frame among the plurality of frames of the model image data, the larger the time difference between the determined frame and the specific frame, the more the guide image appears on the display. Control the timing of displaying data earlier,
Dance practice system.
The signal processing circuit includes:
generating the guide image data by referring to a frame after the predetermined time point by a time corresponding to the time difference among the plurality of frames of the model image data;
The dance practice system according to claim 1.
The signal processing circuit includes:
displaying the model image data on the display via the output interface at a timing when the user wants the user to perform the operation;
displaying the guide image data on the display via the output interface at a timing earlier than the timing;
The dance practice system according to claim 1 or 2.
The guide image data changes at least one of size and shading according to a difference between the user's motion and the performer's motion.
The dance practice system according to claim 1 or 2.
When the time difference is less than or equal to a threshold, the signal processing circuit causes the guide image data based on the performer's movements with increased resolution to be displayed on the display via the output interface.
The dance practice system according to claim 1 or 2.
The guide image data is
a destination point indicating a next location for the user to reach on the display;
and an arrow indicating the direction and amount of movement from the user's current position to the destination point on the display.
The dance practice system according to claim 1 or 2.
Obtaining user image data including a plurality of frames indicating user actions based on image data captured and generated by a camera,
Acquire model image data containing multiple frames showing the performer's movements as a dance model,
generating guide image data that instructs the user so that the user's movements approach those of the performer;
displaying the user image data, the model image data, and the guide image data on a display;
In the process of generating the guide image data,
determining which frame of the plurality of frames of the model image data a specific frame acquired at a predetermined time point among the plurality of frames of the user image data corresponds to;
When the specific frame lags behind the determined frame among the plurality of frames of the model image data, the larger the time difference between the determined frame and the specific frame, the more the guide image appears on the display. Control the timing of displaying data earlier,
How to practice dancing.
one or more processors,
carrying out the dance practice method according to claim 7;
program.