JP2019028509A - Detection device, detection system, image processing device, detection method, image processing program, image display method, and image display system - Google Patents

Abstract

To allow a user to easily comprehend information on a posture of a user.SOLUTION: A detection device comprises: a detection unit which detects a user; a model generation unit which generates a model of the user, based on detection result of the detection unit; and an image processing unit which positions a teaching figure including a plurality of parts for each part, based on a posture of the user acquired from the detection unit, for the model of the user generated by the model generation unit, and generates a teaching image including the model of the user and the teaching figure.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a detection device, a detection system, an image processing device, a detection method, an image processing program, an image display method, and an image display system.

  A technique for providing a user with a posture during training in real time has been proposed (for example, Patent Document 1 below). It is desired that information such as the user's posture is provided so that the user can easily grasp the information.

JP 2011-19627 A

  According to the first aspect of the present invention, a detection unit that detects a user, a model generation unit that generates a user model based on a detection result of the detection unit, and a user model generated by the model generation unit And an image processing unit that positions a teaching figure including a plurality of parts for each part based on the posture of the user obtained from the detecting unit, and generates a teaching image including the user model and the teaching figure. Is provided.

  According to a second aspect of the present invention, there is provided a detection system including the detection device according to the first aspect and a display unit that displays a teaching image generated by the image processing unit.

  According to the fourth aspect of the present invention, for a user model generated based on a detection result of detecting a user, a teaching figure including a plurality of parts is converted into a part based on the posture of the user obtained from the detection result. An image processing apparatus is provided that generates a teaching image including a user's model and a teaching figure by positioning each of them.

  According to the fifth aspect of the present invention, detecting a user, generating a user model based on the detection result of detecting the user, and obtaining the user's model from the detection result A detection method including positioning a teaching figure including a plurality of parts for each part based on the posture and generating a teaching image including a user model and the teaching figure is provided.

  According to the sixth aspect of the present invention, the computer includes a plurality of parts based on the user's posture obtained from the detection result with respect to the user's model generated based on the detection result of detecting the user. An image processing program for positioning a figure for each part and generating a teaching image including a user model and a teaching figure is provided.

  According to the seventh aspect of the present invention, the display unit dynamically displays the user's model that changes with time, and the display unit displays the model of the user at a position determined with respect to the user's model. There is provided an image display method including dynamically displaying a teaching figure selected based on a temporal change.

  According to the eighth aspect of the present invention, a display unit that displays an image, a first display processing unit that dynamically displays a user model that changes over time on the display unit, and a user model in the display unit There is provided an image display system including a second display processing unit that dynamically displays a teaching figure selected based on a temporal change of a user's model at a position determined with respect to.

It is a figure showing a detection system concerning a 1st embodiment. It is a figure which shows the teaching image which concerns on 1st Embodiment. It is a figure which shows the teaching image which concerns on 1st Embodiment. It is a figure which shows the process by the detection apparatus which concerns on 1st Embodiment. It is a conceptual diagram which shows the attitude | position data, teaching figure data, and display condition data which concern on 1st Embodiment. It is a figure which shows the teaching image which concerns on 1st Embodiment. It is a figure which shows the teaching image which concerns on 1st Embodiment. It is a figure which shows operation | movement of the detection system which concerns on 1st Embodiment. It is a figure which shows operation | movement of the detection system which concerns on 1st Embodiment. It is a figure which shows operation | movement of the detection system which concerns on 1st Embodiment. It is a figure which shows operation | movement of the detection system which concerns on 1st Embodiment. It is a figure which shows operation | movement of the detection system which concerns on 1st Embodiment. It is a figure which shows operation | movement of the detection system which concerns on 1st Embodiment. It is a figure which shows operation | movement of the detection system which concerns on 1st Embodiment. It is a figure which shows operation | movement of the detection system which concerns on 1st Embodiment. It is a flowchart which shows operation | movement of the detection system which concerns on 1st Embodiment. It is a flowchart which shows the process by the image process part which concerns on 1st Embodiment. It is a figure which shows the detection system which concerns on 2nd Embodiment. It is a figure which shows the process by the attitude | position identification part which concerns on 2nd Embodiment. It is a figure which shows the process by the image process part which concerns on 2nd Embodiment. It is a flowchart which shows the process by the image process part which concerns on 2nd Embodiment. It is a figure which shows the detection system which concerns on 3rd Embodiment. It is a figure which shows the process by the figure division part which concerns on 3rd Embodiment. It is a figure which shows the detection system which concerns on 4th Embodiment. It is a figure which shows the process of the log | history management part which concerns on 4th Embodiment. It is a figure which shows the user interface which concerns on 5th Embodiment. It is a figure which shows the example of the teaching image which the image process part which concerns on 6th Embodiment produces | generates. It is a figure which shows the example of the teaching image which the image process part which concerns on 6th Embodiment produces | generates. It is a figure which shows the detection system which concerns on 7th Embodiment.

[First Embodiment]
A first embodiment will be described. FIG. 1 is a diagram illustrating a detection system and a detection apparatus according to the first embodiment. FIG. 1 is a block diagram illustrating a detection system 1 (information providing system) according to the embodiment. The detection system 1 includes a detection device 2 and a display unit 3 (display device).

  For example, the detection device 2 detects the user U who moves so as to approach a predetermined posture (eg, target, basic, ideal, sample) (pose, form). The detection device 2 is used in various fields such as sports (eg, baseball, soccer, golf, gymnastics, exercise, body building), fashion shows (eg, walking, posing), games, or person authentication (eg, security). Is available. For example, the user U may be a player of a game on the theme of yoga, and the detection device 2 may detect the user U who plays the game.

  In the following description, it is assumed that the detection device 2 detects a user U taking a predetermined posture in yoga (eg, warrior pose, triangle pose). In addition, when the detection apparatus 2 detects the user U, a state in which an attachment (eg, an object that moves with the user U) such as clothing, shoes, or a protector may be attached. The detection of the user U by the detection device 2 may include detection of a wearing object that moves with the user U.

  The display unit 3 displays various images to the user U. The display unit 3 is communicably connected to the detection device 2 by wire or wireless. The display unit 3 displays an image based on the image data supplied from the detection device 2. The display unit 3 includes, for example, a direct view display unit such as a flat panel display (FPD) or a head mounted display (HMD), or a projection display unit such as a projector. The display unit 3 may be a stationary display unit such as an FPD or a portable (mobile device) or a wearable (wearable device) display unit such as an HMD.

  Hereinafter, an example of an image displayed on the display unit 3 will be described with reference to FIGS. 1 to 3, and then the detection device 2 will be described. 2 and 3 are conceptual diagrams showing images displayed on the display unit 3, respectively. 2 and 3 correspond to warrior poses in yoga (Vilabadra Asana). Image Im1 to Image Im3 in FIG. 2 correspond to a series of operations in which the user opens his / her leg, raises his hand and stretches his spine and inhales deeply.

  The display unit 3 displays a pre-registered posture image (hereinafter referred to as a sample image Sa). For example, the sample image Sa is arranged so as to overlap with a part of the image Im1 (eg, upper left). The sample image Sa is, for example, an image (teacher image) that is referred to when the user takes a target posture. The sample image Sa includes, for example, an image (eg, a still image or a moving image) in which a yoga instructor takes a predetermined pose (eg, a warrior's pose). The user U can confirm the type of pose to be performed next, for example, by looking at the sample image Sa displayed on the display unit 3.

  Note that the detection system 1 may include a notification unit that notifies information on a predetermined pose. The information on the predetermined pose includes, for example, one or both of the name of the predetermined pose and the operation description related to the predetermined pose. The notification unit outputs information on a predetermined pose in a form that can be identified by the user (eg, image, sound). For example, the notification unit includes a display unit 3 that outputs a sample image Sa as information on a predetermined pose. In addition, the notification unit described above is a voice notification unit (e.g., a voice output unit, a voice guide unit) that reports information on a predetermined pose corresponding to the sample image Sa when the display unit 3 displays the sample image Sa. ) May be included.

  For example, when displaying the sample image Sa of a predetermined pose (for example, a warrior's pose), the detection system 1 (for example, the voice notification unit) reads a name of the predetermined pose (for example, “warrior's pose”). A voice (voice guide) may be output. Further, the detection system 1 (eg, a voice notification unit) may output music, music, and BGM that are associated with the warrior pose in advance, instead of the voice “warrior pose”. For example, when proceeding to the next pose of yoga, the detection system 1 (for example, a voice notification unit) may change music, music, BGM, and the like output as voice.

  In addition, for example, when the detection system 1 (eg, a voice notification unit) displays a sample image Sa of a predetermined pose (eg, a warrior's pose), as an explanation of the above operation, at least a part of the human body You may output the voice (for example, “please breathe in”, “stretch your arm horizontally”, “bend your legs vertically”) that reads out the words (phrases) that represent the movement. In addition, the detection system 1 (eg, voice notification unit) may output the above voice (eg, voice guide) instead of displaying the sample image Sa.

  The display unit 3 displays a model of the user U (referred to as a user model UM as appropriate). The detection apparatus 2 in FIG. 1 detects the user U, and generates a user model UM indicating the user's posture and the like based on the detection result. The user model UM is a figure representing the user U's two-dimensional shape or three-dimensional shape. The user model UM is represented by, for example, a silhouette, a polygon, a voxel, a wire frame, a collection of points on the surface of the user U, an avatar, or a skeleton model. Further, for example, the layout of an object (eg, user model UM, teaching image, sample image Sa) displayed on the display unit 3 is such that the sample image Sa is arranged at the upper left of the display unit 3 and the user is displayed at the center of the display unit 3. This is a layout in which a model UM is arranged and a teaching image is arranged around (close to) the user model UM.

  The detection device 2 in FIG. 1 optically detects the user U repeatedly, and the display unit 3 displays (updates) the user model UM in real time based on the detection result of the detection device 2. The display unit 3 displays, for example, an image (for example, the image Im1) including the sample image Sa and the user model UM.

  The display unit 3 also displays an image (eg, image Im1 to image Im6) of the teaching user U including the user model UM and the teaching figure F. The teaching figure F constitutes, for example, a user interface when the user U takes a predetermined posture. The teaching image includes, for example, an image that expresses points to be noted when taking a predetermined posture so that the user U can intuitively understand. As described above, the display unit 3 displays the sample image Sa, the user model UM, the teaching image, and the like as a user interface in a predetermined arrangement.

  The teaching figure F includes, for example, at least one of a figure representing a part of the human body, a figure representing the shape of the human body, and a figure representing the action of the human body. In the present specification, the “graphic” includes at least one of a line segment, a curve, or a shape including a line segment and a curve (eg, geometric shape), a character, and a symbol. Examples of the geometric shape include a circle, an ellipse, a polygon, a shape obtained by rounding the corners of a polygon, and an ellipse.

  Moreover, the character contained in a figure in this specification contains one or both of an ideogram and a phonetic character. The ideographic characters are, for example, Chinese characters, Arabic numerals (1, 2,...), Etc., and are characters that represent meaning with one character. Display characters include, for example, kanji characters that represent parts of the human body such as bones, armpits, shoulders, necks, kanji characters that represent human movements such as vomiting, calling, sucking, and stretching, Examples include kanji characters that represent shapes.

  A phonetic character (phonetic character) is a character that represents a phoneme or syllable with a single character, such as hiragana, kana, alphabet, hangul, and the like. Note that Chinese characters are sometimes used as phonetic characters in China and the like. The number of characters included in the graphic may be one or two or more. For example, the graphic in the present specification may include a character string composed of two or more characters. In addition, the graphic in this specification may include a word, a phrase, or a sentence composed of a plurality of phonetic characters.

  In at least one image of the teaching image, the teaching graphic includes a plurality of parts (eg, elements, parts, parts, segments). For example, in the image Im1, the teaching figure F includes a part p1 and a part p2. The part p1 is a character “sucking” and is a figure representing the motion of the human body. The part p1 “suck” is displayed in association with the action of the user U breathing. For example, when the user U performs yoga with reference to the sample image Sa, “sucking” as the part p1 is displayed on the display unit 3 based on the timing at which the sample image Sa corresponding to the action of the human body breathing is displayed. Is displayed. For example, when the user U who has seen the sample image Sa during yoga performs this operation, the part p1 “suck” is displayed on the display unit 3.

  The part (for example, part p1) of the teaching figure F is positioned with respect to the user model UM by the detection device 2 (processing unit 6). For example, “suction” which is the part p1 in the image Im1 is positioned with respect to a part of the human body of the user U (eg, mouth, chest) related to breathing in the user model UM, and has a predetermined positional relationship with this part. Is displayed. For example, when moving from the image Im1 to the image Im3, it is assumed that the user U stretches his / her back, and the user model UM is such that the “puff” that is the part p1 moves upward with respect to the user model UM. Is positioned with respect to. For example, the user U can intuitively perform the action of extending the spine by having the image that the body is pulled upward as the “sucking” of the part p1 moves upward. For example, the teaching figure according to the embodiment is selected from figures whose actions are deformed so as to remind the user U of an action image.

  Further, the part (eg, part p1) of the teaching figure F may be represented such that the color (at least one of hue, lightness, and saturation) changes entirely or partially. For example, in FIG. 1, the part p1 “sucking” corresponds to the progress (time) of the breathing operation from the state in which the contrast with the surroundings is relatively low (indicated by a dotted line in the image Im1). The contrast changes to a relatively high state (indicated by a solid line in the image Im2). For example, the color of the part p1 “suck” changes so as to fade in when proceeding from the image Im1 to the image Im2. At least one part of the plurality (at least two) parts in the teaching figure F is determined based on, for example, a teacher (eg, yoga instructor) operation (eg, pose of the sample image Sa) in the sample image Sa. The display position is changed at a timing (for example, when moving from the image Im1 to the image Im2) (the target part moves at a predetermined moving speed in the screen of the display unit 3).

  The part p2 of the teaching figure F is a line-shaped (band-like) figure starting from the hand of the user model UM. The length of the part p2 changes so as to extend downward from the position of the hand of the user model UM when proceeding from the image Im1 to the image Im2, and further from the image Im2 to the image Im3. The part p3 of the teaching figure F in the image Im3 is a figure (here, a circle) representing the waist center (base) of the human body. The part p3 is displayed to fade in when the part p2 extends from the hand of the human body to the center of the waist.

  The images Im4 to Im6 in FIG. 3 correspond to a series of operations in which the user U exhales deeply while dropping his / her waist so that the knee of the front foot is bent vertically. In the image Im4, the teaching figure F includes a part p3, a part p4, a part p5, and a part p6. The part p3 is a graphic representing the position of the human waist center in the user model UM. The part p4 is, for example, a graphic indicating the posture of the human body foot. A part p4 in the image Im4 and the image Im5 is a line-shaped (band-like) figure extending horizontally with respect to the user model UM, and represents an image for keeping the thigh of the human body horizontal. Further, the part p4 in the image Im6 is a polygonal line shape extending horizontally from the user model UM and extending further downward, and represents an image in which the knee is bent vertically.

  Further, the part p5 of the teaching figure F is “mouth” which is a part (eg, radical) of the Chinese character “vomit”. The part p6 is “Sat”, which is a part (eg, radical) of the Chinese character “Yo”. The part p5 and the part p6 represent a kanji character called “vomit” as a set. “Vomiting” composed of the parts p5 and p6 is displayed in association with the action of the user exhaling. For example, when the user U performs yoga with reference to the sample image Sa, the part p5 and the part p6 are displayed based on the timing at which the sample image Sa corresponding to the action of the human body exhaling is displayed. For example, when the user U who has seen the sample image Sa performs this operation, “vomit” is displayed.

  The parts p5 and p6 are each positioned with respect to the user model UM. For example, in the image Im4, the part p5 and the part p6 are positioned with respect to a part (eg, mouth, chest) related to respiration in the user model UM, and a predetermined reference position (eg, close to the part) with respect to this part Position).

  The character “vomit” in FIG. 3 is expressed by an image of exhaled breath, for example. For example, “vomit” in the image Im5 is farther from the user model UM than in the image Im4. The “mouth” of the part p5 and the “soil” of the part p6 in the image Im5 are arranged (positioned) at positions farther from the user model UM (reference position in the user model UM) than the image Im4. In addition, the “vomit” in the image Im5 is deformed so as to be dissipated away from the user model UM compared to the image Im4. For example, the “soil” of the part p6 in the image Im5 is arranged (positioned) at a position farther from the “mouth” of the part p5 than the image Im4. Further, for example, the “soil” of the part p6 in the image Im5 is larger (or larger) than the image Im4 so that the shape of the “soil” of the part p6 that is a part of the teaching figure extends in one direction or two directions. As may be changed).

  In addition, “vomit” in the image Im6 is farther from the user model UM than in the image Im5. In the image Im6, the “mouth” of the part p5 and the “soil” of the part p6 are positioned at positions farther from the user model UM than the image Im5. In the image Im6, “vomit” is expressed as an image in which the “mouth” and the “earth” are dissociated and dissipated compared to the image Im5. In the image Im6, the “powder” of the part p6 is positioned at a position farther from the “mouth” of the part p5 than the image Im5. As described above, the teaching figure may include a deformed Chinese character, and at least one of the plurality of parts may include a radical of the deformed Chinese character.

  Further, when proceeding from the image Im5 to the image Im6, “vomiting” is a state in which the contrast with the surroundings is relatively high (indicated by a solid line in the image Im5) and the contrast with the surroundings is relatively low (image Im6). (Indicated by a dotted line). For example, the elements (eg, contrast, hue, lightness, and saturation) of the display (eg, color) change so that “voking” in the image Im6 fades out.

  As described above, the image display method according to the embodiment dynamically displays the model of the user U that changes over time on the display unit 3 (first display process), and the display unit 3 displays the user U's model. Dynamically displaying a teaching figure F selected based on a temporal change in the model of the user U at a position determined with respect to the model (second display processing).

  The image display method according to the embodiment may include dynamically displaying a model of the user U obtained by detecting the user U. The dynamic display of the teaching figure F includes, for example, moving the teaching figure F at a predetermined moving speed in the image (changing the position of the teaching figure in the image with time), and deforming the teaching figure F. And at least one of changing the display (eg, color, hue, lightness, saturation, contrast) of the teaching figure F.

  Further, the image display method according to the embodiment may include displaying a figure dynamically by positioning the figure with respect to the model of the user U that is dynamically displayed. This type of figure may be associated in advance with the operation (eg, posture) of the user U, for example. Moreover, the timing at which this graphic is displayed may be associated in advance with the timing of the operation of the user U, for example. Hereinafter, each part of the detection apparatus 2 according to the embodiment will be described.

  The detection device 2 of FIG. 1 includes a detection unit 5, a processing unit 6, a display control unit 7, and a storage unit 8. In the following description, FIG. 4 will be referred to as appropriate. FIG. 4A is a conceptual diagram illustrating a detection result of the detection unit, and FIG. 4B is a conceptual diagram illustrating a processing result by the processing unit. The detection unit 5 optically detects the user U. For example, the detection unit 5 detects the user U three-dimensionally. The detection unit 5 includes, for example, a depth sensor that detects a distance. The detection unit 5 detects, for example, the distance from a predetermined viewpoint (eg, position of the detection unit 5, viewpoint) to each point on the surface of the user U. For example, the detection unit 5 repeatedly performs detection processing at a predetermined frequency (eg, sampling rate, frame rate).

  The detection unit 5 detects the distance by, for example, the TOF (time of flight) method. The detection unit 5 outputs, for example, position information indicating a distance distribution as the detection result (eg, position information). The position information includes, for example, depth information (eg, depth image data, depth information, depth map). For example, the detection unit 5 outputs data of a depth image Im7 illustrated in FIG. In the depth image Im7, the pixel value of each pixel represents, for example, the distance between a point on the real space corresponding to the position of the pixel on the depth image Im7 and the detection unit 5. For example, the depth image Im7 is expressed in gray scale, and the dark part is more distant from the detection unit 5 than the bright part.

  Note that the detection unit 5 may be a device that detects a distance by a method other than the TOF method. The detection unit 5 may be, for example, a device that includes a laser scanner (for example, a laser range finder) and detects a distance by laser scanning. The detection unit 5 may be, for example, a device that projects a predetermined pattern (eg, a polka dot pattern or a lattice pattern) onto an area where the user U exists, and measures the distance based on the detection result of this pattern. The detection unit 5 may be, for example, a device that includes a phase difference sensor and detects a distance by a phase difference method. The detection unit 5 may be, for example, a device that detects a distance by a DFD (depth from defocus) method.

  The detection unit 5 may be a device that detects a distance by triangulation using captured images obtained by capturing the user U from a plurality of viewpoints using, for example, a stereo camera. The detection unit 5 may be a device that detects the user U by combining two or more of the plurality of detection methods described above. The detection unit 5 may include a sensor that detects a distance and an image sensor. Further, the detection unit 5 may detect the user U by a method other than the optical method (for example, scanning with ultrasonic waves). Further, the detection unit 5 determines the distance from the feature of the user U image obtained from a predetermined viewpoint (eg, a single viewpoint or a plurality of viewpoints) (such as a feature point obtained by image processing of the user U image). It may be estimated.

  The processing unit 6 is an information processing unit that processes information (for example, a detection result) output from the detection unit 5. The processing unit 6 is communicably connected to the detection unit 5 by wired or wireless communication. The processing unit 6 acquires (eg, receives) depth information from the detection unit 5 as a detection result of the detection unit 5. The processing unit 6 may acquire the detection result or the processing result of the detection unit 5 without communication with the detection unit 5. For example, the processing unit 6 may acquire the detection result from the detection unit 5 via a storage medium such as a nonvolatile memory.

  The processing unit 6 includes a model generation unit 11, a part specifying unit 12, and an image processing unit 13. The model generation unit 11 generates a model of the user U (for example, the user model UM in FIG. 2) based on the detection result of the detection unit 5. The model generation unit 11 detects (calculates) the position of each part of the human body of the user U based on the detection result of the detection unit 5. For example, the model generation unit 11 generates (calculates) shape information representing the three-dimensional shape of the user U by processing the data of the depth image Im7 in FIG. 4A (eg, perspective conversion processing, projective conversion processing). ) The shape information includes, for example, point cloud data. The point cloud data includes, for example, a plurality of point data on the surface of the user U. The plurality of point data are each represented by three-dimensional coordinates.

  The model generation unit 11 may calculate surface information as shape information. The surface information is, for example, polygon data, vector data, draw data, or the like. The surface information includes, for example, coordinates of a plurality of points and connection information between the plurality of points. The connection information includes, for example, information that associates points at both ends of a line corresponding to a ridge line (eg, edge) of an object (eg, human body). The connection information includes, for example, information that associates a plurality of lines corresponding to the contour of the surface on the object.

  For example, the model generation unit 11 estimates a surface between a point selected from a plurality of points included in the point cloud data and its neighboring points, and converts the point cloud data into polygon data having plane information between the points. It may be converted. For example, the model generation unit 11 converts the point cloud data into polygon data by an algorithm using a least square method. This algorithm may be, for example, an algorithm published in the point cloud processing library.

  The part specifying unit 12 specifies the part of the user U based on the detection result of the detecting unit 5. The part specifying unit 12 generates position information of a characteristic part (eg, characteristic part, feature point) of the human body of the user U based on the detection result of the detection part 5. The characteristic part of the human body is, for example, a part that can be distinguished from other parts of the human body. The characteristic part of the human body includes, for example, at least one of an end portion of the human body, a joint, or an intermediate portion between the end portion and the joint or between the two joints.

  The part specifying unit 12 generates position information of the part of the user U (eg, (skeleton information SD shown in FIG. 4B)) as the processing result. In the skeleton information SD of FIG. (◯) corresponds to a characteristic part, for example, by the recognition process (eg, pattern recognition, shape recognition, skeleton recognition) using the point cloud data generated by the model generation unit 11 and the like. The position information of the characteristic part includes, for example, the coordinates (eg, three-dimensional coordinates) of the point representing the characteristic part, for example, by the above recognition process. The coordinates of a point representing the characteristic part are calculated.

  For example, the part specifying unit 12 generates data that associates the name of the characteristic part and the position of the characteristic part for each of the characteristic parts. The model generation unit 11 generates table data including the coordinates of one or more preset characteristic parts as position information (eg, skeleton information SD). The position information (eg, skeleton information SD) includes, for example, position information of a plurality of characteristic parts obtained from detection results at the same detection timing.

  The image processing unit 13 uses the detection result (for example, the captured image) to teach the user U's model (for example, the user model UM in FIG. 2) and the teaching figure F (see FIG. 2) (for example, FIG. 2, FIG. 2). An image Im6) is generated from the image Im1 in FIG. When generating the teaching image, the image processing unit 13 applies the teaching figure F (see FIG. 2) to each part (eg, part p1 and part p2 in FIG. 2) for the user model generated by the model generating unit 11. Position to. The image processing unit 13 positions the teaching figure F for each part based on the user's posture obtained from the detecting unit 5. For example, the image processing unit 13 teaches the user's posture obtained from the detection unit 5 based on the position information (for example, the skeleton information SD in FIG. 4B) of the user's part generated by the part specifying unit 12. Position figure F for each part.

  The image processing unit 13 generates a teaching image based on, for example, posture definition data D1, teaching graphic data D2, and display condition data D3. Posture definition data D1, teaching figure data D2, and display condition data D3 are stored in the storage unit 8 in advance. The storage unit 8 includes, for example, a nonvolatile memory or a hard disk.

  FIG. 5 is a conceptual diagram showing posture definition data, teaching graphic data, and display condition data according to the first embodiment. FIG. 5A conceptually shows the posture definition data D1. The posture definition data D1 is, for example, information (eg, table) that is a set of information indicating the type of posture (eg, “posture name”) and information indicating the definition of posture (eg, “sample image data”). Data). The “posture name” is, for example, “warrior pose”, “triangular pose”, or the like.

  In FIG. 5A, reference symbol Sa <b> 1 is a sample image corresponding to “warrior pose”. Reference numeral Sa <b> 2 is a sample image corresponding to “triangular pose”. In the “sample image data” in the row whose “posture name” is “warrior pose”, “warrior pose.dat” which is image data indicating the sample image Sa is used as information indicating the definition of “warrior pose”. The information indicating the location of is stored. The “warrior pose.dat” may be, for example, a still image format file (eg, jpg, bitmap) or a moving image format file (eg, mpeg, avi).

  Note that the user can edit the posture definition data D1 as appropriate. For example, the user can register information (hereinafter referred to as a definition file) in which the posture type and the posture definition are set as one set in the posture definition data D1. Further, the user can delete the definition file from the posture definition data D1. The above definition file may be acquired automatically or manually via an Internet line or the like. The acquired definition file may be registered in the posture definition data D1 automatically or manually.

  FIG. 5B conceptually shows the teaching graphic data D2. The teaching figure data D2 includes, for example, teaching figure identification information (eg, teaching figure identification number), part identification information (eg, part identification number) included in the teaching figure, and information (eg, (Shape data) as a set. “Teaching figure identification number” is an identification number assigned to each teaching image. For example, data whose “teaching figure identification number” is “01” corresponds to the teaching figure F shown in FIG.

  Further, data having “teaching figure identification number” “01” and “part identification number” “01” corresponds to the part p2 (line segment) in FIG. The symbol Imp1 is an image corresponding to the part p2 (line segment). The “shape data” whose “teaching figure identification number” is “01” and whose “part identification number” is “01” includes data defining the shape of the part p2 (eg, “form — 0101.dat”). Stored. “Form — 0101.dat” includes, for example, vector data or raster data representing the image Imp1.

  Further, data having “teaching figure identification number” “01” and “part identification number” “02” corresponds to the part p3 (circle) in FIG. The symbol Imp2 is an image corresponding to the part p3 (circle). In “shape data” whose “teaching figure identification number” is “01” and whose “part identification number” is “02”, data defining the shape of the part p3 (eg, “form — 0102.dat”) is included. Stored. “Form — 0102.dat” includes, for example, vector data or raster data representing the image Imp2.

  Further, data having “teaching figure identification number” “01” and “part identification number” “03” corresponds to “suction” which is the part p1 in FIG. The symbol Imp3 is an image corresponding to the part p3 “suck”. The “shape data” whose “teaching figure identification number” is “01” and whose “part identification number” is “03” includes data defining the shape of the part p1 (eg, “form — 0103.dat”). Stored. “Form — 0103.dat” includes, for example, vector data or raster data representing the image Imp3. “Form — 0103.dat” may include, for example, character data representing the image Imp3. The character data includes, for example, a character code, a font type, a font size, and the like.

  The data with “teach figure identification number” “i” corresponds to the teach figure F (for example, “eject”) shown in FIG. Data with “teach figure identification number” “i” and “part identification number” “01” corresponds to “mouth” of part p5 in FIG. The symbol Imp5 is an image corresponding to the “mouth” of the part p5. In “shape data” whose “teaching figure identification number” is “i” and whose “part identification number” is “01”, data defining the shape of the part p5 (eg, “form_i01.dat”) is included. Stored. “Form_i01.dat” includes, for example, vector data or raster data representing the image Imp5.

  Further, data having “teach figure identification number” “i” and “part identification number” “02” corresponds to “soil” of part p6 in FIG. The symbol Imp6 is an image corresponding to the “soil” of the part p6. In “shape data” whose “teaching figure identification number” is “i” and whose “part identification number” is “02”, data defining the shape of the part p6 (eg, “form_i02.dat”) is included. Stored. “Form_i02.dat” includes, for example, vector data or raster data representing the image Imp6.

  Note that the image Imp5 corresponding to the part p5 and the image Imp6 corresponding to the part p6 are combined to form an image Imp7 corresponding to “discharge”. When there is a character code corresponding to the part (eg, “mouth” or “earth” in “voking”), the shape data of the part may be represented by character data.

  FIG. 5C conceptually shows the display condition data D3. The display condition data D3 includes, for example, identification information (eg, teaching figure identification number) of a teaching figure, identification information (eg, part identification number) of a part included in the teaching figure, and information (eg, an example) that defines display timing. Display start condition, display end condition) and information defining the display position (eg, display start position, display movement condition).

  The “teaching figure identification number” and the “part identification number” are the same as the teaching figure data D2 in FIG. For example, data having “teaching figure identification number” of “01” and “part identification number” of “03” corresponds to “suction” which is the part p1 of FIG. Here, a description will be given by taking, as an example, a part (“pick” that is part p1) whose “teaching figure identification number” is “01” and “part identification number” is “03”.

  The “display start condition” is information that defines a condition (eg, timing, trigger) for starting display of corresponding shape data (eg, “suction” in FIG. 5B). When it is determined that the “display start condition” is satisfied for each part, the image processing unit 13 in FIG. 1 starts displaying the part. For example, the display start condition of “suck” that is the part p1 is defined as the time when a predetermined time T1 has elapsed since the display of the sample image Sa of the pose corresponding to this part (for example, the pose of the warrior in FIG. 2). ing. When the image processing unit 13 determines that the “display start condition” is satisfied for “sucking” that is the part p1, the image processing unit 13 generates a teaching image including the part p1. By supplying the teaching image data to the display unit 3, “suck” as the part p <b> 1 is displayed on the display unit 3.

  As described above, the display unit 3 displays an image of an attitude registered in advance (for example, a sample image) to the user, and the teaching image is displayed on the display unit according to a predetermined time schedule. The teaching image is displayed on the display unit 3 at a timing based on the display timing of the sample image, for example.

  “Display start position” is information that defines the display position at the start of display of the corresponding shape data (eg, “suction” in FIG. 5B). The “display start position” includes, for example, information defining a reference position (eg, “reference part name”) and information defining an offset amount with respect to the reference position (eg, “offset amount”).

  For example, “suck” in FIG. 2 is positioned with respect to the chest position of the user model UM. In this case, information specifying “chest” is stored as “reference part name”. The “offset amount” includes an offset amount with respect to a reference position on the image (eg, the coordinates of “chest”) on a scale (eg, the number of pixels in the horizontal scanning direction and the number of pixels in the vertical scanning direction). It is prescribed. The offset amount may be a fixed value, for example, a variable value based on the scale of the user model UM on the image.

  Further, the offset amount is based on the comparison result between the posture of the user U (for example, the user model UM) obtained from the detection result of the detection unit 5 and the posture registered in advance (for example, the posture in the sample image Sa). It may be set. For example, when the offset amount of the first part is relatively large with respect to the offset amounts of other parts, or when the offset amount of the first part is larger than a predetermined value, the first part is The user U may be shown to increase the operation of the part corresponding to the part.

  The image processing unit 13 in FIG. 1 determines the position of the part specified by the “reference part name” based on the coordinates of the part specified by the part specifying unit 12 (for example, “chest” in the user model UM on the image). Get coordinates). Further, the image processing unit 13 calculates a position shifted from the reference position by the offset amount defined in the “offset amount”, with the position of the portion specified in the “reference portion name” as the reference position. The determined position is determined (positioned) as the “sucking” position of the part p1.

  For example, when the coordinates of “chest” in the user model UM on the image are represented by (X1, Y1) and the offset amount is represented by (ΔX1, ΔY1), the image processing unit 13 performs “suction” which is the part p1. Is determined as (X1 + ΔX1, Y1 + ΔY1)). The image processing unit 13 arranges “suction” of the image Imp3 defined in the shape data (form — 0103.dat) in the teaching graphic data D2 of FIG. 5B at the determined position (X1 + ΔX1, Y1 + ΔY1). 2 is generated as the teaching image. As described above, the image processing unit 13 is configured so that the part of the user among a plurality of parts is relative to the position (for example, (X1, Y1)) on the user model UM corresponding to the part of the user specified by the part specifying unit 12. A part (eg, “suck”) previously associated with (eg, “chest”) is positioned.

  The “display movement condition” in FIG. 5C is information that defines the movement of the corresponding shape data (eg, “suction” in FIG. 5B). The “display movement condition” includes, for example, “V1x” in which the movement amount in the horizontal scanning direction is represented by the number of pixels and “V1y” in which the movement amount in the vertical scanning direction is represented by the number of pixels. When the image processing unit 13 generates an image of the next frame as the teaching image, the movement amount specified by the “display movement condition” from the position in the image of the next frame is changed from the position of the “pick” part p1. Just shift. For example, when generating an image of the next frame of the image Im1 of FIG. 2 as the teaching image, the image processing unit 13 determines the position of “suction” as the part p1 as (X1 + ΔX1 + V1x, Y1 + ΔY1 + V1y). When the part is not moved, for example, the movement amount is set to “0”.

  The “display end condition” is information that defines a condition (eg, timing, trigger) for ending the display of the corresponding shape data (eg, “suction” in FIG. 5B). When the image processing unit 13 of FIG. 1 determines that the “display end condition” is satisfied for each part, the display of the part is ended. For example, the display end condition of “suck” that is the part p1 is defined as the time when the time T2 has elapsed from the display start of the sample image Sa of the pose corresponding to this part (eg, the pose of the warrior in FIG. 2). . For example, when the image processing unit 13 determines that the “display end condition” is satisfied for “sucking” that is the part p1, the image processing unit 13 generates an image that does not include the part p1, for example. By supplying the image data to the display unit 3, the “pickup” that is the part p <b> 1 is not displayed on the display unit 3.

  The display control unit 7 in FIG. 1 displays on the display unit 3 a teaching image (eg, teaching figure, comment, operation time (eg, remaining time for maintaining a pose)) including the parts generated by the image processing unit 13. Let For example, the display control unit 7 causes the display unit 3 to display the teaching image by supplying the display unit 3 with the data of the teaching image generated by the image processing unit 13. The teaching image data may be stored in the storage unit 8, and the display control unit 7 may supply the teaching image data stored in the storage unit 8 to the display unit 3.

  6 and 7 are diagrams showing teaching images according to the first embodiment, respectively. 6 and 7, it is assumed that the user performs a “triangular pose” of yoga. A teaching image Im11 illustrated in FIG. 6A includes a sample image Sa and a user model UM, and does not include a teaching figure. In the embodiment, the plurality of teaching images generated by the image processing unit 13 in FIG. 1 may include at least a part of the teaching figure, and the part of the teaching image may not include the teaching figure. Note that the teaching image may not include the sample image Sa.

  In FIG. 6B, the teaching image Im12 includes a user model UM and a teaching figure F. The teaching figure F includes a plurality of parts (part p11, part p12, part p13, part p14, and part p15). The part p11 is a triangle positioned on the right foot of the user model UM. The part p12 is a triangle positioned on the left foot of the user model UM. The part p13 is a line segment connecting the parts p11 and p12. Each of the parts p14 and p15 is a character (character string) “50”. The parts p14 and p15 represent values recommended as the ratio of the weight applied to the right foot and the left foot, for example. For example, the part p14 represents applying 50% of the weight to the right foot, and the part p15 represents applying 50% of the weight to the left foot. For example, the user has an image of applying 50% of his / her weight to the left foot and the right foot, so that the user can easily approach the target pose. The character “50” of the teaching figure F in FIG. 6B allows the user U to grasp the proportion of weight and to image the center of gravity of the human body, and to easily approach the target pose.

  In FIG. 6C, the teaching image Im13 includes a user model UM and a teaching figure F. The teaching figure F includes a plurality of parts (part p11, part p12, and part p16). The part p16 is, for example, a triangle that represents the posture (relative position) between the right foot and the left foot. The part p16 is positioned with respect to the part p11 (right foot position) and the part p12 (left foot position), for example. The bottom side of the part p16 is disposed in parallel with the horizontal direction, for example. The two hypotenuses of the part p16 represent images of the left and right foot inclinations as targets. The user can easily approximate the target pose by imaging the left foot inclination and the right foot inclination so as to match the part p16.

  In FIG. 7A, the teaching image Im14 includes a user model UM and a teaching figure F. The teaching figure F includes a plurality of parts (part p17 and part p18). The part p17 is, for example, a triangle representing an image of the positional relationship between the head, the right foot tip, and the waist center. The part p17 is a triangle that represents an image of the positional relationship between the right hand tip and the left hand tip, for example. The user can easily approach the target pose by imaging the positional relationship of each part of the body so as to match the parts p17 and p18.

  In FIG. 7B, the teaching image Im15 includes a user model UM and a teaching figure F. The teaching figure includes a plurality of parts (part p21 to part p30). The part p21 is a triangle whose apex is downward and represents an image in which the right hand is extended downward. The part p22 is a triangle whose apex is upward and represents an image in which the left hand is extended upward. The part p23 is a line segment connecting the parts p21 and p22.

  Part p24 is a triangle positioned with respect to the head. The part p24 is a triangle whose vertex is upward and represents an image in which the face is directed upward. The part p25 is a triangle which is positioned with respect to the waist and whose apex is downward. The part p26 is a line segment connecting the parts p24 and p25. The part p26 is arranged in parallel with the horizontal direction and represents an image in which the line connecting the head and the waist is horizontal.

  Parts p27 to p30 are each a part of the kanji “Waki”. This Chinese character “side” is positioned with respect to the left side of the user model UM. In a triangular pose, the left side should be extended. The part p27 is “month” which is a part of “side”, and the parts p28 to p30 are “power” which is a part of “side” (part of making). The part p30 is positioned along the left side (left waist) of the user model UM, and the Chinese character “side” is deformed along the left side (left waist) of the user model UM.

  In FIG. 7C, the teaching image Im15 includes a user model UM and a teaching figure F. The teaching figure F includes a plurality of parts (parts p31 to p37). Parts p31 to p37 are alphabets (phonetic characters) of “s”, “t”, “r”, “e”, “t”, “c”, and “h”, respectively. Parts p31 to p37 represent the word “stretch” as a set. Parts p31 to p37 are each positioned with respect to the left side of the user model UM, and “stretch” is deformed so as to be along the left side of the user model UM. Thus, the teaching figure may include a deformed word, and at least one of the plurality of parts may include a phonetic character that constitutes the deformed word.

  When a character is used for the teaching graphic, the teaching graphic may include a kanji character (eg, side) as shown in FIG. 7B, or an alphabet (eg, “stretch”) as shown in FIG. 7C. Good. Further, in the case where characters are used for the teaching graphic, the teaching graphic may include one or both of a hiragana such as “Waki” and a katakana such as “Waki”. When the teaching figure includes a word (eg, “Waki”) composed of a phonetic character string, each character (eg, “Wa”, “Ki”) constituting the word may be a part of the teaching figure. The character string of phonograms (for example, “Waki”) may be the whole teaching figure or one part. When characters are used for teaching graphics, the language is arbitrary. For example, teaching graphics include characters used in Korean (eg, Korean), artificially created characters (eg, font), etc. But you can.

  Next, the operation of the detection system 1 according to the above-described embodiment will be described with reference to FIGS. 8 to 15 are diagrams illustrating the operation of the detection system according to the embodiment. Each of FIGS. 8 to 15 shows a teaching image displayed on the display unit 3. 8 to 11 correspond to “warrior poses”, FIGS. 12 and 13 correspond to “triangular poses”, and FIGS. 14 and 15 correspond to “tree poses”.

  The schedule (timing, order, time) for performing each pose in a series of yoga operations is determined, for example, in posture definition data D1 stored in advance in the storage unit 8 of FIG. For example, the posture definition data D1 stipulates that “warrior pose” is first performed, then “triangular pose” is performed, and then “tree pose” is performed. In this case, sample images of each pose are displayed in the order of poses to be performed. In addition, a sample image of this pose is displayed over a period during which each pose is assumed to be performed.

  8 to 15, the user model UM is represented by a black silhouette. This user model UM represents, for example, each point included in the point cloud data by black dots. The user model UM is displayed by being synthesized with, for example, a preset background image (e.g., coast image, dusk image, forest image) or the like, but the background image is omitted in FIGS. 8 to 15. . Moreover, in the following description, FIG. 1 is suitably referred for about each part of the detection system 1.

  8 to 11, the image processing unit 13 (see FIG. 1) is a sample image Sa corresponding to “warrior pose”, the user model UM obtained by detecting the user by the detection unit 5 (see FIG. 1), And a teaching image Im including the teaching figure F is generated. The display control unit 7 (see FIG. 1) supplies the data of the teaching image Im to the display unit 3 (see FIG. 1), and the display unit 3 displays the teaching image Im.

  In FIG. 8A, the teaching figure F is a Chinese character “suck”. The teaching figure F “sucking” is displayed at a predetermined position with respect to the user model UM based on, for example, the timing at which the sample image Sa in a posture with a breathing action is displayed. In this case, for example, at the timing when the user U inhales with reference to the sample image Sa, the teaching figure F “inhalation” is positioned and displayed with respect to the chest of the user model UM.

  In FIG. 8B, the teaching figure F includes a part p41, a part p42, and a part p43. The part p41 is, for example, a belt-like figure representing an image of extending the back. The part p42 is, for example, a circle graphic representing an image in which the center of gravity is placed at a low position. In FIG. 8 (B), the user U has not fully lowered his / her waist, and the user model UM has a higher waist position than the sample image Sa. For example, the user U can intuitively obtain an image of lowering the center of gravity by the part p42 of the teaching figure F.

  Also, the part p43 is a Chinese character “vomit”. The part “p43” “exhale” is displayed at a predetermined position with respect to the user model UM based on, for example, the timing at which the sample image Sa in a posture with a breathing operation is displayed. In this case, for example, at the timing when the user U exhales with reference to the sample image Sa, the teaching figure F “exhale” is positioned and displayed with respect to the chest of the user model UM.

  In FIG. 8C, the teaching figure F includes a part p43 and a part p44. The part p43 is a Chinese character “vomit” as in FIG. 8B, but is displayed at a position farther from the user model UM than in FIG. 8B. For example, the user U can intuitively perform an operation of exhaling deeply by intuitively obtaining an image that makes the exhaled breath reach far from the part p43. The part p44 is a figure obtained by deforming the part p41 of FIG. Part p44 represents an image in which the back is stretched and the knee is bent vertically.

  In FIG. 9A, the teaching figure F includes a part p45, a part p46, and a part p47. Each of the parts p45 to p47 is a triangular figure (eg, the shape of the tip of the arrow). The part p45 is positioned with respect to the head of the user model UM. The part p45 represents an image in a direction in which the user's head is directed, for example. The part p46 is positioned with respect to the right foot of the user model UM. The part p46 represents an image in a direction in which the user's right foot is directed, for example. The part p47 is positioned with respect to the left foot of the user model UM. The part p47 represents an image in a direction in which the user's left foot is directed, for example.

  9B, the teaching figure F includes a part p48, a part p49, a part p50, and a part p51. The part p48 is positioned with respect to the center of the right shoulder and the left shoulder of the user model UM, for example. The part p48 is a bar-like figure extending in the horizontal direction. The part p48 represents an image of extending the right arm and the left arm in the horizontal direction, for example. The part p49 is a polygonal figure and is positioned with respect to the leg portion of the user model UM. The part p49 represents an image in which the leg is bent with an ankle and a knee, for example. Each of the parts p50 and p51 is a square figure. For example, the part p49 is positioned with respect to the ankle of the user model UM and represents an image in which the ankle is bent vertically. For example, the part p50 is positioned with respect to the knee of the user model UM and represents an image in which the knee is bent vertically.

  In FIG. 9C, the teaching figure F includes a part p52, a part p53, and a part p54. The part p52 is positioned with respect to the right foot and the left foot of the user model UM, for example. The part p52 is a bar-like figure extending in the horizontal direction, and includes a triangle representing a position where the weight is applied to both ends thereof. The part p53 is a number “70” and represents an image in which the proportion of the weight on the right foot is 70%. The part p54 is a number “30” and represents an image in which the proportion of the weight on the left foot is 30%.

  In FIG. 10A, the teaching figure F is a frame-like figure whose outer shape is a rectangle. The teaching figure F represents, for example, an image that is positioned with respect to the chest of the user model UM and stretches the chest so that the outer shape of the chest becomes a rectangular shape. In FIG. 10B, the teaching figure F is a Chinese character “suck”. The teaching figure F “inhalation” is positioned with respect to the chest of the user model UM and represents an inhalation image, for example.

  In FIG. 10C, the teaching figure F includes a part p55 and a part p56. The part p55 is three triangles with one vertex facing upward. The part p55 represents, for example, an image positioned with respect to the right arm of the user model UM and extending the right arm upward. The part p56 is a Chinese character “vomit”. The part p56 “exhale” is positioned with respect to the chest of the user model UM, for example, and represents an image of exhaling.

  FIG. 11A corresponds to the operation continued from FIG. In FIG. 11A, the teaching figure F includes a part p55 and a part p56. For example, the part p56 called “part” is positioned at a position distant from the user model UM as compared with FIG. In FIG. 11B, the teaching figure F is a plurality of round figures arranged in an arc. This teaching figure F represents, for example, an image in which the right side is curved and extended.

  12 and 13, the image processing unit 13 (see FIG. 1) includes a sample image Sa corresponding to “triangular pose”, and a user model UM obtained by detecting the user by the detecting unit 5 (see FIG. 1). And a teaching image Im including the teaching figure F is generated. The display control unit 7 (see FIG. 1) supplies the data of the teaching image Im to the display unit 3 (see FIG. 1), and the display unit 3 displays the teaching image Im.

  In FIG. 12A, the teaching figure F is a Chinese character “vomit”. In FIG. 12B, the teaching figure F includes a part p57, a part p58, and a part p59. The part p57 is positioned with respect to the right foot and the left foot of the user model UM, for example. The part p57 is a bar-like figure extending in the horizontal direction, and includes a triangle representing a position where the weight is applied to both ends thereof. The part p58 is a number “50” and represents an image in which the proportion of the weight on the right foot is 50%. The part p59 is a number “50” and represents an image in which the ratio of the weight applied to the left foot is 50.

  In FIG. 13A, the teaching figure F includes a part p57, a part p58, a part p59, a part p60, and a part p61. The part p60 is, for example, a triangle representing an image of the posture (relative position) between the right foot and the left foot. The part p60 is positioned with respect to the right foot and the left foot of the user model UM, for example. The bottom side of the part p60 is disposed in parallel with the horizontal direction, for example. The two hypotenuses of the part p60 each represent an image of the left and right foot inclination as a target. The user U can easily approximate the target pose by imaging the left foot and the right foot so as to match the part p60. The part p61 is a triangle whose apex is upward (the tip of an upward arrow). For example, the part p61 is positioned with respect to the head of the user model UM and represents an image in a direction in which the face is directed.

  In FIG. 13B, the teaching figure F includes a part p62 and a part p63. The part p62 represents, for example, an image of the positional relationship between the head, right foot tip, and waist center. The part p63 represents an image of the positional relationship between the right hand tip and the left hand tip, for example. The user U can easily approach the target pose by imaging the positional relationship of each part of the body to match the teaching figure.

  14 and 15, the image processing unit 13 (see FIG. 1) includes a sample image Sa corresponding to “a tree pose”, and a user model UM obtained by detecting the user by the detecting unit 5 (see FIG. 1). And a teaching image Im including the teaching figure F is generated. The display control unit 7 (see FIG. 1) supplies the data of the teaching image Im to the display unit 3 (see FIG. 1), and the display unit 3 displays the teaching image Im.

  In FIG. 14A, the teaching figure F includes a part p65 and a part p66. The part p65 is, for example, a broken line representing an image of the positional relationship between the right foot and the left foot. The part p66 is a square indicating an image in which the right foot and the shin of the left foot are vertical. In FIG. 14B, the teaching figure F is a Chinese character “suck”. FIG. 14C corresponds to the operation continued from FIG. In FIG. 14C, the “sucking” of the teaching figure F has moved upward as compared with FIG. 14B so as to give an image of inhaling while stretching the back muscles, for example.

  In FIG. 15A, the teaching figure F includes a part p67 and a part p68. The part p67 is a bar-like figure extending in the vertical direction. The part p68 is a triangle whose top is upward (a figure with a tip of an arrow). The part p67 and the part p68 represent, for example, an image in which the right foot, the spine, and the head are aligned and extended upward. 15B and 15C, the teaching figure F includes a part p69 and a part p70. Part p69 is a Chinese character “call”, and part p70 is a Chinese character “suck”. The “call” of the part p69 is positioned on the left side of the user model UM, and the part p70 is positioned on the right side of the user model UM. Parts p69 and p70 represent the word “breathing” as a set.

  In each of FIG. 15B and FIG. 15C, the relatively high contrast of “call” and “suck” is represented by a solid line, and “call” and “suck” are relatively The lower contrast is represented by a dotted line. FIG. 15B corresponds to an operation in which the user exhales, and the “call” of the part p69 is displayed with a higher contrast than the “suck” of the part p70. FIG. 15C corresponds to the operation in which the user U inhales, and the “puff” of the part p70 is displayed with a higher contrast than the “pump” of the part p69. As described above, the image processing unit 13 generates an image representing the first part (eg, part p69) of the plurality of parts in a display different from the second part (eg, part p70) as the teaching image. Good. In the present embodiment, “different display” includes that at least one of display elements (eg, color, hue, brightness, saturation, contrast to surroundings) is different.

  Next, a detection method according to the embodiment will be described based on the configuration of the detection system as described above. FIG. 16 is a flowchart showing the operation of the detection system according to the first embodiment. For each part of the detection system, refer to FIG. 1 as appropriate.

  In step S <b> 1, the detection device 2 (for example, the detection unit 5) starts detecting the user U. In step S <b> 2, the detection device (e.g., model generation unit 11) generates the user U model based on the detection result in step S <b> 1. In step S <b> 3, the detection device (for example, the display control unit 7) transmits a command (display start command) for starting display of the model of the user U to the display unit 3. For example, the display control unit 7 supplies image data of an image (eg, an image including the user model UM in FIG. 2) corresponding to the model of the user U generated in step S2 to the display unit 3 as a display start command. . In step S4, the display unit 3 starts displaying the model of the user U based on the data supplied from the detection device.

  In step S5, the detection apparatus sets a target pose. The type and order of the poses to be performed and the time set as the time for performing the poses are determined, for example, in the posture definition data D1 shown in FIG. In the following description, the order set as the order in which pauses are performed as appropriate is referred to as a pause setting order. In the following description, a time set as a time for performing a pause (eg, 15 seconds, 30 seconds) is referred to as a pause setting time. In FIG. 5A, for example, “warrior pose” is set in advance as the first pose, and “triangular pose” is set as the pose to be performed thereafter. Here, it is assumed that “warrior pose” is set as the target pose. When the set time of the set pose elapses after the target pose is set, the target pose is changed to the next pose based on the pose setting order through the process of step S15 described later.

  In step S <b> 6, the detection device (for example, the display control unit 7) transmits a command (display start command) for starting display of the sample image of the pose set in step S <b> 5 to the display unit 3. For example, the image processing unit 13 acquires the data of the sample image Sa of “warrior's pose” based on the posture definition data D1 in FIG. 5A, and obtains image data including the sample image Sa and the user model UM. Generate. The display control unit 7 supplies the display unit 3 with image data including the sample image Sa and the user model UM generated by the image processing unit 13 as a sample image display start command. In step S7, the display unit 3 starts displaying the sample image of the pose set in step S5 based on the data supplied from the detection device.

  In step S <b> 8, the detection device (for example, the image processing unit 13) determines whether or not the display start condition for the teaching graphic is satisfied. For example, the image processing unit 13 determines whether or not the display start condition for the teaching graphic is satisfied based on the display condition data D3 illustrated in FIG. For example, the image processing unit 13 determines whether or not the condition defined in the “display start condition” of the display condition data D3 is satisfied for each part of the teaching graphic associated with the pose set in step S5.

  For example, in the teaching figure data shown in FIG. 5B, the teaching figure associated with the “warrior pose” has “teaching figure identification number” “01”, and the image processing unit 13 The above determination is performed for each of the parts whose number is “01”. For example, a part whose “teaching figure identification number” is “01” and whose “part identification number” is “03” is “suck” of the part p1. As shown in FIG. 5C, the “display start condition” of this part p1 “suck” is that the time T1 has elapsed from the start of the display of the pose. The image processing unit 13 determines that the teaching figure display start condition is satisfied when it is determined in step S6 that the time T1 has elapsed since the instruction to start displaying the pause sample image has been transmitted (step S8; Yes). . If the image processing unit 13 determines that the teaching graphic display start condition is not satisfied (step S8; No), the process of step S8 is repeated.

  When it is determined that the display start condition for the teaching graphic is satisfied (step S8; Yes), the detection device (for example, the image processing unit 13) generates a teaching image in step S9. The image processing unit 13 positions the teaching figure part determined to satisfy the display start condition in step S8 with respect to the user model UM, and generates a teaching image. The processing for generating the teaching image will be described later with reference to FIG.

  In step S <b> 10, the detection device (for example, the display control unit 7) transmits a command (display start command) for starting display of the teaching image to the display unit 3. For example, the display control unit 7 supplies the display unit 3 with the display image data generated by the image processing unit 13 in step S9 as a teaching image display start command (signal). The display control unit 7 displays the teaching figure F on the display unit 3 as a guide display indicating the current operation or the next operation in a predetermined positional relationship with respect to a position (part) related to the human body of the user U. In step S11, the display unit 3 displays a display image based on the data supplied from the detection device.

  In step S <b> 12, the detection device (for example, the image processing unit 13) determines whether or not a teaching graphic display end condition is satisfied. For example, the image processing unit 13 determines whether or not the display end condition of the teaching graphic is satisfied based on the display condition data D3 illustrated in FIG. For example, the image processing unit 13 determines whether or not the condition (data) defined in the “display end condition” of the display condition data D3 is satisfied for each part of the teaching graphic associated with the pose set in step S5. judge.

  For example, in the teaching figure data shown in FIG. 5B, the teaching figure associated with the “warrior pose” has “teaching figure identification number” “01”, and the image processing unit 13 The above determination is performed for each of the parts whose number is “01”. For example, a part whose “teaching figure identification number” is “01” and whose “part identification number” is “03” is “suck” of the part p1.

  As shown in FIG. 5C, the “display end condition” of this part p1 “suck” is that the time T2 has elapsed from the start of the display of the pose. If it is determined in step S6 that the time T2 has elapsed since the instruction to start displaying the pause sample image is transmitted, the image processing unit 13 determines that the teaching figure display end condition is satisfied (step S12; Yes). . If the image processing unit 13 determines that the teaching graphic display end condition is not satisfied (step S12; No), the process of step S12 is repeated.

  When the detection device (eg, the image processing unit 13) determines that the display end condition of the teaching graphic is satisfied (step S12; Yes), the detection device stops displaying the teaching graphic. For example, if the image processing unit 13 determines that the display end condition of the teaching graphic is satisfied (step S12; Yes), the image processing unit 13 generates a teaching image that does not include the part determined to satisfy the display end condition. In step S <b> 13, the display control unit 7 supplies the teaching image data including no parts to the display unit 3 as a display stop command. In step S <b> 14, the display unit 3 stops displaying the teaching graphic by displaying an image based on the data supplied from the detection device. Here, the detection device (for example, the display control unit 7) supplies the teaching image data that does not include the teaching graphic to the display unit 3, and the display unit 3 displays the teaching image that does not include the teaching graphic.

  In step S15, the detection device determines whether or not to change the target pose. For example, the detection apparatus determines whether there is a next target pose based on the attitude definition data D1 illustrated in FIG. When it is determined that there is a next target pose, the detection apparatus determines to change the target pose when the set time of the target pose set in step S5 has elapsed (step S15; Yes). When it is determined that the target pose is to be changed (step S15; Yes), the detection apparatus returns to the process of step S5, sets the target pose to the next pose, and repeats the processes after step S6. In addition, when the detection device determines that the target pose is not changed (step S15; No), for example, the detection device and the display unit are in a standby state.

  The detection device 2 (for example, the processing unit 6, the image processing unit 13, and the display control unit 7) displays a specific (plural) display when a display start instruction (trigger signal) of a teaching image (teaching figure) is detected, for example. The display unit 3 is caused to execute display movement processing for moving an element (eg, part) in a predetermined direction at a display position on the display screen (display unit 3). In addition, the detection device 2 (for example, the processing unit 6, the image processing unit 13, and the display control unit 7) executes the above display movement process and sequentially displays display elements based on a signal for switching the sample image Sa. The display unit 3 is caused to execute display update processing for updating and displaying on the display screen. The detection device 2 (eg, the processing unit 6, the image processing unit 13, and the display control unit 7) moves the display position of a specific display element when, for example, a display end command (display end instruction) is detected. The display unit is caused to execute display stop processing to be stopped.

  Note that the detection apparatus may include a voice input unit that can input various types of information by voice. The detection device may determine whether to change the pose based on a user input (eg, voice input). For example, the user may input the next target pose type by voice. For example, the detection device may identify the name of a pose uttered by the user (for example, “triangular pose”) The detection device 2 uses the type of pose previously associated with the identified speech in step S5. In the processing, it may be set as the type of the next target pose.

  Next, based on the operation of the image processing unit 13, a process for generating a teaching image in step S9 of FIG. 16 will be described. FIG. 17 is a flowchart showing processing by the image processing unit according to the first embodiment. In addition, about each part of the detection system 1, FIG. 1 is referred suitably. For the processing from step S1 to step S15, refer to FIG. 16 as appropriate.

  In step S21, the image processing unit 13 acquires the user model generated by the model generation unit 11 (see FIG. 1) in step S2 of FIG. In step S22, the image processing unit 13 selects a part of the teaching graphic. For example, with respect to the target pose set in step S5, the image processing unit 13 refers to the teaching graphic data D2 shown in FIG. 5B, and identifies parts belonging to the teaching graphic associated with the target pose. To do. For example, when “warrior pose” is set as “target pose”, the “teaching figure identification number” corresponding to “warrior pose” is “01”, and the image processing unit 13 performs “teaching figure identification”. A part whose number is “01” is specified. In FIG. 5B, the parts belonging to the teaching graphic whose “teaching graphic identification number” is “01” have the “part identification number” “01”, “02”, and “03”. In step S22, the image processing unit 13 selects “part identification number” from “01”, “02”, and “03”.

  In step S23, the image processing unit 13 acquires the display start position of the part selected in step S22. For example, in step S22, it is assumed that a part (“Suck”) whose “part identification number” shown in FIG. 5B is “03” is selected. Based on the display condition data shown in FIG. 5C, the image processing unit 13 displays the “display start position” of the part whose “teaching figure identification number” is “01” and whose “part identification number” is “03”. Is obtained. “Display start position” includes, for example, “reference part name” and “offset amount” (offset of display position).

  In step S <b> 24, the image processing unit 13 acquires a position corresponding to “reference part name” of “display start position”. The image processing unit 13 acquires the position of the part specified in the “reference part name” based on the coordinates of the part specified by the part specifying unit 12. For example, it is assumed that “reference part name” is “chest”. In this case, the image processing unit 13 acquires the position (for example, coordinates) of the part specified by the part specifying unit 12 as “chest”.

  In step S25, the image processing unit 13 positions the part on the teaching image based on the offset amount. For example, the image processing unit 13 defines the “offset amount” from the reference position to the “display start position” with the position of the part specified in the “reference part name” (see step S24) as the reference position. The position shifted by the amount is calculated, and the calculated position is determined (positioned) as the position of “suction” as the selected part. The image processing unit 13 arranges “suction” of Imp3 defined in the shape data (eg, “form — 0103.dat) in the teaching graphic data D2 of FIG. 5B at the determined position.

  In step S26, the image processing unit 13 determines whether there is a next part. The image processing unit 13 determines that there is a next part when the positioning process in step S25 is not completed for at least one part belonging to the target pose set in step S5 (step S26; Yes). ). If it is determined that there is a next part (step S26; Yes), the image processing unit 13 returns to step S22, selects the next part, and repeats the subsequent processes. If the image processing unit 13 determines that there is no next part (step S26; No), the image processing unit 13 outputs teaching image data including the part positioned in step S25 as a teaching figure, and ends a series of processing.

[Second Embodiment]
A second embodiment will be described. In the present embodiment, the same components as those in the above-described embodiment are denoted by the same reference numerals, and the description thereof is omitted or simplified. FIG. 18 is a diagram illustrating a detection system according to the second embodiment. The detection system 1 according to the present embodiment displays a teaching graphic at a timing based on the detection result of the detection unit 5. For example, the detection device 2 identifies the posture of the user U and displays the teaching figure at a timing associated with the identified posture.

  The detection device 2 according to the present embodiment includes a posture specifying unit 15. The posture specifying unit 15 is provided in the processing unit 6, for example. The posture specifying unit 15 specifies the type of posture of the user U obtained from the detection unit 5 from a plurality of types registered in advance. For example, the posture specifying unit 15 specifies the type of posture of the user U based on the posture definition data D1 stored in the storage unit 8.

  FIG. 19 is a diagram illustrating processing by the posture specifying unit according to the second embodiment. FIG. 19A conceptually shows posture definition data according to the present embodiment. FIG. 19B conceptually shows processing by the posture specifying unit according to the embodiment. The posture definition data D1 includes a plurality of types of posture data (D1a, D1b). Each of the posture data D1a and the posture data D1b includes “posture name” information and “definition data” information.

  The “posture name” is information (for example, identification information) indicating the name of the kind of posture defined by each piece of posture data. For example, the posture data D1a is data indicating the definition of “warrior pose”, and information “warrior pose” is stored in “posture name” of the posture data D1a. The posture data D1b is data indicating the definition of “triangular pose”, and information “triangular pose” is stored in the “posture name” of the posture data D1b.

  “Definition data” is data indicating the definition of each posture. “Definition data” includes, for example, “condition of part 01”, “condition of part 02”,... “Condition of part n”. Site 01, site 02,... Site n each correspond to one of the types of sites specified by site specifying unit 12. The types of parts specified by the part specifying unit 12 include, for example, the head, neck, right shoulder, left shoulder, shoulder center, right elbow, right wrist, right hand, left elbow, left wrist, left hand, spine, right hip, Left waist, middle waist, right knee, right ankle, right foot tip, left knee, left ankle, left foot tip, etc.

  In ““ Condition for part 01 ”,“ Condition for part 02 ”,...,“ Condition for part n ”, conditions that each part satisfies are stored. For example, assuming that “part 02” is “right elbow”, the “condition of part 02” of “definition data” in the posture data D1a includes the position of the right elbow when taking the “warrior pose” (eg, The condition of the relative position) with respect to other parts is stored. For example, the condition of the position of the right elbow includes, for example, a range of the bending angle of the right elbow. The bending angle of the right elbow is, for example, an angle between a line connecting the right elbow and the right hand (forearm) and a line connecting the right elbow and the right shoulder (upper arm).

  For example, the posture specifying unit 15 acquires the position of each part specified by the part specifying unit based on the detection result of the detecting unit 5. Then, the posture specifying unit 15 calculates the similarity between the positional relationship of each part obtained from the detection result of the detecting unit 5 and the positional relationship of each part defined in the “definition data”. The posture specifying unit 15 calculates the similarity for each of a plurality of types of posture data. The posture specifying unit 15 specifies the type of posture of the user U from a plurality of types by comparing the above similarities for a plurality of types of postures.

  For example, in FIG. 11B, the symbol SD is skeleton information indicating the position of each part of the user U obtained from the detection result of the detection unit 5. The posture identifying unit 15 calculates the degree of similarity for the “warrior pose” using the skeleton information SD and the definition data D1a of the “warrior pose”. In addition, the posture specifying unit 15 calculates the degree of similarity for “triangular pose” using the skeleton information SD and the definition data D1b of “triangular pose”. The posture specifying unit 15 compares the similarity for the “warrior pose” and the similarity for the “triangular pose”, and specifies the posture with the higher similarity as the posture of the user.

  FIG. 20 is a diagram illustrating processing by the image processing unit according to the second embodiment. Here, the pose of the warrior will be described. In FIG. 20A, reference sign Q1 is the hand specified by the part specifying part 12, reference sign Q2 is the elbow specified by the part specifying part 12, and reference sign Q3 is the shoulder specified by the part specifying part 12. Further, symbol α is an angle (elbow angle) between a line connecting the hand Q1 and the elbow Q2 (forearm Q4) and a line connecting the elbow Q2 and the shoulder Q3 (upper arm Q5).

  The detection apparatus 2 (refer FIG. 18) which concerns on this embodiment displays a teaching figure based on the elbow angle (alpha) obtained from the detection result of the detection part 5, for example. For example, the processing unit 6 of the detection device 2 causes the display control unit 7 to display the teaching figure with a trigger that the elbow angle α satisfies a predetermined condition. For example, in a warrior's pose, the state ST2 (two-dot chain line) in which the elbow is extended may be more desirable than the state ST1 (solid line) in which the elbow is bent (see FIG. 2).

  Symbol G1 in FIG. 20A is a graph showing the elbow angle α with respect to time. When the elbow angle α increases, it is estimated that the user is moving the elbow. For example, the image processing unit 13 monitors the amount of change Δα of the elbow angle α at time Δt. For example, the image processing unit 13 determines that the user is performing an action of extending the elbow when Δα / Δt, which is the amount of time change in the elbow angle, is equal to or greater than a threshold value. This threshold value is stored in, for example, “display start condition” in the display condition data D3 shown in FIG. When the image processing unit 13 determines that the user is performing an action of extending the elbow, the image processing unit 13 displays a teaching figure (eg, “suction” in FIG. 2) associated with the operation of extending the elbow. Let

  In FIG. 20B, reference sign Q6 is the ankle specified by the part specifying part 12, reference sign Q7 is the knee specified by the part specifying part 12, and reference sign Q8 is the waist specified by the part specifying part 12. . Further, symbol β is an angle (a knee angle) between a line connecting the ankle Q6 and the knee Q7 (shin portion Q9) and a line connecting the knee Q7 and the waist Q8 (thigh Q10).

  The detection device 2 (see FIG. 18) according to the present embodiment displays a teaching figure based on, for example, the knee angle β obtained from the detection result of the detection unit 5. For example, the detection device 2 displays a teaching figure with a trigger that the knee angle β satisfies a predetermined condition. For example, in a warrior's pose, a state ST4 (two-dot chain line) in which the knee is bent vertically may be preferable to a state ST3 (solid line) in which the knee is open (see FIG. 3 for example).

  A symbol G2 in FIG. 20B is a graph showing the knee angle β with respect to time. When the elbow angle β decreases, it is estimated that the user is bending the knee. For example, the image processing unit 13 monitors the change amount Δβ of the knee angle β at time Δt. For example, when Δβ / Δt, which is a temporal change amount of the knee angle, is equal to or greater than a threshold value, the image processing unit 13 determines that the user is bending the knee. This threshold value is stored in, for example, “display start condition” in the display condition data D3 shown in FIG. When the image processing unit 13 determines that the user is performing an operation of bending the knee, the image processing unit 13 displays a teaching graphic (for example, “vomit” in FIG. 2) associated with the operation of bending the knee.

  Next, processing for generating a teaching image according to the present embodiment will be described based on the operation of the processing unit 6. FIG. 21 is a flowchart showing processing by the processing unit according to the second embodiment. For each part of the detection system 1, refer to FIG. 18 as appropriate. In step S31, the posture identifying unit 15 identifies the type of user posture obtained from the detection. For example, the detection unit 5 in FIG. 18 detects the user U, and the part specifying unit 12 specifies the position of each part of the user U based on the detection result of the detection unit 5. In addition, as described with reference to FIG. 19, the posture specifying unit 15 specifies information (for example, skeleton information SD in FIG. 19B) that is specified by the part specifying unit 12 and indicates the positional relationship of each part of the user, and is registered in advance. A plurality of types of posture information (for example, posture definition data D1 in FIG. 19A) are compared to identify the type of posture of the user.

  In step S <b> 32, the image processing unit 13 specifies the teaching graphic associated with the posture specified by the posture specifying unit 15 in step S <b> 31. For example, as described with reference to FIG. 5B, the teaching figure is associated with a registered posture type (eg, “posture name” in FIG. 5A). For example, when the posture identified by the posture identifying unit 15 is “warrior's pose”, the image processing unit 13 teaches the teaching graphic whose teaching graphic identification number is “01” in the teaching graphic data D2 of FIG. Is identified as the teaching figure associated with the “warrior pose”.

  In step S33, the image processing unit 13 selects the part of the teaching graphic identified in step S32. For example, the image processing unit 13 selects a teaching graphic part using the teaching graphic data D2 of FIG. For example, if the image processing unit 13 specifies a teaching figure whose “teaching figure identification number” is “01” in step S32, the “part identification number” belonging to the teaching figure whose “teaching figure identification number” is “01” is displayed. Select one of the parts “01”, “02”, and “03”.

  In step S34, the image processing unit 13 acquires display conditions for the parts selected in step S33. The display conditions for parts are defined in, for example, display condition data D3 in FIG. For example, as described with reference to FIG. 20A, the display condition of “sucking”, which is a teaching figure related to “warrior pose” (see FIG. 2), is expressed by Δα / Δt, which is the amount of time change of the elbow angle α. The threshold value is stored in advance in the “display start condition” of the display condition data D3 in FIG. For example, the image processing unit 13 acquires the display conditions of the parts from the display condition data D3 stored in the storage unit 8.

  In step S35, the image processing unit 13 determines whether or not the part selected in step S33 satisfies the display condition. For example, as described with reference to FIG. 20A, the image processing unit 13 calculates Δα / Δt, which is a temporal change amount of the elbow angle α, based on the position of each part specified by the part specifying unit 12. . The part specifying unit 12 compares the calculated Δα / Δt with the threshold acquired in step S34, and determines that the selected part satisfies the display condition if Δα / Δt is equal to or greater than the threshold (step) S35; Yes). If the image processing unit 13 determines that the selected part satisfies the display condition (step S35; Yes), in step S36, the image processing unit 13 determines to display the part selected in step S33, and the process from step S23 in FIG. The parts are positioned in the same manner as in S25.

  When it is determined that the selected part does not satisfy the display condition (step S35; No), the image processing unit 13 determines in step S37 that the part selected in step S33 is not displayed. After the process of step S36 is completed or after the process of step S37 is completed, the image processing unit 13 determines whether there is a next part in step S38. The process of step S38 is the same as the process of step S26.

[Third Embodiment]
A third embodiment will be described. In the present embodiment, the same components as those in the above-described embodiment are denoted by the same reference numerals, and the description thereof is omitted or simplified. FIG. 22 is a diagram illustrating a detection system according to the third embodiment. The detection apparatus 2 according to the present embodiment includes a figure dividing unit 16. The figure dividing unit 16 divides the teaching figure into a plurality of parts. The detection device 2 is provided with an input unit 17. The input unit 17 is, for example, a keyboard, a mouse, a touch panel, a voice input device, or the like. The user can input various types of information to the detection device 2 through the input unit 17. For example, the user can register the teaching graphic via the input unit 17. For example, the graphic dividing unit 16 divides the teaching graphic input by the user into a plurality of parts, and stores information of the divided parts in the teaching graphic data D2 of the storage unit 8.

  FIG. 23 is a diagram illustrating processing by the graphic dividing unit according to the third embodiment. In FIG. 23A, the teaching figure F before division is a Chinese character (eg, “extension”) including two or more radicals. For example, the figure dividing unit 16 searches for a route that divides the region AR by searching for a region AR including the teaching figure F by a route search using a search tree (binary search tree).

  For example, as shown in the middle left diagram of FIG. 23A, the figure dividing unit 16 searches for the route R1 with the horizontal end as the start position. The figure dividing unit 16 searches for a route from the start position to the end of the area AR without intersecting with the line of the teaching figure F. For example, the graphic dividing unit 16 optimizes the route R1 so that the route R1 passes through the center of the area AR and the length of the route R1 is minimized.

  The figure dividing unit 16 divides the area AR into a plurality of areas (for example, the area A1 and the area A2) with the route R1 as a boundary. Each of the plurality of divided areas is a candidate for a part of the teaching figure. The figure dividing unit 16 determines whether or not a part of the teaching figure F is arranged in each of the area A1 and the area A2. The graphic dividing unit 16 rejects candidate parts for the teaching graphic F when at least one of the teaching graphic F is not arranged in at least one of the area A1 and the area A2. The graphic dividing unit 16 determines whether each of the area A1 and the area A2 is equal to or larger than a predetermined size. When at least one of the area A1 and the area A2 is smaller than a predetermined size, the figure dividing unit 16 rejects the candidate parts for these teaching figures.

  Further, as shown in the middle right diagram of FIG. 23A, the figure dividing unit 16 searches for the route R2 with the end in the vertical direction as the start position. The figure dividing unit 16 searches for a route R2 from the start position to the end of the area AR without intersecting with the line of the teaching figure F. For example, the graphic dividing unit 16 optimizes the route R2 so that the route R2 passes through the center of the area AR and the length of the route R2 is minimized. The figure dividing unit 16

  The graphic dividing unit 16 divides the region R2 into a plurality of regions (eg, region A3, region A4) with the route R2 as a boundary. Each of the divided areas is a candidate for a part of the teaching figure F. The figure dividing unit 16 determines whether or not a part of the teaching figure F is arranged in each of the area A3 and the area A4. When a part of the teaching figure F is not arranged in at least one of the area A3 and the area A4, the figure dividing unit 16 rejects these teaching figure part candidates. The graphic dividing unit 16 determines whether each of the area A3 and the area A4 is equal to or larger than a predetermined size. When at least one of the area A3 and the area A4 is less than a predetermined size, the figure dividing unit 16 rejects the candidate parts for the teaching figure F.

  In FIG. 23 (A), teaching is performed on both the path R1 whose start position is the horizontal end as shown in the middle left figure and the path R2 whose start position is the vertical end as shown in the middle right figure. The candidate for the part of figure F is effective. In such a case, the graphic dividing unit 16 employs the route R1 or the route R2 as a boundary. For example, the graphic dividing unit 16 employs the route R1 as a boundary, and employs a portion pa and a portion pb of the teaching graphic F that are divided with the route R1 as the boundary as parts of the teaching graphic F, respectively. The figure dividing unit 16 registers the information of the parts of the adopted teaching figure F in the teaching figure data D2 described with reference to FIG.

  FIG. 23B shows a case where the image is divided by a method different from that shown in FIG. For example, the teaching figure F in FIG. 23B is a Chinese character “neck” and does not have an independent radical (a radical discontinuous with other parts). In such a case, as shown in the middle left figure, when the end of the area AR including the teaching figure F is set as the start position and a path reaching the end of the area AR without intersecting with the line of the teaching figure F is searched, for example, the path R3 Is obtained. When the area AR is divided into the area A5 and the area A6 with the route R3 as a boundary, the area A6 does not include a part of the teaching figure F, and the teaching figure F is not divided.

  In such a case, the graphic dividing unit 16 sets the condition that the route does not intersect with the line of the teaching graphic F, and the condition that the path intersects with the line of the teaching graphic F and the length of the path at the intersection is minimum. Change to The figure dividing unit 16 searches for a route under the changed conditions, and obtains a route R4 as shown in the middle right diagram. The figure dividing unit 16 divides the area AR into an area A7 and an area A8 with the route R4 as a boundary. The figure dividing unit 16 employs a part pc included in the area A7 of the teaching figure F and a part pd included in the area A8 of the teaching figure F as parts of the teaching figure F, respectively. The figure dividing unit 16 registers the information of the parts of the adopted teaching figure F in the teaching figure data D2 described with reference to FIG.

[Fourth Embodiment]
A fourth embodiment will be described. In the present embodiment, the same components as those in the above-described embodiment are denoted by the same reference numerals, and the description thereof is omitted or simplified. FIG. 24 is a diagram illustrating a detection system according to the fourth embodiment. In the present embodiment, the detection device 2 can display information on the posture of the user U acquired in the past (for example, the user model UM in FIG. 2).

  In the present embodiment, the detection device 2 includes a posture specifying unit 15 and a history management unit 18. As described with reference to FIG. 19, the posture specifying unit 15 determines the positional relationship between the parts of the user U obtained from the detection result of the detection unit 5 and the “definition data” of the posture definition data D <b> 1 stored in the storage unit 8. The similarity with the positional relationship of each part defined in FIG. 19 (A) is calculated. The posture identifying unit 15 identifies the type of posture of the user based on the calculated similarity, and based on the identified posture type, the calculated similarity (eg, score), and the posture type. Posture history data D4 in which the model of the user U (for example, the user model UM in FIG. 2) obtained from the detection result of the detection unit 5 is associated is generated. The posture identifying unit 15 stores the generated posture history data D4 in the storage unit 8.

  The history management unit 18 causes the display unit 3 to display information related to the posture of the user U acquired in the past based on the posture history data D4 stored in the storage unit 8. For example, the history management unit 18 displays information on the posture of the user U based on a command (display request command for requesting display) input to the input unit 17. The history management unit 18 generates a posture list image (shown later in FIG. 24) when a display request command is input to the input unit 17. The posture list image is, for example, a user interface image showing a list of user models stored in the storage unit 8. The display control unit 7 supplies the posture list image data generated by the history management unit 18 to the display unit 3 and causes the display unit 3 to display the posture list image.

  FIG. 25 is a diagram illustrating processing by the history management unit according to the fourth embodiment. For each part of the detection system 1, refer to FIG. 24 as appropriate. FIG. 25A shows an attitude list image Im21 displayed on the display unit 3 by the process of the history management unit 18 shown in FIG. In the posture list image Im21, a symbol D5 is a date when the display request command is input. The thumbnail images displayed in the “Teacher” column are sample images Sa included in the posture definition data D1 (see FIG. 5A) stored in the storage unit 8, respectively.

  Further, in the posture list image Im21, the thumbnail image Tn1 in the “Today” column displayed in the horizontal scanning direction (hereinafter referred to as the same row) with respect to the sample image Sa of “Teacher” is, for example, “Teacher” in the same row. ] Is an image showing the latest user model in the same posture. The thumbnail image Tn1 is, for example, an image showing a user model acquired on the date when the display request command is input. Also, the thumbnail image Tn2 in the “Your Best” column is an image showing the user model having the highest similarity calculated by the posture specifying unit 15 among the user models acquired in the past. For example, the thumbnail image Tn2 is an image indicating a user model having the highest evaluation value (for example, the above-described similarity and score) with the sample image Sa of “Teacher” as an evaluation criterion.

  A column SP1 next to “Your Best” is a space for displaying information (eg, detailed information, comments) about the thumbnail image Tn1 in the “Today” column or the thumbnail image Tn2 in the “Your Best” column. . For example, when the cursor is overlaid on the thumbnail image Tn1 in the “Today” column or the thumbnail image Tn2 in the “Your Best” column, the similarity D6 (eg, score) corresponding to the image in the column where the cursor is overlaid, Text data D7 (eg, comment) is displayed. A tag “Tg” of “past log” is a user interface for switching displayed information. For example, when the tag “Tg” of “past log” is clicked, the posture list image Im22 shown in FIG.

  The posture list image Im22 is an image showing a list of thumbnail images Tn3 indicating the user's model acquired in the past for one posture (eg, warrior's pose). Each thumbnail image Tn3 is given a date D8 when the user's model is acquired. When the thumbnail image Tn3 is clicked, information (eg, score, comment) associated with the thumbnail image Tn3 is displayed. For example, when the history management unit 18 (or the display control unit 7) receives a signal (selection signal) selected by clicking the thumbnail image Tn3 by the user, information associated with the thumbnail image Tn3 (eg, score, Comment) is displayed on the display unit 3. Further, for example, when the history management unit 18 (or the display control unit 7) receives the selection signal, in addition to the information associated with the thumbnail image Tn3, the history management unit 18 (or the display control unit 7) displays the 3D image (three-dimensionally) of the thumbnail image Tn3. It is also possible to display (image) on the display unit 3.

[Fifth Embodiment]
A fifth embodiment will be described. In the present embodiment, the same components as those in the above-described embodiment are denoted by the same reference numerals, and the description thereof is omitted or simplified. FIG. 26 is a diagram showing a user interface according to the fifth embodiment. FIG. 26 illustrates an example of a teaching image displayed on the display unit 3 as a user interface according to the embodiment.

  The teaching image Im30a to the teacher image Im30c are displayed in time sequence. For example, the teaching image Im30a is displayed first, then the teaching image Im30b is displayed, and then the teaching image Im30c is displayed. In the present embodiment, when the teaching image Im30c is not distinguished from the teaching image Im30a, the teaching image Im30 is appropriately described.

  In the present embodiment, the teaching image Im30 includes a first area AR1, a second area AR2, and a third area AR3. In FIG. 26, the first area AR1, the second area AR2, and the third area AR3 are each displayed side by side (collectively) in one area (eg, screen, window, information window) on the display unit 3. The In FIG. 26, the first area AR1 is a lower left area in the teaching image Im30. The second area AR2 is an upper left area (area above the first area AR1) in the teaching image Im30. The third area AR3 is an area on the right side of the first area AR1 and the second area AR2 in the teaching image Im30.

  In the first area AR1, a plurality of sample images Sa registered in advance are displayed. The plurality of sample images Sa include sample images Sa1 to Sa5. In the first area AR1, the sample images Sa1 to Sa5 are arranged in the order in which the posture is scheduled to be performed. The order in which postures are scheduled to be performed is, for example, the order in which poses are set in the posture definition data D1 shown in FIG.

  Sample image Sa1 to sample image Sa5 are arranged from left to right in the order of postures to be performed. For example, the sample image Sa1 is arranged on the leftmost side in the first area AR1, and corresponds to the posture that is first performed among the sample images Sa1 to Sa5. Further, for example, the sample image Sa2 is arranged on the right side of the sample image Sa1, and corresponds to the posture performed next to the sample image Sa1.

  A mark MK is displayed in the first area AR1. The mark MK is positioned with respect to the sample image Sa corresponding to the current posture. The current posture is, for example, the target pose set in the process of step S5 in FIG. The mark MK is a figure represented by a color and brightness that can be easily distinguished from the surroundings. The mark MK is, for example, a rectangular frame shape, and is arranged at a position surrounding the sample image Sa corresponding to the current posture.

  For example, in the teaching image Im30a, the sample image Sa1 corresponds to the current posture, and the mark MK is positioned with respect to the sample image Sa1. In the second area AR2, a sample image Sb obtained by enlarging the sample image Sa corresponding to the current posture is displayed. For example, in the teaching image Im30a, the sample image Sb obtained by enlarging the sample image Sa1 is displayed in the second area AR2. The user model UM and the teaching figure F are displayed in the third area AR3.

  When this posture setting time elapses after the posture of the sample image Sa1 is set as the target, the posture of the sample image Sa2 is set as the next target as shown in the teaching image Im30b. In the first area AR1 of the teaching image Im30b, the mark MK is positioned and displayed with respect to the sample image Sa2. In the teaching image Im30b, the sample image Sb obtained by enlarging the sample image Sa2 is displayed in the second area AR. The user model UM and the teaching figure F are displayed in the third area AR3.

  Further, when the set time of the pose elapses after the posture of the sample image Sa2 is set as the target, the posture of the sample image Sa3 is set as the next target as shown in the teaching image Im30c. In the first area AR1 of the teaching image Im30c, the mark MK is positioned and displayed with respect to the sample image Sa3. In the teaching image Im30c, the sample image Sb obtained by enlarging the sample image Sa3 is displayed in the second area AR. The user model UM and the teaching figure F are displayed in the third area AR3.

  Note that the first area AR1, the second area AR2, and the third area AR3 may be areas that are divided into two or more areas (for example, a screen, a window, and an information window). In addition, the arrangement of the first area AR1, the second area AR2, and the third area AR3 is not limited to the example illustrated in FIG. 26, and is arbitrarily set. Further, the arrangement of the first area AR1, the second area AR2, and the third area AR3 may be set by the user or may be changed by the user's designation.

[Sixth Embodiment]
Next, a sixth embodiment will be described. In the description of this embodiment, FIG. 18, FIG. 19, FIG. 24, FIG. In the present embodiment, the image processing unit 13 determines the size and color of at least some of the plurality of parts based on the comparison result between the user posture obtained from the detection result of the detection unit 5 and the posture registered in advance. Change one or both.

  The detection system 1 according to the present embodiment (see FIG. 18) calculates the similarity between the posture of the user U obtained from the detection result of the detection unit 5 and the posture registered in advance for each part of the human body. A part. The detection system according to the present embodiment includes the posture specifying unit 15 illustrated in FIG. 18 as the similarity calculation unit. The posture specifying unit 15, as described with reference to FIGS. 24 and 25, describes the positional relationship of each part of the user U obtained from the detection result of the detection unit 5 and the “definition” of the posture definition data D <b> 1 stored in the storage unit 8. The degree of similarity with the positional relationship of each part defined in “Data” (see FIG. 19A) is calculated.

  In the present embodiment, the image processing unit 13 (see FIG. 18) adjusts the display of the teaching graphic based on the similarity of each part calculated by the posture specifying unit 15. The processing of the image processing unit 13 according to the present embodiment will be described later with reference to FIG. 27, FIG. 28, and the like, but the image processing unit 13 uses at least a plurality of parts based on the similarity calculated by the posture specifying unit 15. Partial size (size), color (eg, hue, saturation, brightness, contrast), thickness (eg, line thickness), display position offset, movement speed, and typeface (eg, text) At least one of the fonts). The image processing unit 13 uses a size, a color, a thickness (eg, a line thickness), a display position based on the similarity for a part associated with a human body part corresponding to the similarity among the plurality of parts. At least one of the offset amount, the moving speed, and the typeface (for example, character font) is determined. For example, for each of a plurality of parts included in the teaching graphic, the image processing unit 13 determines the size, color, and thickness (eg, line thickness) of each part based on the similarity of the part associated with each part. A), adjusting (changing) at least one of the offset amount of the display position, the moving speed, and the typeface (eg, the font of the character).

  27 and 28 are diagrams each illustrating an example of a teaching image generated by the image processing unit according to the present embodiment. In the teaching image Im41 shown in FIG. 27A, for example, the size of a part is adjusted based on the degree of similarity. The teaching image Im41 includes a user model UM and a teaching figure F. The teaching figure F includes a plurality of parts (p81, p82, p83).

  The part p81 is positioned with the head of the user model UM. The part p81 has a tip shape of an arrow and represents an image of the direction of the head (face). The part p82 is positioned with the right foot of the user model UM. The part p82 has an arrow tip shape and represents an image of the right foot. The part p83 is positioned with the left foot of the user model UM. A part p83 has a tip shape of an arrow and represents an image of the direction of the left foot.

  When the sample image Sa and the user model UM are compared in the teaching image Im41, the similarity degree of the right foot associated with the part p82 is low. In such a case, the image processing unit 13 (see FIG. 18) generates a teaching image Im41 whose display is different between the part p82 and the other parts (p81, p83). In the teaching image Im41, the part p81 and the part p83 are displayed in the same size. In the teaching image Im41, the part p82 is highlighted and displayed with a size larger than both the part p81 and the part p83.

  For example, the image processing unit 13 calculates and compares the similarity of a part and a threshold for each part, and sets the part to the first size when the similarity is equal to or greater than the threshold. In addition, for example, the image processing unit 13 compares the similarity between the parts for each part and a threshold, and sets the part to a second size larger than the first size when the similarity is less than the threshold. To do.

  The image processing unit 13 may set the size of the part to a value calculated by converting the similarity of the part corresponding to the part (for example, a relative value obtained by relatively converting each similarity). For example, the image processing unit 13 may determine the size of the part so as to be inversely proportional to the similarity. Further, the image processing unit 13 may change the size of the parts based on the degree of correlation obtained by using statistical processing (eg, multivariate analysis) based on the above-described similarity. For example, the image processing unit 13 sets a part having a low correlation degree to a size larger than the size of other parts.

  In the teaching image Im41 shown in FIG. 27B, as an example, the color of a part is adjusted based on the degree of similarity. In FIG. 27B, the sample image Sa and the user model UM are the same as those in FIG. In FIG. 27B, part p81, part p82, and part p83 are displayed in the same size. In the teaching image Im41, the part p81 and the part p83 are displayed in the same color. In the teaching image Im41, the part p82 is highlighted and displayed in a color different from both the part p81 and the part p83.

  For example, the image processing unit 13 compares the similarity of a part with a threshold for each part, and sets the part to the first color when the similarity is equal to or higher than the threshold. In addition, for example, the image processing unit 13 compares the similarity of a part with a threshold value for each part, and sets the part to a second color different from the first color when the similarity is less than the threshold value. .

  Note that the image processing unit 13 may set the color (eg, contrast, brightness) of the part to a value (eg, gradation value) calculated from the similarity of the part corresponding to the part. For example, the image processing unit 13 may determine the brightness (contrast) of the parts so as to be inversely proportional to the similarity. The image processing unit 13 may change the color and size of the parts based on the similarity. For example, the part p82 may be displayed with a different size and a different color from the parts p81 and p83.

  In the teaching image Im42 shown in FIG. 28A, the size of the part is adjusted based on the similarity. The teaching image Im42 includes a user model UM and a teaching figure F. The teaching figure F includes a plurality of parts (p91 to p98). The plurality of parts (p91 to p98) are each positioned on the left body side of the user model UM. Parts p91, parts p95 to p98 are displayed in the first size. For part p92 and part p94, for example, the similarity of corresponding parts is less than the first threshold. The part p92 and the part p94 are highlighted and displayed with a second size larger than the first size. Further, for the part p93, for example, the similarity of the corresponding part is less than the second threshold value. The second threshold is smaller than the first threshold. The part p93 is highlighted and displayed in a third size larger than the second size.

  In the teaching image Im42 shown in FIG. 28B, the colors of the parts are adjusted based on the similarity. In FIG. 28B, the sample image Sa and the user model UM are the same as those in FIG. In FIG. 28B, parts p91 to p98 are displayed in the same size. Further, the parts p91 and parts p95 to p98 are displayed in the first color. Further, the parts p92 to p94 are displayed with being emphasized with a second color different from the first color.

  For example, the image processing unit 13 includes a posture of the user U (for example, the shape of the user model UM) obtained from the detection result of the detection unit 5 and a posture registered in advance (for example, a posture defined in the posture data D1). Based on the comparison result, at least one of the plurality of parts is emphasized so as to be visually distinguishable from other parts. As described above, in the present embodiment, the image processing unit 13 emphasizes so that at least one of a plurality of parts can be visually distinguished from other parts based on the similarity calculated by the posture specifying unit 15. Then, the highlighted part is displayed on the display unit 3 together with other parts as identification information (emphasis information) that can be distinguished from other parts.

For example, the image processing unit 13 compares the posture of the user U (for example, the shape of the user model UM) obtained from the detection result of the detection unit 5 with a pre-registered posture example, the posture defined in the posture data D1). Based on the result, at least one of the size, color, thickness, display position offset amount, moving speed, and typeface of the plurality of parts is changed. Based on the size (size), color (e.g., hue, saturation, brightness, contrast), thickness (e.g., line thickness) of at least some of the parts, display position offset, At least one of the moving speed and the typeface (for example, the font of the character) is changed to identification information different from the other parts and emphasized.

[Seventh Embodiment]
A seventh embodiment will be described. In the present embodiment, the same components as those in the above-described embodiment are denoted by the same reference numerals, and the description thereof is omitted or simplified. FIG. 29 is a diagram illustrating a detection system according to the seventh embodiment. The detection system 1 includes a detection device 2, a display unit 3 (display device), a storage unit 8 (storage device), and an image processing device 20. The detection device 2 includes, for example, a motion capture device. The detection device 2 includes a detection unit 5, a model generation unit 11, and a part specifying unit 12. The detection device 2 outputs model information (eg, user model, skeleton information, skeleton data) of the user U as the detection result.

  The image processing device 20 positions a teaching figure including a plurality of parts for each part based on the user's posture obtained from the detection result with respect to the user's model generated based on the detection result of detecting the user U. Then, a teaching image including a user model and a teaching figure is generated. The image processing device 20 includes, for example, a display control unit 7 and an image processing unit 13. The image processing apparatus 20 uses the model information output from the detection apparatus 2 as a user model generated based on the detection result of detecting the user U. The image processing device 20 may include, for example, a cloud computer that is communicably connected to the detection device 2 via an Internet line.

  The storage unit 8 stores various data used for processing executed by the image processing apparatus 20. The storage unit 8 stores, for example, the posture definition data D1, the teaching figure data D2, and the display condition data D3 described with reference to FIG. The display unit 3 displays an image based on the image data supplied from the image processing device 20. The display unit 3 may be, for example, a stationary display or at least a part of a terminal that can be configured by the user. Examples of the terminal include a smartphone, a tablet, a head mounted display, and a wearable terminal.

  In the above-described embodiment, the image processing unit 13 (image processing apparatus 20) includes, for example, a computer system. The image processing unit 13 reads an image processing program stored in the storage unit 8 and executes various processes according to the image processing program. This image processing program stores, for each part, a teaching figure including a plurality of parts based on the user's posture obtained from the detection result for a user model generated based on the detection result obtained by detecting the user. Positioning is performed to generate a teaching image including a user model and a teaching figure. The image processing program may be provided by being recorded on a computer-readable storage medium (eg, non-transitory tangible media).

  In the above-described embodiment, the detection system 1 includes an image display system. This display system includes a display unit 3 that displays an image, and a first display processing unit (e.g., model generation) that dynamically displays a user model (e.g., user model UM) that changes over time on the display unit 3. Unit 11) and a second display processing unit (example) that dynamically displays a teaching figure selected based on a temporal change of the user's model at a position determined with respect to the user's model in the display unit 3 And image processing unit 13).

  The system (detection system, image display system, information providing system) according to the above-described embodiments can be widely used for lessons (activity, practice) for improving physical skills such as yoga and golf. For example, in a lesson facility, when it is difficult to see an instructor (teacher) or a mirror reflecting the user, the system according to the above embodiment uses a display unit (eg, display, tablet) instead of a mirror, It can be used to check the posture of the teacher and the posture of the teacher in real time while watching the teacher video. In addition, the system according to the above-described embodiment can be used for reviewing and preparing for a lesson target operation (for example, a pose that I am not good at). In addition, when it is difficult to go to a desired lesson facility due to time restrictions or geographical restrictions, the user U can take lessons using the system according to the embodiment at home or the like.

  The technical scope of the present invention is not limited to the aspects described in the above-described embodiments. One or more of the requirements described in the above embodiments and the like may be omitted. In addition, the requirements described in the above-described embodiments and the like can be combined as appropriate. In addition, as long as it is permitted by law, the disclosure of all documents cited in the above-described embodiments and the like is incorporated as a part of the description of the text.

DESCRIPTION OF SYMBOLS 1 ... Detection system, 2 ... Detection apparatus, 3 ... Display part, 5 ... Detection part, 6 ... Processing part, 7 ... Display control part, 8 ... Memory | storage part, DESCRIPTION OF SYMBOLS 11 ... Model production | generation part, 12 ... Part specific | specification part, 13 ... Image processing part, 15 ... Posture specific part, 16 ... Graphic division | segmentation part, 17 ... Input part, 18 ...・ History management unit, F ... Teach figure, U ... User

Claims (22)

A detection unit for detecting a user;
A model generation unit that generates a model of the user based on a detection result of the detection unit;
For the user model generated by the model generation unit, a teaching figure including a plurality of parts is positioned for each part based on the posture of the user obtained from the detection unit, and the model of the user and the user An image processing unit that generates a teaching image including a teaching figure. The teaching graphic includes at least one of a graphic representing a part of the human body, a graphic representing the shape of the human body, and a graphic representing the action of the human body.
The detection device according to claim 1. The figure includes characters,
The detection device according to claim 2. The teaching figure includes a deformed Chinese character,
At least one of the plurality of parts includes the radical of the deformed Chinese character,
The detection device according to any one of claims 1 to 3. The teaching figure includes a deformed word,
At least one of the plurality of parts includes a phonetic character constituting the deformed word.
The detection device according to any one of claims 1 to 4. A graphic dividing unit that divides the teaching graphic into the plurality of parts;
The detection device according to any one of claims 1 to 5. The image processing unit generates a plurality of images having different relative positions of the plurality of parts as the teaching image based on a change in the posture of the user obtained from the detection unit.
The detection device according to any one of claims 1 to 6. The image processing unit generates, as the teaching image, an image in which a first part of the plurality of parts is expressed in a color different from that of a second part;
The detection device according to any one of claims 1 to 7. The image processing unit visually compares at least one of the plurality of parts with another part based on a comparison result between the posture of the user obtained from the detection result of the detection unit and a pre-registered posture. Emphasize so that it can be identified
The detection device according to any one of claims 1 to 8. The image processing unit, based on a comparison result between the user posture obtained from the detection result of the detection unit and a posture registered in advance, the size, color, thickness of at least some of the plurality of parts, Change at least one of the offset amount of the display position, the moving speed, and the typeface.
The detection device according to any one of claims 1 to 9. A posture identifying unit that calculates the similarity between the posture of the user obtained from the detection result of the detection unit and the posture registered in advance for each part of the human body;
The image processing unit includes at least a size, a color, a thickness, a display position offset amount, a moving speed, and a typeface of the plurality of parts based on the similarity calculated by the posture specifying unit. Determine at least one,
The detection device according to claim 9 or 10. The image processing unit, for the parts associated with the part of the human body corresponding to the similarity among the plurality of parts, based on the similarity, size, color, thickness, display position offset amount, Determine the speed of movement and at least one of the typefaces;
The detection device according to claim 11. Based on the detection result of the detection unit, comprising a site specifying unit for specifying the user site,
The image processing unit
Positioning a part previously associated with the user part of the plurality of parts with respect to a position on the user model corresponding to the user part specified by the part specifying unit;
The detection device according to any one of claims 1 to 12. A posture identifying unit that identifies the type of posture of the user obtained from the detection unit from a plurality of types registered in advance;
The image processing unit generates the teaching image using the teaching figure previously associated with the type of posture of the user based on the type of posture of the user specified by the posture specifying unit.
The detection device according to any one of claims 1 to 13. The teaching image is displayed in a predetermined schedule on a display unit that displays an image of a posture registered in advance to the user.
The detection device according to any one of claims 1 to 13. A display control unit that causes the display unit to display the teaching image generated by the image processing unit;
The detection device according to any one of claims 1 to 15. The detection device according to any one of claims 1 to 16,
And a display unit that displays the teaching image generated by the image processing unit.   The teaching figure including a plurality of parts is positioned for each of the parts based on the posture of the user obtained from the detection result with respect to the model of the user generated based on the detection result of detecting the user, An image processing apparatus that generates a teaching image including a user model and the teaching figure. Detecting users,
Generating a model of the user based on a detection result of detecting the user;
A teaching graphic including a plurality of parts is positioned for each of the parts based on the posture of the user obtained from the detection result with respect to the user model, and a teaching image including the user model and the teaching graphic is obtained. Generating a detection method. On the computer,
The teaching figure including a plurality of parts is positioned for each of the parts based on the posture of the user obtained from the detection result with respect to the model of the user generated based on the detection result of detecting the user, An image processing program for executing generation of a teaching image including a user model and the teaching figure. Dynamically displaying a user's model that changes over time on the display;
An image display method comprising: dynamically displaying a teaching figure selected based on a temporal change of the user's model at a position determined with respect to the user's model in the display unit. A display for displaying an image;
A first display processing unit for dynamically displaying a time-varying user model on the display unit;
An image including a second display processing unit that dynamically displays a teaching figure selected based on a temporal change of the user's model at a position determined with respect to the user's model in the display unit; Display system.