WO2021192908A1 - 追跡方法 - Google Patents

追跡方法 Download PDF

Info

Publication number
WO2021192908A1
WO2021192908A1 PCT/JP2021/008569 JP2021008569W WO2021192908A1 WO 2021192908 A1 WO2021192908 A1 WO 2021192908A1 JP 2021008569 W JP2021008569 W JP 2021008569W WO 2021192908 A1 WO2021192908 A1 WO 2021192908A1
Authority
WO
WIPO (PCT)
Prior art keywords
unit
person
image data
tracking
information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2021/008569
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
宏紀 寺島
克幸 永井
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NEC Solution Innovators Ltd
Original Assignee
NEC Solution Innovators Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by NEC Solution Innovators Ltd filed Critical NEC Solution Innovators Ltd
Priority to JP2022509484A priority Critical patent/JP7343237B2/ja
Priority to CN202180019431.0A priority patent/CN115280362A/zh
Publication of WO2021192908A1 publication Critical patent/WO2021192908A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments

Definitions

  • the present invention relates to a tracking method, a tracking device, and a recording medium.
  • Patent Document 1 is a document that describes walking analysis of a person.
  • Patent Document 1 is created by a data acquisition unit that acquires two types of image data from a depth sensor, a skeleton information creation unit that creates skeleton information based on the image data acquired by the data acquisition unit, and a skeleton information creation unit.
  • a gait analyzer having a correction processing unit that corrects skeleton information and an analysis processing unit that analyzes the user's gait using the corrected skeleton information is described.
  • Patent Document 1 it is essential to use a depth sensor (3D sensor), but it is desired to perform gait analysis based on image data acquired by a camera without using a 3D sensor such as a depth sensor. There is a need.
  • 3D sensor 3D sensor
  • an object of the present invention is to provide a tracking method, a tracking device, and a recording medium that solve the problem that it may be difficult to track the same person between image data.
  • the tracking method which is a form of the present disclosure to achieve such an object,
  • Information processing device By recognizing the skeleton of a person in the image data, information indicating multiple parts of the recognized person is acquired, and information is obtained. Based on the acquired information, the same person is tracked among a plurality of image data.
  • the tracking device which is another form of the present disclosure, An acquisition unit that acquires information indicating multiple parts of the recognized person by recognizing the skeleton of the person in the image data, and an acquisition unit. A tracking unit that tracks the same person among a plurality of image data based on the information acquired by the acquisition unit. It has a structure of having.
  • the recording medium which is another form of the present disclosure is In the tracking device, An acquisition unit that acquires information indicating multiple parts of the recognized person by recognizing the skeleton of the person in the image data, and an acquisition unit. A tracking unit that tracks the same person among a plurality of image data based on the information acquired by the acquisition unit. It is a computer-readable recording medium on which a program for realizing the above is recorded.
  • FIG. 1st Embodiment of this disclosure It is a figure which shows the structural example of the walking posture measurement system in 1st Embodiment of this disclosure. It is a figure which shows an example at the time of photographing the walking posture in the front-back direction. It is a figure which shows an example at the time of photographing the walking posture in the left-right direction. It is a block diagram which shows the configuration example of the smartphone shown in FIG. It is a block diagram which shows the structural example of the photographing auxiliary part shown in FIG. It is a figure which shows the example of the shooting assistance display at the time of holding horizontally. It is a figure which shows the example of the shooting assistance display at the time of holding vertically. It is a figure which shows the operation example of the angle adjustment information output part shown in FIG.
  • FIG. 1 It is a block diagram which shows the structural example of the walking posture measuring apparatus shown in FIG. It is a figure which shows an example of the skeleton information shown in FIG. It is a figure which shows an example of the measurement result information shown in FIG. It is a figure for demonstrating the processing of the measured value calculation part. It is a figure for demonstrating the processing of the measured value calculation part. It is a figure for demonstrating the processing of the measured value calculation part. It is a figure for demonstrating the processing of the measured value calculation part. It is a figure for demonstrating the processing of the measured value calculation part. It is a figure for demonstrating the processing of the measured value calculation part. It is a figure for demonstrating the processing of the measured value calculation part. It is a figure for demonstrating the processing of the measured value calculation part. It is a figure for demonstrating the processing of the measured value calculation part. It is a figure for demonstrating the processing of the measured value calculation part. It is a figure for demonstrating the processing of the measured value calculation part.
  • FIG. 1 It is a figure which shows an example of the figure generated by the inclusion figure generation part shown in FIG. It is a flowchart which shows the operation example of the tracking part. It is a figure which shows an example of the hardware composition which the photographing apparatus has in 3rd Embodiment of this disclosure. It is a block diagram which shows the structural example of the photographing apparatus. It is a block diagram which shows the structural example of the information processing apparatus in 4th Embodiment of this disclosure. It is a block diagram which shows the structural example of the information processing apparatus in 5th Embodiment of this disclosure. It is a block diagram which shows the structural example of the tracking apparatus in 6th Embodiment of this disclosure.
  • FIG. 1 is a diagram showing a configuration example of the walking posture measurement system 100.
  • FIG. 2 is a diagram showing an example of photographing a walking posture in the front-back direction.
  • FIG. 3 is a diagram showing an example of photographing a walking posture in the left-right direction.
  • FIG. 4 is a block diagram showing a configuration example of the smartphone 200.
  • FIG. 5 is a block diagram showing a configuration example of the photographing auxiliary unit 212.
  • FIG. 6 is a diagram showing an example of a shooting assistance display when the camera is held horizontally.
  • FIG. 7 is a diagram showing an example of a shooting auxiliary display when the camera is held vertically.
  • FIG. 1 is a diagram showing a configuration example of the walking posture measurement system 100.
  • FIG. 2 is a diagram showing an example of photographing a walking posture in the front-back direction.
  • FIG. 3 is a diagram showing an example of photographing a walking posture in the left-right direction.
  • FIG. 4 is a block diagram
  • FIG. 8 is a diagram showing an operation example of the angle adjustment information output unit 2124.
  • FIG. 9 is a block diagram showing a configuration example of the walking posture measuring device 300.
  • FIG. 10 is a diagram showing an example of skeleton information 334.
  • FIG. 11 is a diagram showing an example of measurement result information 336.
  • FIG. 12 is a diagram for explaining the processing of the actually measured value calculation unit 343.
  • 13 to 19 are diagrams for explaining the processing of the actually measured value calculation unit 343.
  • FIG. 20 is a flowchart showing an operation example of the photographing assisting unit 212 in the smartphone 200.
  • FIG. 21 is a flowchart showing an operation example of the walking posture measuring device 300.
  • FIG. 22 is a flowchart showing an example of processing for calculating the measured value information 335.
  • the walking posture measuring system 100 that measures the walking posture of a person based on a moving image acquired by using a photographing device such as a smartphone 200 will be described.
  • a moving image showing a person walking in the front-back direction of an image that is, a plurality of image data
  • a moving image showing a person walking in the left-right direction of the image and a moving image showing a person walking in the left-right direction of the image, using a smartphone 200
  • the walking posture measuring system 100 measures the walking posture such as stride length, walking speed, straightness according to blurring of the head during walking, etc., based on a plurality of image data which are moving images acquired by shooting.
  • the walking posture measurement system 100 includes a mechanism for aligning shooting conditions as much as possible when acquiring image data using the smartphone 200, a mechanism for measuring an actually measured value from the image data, and the like. I'm out.
  • FIG. 1 shows a configuration example of the walking posture measurement system 100.
  • the walking posture measuring system 100 includes, for example, a smartphone 200 and a walking posture measuring device 300.
  • the smartphone 200 and the walking posture measuring device 300 are connected so as to be able to communicate with each other by, for example, wirelessly or by wire.
  • the smartphone 200 functions as a photographing device for photographing a person walking.
  • the smartphone 200 may be a smartphone having general functions such as a camera function, a touch panel 201 for displaying a screen, and various sensors such as a GPS sensor and an acceleration sensor.
  • the photographer takes a picture of a person walking from the back of the screen to the front, etc., while holding the smartphone 200 vertically (holding it vertically).
  • the photographer takes a picture of a person walking in the front-back direction while the short side of the rectangular-shaped smartphone 200 is held vertically so as to be horizontal to the ground.
  • the photographer takes a picture of a person walking in the left-right direction such as from the left to the right of the screen while the smartphone 200 is held sideways (sideways).
  • the photographer takes a picture of a person walking in the front-back direction while the smartphone 200 is held sideways with the long side horizontal to the ground.
  • the photographer shoots a type of moving image (a plurality of image data) according to the orientation of the smartphone 200.
  • the appearance of a person walking in the front-back direction and the appearance of a person walking in the left-right direction may be divided into two shots using, for example, one smartphone 200, or two smartphones 200 may be used. You may shoot at the same time.
  • FIG. 4 shows a characteristic configuration example in the present embodiment of the smartphone 200.
  • the smartphone 200 has a measurement motion photographing unit 210 in addition to a general configuration as a smartphone such as an acceleration sensor and a gyro sensor for detecting the orientation (vertical orientation, horizontal orientation) of the smartphone 200. doing.
  • the measurement motion photographing unit 210 includes an image data photographing unit 211 and an imaging assisting unit 212.
  • the smartphone 200 has an arithmetic unit such as a CPU (Central Processing Unit) and a storage device.
  • the smartphone 200 realizes the above-mentioned processing unit by, for example, executing a program stored in the storage device by the arithmetic unit.
  • the measurement motion photographing unit 210 photographs a person walking, which is the measurement motion in the present embodiment, in response to the photographer's operation on the smartphone 200.
  • the measurement motion photographing unit 210 includes a function for assisting shooting so that shooting can be performed under the same conditions as possible when performing shooting.
  • the measurement motion photographing unit 210 has a camera operation function for photographing a person walking, and also captures the walking direction, angle, size reflected on the screen, etc. when photographing the person walking. It is a shooting application that has a guide function to meet the conditions as much as possible.
  • the measurement motion photographing unit 210 includes an image data photographing unit 211 and an imaging assisting unit 212.
  • the image data shooting unit 211 acquires a moving image (a plurality of image data) by shooting a person using the camera of the smartphone 200. Further, the image data shooting unit 211 associates the moving image (image data) shot by the image data shooting unit 211 with information indicating the acquisition date and time of the image data, information acquired by the shooting assistance unit 212, which will be described later, and the like. Can be done.
  • the shooting assistance unit 212 assists the image data shooting unit 211 to arrange the shooting conditions as much as possible when acquiring the image data.
  • FIG. 5 shows an example of the configuration included in the photographing assisting unit 212.
  • the photographing auxiliary unit 212 includes, for example, a guide line display unit 2121, an angle information display unit 2122, a height information input unit 2123, and an angle adjustment information output unit 2124.
  • the guide line display unit 2121 displays the guide line 2011 on the touch panel 201, which serves as a guideline for the position where the person walks, such as the position of the person's feet when shooting the person.
  • the guide line 2011 displayed on the touch panel 201 as much as possible, it is possible to align the walking direction, angle, size displayed on the screen, etc. when the person walks. ..
  • a mark or the like may be placed in the real world so that the position coincides with the position indicated by the guide line 2011, and the person may walk using the mark.
  • the guide line display unit 2121 displays different guide lines 2011 on the touch panel 201 depending on whether the smartphone 200 is held vertically or horizontally. For example, when it is determined that the smartphone 200 is held horizontally based on the information acquired from the acceleration sensor or the like, the guide line display unit 2121 is for photographing a person walking in the left-right direction of the screen as shown in FIG.
  • the guide line 2011 is displayed on the touch panel 201.
  • the guide line display unit 2121 displays a guide line 2011 for guiding a person walking in the left-right direction on the touch panel 201.
  • the shooting area which is the area shot by the camera of the smartphone 200
  • the guide line 2011 is displayed.
  • the guide line display unit 2121 photographs a person walking in the front-back direction of the screen as shown in FIG.
  • the guide line 2011 for this purpose is displayed on the touch panel 201.
  • the guide line display unit 2121 displays a guide line 2011 for guiding a person walking in the front-back direction on the touch panel 201.
  • the position where the guide line display unit 2121 displays the guide line 2011 is, for example, predetermined.
  • the guide line display unit 2121 displays the guide line 2011 below the center of the photographing area (for example, about the middle of the lower half area) as shown in FIG.
  • the guide line display unit 2121 displays the guide line 2011 in the center of the photographing area as shown in FIG. 7.
  • the position where the guide line display unit 2121 displays the guide line 2011 may be other than those illustrated above.
  • the angle information display unit 2122 acquires information indicating the angle of the smartphone 200, such as an inclination in the left-right direction and an inclination in the front-back direction, from an acceleration sensor or a gyro sensor of the smartphone 200. Then, the angle information display unit 2122 displays the acquired information on the touch panel 201 as angle information 2012 indicating the angle of the smartphone 200.
  • angle information 2012 When shooting a person walking, it is desirable that the smartphone 200 is not tilted as much as possible.
  • By displaying the angle information 2012 on the touch panel 201 by the angle information display unit 2122 it is possible to correct the angle of the smartphone 200 when the photographer takes a picture of a person walking, and it is desirable that the smartphone 200 is not tilted. It is possible to take a picture of a person walking in a state.
  • the angle information display unit 2122 displays information indicating the inclination in the left-right direction and information indicating the inclination in the front-back direction on the touch panel 201.
  • the angle information display unit 2122 displays information indicating the inclination of the smartphone 200 in the horizontal direction and the vertical direction.
  • the angle information display unit 2122 displays the angle information 2012 at a predetermined position on the touch panel 201.
  • the display of the angle information 2012 by the angle information display unit 2122 is the same (display according to the inclination) although the display position changes depending on whether the smartphone 200 is held vertically or horizontally. It doesn't matter.
  • the height information input unit 2123 receives input of height information indicating the height h from the ground of the smartphone 200 from a person, and displays the height display unit 2013 displaying the input height h on the touch panel 201 on the touch panel 201. do.
  • the height h received by the height information input unit 2123 can be utilized when the walking posture measuring device 300 calculates the actually measured value W.
  • the height information input unit 2123 displays the height display unit 2013 at a predetermined position on the touch panel 201. Then, the height information input unit 2123 receives, for example, an input of information indicating the height h from the person by touching the height display unit 2013 by a person who operates the smartphone 200. After that, the height information input unit 2123 displays the information indicating the received height h on the height display unit 2013.
  • the display of the height display unit 2013 by the height information input unit 2123 may be the same as the display content although the display position changes depending on whether the smartphone 200 is held vertically or horizontally.
  • the angle adjustment information output unit 2124 outputs information for adjusting the inclination of the smartphone 200.
  • the angle adjustment information output unit 2124 outputs information according to the inclination of the smartphone 200, which differs depending on the inclination of the smartphone 200.
  • the angle adjustment information output unit 2124 outputs a sound adjusted according to the inclination of the smartphone 200 as information for adjusting the inclination of the smartphone 200.
  • FIG. 8 shows an example of processing by the angle adjustment information output unit 2124.
  • the angle adjustment information output unit 2124 outputs a sound whose length is adjusted according to the inclination of the smartphone 200 in the left-right direction, and outputs the sound whose length is adjusted according to the inclination of the smartphone 200 in the front-back direction. Outputs a sound with adjusted pitch.
  • the angle adjustment information output unit 2124 makes different adjustments depending on how the smartphone 200 is tilted. For example, referring to FIG. 8, the angle adjustment information output unit 2124 adjusts the length of one sound so that the more the smartphone 200 is tilted to the left, the shorter the length of one sound is.
  • the angle adjustment information output unit 2124 adjusts the length of the sound so that the longer the smartphone 200 is tilted to the right, the longer the length of one sound is. Further, the angle adjustment information output unit 2124 adjusts the pitch so that the pitch is lowered as the smartphone 200 is tilted toward the front (for example, the photographer side). Further, the angle adjustment information output unit 2124 adjusts the pitch so that the pitch increases as the smartphone 200 tilts toward the back (for example, the side opposite to the photographer).
  • the angle adjustment information output unit 2124 adjusts at least one of the length and pitch of the sound according to the tilt. Outputs two types of sounds.
  • the smartphone 200 is not tilted, the length and pitch of the sound are not adjusted, so that the angle adjustment information output unit 2124 outputs one type of sound.
  • the information for adjusting the tilt of the smartphone 200 which is output by the angle adjustment information output unit 2124, is not necessarily limited to the case of sound.
  • the angle adjustment information output unit 2124 may illuminate a light or vibrate the smartphone 200 instead of the sound.
  • the angle adjustment information output unit 2124 may be configured to perform processing such as illuminating a light or vibrating the smartphone 200 when approaching an angle having no inclination, which is a correct angle.
  • the angle adjustment information output unit 2124 outputs a sound according to the inclination in the left-right direction and illuminates the light according to the inclination in the front-back direction, and outputs a sound and illuminates the light according to the inclination in the left-right direction.
  • the above-mentioned processes may be combined in various combinations such as.
  • the above is a configuration example of the smartphone 200, which is characteristic of this embodiment.
  • the walking posture measuring device 300 is a server device that measures the walking posture such as stride length, walking speed, straightness, etc., based on a moving image (that is, a plurality of image data) taken by the smartphone 200.
  • FIG. 9 shows a configuration example of the walking posture measuring device 300.
  • the walking posture measuring device 300 has, for example, a screen display unit 310, a communication I / F unit 320, a storage unit 330, and an arithmetic processing unit 340 as main components. There is.
  • the screen display unit 310 includes a screen display device such as a touch panel or a liquid crystal display.
  • the screen display unit 310 displays the position of the skeleton indicated by the skeleton information 334 in the image data included in the image information 333, the skeleton information 334, the measurement result information 336, and the image information 333 in response to an instruction from the arithmetic processing unit 340. It is possible to display superimposed images, etc.
  • the communication I / F unit 320 includes a data communication circuit.
  • the communication I / F unit 320 performs data communication with an external device, a smartphone 200, or the like connected via a communication line.
  • the storage unit 330 is a storage device such as a hard disk or a memory.
  • the storage unit 330 stores processing information and a program 337 required for various processes in the arithmetic processing unit 340.
  • the program 337 realizes various processing units by being read and executed by the arithmetic processing unit 340.
  • the program 337 is read in advance from an external device or a recording medium via a data input / output function such as the communication I / F unit 320, and is stored in the storage unit 330.
  • the main information stored in the storage unit 330 includes, for example, the trained model 331, the camera setting information 332, the image information 333, the skeleton information 334, the measured value information 335, and the measurement result information 336.
  • the trained model 331 is a trained model used by the skeleton recognition unit 342 when performing skeleton recognition.
  • the trained model 331 is generated in advance by performing machine learning using teacher data such as image data containing skeleton coordinates in an external device or the like, and is externally generated via a communication I / F unit 320 or the like. It is acquired from a device or the like and stored in the storage unit 330.
  • the trained model 331 may be updated by a re-learning process using additional teacher data.
  • the camera setting information 332 includes information indicating the parameters of the camera possessed by the smartphone 200, which is used when the smartphone 200 captures the walking of a person.
  • the camera setting information 332 includes, for example, information indicating a vertical viewing angle ⁇ and a horizontal viewing angle ⁇ of the camera.
  • the camera setting information 332 is acquired in advance from the smartphone 200 or the like via, for example, the communication I / F unit 320 or the like, and is stored in the storage unit 330.
  • the camera setting information 332 may be acquired from the smartphone 200 together with the image data and stored in the storage unit 330.
  • the image information 333 includes image data (video) acquired by the camera of the smartphone 200.
  • image data for example, image data, information indicating the date and time when the smartphone 200 acquired the image data, and information indicating the height input by the height information input unit 2123 are provided for each unit of moving image. Etc. are associated with each other.
  • a moving image of a person walking in the left-right direction and a moving image of a person walking in the front-back direction are associated with each other.
  • the measuring unit 344 measures the walking posture corresponding to the moving image of the person walking in the left-right direction and the moving image of the person walking in the front-back direction.
  • the skeleton information 334 includes information indicating the coordinates of each part of the person recognized by the skeleton recognition unit 342.
  • FIG. 10 shows an example of skeleton information 334.
  • the time and the position information of each part are associated with each person to be recognized. The time indicates the elapsed time from the start of movie shooting, the time when the movie was shot, and the like.
  • the position information of each part includes information indicating the coordinates of each part in the image data such as the position of the pelvis.
  • the parts included in the position information of each part correspond to the trained model 331.
  • the pelvis, the center of the spine, ..., Are illustrated.
  • the position information of each part can include, for example, about 30 parts such as right shoulder, ..., left elbow, ..., right knee, ..., etc. (other than the examples may be used).
  • the part included in the position information of each part may be other than those illustrated in FIG. 10 and the like.
  • the actual measurement value information 335 includes information indicating the actual measurement value W calculated by the actual measurement value calculation unit 343.
  • the actually measured value W is associated with the reference line, the identification information of the image data, and the like.
  • the measured value information 335 may include information indicating the stride length and the like. A detailed description of the processing of the measured value calculation unit 343 will be described later.
  • the measurement result information 336 shows the result of the walking posture measurement measured by the measuring unit 344.
  • FIG. 11 shows an example of the measurement result information 336.
  • time indicates the elapsed time from the start of moving image shooting, the time when the moving image was taken, and the like.
  • the walking speed indicates the speed at which a person walks.
  • stride length indicates the length of the toe gap (or heel gap) between the right foot and the left foot when the person walks.
  • the straightness indicates the degree of shaking and blurring of the head and body when a person walks.
  • the measurement result information 336 may include information indicating a pitch or the like indicating the time required for one step.
  • the walking speed and the stride are the information measured by the measured value calculation unit 343 and the measurement unit 344 based on the moving image (image data) of the person walking in the left-right direction. ..
  • the straightness is information measured by the measuring unit 344 based on a moving image (image data) in which a person walks in the front-back direction.
  • the measurement result information 336 is measured by the measuring unit 344 based on the information measured by the measuring unit 344 based on the moving image of the person walking in the left-right direction and the measuring unit 344 based on the moving image of the person walking in the front-back direction. Information and is included.
  • the arithmetic processing unit 340 has a microprocessor such as an MPU and its peripheral circuits.
  • the arithmetic processing unit 340 reads the program 337 from the storage unit 330 and executes it, thereby realizing various processing units in cooperation with the hardware and the program 337.
  • the main processing units realized by the arithmetic processing unit 340 include, for example, an image acquisition unit 341, a skeleton recognition unit 342, an actual measurement value calculation unit 343, a measurement unit 344, and an output unit 345.
  • the image acquisition unit 341 acquires a moving image (a plurality of image data) acquired by the smartphone 200 from the smartphone 200 via the communication I / F unit 320. Then, the image acquisition unit 341 stores the acquired image data in the storage unit 330 as image information 333 in association with, for example, information indicating the acquisition date and time and height of the image data.
  • the image acquisition unit 341 associates or supports a moving image (image data) of a person walking in the left-right direction and a moving image (image data) of the person walking in the front-back direction from the smartphone 200. Get it so that it can be attached. Then, the image acquisition unit 341 associates the acquired two types of moving images and stores them in the storage unit 330 as image information 333.
  • the skeleton recognition unit 342 uses the trained model 331 to recognize the skeleton of the person whose walking posture is to be measured in the image data. For example, the skeleton recognition unit 342 recognizes each part such as the upper part of the spine, the right shoulder, the left shoulder, the right elbow, the left elbow, the right wrist, the left wrist, the right hand, the left hand, and so on. Further, the skeleton recognition unit 342 calculates the coordinates in the screen data of each recognized part. Then, the skeleton recognition unit 342 stores the recognition / calculation result in the storage unit 330 as skeleton information 334 for each person by associating it with the identification information for identifying the person.
  • the portion recognized by the skeleton recognition unit 342 corresponds to the trained model 331 (teacher data used when learning the trained model 331). Therefore, the skeleton recognition unit 342 may recognize a part other than the above-exemplified parts according to the trained model 331.
  • the measured value calculation unit 343 has a height h of the camera of the smartphone 200 from the floor surface, a vertical viewing angle ⁇ of the camera of the smartphone 200, a horizontal viewing angle ⁇ , and a reference line screen having an arbitrary height on the screen. Based on the ratio ⁇ from half, the measured value W from one end of the screen to the other of the reference line is calculated. In addition, the measured value calculation unit 343 can calculate the stride length of a person or the like by using the calculated measured value W. Then, the actual measurement value calculation unit 343 stores the calculated actual measurement value W and the like as the actual measurement value information 335 in the storage unit 330. The actual measurement value calculation unit 343 can perform the process of calculating the actual measurement value W for each frame (that is, for each image data) in the moving image.
  • the measured value calculation unit 343 makes it possible to calculate the measured value W, which is the actual length (that is, the length in the real world), by using the various values described above.
  • FIG. 12 shows a mathematical formula used by the measured value calculation unit 343 to calculate the measured value W.
  • the actual measurement value calculation unit 343 calculates the actual measurement value W by solving the equation shown in Equation 1. That is, the measured value calculation unit 343 determines the height h of the camera of the smartphone 200 from the floor surface, the parameters of the camera of the smartphone 200, and the ratio ⁇ from half of the screen of the reference line for calculating the measured value W.
  • the measured value W is calculated based on.
  • the actually measured value W indicates, for example, the actual length of the reference line in the image data from the screen edge to the edge.
  • h is the height of the camera (smartphone 200) from the floor surface.
  • the value of height h is input by the photographer when acquiring image data using the height information input unit 2123. Further, ⁇ is a vertical viewing angle, and ⁇ is a horizontal viewing angle. ⁇ is the ratio of the reference line of any height on the screen from half the screen.
  • Equation 1 will be described in more detail with reference to FIGS. 13 to 18.
  • FIG. 13 it is assumed that a person on a reference line at an arbitrary height on the screen is photographed by using a camera of a smartphone 200 located at a height h.
  • the position of the smartphone 200 located at the height h is set as the origin O.
  • the point of contact with the floor when the smartphone 200 is lowered vertically toward the floor is defined as the point G
  • the position of the person is defined as the point P.
  • the angle formed by the line connecting the origin O and the point P and the ground (or a line horizontal to the ground) is defined as an angle ⁇ .
  • the angle ⁇ is the ratio of the positions of the arbitrary points P on the screen, and the vertical viewing angle ⁇ is the equation 4.
  • FIG. 17 one end of a line having an arbitrary height on the screen passing through the point P is designated as the point Q.
  • the number is 6 rather than 5.
  • substituting the number 7 into the number 6 gives the number 8.
  • the measured value W which is the full length of the horizontal edge of the screen at the distance d from the smartphone 200, is 2w, the above-mentioned number 8 is doubled to the above-mentioned number 1. From the above, it can be seen that the measured value W can be calculated by calculating Equation 1.
  • the reference line at an arbitrary height on the screen which is the calculation source of the ratio ⁇
  • the actually measured value calculation unit 343 can perform a process of calculating the ratio ⁇ after specifying the reference line based on the position of the foot of the person in the image data.
  • the measured value calculation unit 343 can use a line having a slope of 0 passing through the average value of the Y coordinates of the left and right feet of the person, the Y coordinate of any of the feet, and the like as a reference line (other than the example). It doesn't matter).
  • the above processing by the actually measured value calculation unit 343 may be performed, for example, for each frame (that is, for each image data). Further, the height of the reference line on the screen may be predetermined according to the position of the guide line 2011 displayed by the guide line display unit 2121 or the like.
  • the actual measurement value calculation unit 343 can be calculated by calculating 100 ⁇ k.
  • the measured value calculation unit 343 can calculate the stride length based on the calculated measured value W, the resolution, and the number of pixels.
  • the actual measurement value calculation unit 343 can calculate the actual measurement value W of the length from the screen edge to the end of the reference line, and can calculate the stride length of the person based on the actual measurement value W. ..
  • the measurement unit 344 measures the walking posture of the person by using the calculation result by the actual measurement value calculation unit 343 and the recognition result by the skeleton recognition unit 342. Then, the measurement unit 344 stores the measurement result and the like as the measurement result information 336 in the storage unit 330.
  • the measurement unit 344 can perform measurement based on a moving image of a person walking in the left-right direction and measurement based on a moving image of a person walking in the front-back direction.
  • the measuring unit 344 can calculate the walking speed, pitch, and the like based on a moving image (image data) of a person walking in the left-right direction.
  • the measuring unit 344 calculates the moving distance between frames of the portion recognized by the skeleton recognition unit 342, and calculates the walking speed based on the calculated moving distance and the time of each frame (image data). Can be done.
  • the measuring unit 344 may use the calculation result by the measured value calculation unit 343 such as the stride length.
  • the measuring unit 344 can calculate the straightness and the like based on the moving image (image data) in which the person walks in the front-back direction. For example, the measuring unit 344 can calculate the straightness based on the fluctuation of the coordinates of the head recognized by the skeleton recognition unit 342.
  • the measurement unit 344 may be configured to perform measurements other than those illustrated above.
  • the output unit 345 can output the skeleton information 334, the measured value information 335, the measurement result information 336, the moving image included in the image information 333, and the skeleton information 334 superimposed on the moving image.
  • the output by the output unit 345 is performed by displaying each of the above information on the screen display unit 310 or transmitting the information to an external device connected via the communication I / F unit 320.
  • the above is a configuration example of the walking posture measuring device 300.
  • FIG. 20 shows an operation example of the angle adjustment information output unit 2124.
  • the angle adjustment information output unit 2124 corrects the pitch according to the tilt (step S102). That is, the angle adjustment information output unit 2124 corrects the pitch more greatly as the smartphone 200 is tilted toward the front or the back. Then, the angle adjustment information output unit 2124 outputs the corrected sound (step S103).
  • the angle adjustment information output unit 2124 corrects the sound length according to the tilt (step S102). That is, the angle adjustment information output unit 2124 corrects the length of the sound more as the smartphone 200 is tilted to the left or right. Then, the angle adjustment information output unit 2124 outputs the corrected sound (step S103).
  • the sound output by the angle adjustment information output unit 2124 ends when the end condition is satisfied (step S107, Yes).
  • the conditions for termination include, for example, that the shooting of the moving image has started, the shooting of the moving image has ended, a predetermined time has passed since the tilt of the smartphone 200 disappeared and the sounds matched, and the output was stopped by a person. And so on.
  • the conditions for termination may be other than those illustrated above.
  • the image acquisition unit 341 acquires a moving image (a plurality of image data) acquired by the smartphone 200 from the smartphone 200 via the communication I / F unit 320 (step S201).
  • the image acquisition unit 341 associates or supports a moving image (image data) of a person walking in the left-right direction and a moving image (image data) of the person walking in the front-back direction from the smartphone 200. Get it so that it can be attached.
  • the skeleton recognition unit 342 recognizes the skeleton of the person whose walking posture is to be measured in the image data by using the trained model 331 (step S202).
  • the measuring unit 344 acquires the measured value W or the like indicated by the measured value information 335 (step S203).
  • the actual measurement value W indicated by the actual measurement value information 335 may be calculated in advance by the actual measurement value calculation unit 343, for example, in parallel with the skeleton recognition process by the skeleton recognition unit 342, or by the skeleton recognition unit. After the recognition process by 342, it may be calculated by the actual measurement value calculation unit 343.
  • the measurement unit 344 measures the walking posture of the person by using the calculation result by the actual measurement value calculation unit 343 (step S204). For example, the measurement unit 344 performs measurement based on a moving image of a person walking in the left-right direction and measurement based on a moving image of a person walking in the front-back direction.
  • the output unit 345 outputs the skeleton information 334, the measured value information 335, the measurement result information 336, the moving image included in the image information 333, and the skeleton information 334 superimposed on the moving image (step S205).
  • the measured value calculation unit 343 acquires information indicating the height input by the height information input unit 2123 included in the image information 333. Further, the measured value calculation unit 343 acquires information indicating the vertical viewing angle ⁇ and the horizontal viewing angle ⁇ of the camera with reference to the camera setting information 332 (step S301).
  • the measured value calculation unit 343 determines the reference line for calculating the measured value W and calculates the ratio ⁇ (step S302). For example, the measured value calculation unit 343 specifies the reference line based on the position of the foot of a person in the image data. Further, the measured value calculation unit 343 calculates the ratio ⁇ from the half of the screen of the reference line based on the specified reference line.
  • the measured value calculation unit 343 calculates the measured value W based on the height h, the vertical viewing angle ⁇ , the horizontal viewing angle ⁇ , and the ratio ⁇ (step S303). For example, the actually measured value calculation unit 343 calculates the actually measured value W by solving the equation shown by the above-mentioned equation 1. Then, the actual measurement value calculation unit 343 stores the calculated actual measurement value W as the actual measurement value information 335 in the storage unit 330.
  • the measured value calculation unit 343 takes the difference between the x-coordinate values of the left and right feet (for example, the toes) in the image data. Then, the stride is calculated based on the difference between the x-coordinate values, the actually measured value W, and the resolution (step S304).
  • the above is a processing example of the measured value calculation unit 343.
  • the walking posture measuring device 300 has a skeleton recognition unit 342 and a measuring unit 344. According to such a configuration, the measuring unit 344 can measure the walking posture based on the result of the skeleton recognition by the skeleton recognition unit 342. As a result, the measuring unit 344 can measure the walking posture based on the image data acquired by using the camera or the like of the smartphone 200 without using the depth sensor or the like.
  • the walking posture measuring device 300 is configured to acquire a moving image showing a person walking in the front-back direction of the image and a moving image showing a person walking in the left-right direction of the image.
  • the measuring unit 344 performs measurement based on a moving image showing a person walking in the front-back direction of the image, and also performs measurement based on a moving image showing a person walking in the left-right direction of the image. Can be done.
  • the measuring unit 344 can measure various walking postures, which is difficult from one moving image, based on the image data acquired by using the camera or the like of the smartphone 200 without using the depth sensor or the like.
  • the walking posture measuring device 300 has an actually measured value calculation unit 343. According to such a configuration, the measured value calculation unit 343 can calculate the measured value. As a result, it becomes possible to calculate the stride length and the like based on the moving image taken by the camera of the smartphone 200. As a result, the walking posture measuring device 300 can accurately measure the walking posture based on the moving image taken by the camera of the smartphone 200.
  • the smartphone 200 has a shooting assistance unit 212.
  • the smartphone 200 includes a video showing a person walking in the front-back direction of the image and a video showing a person walking in the left-right direction of the image with the assistance of the shooting assistance unit 212. Can be obtained.
  • the smartphone 200 can acquire the above two types of moving images while keeping the shooting conditions as close as possible. Thereby, when the measurement using two kinds of moving images is performed, the accuracy of the measurement by the measuring unit 344 can be improved.
  • the photographing auxiliary unit 212 has an angle adjustment information output unit 2124.
  • the angle adjustment information output unit 2124 can output information such as a sound for adjusting the inclination of the smartphone 200.
  • the angle of the smartphone 200 can be easily adjusted even when it is difficult to see the screen of the smartphone 200.
  • the photographing device included in the walking posture measuring system 100 is not limited to the smartphone 200.
  • the function as the walking posture measuring device 300 is realized by one information processing device.
  • the function of the walking posture measuring device 300 may be realized by, for example, a plurality of information processing devices connected via a network.
  • the function of the walking posture measuring device 300 is not limited to the case where it is realized by one information processing device, and may be realized, for example, on the cloud.
  • the photographing assisting unit 212 may have all of the plurality of functions illustrated in FIG. 5, or may have some (at least one) of the functions illustrated in FIG.
  • the photographing assisting unit 212 may have only the function as the angle adjustment information output unit 2124 without displaying the angle information 2012 by the angle information display unit 2122.
  • the auxiliary function by the photographing assisting unit 212 described in the present embodiment may be applied to a system other than the walking posture measurement system 100.
  • the auxiliary function by the shooting auxiliary unit 212 may be applied to various scenes in which it is necessary to arrange the shooting conditions as much as possible when acquiring the image data.
  • the calculation process of the actual measurement value W by the actual measurement value calculation unit 343 may be applied to a system other than the walking posture measurement system 100.
  • the calculation process of the measured value W by the measured value calculation unit 343 can be applied to various situations in which the measured value is calculated based on the image data.
  • FIG. 23 is a diagram for explaining a tracking example of the walking posture measuring device 300.
  • FIG. 24 is a block diagram showing a configuration example of the walking posture measuring device 300 according to the second embodiment.
  • FIG. 25 is a diagram showing a configuration example of the tracking unit 346.
  • 26 and 27 are diagrams showing an example of a figure generated by the inclusion figure generation unit 3461.
  • FIG. 28 is a flowchart showing an operation example of the tracking unit 346.
  • the tracking unit 346 is provided.
  • the tracking unit 346 is configured to perform tracking based on the result of recognition by the skeletal recognition unit 342.
  • FIG. 24 shows a configuration example of the walking posture measuring device 300 according to the second embodiment.
  • the walking posture measuring device 300 has a tracking unit 346 in addition to the configuration described in the first embodiment.
  • a configuration characteristic of the present embodiment will be described.
  • the tracking unit 346 tracks the person based on the recognition result by the skeleton recognition unit 342. For example, the tracking unit 346 tracks a person by assigning a recognition number to the recognized person. That is, the tracking unit 346 tracks the person by assigning the same recognition number to the person determined to be the same between the image data one frame before and the image data of the current frame.
  • FIG. 25 shows a more detailed configuration example of the tracking unit 346. Referring to FIG. 25, the tracking unit 346 includes, for example, an inclusion figure generation unit 3461, an average skeleton coordinate calculation unit 3462, and a comparison tracking unit 3464.
  • the inclusion figure generation unit 3461 generates a figure including the coordinates of all the parts recognized by the skeleton recognition unit 342 (coordinates included in the skeleton information 334) for each person. For example, the inclusion figure generation unit 3461 generates one of the smallest, convex hull, rectangle, and circle that includes all the coordinates. In addition, the inclusion figure generation unit 3461 calculates the area of the generated figure.
  • the inclusion figure generation unit 3461 performs the generation of the inclusion figure and the calculation process of the area for the image data of each frame.
  • the inclusion figure generation unit 3461 performs the generation of the inclusion figure and the calculation process of the area for each of the plurality of persons included in the image data.
  • FIG. 26 shows an example of a figure including the coordinates of a person walking in the front-back direction of the screen.
  • the inclusion figure generation unit 3461 can generate any of the smallest convex hulls, rectangles, and circles that include all the coordinates.
  • FIG. 27 shows an example of a figure including the coordinates of a person walking in the left-right direction of the screen. In the case shown in FIG. 27 as in the case shown in FIG. 26, the inclusion figure generation unit 3461 can generate any one of the smallest convex hull, rectangle, and circle that includes all the coordinates.
  • the inclusion figure generation unit 3461 is predetermined, for example.
  • the inclusion figure generation unit 3461 is defined to generate a circle as a figure containing coordinates.
  • the average skeleton coordinate calculation unit 3462 calculates the average value of the coordinates (coordinates included in the skeleton information 334) of all the parts recognized by the skeleton recognition unit 342 for each person. As a result, the average skeleton coordinate calculation unit 3462 calculates the average skeleton coordinates based on the coordinates of each part of the person recognized by the skeleton recognition unit 342.
  • the average skeleton coordinate calculation unit 3462 performs the calculation process of the average skeleton coordinates for the image data of each frame.
  • the average skeleton coordinate calculation unit 3462 performs the calculation process of the average skeleton coordinates for each of the plurality of persons included in the image data.
  • the comparison tracking unit 3464 tracks a person based on the area of the included figure calculated by the inclusion figure generation unit 3461 and the average skeleton coordinates calculated by the average skeleton coordinate calculation unit 3462. For example, the comparison tracking unit 3464 compares the area corresponding to the person to be tracked calculated in the current frame and the area calculated one frame before. Further, the comparison tracking unit 3464 performs a comparison between the average skeleton coordinates corresponding to the person to be tracked and the average skeleton coordinates calculated one frame before, according to the result of the comparison.
  • the comparison tracking unit 3464 determines that the person one frame before and the person in the current frame whose area difference belongs to the first allowable value are the same person. As a result, the comparison tracking unit 3464 sets, for example, the recognition number corresponding to the person one frame before determined to be the same as the recognition number corresponding to the person in the current frame.
  • the comparison tracking unit 3464 has a difference between the average skeleton coordinates of the person to be tracked and the area. A comparison is made with the average skeletal coordinates of the person one frame before, which was within the first permissible value. Then, the comparison tracking unit 3464 determines that the person one frame before and the person in the current frame whose difference in the average skeleton coordinates belongs to the second allowable value are the same person. As a result, the comparison tracking unit 3464 sets, for example, the recognition number corresponding to the person one frame before determined to be the same person as the recognition number corresponding to the person in the current frame.
  • the comparison tracking unit 3464 determines that the person whose average skeleton coordinate difference is closest to the second allowable value is the same person. Can be done.
  • the comparison tracking unit 3464 may be configured to perform a process other than the above example, such as determining an error.
  • the comparison tracking unit 3464 determines that the person to be tracked is a newly recognized person. In this case, the comparison tracking unit 3464 assigns a new recognition number to the newly recognized person.
  • the comparison tracking unit 3464 can use, as the first allowable value, an estimated area value estimated based on the rate of increase / decrease in the area corresponding to the person determined to be the same person in the past several frames. ..
  • the first permissible value may be a predetermined value.
  • the comparison tracking unit 3464 can use, as the second allowable value, an estimated coordinate value estimated from the moving speed of the average skeleton coordinates corresponding to the person determined to be the same person in the past several frames.
  • the second permissible value may be a predetermined value like the first permissible value.
  • the above is a configuration example of the tracking unit 346. Subsequently, an operation example of the tracking unit 346 will be described with reference to FIG. 28.
  • the inclusion figure generation unit 3461 generates a figure including the coordinates of all the parts recognized by the skeleton recognition unit 342 (coordinates included in the skeleton information 334) for each person. In addition, the inclusion figure generation unit 3461 calculates the area of the generated figure (step S401).
  • the average skeleton coordinate calculation unit 3462 calculates the average skeleton coordinates by calculating the average value of the coordinates (coordinates included in the skeleton information 334) of all the parts recognized by the skeleton recognition unit 342 for each person (step S402). ).
  • the comparison tracking unit 3464 compares the area corresponding to the person to be tracked calculated in the current frame and the area calculated one frame before (step S403).
  • the comparative tracking unit. 3463 determines that the person one frame before and the person in the current frame whose area difference belongs to the first permissible value are the same person (step S404). As a result, the comparison tracking unit 3464 sets the recognition number corresponding to the person one frame before as the recognition number of the current frame, for example.
  • the comparative tracking unit 3464 has the average skeletal coordinates of the person to be tracked. And the average skeleton coordinates of the person one frame before the difference in area was within the first permissible value (step S405). Then, when there is a person whose difference from the average skeleton coordinates of the tracking target is within the second allowable value one frame before (step S405, Yes), the comparative tracking unit 3464 is the person one frame before and the tracking target. It is determined that the person is the same person (step S404).
  • the comparison tracking unit 3464 uses, for example, the same person whose difference is closest to the second allowable value. It can be judged that it is a person. Further, when there is no person whose difference from the average skeleton coordinates of the tracking target is within the second allowable value one frame before (step S405, No), the comparative tracking unit 3464 newly recognizes the person to be tracked. It is determined that the person is a person (step S406). In this case, the comparison tracking unit 3464 assigns a new recognition number to the newly recognized person.
  • the comparative tracking unit 3464 newly sets the person to be tracked. (Step S406). In this case, the comparison tracking unit 3464 assigns a new recognition number to the newly recognized person.
  • the above is an operation example of the tracking unit 346.
  • the walking posture measuring device 300 in the present embodiment has a tracking unit 346.
  • the tracking unit 346 can track the same person based on the recognition result by the skeleton recognition unit 342. As a result, it is possible to suppress the erroneous calculation of the walking speed, and it is possible to improve the accuracy of the walking posture measurement.
  • the inclusion figure generation unit 3461 may be configured to calculate a value other than the area based on the generated inclusion figure.
  • the inclusion figure generation unit 3461 may calculate the height, diameter, and the like of the generated inclusion figure instead of the area.
  • the comparison tracking unit 3464 will compare values such as the height calculated based on the figure generated by the inclusion figure generation unit 3461 instead of the area.
  • the inclusion figure generation unit 3461 is supposed to generate a figure included in all coordinates.
  • the included figure generation unit 3461 may be configured to generate a figure including a part of the coordinates recognized by the skeleton recognition unit 342, for example, generating a figure included in the coordinates corresponding to the upper body of the person. I do not care.
  • the average skeleton coordinate calculation unit 3462 may also calculate the average coordinates based on a part of the coordinates recognized by the skeleton recognition unit 342, such as the average coordinates of the coordinates corresponding to the upper body of the person.
  • comparison tracking unit 3464 may be configured to perform only one of the comparison of the values based on the inclusion figure generated by the inclusion figure generation unit 3461 and the comparison of the average skeleton coordinates.
  • the tracking using the recognition result by the skeleton recognition unit 342 described in the present embodiment may be applied to the case of tracking a person other than the scene of measuring the walking posture. That is, the function as the tracking unit 346 may be applied to a device other than the walking posture measuring device 300 that requires tracking a person.
  • the method of tracking a person using skeletal information described in the present embodiment is not limited to the case of measuring the walking posture, and can be used in various situations.
  • FIGS. 29 and 30 show a configuration example of the photographing apparatus 400.
  • the photographing device 400 photographs a person walking.
  • FIG. 29 shows an example of the hardware configuration of the photographing device 400.
  • the photographing apparatus 400 has the following hardware configuration as an example in addition to a camera for performing photographing.
  • -CPU Central Processing Unit
  • -ROM Read Only Memory
  • RAM Random Access Memory
  • storage device storage device
  • -Program group 404 loaded into RAM 403
  • a storage device 405 that stores a program group 404.
  • -Drive device 406 that reads and writes the recording medium 410 external to the information processing device.
  • -Communication interface 407 that connects to the communication network 411 outside the information processing device -I / O interface 408 that inputs and outputs data -Bus 409 connecting each component
  • the photographing device 400 can realize the functions as the detection unit 421 and the display unit 422 shown in FIG. 30 by the CPU 401 acquiring the program group 404 and executing the program group 404.
  • the program group 404 is stored in, for example, a storage device 405 or a ROM 402 in advance, and the CPU 401 loads the program group 404 into a RAM 403 or the like and executes the program group 404 as needed. Further, the program group 404 may be supplied to the CPU 401 via the communication network 411, or may be stored in the recording medium 410 in advance, and the drive device 406 may read the program and supply the program to the CPU 401.
  • FIG. 29 shows an example of the hardware configuration of the photographing device 400.
  • the hardware configuration of the photographing device 400 is not limited to the above-mentioned case.
  • the photographing device 400 may be composed of a part of the above-described configuration, such as not having the drive device 406.
  • the detection unit 421 detects the orientation of the photographing device 400. For example, the detection unit 421 detects whether the photographing device 400 is in the vertical orientation or in the horizontal orientation.
  • the display unit 422 displays a guide line indicating the position where the person walks on the screen display unit. For example, the display unit 422 displays different guide lines according to the orientation of the photographing device 400 detected by the detection unit 421.
  • the photographing device 400 has a detection unit 421 and a display unit 422.
  • the display unit 422 can display different guide lines according to the orientation of the photographing device 400 detected by the detection unit 421.
  • the display unit 422 can display different guide lines according to the orientation of the photographing device 400 detected by the detection unit 421.
  • the above-mentioned photographing device 400 can be realized by incorporating a predetermined program into the photographing device 400.
  • the photographing device 400 has a detection unit 421 that detects the direction of the photographing device and a display unit that displays a guide line indicating a position where a person walks on the screen display unit.
  • the display unit 422 is a program that realizes 422 and displays the guide lines that differ depending on the orientation of the photographing device detected by the detection unit 421.
  • the photographing device 400 that photographs the state of walking of a person detects the direction of the photographing device and displays a guide line indicating the position where the person walks on the screen display unit.
  • a different guide line is displayed according to the detected orientation of the photographing device.
  • FIG. 31 shows A configuration example of the information processing device 500 is shown.
  • the information processing device 500 has, for example, a configuration similar to the hardware configuration of the photographing device 400 described with reference to FIG. 29. Further, the information processing device 500 can realize the function as the calculation unit 521 shown in FIG. 31 by the CPU acquiring the program group included in the information processing device 500 and executing the program group by the CPU.
  • the calculation unit 521 calculates the actual length at a predetermined position in the image data based on the parameters of the photographing device for acquiring the image data and the information indicating the height of the photographing device when acquiring the image data. do.
  • the information processing device 500 has a calculation unit 521.
  • the calculation unit 521 can calculate the actual length at a predetermined position in the image data based on various information. As a result, it becomes possible to perform analysis using the actual length based on the image data acquired by a photographing device such as a smartphone.
  • the information processing device 500 described above can be realized by incorporating a predetermined program into the information processing device 500.
  • the program according to another embodiment of the present invention provides the information processing device 500 with information indicating the parameters of the photographing device for acquiring image data and the height of the photographing device when acquiring the image data. Based on this, it is a program for realizing the calculation unit 521 that calculates the actual length at a predetermined position in the image data.
  • the information processing device 500 obtains the parameters of the photographing device for acquiring the image data and the information indicating the height of the photographing device when acquiring the image data. It is a method of acquiring and calculating the actual length at a predetermined position in the image data based on the acquired information.
  • FIG. 32 shows a configuration example of the information processing device 600.
  • the information processing device 600 has, for example, a configuration similar to the hardware configuration of the photographing device 400 described with reference to FIG. 29. Further, the information processing device 600 can realize the functions as the detection unit 621 and the output unit 622 shown in FIG. 32 by the CPU acquiring the program group possessed by the information processing device 600 and executing the program group by the CPU. You can.
  • the detection unit 621 detects the tilt of the information processing device.
  • the output unit 622 outputs information according to the inclination of the information processing device detected by the detection unit 621, which differs depending on the inclination of the information processing device.
  • the information processing device 600 has a detection unit 621 and an output unit 622. According to such a configuration, the output unit 622 can output information according to the inclination of the information processing device detected by the detection unit 621. As a result, the operator who operates the information processing apparatus 600 can correct the inclination according to the output information.
  • the information processing device 600 described above can be realized by incorporating a predetermined program into the information processing device 600.
  • the information processing device 600 has a detection unit 621 that detects the tilt of the information processing device 600 and a detection unit 621 that differs depending on how the information processing device 600 is tilted.
  • This is a program for realizing an output unit 622 that outputs information according to the inclination of the information processing apparatus 600 detected by the information processing apparatus 600.
  • the information processing device 600 detects the tilt of the information processing device 600, and the detected tilt of the information processing device differs depending on how the information processing device is tilted. It is a method of outputting information according to.
  • FIG. 33 shows a configuration example of the tracking device 700.
  • the tracking device 700 has, for example, a configuration similar to the hardware configuration of the photographing device 400 described with reference to FIG. 29. Further, the tracking device 700 can realize the functions as the acquisition unit 721 and the tracking unit 722 shown in FIG. 33 by the CPU acquiring the program group included in the tracking device 700 and executing the program group.
  • the acquisition unit 721 acquires information indicating a plurality of parts of the recognized person by recognizing the skeleton of the person in the image data.
  • the tracking unit 722 tracks the same person among a plurality of image data based on the information acquired by the acquisition unit 721.
  • the tracking device 700 has an acquisition unit 721 and a tracking unit 722. With such a configuration, the tracking unit 722 can perform tracking based on the information indicating the portion acquired by the acquisition unit 721. This makes it possible to realize easy tracking.
  • the tracking device 700 described above can be realized by incorporating a predetermined program into the tracking device 700.
  • the tracking device 700 has an acquisition unit 721 that acquires information indicating a plurality of recognized parts of the person by recognizing the skeleton of the person in the image data.
  • This is a program for realizing a tracking unit 722 that tracks the same person among a plurality of image data based on the information acquired by the acquisition unit 721.
  • the tracking device 700 acquires information indicating a plurality of parts of the recognized person by recognizing the skeleton of the person in the image data, and the acquired information is used. Based on this, it is a method of tracking the same person among a plurality of image data.
  • Appendix 1 A shooting device that shoots a person walking Detects the orientation of the shooting device and A guide line indicating the position where the person walks is displayed on the screen display.
  • a guide method for displaying the guide lines that differ depending on the direction of the detected photographing device when the guide lines are displayed on the screen display unit.
  • Appendix 2 The guide method described in Appendix 1 A guide method for displaying different guide lines depending on whether the photographing device is in portrait orientation or landscape orientation.
  • Appendix 3) The guide method described in Appendix 1 or Appendix 2, A guide method for displaying the guide line for guiding a person walking in the front-back direction of the screen when the photographing device is oriented vertically.
  • Appendix 7 The guide method described in Appendix 6 A guide method that detects the tilt in the left-right direction and the tilt in the front-back direction, and outputs different information when the tilt in the left-right direction is detected and when the tilt in the front-back direction is detected.
  • Appendix 8 The guide method according to Appendix 6 or Appendix 7. A guide method that outputs the sound adjusted by different methods when the tilt in the left-right direction is detected and when the tilt in the front-back direction is detected.
  • Appendix 9 It is a shooting device that shoots a person walking.
  • the display unit is an imaging device that displays different guide lines according to the orientation of the imaging device detected by the detection unit. (Appendix 10) For a shooting device that shoots a person walking A detector that detects the orientation of the imaging device and A display unit that displays a guide line indicating the position where a person walks on the screen display unit, Realized,
  • the display unit is a program that displays the guide lines that differ depending on the orientation of the photographing device detected by the detection unit.
  • (Appendix 11) Information processing device The parameters of the photographing device for acquiring the image data and the information indicating the height of the photographing device when acquiring the image data are acquired, and based on the acquired information, the actual length at a predetermined position in the image data is obtained. Calculation method to calculate.
  • Information processing device (Appendix 20) For information processing equipment Realized a calculation unit that calculates the actual length at a predetermined position in the image data based on the parameters of the imaging device that acquires the image data and the information that indicates the height of the imaging device when acquiring the image data. Program to do. (Appendix 21) Information processing device Detects the tilt of the information processing device and An output method that outputs information according to the tilt of the information processing device, which differs depending on the tilt of the information processing device.
  • Appendix 22 The output method described in Appendix 21. Detects the tilt of the information processing device in the left-right direction and the tilt in the front-back direction, An output method that outputs different information when the tilt in the left-right direction is detected and when the tilt in the front-back direction is detected.
  • Appendix 23 The output method according to Appendix 21 or Appendix 22. An output method that outputs sound adjusted by different methods when the tilt in the left-right direction is detected and when the tilt in the front-back direction is detected.
  • Appendix 24 The output method described in Appendix 23.
  • Appendix 27 A detector that detects the tilt of the information processing device and An output unit that outputs information according to the inclination of the information processing device detected by the detection unit, which differs depending on the inclination of the information processing device.
  • Information processing device with. (Appendix 28) The information processing device according to Appendix 27.
  • the detection unit detects the tilt of the information processing device in the left-right direction and the tilt in the front-back direction.
  • the output unit is an information processing device that outputs different information depending on whether the detection unit detects a tilt in the left-right direction or a tilt in the front-back direction.
  • Appendix 29 For information processing equipment A detector that detects the tilt of the information processing device and An output unit that outputs information according to the inclination of the information processing device detected by the detection unit, which differs depending on the inclination of the information processing device. A program to realize.
  • Appendix 30 The program described in Appendix 29.
  • the detection unit detects the tilt of the information processing device in the left-right direction and the tilt in the front-back direction.
  • the output unit is a program that outputs different information depending on whether the detection unit detects a tilt in the left-right direction or a tilt in the front-back direction.
  • Information processing device By recognizing the skeleton of a person in the image data, information indicating multiple parts of the recognized person is acquired, and information is obtained.
  • a tracking method that tracks the same person among multiple image data based on the acquired information (Appendix 32) The tracking method described in Appendix 31. Generate an inclusion figure that includes at least a part of the recognized part, A tracking method for tracking the same person based on a value corresponding to the generated inclusion figure. (Appendix 33) The tracking method described in Appendix 32. A tracking method for tracking the same person based on the difference in values between image data according to the included figure. (Appendix 34) The tracking method described in Appendix 33. The number at which the difference between the value corresponding to the included figure of the tracking target and the value corresponding to the included figure corresponding to the person included in the image data different from the image data to which the tracking target belongs is within a predetermined value.
  • a tracking method that determines that a person whose difference is within a predetermined value is the same person as the person to be tracked.
  • the predetermined value is a tracking method determined according to the degree of change in the value according to the included figure among a plurality of image data.
  • Appendix 36 The tracking method according to any one of Appendix 31 to Appendix 35. Calculate the average value of the coordinates of at least a part of the coordinates of the recognized part, A tracking method that tracks the same person based on the calculated results.
  • Appendix 37 The tracking method described in Appendix 36. A tracking method for tracking the same person based on the difference in the average value between image data.
  • Appendix 38 The tracking method according to Appendix 36 or Appendix 37.
  • Appendix 39 An acquisition unit that acquires information indicating multiple parts of the recognized person by recognizing the skeleton of the person in the image data, and an acquisition unit.
  • a tracking unit that tracks the same person among a plurality of image data based on the information acquired by the acquisition unit.
  • Appendix 40 In the tracking device, An acquisition unit that acquires information indicating multiple parts of the recognized person by recognizing the skeleton of the person in the image data, and an acquisition unit.
  • the programs described in each of the above embodiments and appendices may be stored in a storage device or recorded in a computer-readable recording medium.
  • the recording medium is a portable medium such as a flexible disk, an optical disk, a magneto-optical disk, and a semiconductor memory.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
  • Image Analysis (AREA)
PCT/JP2021/008569 2020-03-25 2021-03-04 追跡方法 Ceased WO2021192908A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2022509484A JP7343237B2 (ja) 2020-03-25 2021-03-04 追跡方法
CN202180019431.0A CN115280362A (zh) 2020-03-25 2021-03-04 追踪方法

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2020-053977 2020-03-25
JP2020053977 2020-03-25

Publications (1)

Publication Number Publication Date
WO2021192908A1 true WO2021192908A1 (ja) 2021-09-30

Family

ID=77891280

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/008569 Ceased WO2021192908A1 (ja) 2020-03-25 2021-03-04 追跡方法

Country Status (3)

Country Link
JP (1) JP7343237B2 (https=)
CN (1) CN115280362A (https=)
WO (1) WO2021192908A1 (https=)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7462178B1 (ja) 2022-09-21 2024-04-05 株式会社バイオネット研究所 動体自動追跡装置、動体自動追跡方法、及び動体自動追跡プログラム

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2015061577A (ja) * 2013-01-18 2015-04-02 株式会社東芝 動作情報処理装置
JP2018057596A (ja) * 2016-10-05 2018-04-12 コニカミノルタ株式会社 関節位置推定装置および関節位置推定プログラム
JP2019012453A (ja) * 2017-06-30 2019-01-24 キヤノンマーケティングジャパン株式会社 情報処理装置、情報処理装置の制御方法、プログラム
WO2019064947A1 (ja) * 2017-09-28 2019-04-04 京セラドキュメントソリューションズ株式会社 監視端末装置及び表示処理方法

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019032307A1 (en) * 2017-08-07 2019-02-14 Standard Cognition, Corp. PREDICTING INVENTORY EVENTS THROUGH PREVIOUS / BACKGROUND TREATMENT
CN109784130B (zh) * 2017-11-15 2023-04-28 株式会社日立制作所 行人重识别方法及其装置和设备
JP7039409B2 (ja) * 2018-07-18 2022-03-22 株式会社日立製作所 映像解析装置、人物検索システムおよび人物検索方法
CN110472613B (zh) * 2019-08-22 2022-05-10 海信集团有限公司 对象行为识别方法和装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2015061577A (ja) * 2013-01-18 2015-04-02 株式会社東芝 動作情報処理装置
JP2018057596A (ja) * 2016-10-05 2018-04-12 コニカミノルタ株式会社 関節位置推定装置および関節位置推定プログラム
JP2019012453A (ja) * 2017-06-30 2019-01-24 キヤノンマーケティングジャパン株式会社 情報処理装置、情報処理装置の制御方法、プログラム
WO2019064947A1 (ja) * 2017-09-28 2019-04-04 京セラドキュメントソリューションズ株式会社 監視端末装置及び表示処理方法

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7462178B1 (ja) 2022-09-21 2024-04-05 株式会社バイオネット研究所 動体自動追跡装置、動体自動追跡方法、及び動体自動追跡プログラム

Also Published As

Publication number Publication date
JPWO2021192908A1 (https=) 2021-09-30
CN115280362A (zh) 2022-11-01
JP7343237B2 (ja) 2023-09-12

Similar Documents

Publication Publication Date Title
US10650533B2 (en) Apparatus and method for estimating eye gaze location
US10297076B2 (en) Apparatus and method for generating 3D face model using mobile device
CN101601276B (zh) 抖动测定系统及抖动测定方法
JP5659305B2 (ja) 画像生成装置および画像生成方法
JP5769813B2 (ja) 画像生成装置および画像生成方法
JP5865388B2 (ja) 画像生成装置および画像生成方法
US20190110039A1 (en) Head-mounted display tracking system
WO2021192906A1 (ja) 算出方法
JP6981531B2 (ja) 物体同定装置、物体同定システム、物体同定方法およびコンピュータプログラム
WO2014103094A1 (ja) 情報処理装置、情報処理システム、および情報処理方法
KR101480410B1 (ko) 디지털 영상 처리기에서 스포츠 자세 교정 장치 및 방법
CN106713740B (zh) 定位跟踪摄像方法与系统
US10671173B2 (en) Gesture position correctiing method and augmented reality display device
US20130069939A1 (en) Character image processing apparatus and method for footskate cleanup in real time animation
JP6879375B2 (ja) 情報処理装置、長さ測定システム、長さ測定方法およびコンピュータプログラム
JP2016099759A (ja) 顔検出方法、顔検出装置、及び顔検出プログラム
JP7356927B2 (ja) 肌解析方法、肌解析システム及び肌解析プログラム
US9492748B2 (en) Video game apparatus, video game controlling program, and video game controlling method
WO2021192908A1 (ja) 追跡方法
JP2011118767A (ja) 表情モニタリング方法および表情モニタリング装置
KR101837142B1 (ko) 사용자와의 인터렉션을 이용한 트레드밀 콘텐츠 제공 장치 및 그 방법
JP7323234B2 (ja) ガイド方法
WO2021192907A1 (ja) 出力方法
US20200226787A1 (en) Information processing apparatus, information processing method, and program
US12350568B2 (en) Golf swing analysis system, golf swing analysis method, and information storage medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21775004

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022509484

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21775004

Country of ref document: EP

Kind code of ref document: A1