KR20220158322A - Apparatus and method for tracking patients with dementia - Google Patents

Abstract

Provided is a tracking device. The tracking device includes: an acquisition unit for acquiring a video of a pedestrian; and a tracking unit for determining whether the pedestrian is a dementia patient or tracks a specific pedestrian through analysis of the video. The tracking device of the present invention can determine whether a pedestrian has dementia through analysis of a video taken by a camera.

Description

Apparatus and method for tracking patients with dementia}

The present invention relates to devices and methods for classifying and tracking patients with dementia.

Dementia is a condition in which memory and behavioral functions are impaired due to various causes and it is difficult to maintain daily life. In particular, the 'wandering' symptom of dementia patients is one of the biggest concerns in nursing. Dementia patients not only wander around the house, but sometimes go out of the house and get lost. If dementia patients get lost, they may be in danger due to traffic accidents, malnutrition, health deterioration, etc. if not detected early.

Korean Patent Registration No. 1792339 discloses a technology for tracking the location of a beacon possessed by a patient with dementia.

Korean Registered Patent Publication No. 1792339

An object of the present invention is to provide a tracking device and method capable of analyzing the presence or absence of dementia by analyzing a person's gait using an installed CCTV and at the same time tracking the location of a dementia patient by applying re-recognition technology.

The tracking device of the present invention includes an acquisition unit for obtaining a video of a pedestrian; It may include; a tracking unit that determines whether the pedestrian is a dementia patient or tracks a specific pedestrian through analysis of the video.

The tracking method of the present invention includes the steps of obtaining a video recording a set area; extracting pedestrians included in the video; forming a skeleton of the pedestrian; measuring a step length and walking speed of the pedestrian through analysis of the skeleton; generating a spectrogram image including the stride length and the walking speed; Analyzing the spectrogram image using a discrimination model generated through a deep learning algorithm, and determining whether the pedestrian has dementia.

The tracking method of the present invention includes the steps of obtaining a video recording a set area; extracting an object image representing a pedestrian included in the video; extracting a gait characteristic including a step length and a gait speed of the pedestrian through analysis of the object image; generating a spectrogram image including the stride length and the walking speed; determining whether or not the pedestrian matches a specific pedestrian entered using the object image, the gait characteristics, and the spectrogram image; and displaying a movement path of a candidate object corresponding to the pedestrian determined to match the specific pedestrian on a map.

The tracking device of the present invention can determine whether a pedestrian has dementia through analysis of a video taken by a camera.

In particular, the tracking device of the present invention can determine whether the pedestrian is a dementia patient by using the walking characteristics of the pedestrian included in the video.

In addition, the tracking device of the present invention can provide a way to continuously track a patient with dementia using a plurality of cameras installed in different places.

The tracking device of the present invention can communicate with a plurality of cameras, and can analyze each video received from the plurality of cameras. The tracking device can classify or specify dementia patients through video analysis. In addition, when a specific pedestrian is determined to be a patient with dementia through analysis of a video captured by a specific camera, the tracking device may store the gait characteristics or spectrogram of the specific pedestrian. The tracking device may recognize a specific pedestrian determined to be a dementia patient from a video clip of another camera by comparing the gait characteristics or spectrogram of the pedestrian photographed by another camera with a previously stored one. When the recognition of a specific pedestrian by other cameras is completed, the estimating device may estimate or track the movement of a specific pedestrian by integrating location information of each camera, photographing time information, moving direction of the specific pedestrian in the video, and the like.

According to the tracking device and tracking method of the present invention, the location of a patient with dementia can be automatically estimated while an early detection of a patient with dementia is possible. Through this, the safety of dementia patients with wandering symptoms can be secured.

1 is a schematic diagram showing a tracking device of the present invention.
2 is a block diagram illustrating a tracking unit.
3 is a schematic diagram showing a first operation of the tracking unit.
4 is a schematic diagram showing a second operation of the tracking unit.
5 is a schematic diagram showing the tracking operation of the tracking unit.
6 is a flowchart illustrating the tracking method of the present invention.
7 is a flowchart illustrating another tracking method of the present invention.
8 is a flowchart illustrating operations of a detection unit, a calculation unit, and a conversion unit.
9 is a schematic diagram showing the operation of the detection unit.
10 is a flowchart illustrating the operation of the determination unit.
11 is a schematic diagram showing the operation of the determining unit.
12 is a schematic diagram showing the operation of the tracking unit.
13 is a schematic diagram showing another operation of the tracking unit.
14 is a diagram illustrating a computing device according to an embodiment of the present invention.

Hereinafter, with reference to the accompanying drawings, embodiments of the present invention will be described in detail so that those skilled in the art can easily carry out the present invention. However, the present invention may be embodied in many different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

In this specification, redundant descriptions of the same components are omitted.

In addition, in this specification, when a component is referred to as being 'connected' or 'connected' to another component, it may be directly connected or connected to the other component, but another component in the middle It should be understood that may exist. On the other hand, in this specification, when a component is referred to as 'directly connected' or 'directly connected' to another component, it should be understood that no other component exists in the middle.

In addition, terms used in this specification are only used to describe specific embodiments and are not intended to limit the present invention.

Also, in this specification, a singular expression may include a plurality of expressions unless the context clearly indicates otherwise.

In addition, in this specification, terms such as 'include' or 'having' are only intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more It should be understood that the presence or addition of other features, numbers, steps, operations, components, parts, or combinations thereof is not precluded.

Also in this specification, the term 'and/or' includes a combination of a plurality of listed items or any item among a plurality of listed items. In this specification, 'A or B' may include 'A', 'B', or 'both A and B'.

Also, in this specification, detailed descriptions of well-known functions and configurations that may obscure the subject matter of the present invention will be omitted.

The tracking device of the present invention analyzes the gait of a pedestrian 90 using various cameras 10 such as a CCTV camera, a camera installed in a mobile communication terminal, and a portable camera to determine the presence or absence of dementia and to determine the presence or absence of dementia. You can track your route. Dementia disease is accompanied by gait disturbance due to neurological damage, so the presence or absence of dementia can be estimated by analyzing gait. Thereafter, the movement path of the pedestrian 90 determined to have dementia through re-recognition may be tracked.

1 is a schematic diagram showing a tracking device of the present invention.

The tracking device shown in FIG. 1 may include an acquisition unit 110 and a tracking unit 130 .

A camera 10 may be provided that captures a pedestrian 90 walking along a walking path 80 or the like and generates a video 20 .

The acquisition unit 110 may communicate with a plurality of cameras 10 disposed at different locations. The acquisition unit 110 may obtain the video 20 of the pedestrian 90 through communication with the camera 10 . For example, the video 20 obtained from the camera 10 may be captured at several tens of frames per second.

Each camera 10 may capture a setting location including the walking path 80 where the pedestrian 90 travels. For example, the camera 10 may include a closed circuit TV (CCTV) for capturing a set area and a set position, a camera of a mobile communication terminal, and a portable camera.

A communication module capable of wired/wireless communication with each camera 10 may be provided in the acquisition unit 110 . The plurality of cameras 10 may communicate with a server provided in the central control room. At this time, the acquisition unit 110 may be integrally formed in the corresponding server, or may receive the video 20 captured by each camera 10 from the corresponding server.

The tracking unit 130 may determine whether the pedestrian 90 included in the video 20 is a dementia patient or track a specific pedestrian presumed to be a dementia patient through analysis of the video 20 .

The tracking unit 130 may analyze the gait of the pedestrian 90 included in the video 20 and determine whether the pedestrian 90 has dementia using the analysis result of the gait.

For example, the tracking unit 130 may analyze at least one of the gait characteristics of the pedestrian 90, for example, a step length, a gait speed, a change in step length, and a change in gait speed through the analysis of the video 20. The tracking unit 130 may determine whether the pedestrian 90 has dementia by using at least one of step length, walking speed, change in step length, and change in walking speed. In general, the gait of dementia patients is different from that of normal people. As a result, the gait characteristics of a patient with dementia and the gait characteristics of a normal person are also different, and whether or not the pedestrian 90 has dementia can be estimated using this point.

For example, the tracking unit 130 may determine the pedestrian 90 as a patient with dementia when the amount of change in step length or change in walking speed satisfies a set value.

The tracking unit 130 may determine whether the pedestrian 90 has dementia through machine learning.

For example, the tracking unit 130 may be provided with an extraction model 31, a classification model 32, and a discrimination model 33 generated through machine learning.

When the video 20 is input, the extraction model 31 may extract the pedestrian 90 included in the video 20 .

The classification model 32 may analyze key points of the joints of the pedestrian 90 extracted through the extraction model 31 and form skeleton S of the pedestrian 90 using the key points.

The discrimination model 33 may determine whether the pedestrian 90 has dementia using the walking characteristics including the step length and walking speed of the pedestrian 90 calculated through the analysis of the skeleton S.

The tracking unit 130 includes first location information including a location where the camera 10 is installed, second location information indicating a location captured by the camera 10, and movement of the pedestrian 90 in the video 20. A patient with dementia may be tracked using at least one of the direction information.

Tracking of the dementia patient may include determining at least one of whether the pedestrian 90 has dementia, the location of the dementia patient, and the movement direction of the dementia patient. Alternatively, the tracking of the patient with dementia may include displaying at least one of the identified dementia or not information, location information, and movement direction information on the manager's terminal. Alternatively, the tracking of the dementia patient may include displaying at least one of the identified location information and movement direction information on a map.

2 is a block diagram illustrating the tracking unit 130. 3 is a schematic diagram showing a first operation of the tracking unit 130 . 4 is a schematic diagram showing a second operation of the tracking unit 130.

The tracking unit 130 may include a detection unit 131 , a calculation unit 133 , a conversion unit 135 , a determination unit 137 , and a tracking unit 139 .

In order to determine dementia based on the gait, it is necessary to extract gait characteristics that are characteristic of the gait from the pedestrian 90 . Changes in gait such as stride length and gait speed appear according to aging and disease. In order to obtain these gait characteristics, it is necessary to detect and track the foot position of the pedestrian 90 .

In order to obtain the stride characteristics, both feet must first be detected and detected, and the detection unit 131 uses the video 20 corresponding to the input image as shown in FIG. 3 or the object extraction model 31 in each frame constituting the video 20 It is possible to detect or detect the pedestrian 90 and extract the pedestrian 90 using .

The detection unit 131 extracts only the area of the pedestrian 90 from the video 20 and the like, and then forms skeleton s using the classification model 32 capable of setting previously learned joint key points (tracking unit 130). Corresponds to the first operation of).

The detection unit 131 may measure the distance between the feet of the pedestrian 90 using the joint key points of the pedestrian 90 included in the video 20 . For example, the detection unit 131 may measure the distance (interval) between points 11 and 14 representing each foot in the skeleton s of FIG. 3 .

The calculation unit 133 may calculate the stride length and walking speed of the pedestrian 90 using the distance between both feet.

The detection unit 131 may initially measure the distance between the feet once when the pedestrian 90 is standing, and then measure the distance between the feet only when the pedestrian 90 moves. The distance between the feet measured by the detection unit 131 may form signals representing the stride length and walking speed. And, the detector 131 measures the degree of movement of the arm a and the leg b at points (2, 3, 4), points (5, 6, 7), points (9, 10, 11), points (12, 13, 14) can measure the initial position value.

The calculation unit 133 may also measure or calculate the amount of change of each point indicating the arm a and the leg b only when the step length value (distance between points 11 and 14) is measured. In other words, the calculation unit 133 may extract or calculate a joint motion including a change amount of an arm joint and a change amount of a leg joint of the pedestrian 90 when the pedestrian 90 walks.

The conversion unit 135 may generate one test signal by summing each signal representing the stride length, walking speed, and joint motion obtained after the pedestrian 90 continuously walks for a set time or more. The test signal may be converted into an image for dementia determination by the conversion unit 135 .

For example, the transform unit 135 may use a short-time Fourier transform (STFT) algorithm to convert the test signal into a processing signal representing a step length variation with time. The conversion unit 135 may convert the processed signal into a spectrogram image including the stride length and walking speed. As a result, the conversion unit 135 may generate a spectrogram image h including the stride length and walking speed (corresponding to the second motion of the tracking unit 130).

The spectrogram is a tool for visualizing and grasping the gait characteristic i1 of the pedestrian 90 as a wave, and the characteristics of a waveform and a spectrum may be combined.

In waveform i2, the change of the amplitude axis according to the change of the time axis can be seen, and in the spectrum i3, the change of the amplitude axis according to the change of the frequency axis can be seen, while in the spectrogram image h, the difference in amplitude is printed according to the change of the time axis and the frequency axis. It can be expressed as a difference in density or display color.

The determining unit 137 may determine whether the pedestrian 90 has dementia through analysis of the spectrogram image h.

For example, the determining unit 137 may calculate the probability that the pedestrian 90 has dementia through analysis of the spectrogram image h. The determination unit 137 may determine that the pedestrian 90 is a dementia patient when the probability of dementia satisfies a set value. The determining unit 137 may determine that the pedestrian 90 is a normal person if the probability of having dementia does not satisfy the set value.

Specifically, a deep learning algorithm may be loaded into the determination unit 137 to realize a gait-based dementia determination technology. The deep learning algorithm may analyze the extracted spectrogram image to calculate the probability of dementia or a normal person. If the estimated probability of dementia exceeds a certain value, it can be judged as dementia. A dementia discrimination model 33 based on a deep learning algorithm specialized for dementia patients may be loaded into the discrimination unit 137 . The discrimination model 33 may be trained to determine whether or not there is dementia by using the gait characteristics of a normal person and a patient with dementia as an input data set. For example, the discrimination model 33 may be trained to determine whether a person has dementia by using a spectrogram image visualizing gait characteristics as an input dataset. The discriminant model 33 may be created using a convolutional neural network for analysis of the spectrogram image h. The discriminant model 33 may divide the spectrogram image h into set intervals in the flow of time and analyze the gait characteristics according to the change in time. Through this, the discrimination model 33 can analyze the change in stride length and walking speed according to the change in time and determine whether or not there is dementia. As a result, the determining unit 137 may determine whether the pedestrian 90 has dementia by analyzing the amount of change in the step length and the amount of change in the walking speed of the pedestrian 90 through analysis of the gait characteristics.

5 is a schematic diagram showing the tracking operation of the tracking unit 130.

The tracking unit 130 may continuously track a specific pedestrian determined to be a dementia patient using object recognition.

Object re-recognition is a technique for uniquely distinguishing an object by comparing objects photographed from various locations. In this case, the object may include the pedestrian 90 included in the video 20 .

For object re-recognition, the current object must be re-identified by extracting and comparing identification information (feature value) that distinguishes it from other objects from the current object. For object re-recognition, it is preferable that the same identification information be obtained even if the photographing angle, position, and brightness are changed.

The tracking unit 139 provided in the tracking unit 130 may search for an object image (the pedestrian 90 included in the video 20) having a similar appearance to a specific pedestrian determined to be a dementia patient through object recognition.

The tracking unit 139 may find an object image most similar to a target object by extracting features from the object image and comparing feature values (identification information).

Since identification information is extracted from the frame image of the video 20, search may be performed focusing on objects having a similar appearance to a specific pedestrian to be found. When recognizing an object, a feature model 34 generated through a convolutional neural network specialized in extracting object identification information may be loaded into the tracking unit 139 . A deep learning algorithm model may be configured to detect from a large part to a small part when extracting features. Therefore, the recognition performance may be improved without biasing the extraction ratio in the outer appearance of a coat or pants having a high specific gravity.

In the case of object re-recognition described above, there are many problems because re-recognition is performed by analyzing only the object image. It is difficult to extract a uniform feature value because a change occurs in an object image due to the influence of a photographing environment such as an angle, resolution, and brightness of a photographing device such as the camera 10 . Therefore, in order to improve re-recognition accuracy, it is preferable to use data less affected by the photographing environment. In order to solve this problem, the tracking unit 139 may perform re-recognition as an element having a minimal influence of the photographing environment. For example, the tracking unit 139 may perform re-recognition using a stride length characteristic and a spectrogram image. The stride length characteristic and the spectrogram image are data that are less affected by the shooting environment. These data may be already obtained by the detection unit 131 , the calculation unit 133 , and the conversion unit 135 .

According to the tracking unit 139 that performs re-recognition using the stride length characteristics and the spectrogram image, a specific pedestrian can be accurately re-recognized even if the observation position of the photographing device changes.

The tracking unit 139 may perform re-recognition with an object image and one or more elements. Using many elements improves recognition accuracy but slows down analysis. Accordingly, the tracking unit 139 may perform re-recognition using only some main elements. At this time, a deep learning algorithm configured to receive and process two or more pieces of data for re-recognition may be used. As shown in FIG. 5, a convolutional neural network model that receives a plurality of data and extracts one feature value may be used. The tracking unit 139 may obtain feature values of the pedestrian 90 clearly distinguished from other objects by using the object image and element information. The tracking unit 139 may perform accurate re-recognition by additionally analyzing an element having a minimal influence of the photographing environment, for example, a gait characteristic or a spectrogram image.

In summary, the tracking unit 139 determines whether the pedestrian 90 matches a specific pedestrian through analysis of the walking characteristics of the pedestrian 90 including the step length and walking speed of the pedestrian 90 and the spectrogram image h. can In this case, the specific pedestrian may indicate a pedestrian 90 determined or determined or estimated as a patient with dementia.

When the pedestrian 90 included in the video 20 is determined to be a specific pedestrian, the tracking unit 139 may track the movement of the pedestrian 90 determined to be a specific pedestrian. Tracking of the movement line may mean displaying the hourly location of the pedestrian 90 on a map or displaying the moving direction of the pedestrian 90 on a map.

6 is a flowchart illustrating the tracking method of the present invention.

The tracking method of FIG. 6 may be performed by the tracking device of FIG. 1 . The tracking method of FIG. 6 may represent a method of determining whether the pedestrian 90 is a dementia patient.

First, a step (S510) of obtaining a video 20 in which a set area is captured may be performed. This step may be performed by the acquisition unit 110.

A step of extracting the pedestrian 90 included in the video 20 (S520) may be performed. This step may be performed by the detection unit 131.

Forming a skeleton of the pedestrian 90 (S530) may be performed. This step may be performed by the detection unit 131.

A step of measuring the stride length and walking speed of the pedestrian 90 through analysis of the skeleton (S540) may be performed. This step may be performed by the calculator 133.

A step of generating a spectrogram image including the stride length and walking speed (S550) may be performed. This step may be performed by the conversion unit 135.

A step of analyzing the spectrogram image using the discrimination model 33 generated through the deep learning algorithm and determining whether the pedestrian 90 has dementia (S560) may be performed. This step may be performed by the determination unit 137.

7 is a flowchart illustrating another tracking method of the present invention.

The tracking method of FIG. 7 may be performed by the tracking device of FIG. 1 . The tracking method of FIG. 7 may represent a method of tracking a movement line of a specific pedestrian.

A step (S610) of obtaining a video 20 in which the set region is captured may be performed. This step may be performed by the acquisition unit 110.

A step of extracting an object image representing the pedestrian 90 included in the video 20 (S620) may be performed. This step may be performed by the detection unit 131.

A step of extracting gait characteristics including the stride length and gait speed of the pedestrian 90 through the analysis of the object image (S630) may be performed. This step may be performed by the calculator 133.

A step of generating a spectrogram image including the stride length and walking speed (S conversion unit 135) may be performed (S640). This step may be performed by the conversion unit 135.

A step of determining whether the pedestrian 90 corresponds to a specific pedestrian entered in the object image, the gait characteristics, and the spectrogram image (S650) may be performed. This step may be performed by the tracking unit 139 .

A step of displaying a movement path of a candidate object corresponding to the pedestrian 90 determined to match a specific pedestrian on a map (S660) may be performed. This step may be performed by the tracking unit 139 .

Looking at the process of determining whether the pedestrian 90 is a patient with dementia and the process of tracking the movement of the patient with dementia, the operation of the detection unit 131, calculation unit 133, and conversion unit 135 (S 510, S 520 , S 530, S 540, S 610, S 620, S 630, S 640) are the same as each other. Using this, operations of the detection unit 131, the calculation unit 133, and the conversion unit 135 in the dementia determination process and the movement tracking process can be shared between both processes.

8 is a flowchart illustrating operations of the detection unit 131, the calculation unit 133, and the conversion unit 135. 9 is a schematic diagram showing the operation of the detection unit 131.

The identification information ID of the new video 20 may be received through the obtaining unit 110 or a separate input unit (S561).

The obtaining unit 110 may retrieve the video 20 file from the video 20 database with the received new video 20 ID (S562).

The detection unit 131 may read one frame from the video 20 (S563).

The detection unit 131 may input a frame to an object detection model corresponding to the extraction model 31, and may receive coordinates and an object ID of an object detected and tracked. The detection unit 131 may generate an object image by cutting out the remaining area except for the entire object in the frame for each detected object (S564).

The detection unit 131 may input object images to a pose estimation model corresponding to the classification model 32 and extract a skeleton of the object (S565).

The calculation unit 133 may measure the change amount of the object's stride length and limb movements when the object stops and after walking begins (S566).

The detection unit 131 may read the next frame from the video 20 (S567).

When the measurement is finished (S 567), the conversion unit 135 may integrate the step length and movement variation of the limbs and convert them into a spectrogram image (S 568).

Object ID, image, detection time, and step length information are stored in an object database, and the converted spectrogram image and step length information may be stored in a pose database.

The conversion unit 135 transfers IDs of detected objects to the determination unit 137 . The detector 131 may detect an object from an image and obtain gait characteristics. The measured gait characteristics may be converted into a spectrogram image for use in the determining unit 137 . Through this process, the determination unit 137 analyzes the gait characteristics to determine dementia.

10 is a flowchart showing the operation of the determination unit 137. 11 is a schematic diagram showing the operation of the determination unit 137.

The determination unit 137 may determine whether the pedestrian 90 has dementia based on the gait characteristics of the object corresponding to the pedestrian 90 . The discrimination model 33 mounted in the determination unit 137 may determine whether or not the object has dementia by analyzing the amount of change in the step length and limb movement.

The operation sequence of the determining unit 137 may be as follows.

The determination unit 137 may receive an object ID for which dementia determination is required (S571).

The determination unit 137 may receive the spectrogram image of the object from the pose database (S572).

The determination unit 137 may pre-process the spectrogram image before determining dementia.

The determination unit 137 may input the pre-processed spectrogram image to the pre-trained discrimination model 33 and receive a return of dementia (S573).

The determination unit 137 may store the dementia determination result in the object database (S574).

The determination unit 137 may transmit the dementia determination result to the tracking unit 139 . As shown in FIG. 11 , the determination unit 137 may receive a spectrogram image and determine whether or not there is dementia. Pre-processing, such as dividing the spectrogram image by size, may be performed before being input to the discrimination model 33 . Through the preprocessing process, the gait characteristics for each set time are extracted and can be used as dementia discrimination data.

12 is a schematic diagram showing the operation of the tracking unit 139.

The tracking unit 139 may perform re-recognition of a specific pedestrian when the pedestrian 90 photographed by the specific camera 10 is determined to be a patient with dementia and the specific pedestrian is out of the field of view of the specific camera 10 .

For example, the tracking unit 139 may compare objects detected from the video 20 captured by the camera 10 within a set radius from a location where a specific pedestrian corresponding to a dementia patient is detected, with the specific pedestrian.

Tracker 139 may use two models to perform fast and accurate re-identification.

The tracking unit 139 may use an object recognition model and an analysis model corresponding to the feature model 34 for re-identification.

The object recognition model compares only object appearance information to select a candidate group that has a similar appearance to a specific pedestrian among numerous objects. The analysis model can rearrange the rankings of the candidate groups by performing re-recognition one more time using more elements than the object re-recognition model. The analysis model can quickly and accurately find a dementia object (a specific pedestrian) among numerous objects using two models.

The processing process of the object recognition model and analysis model is as follows.

The tracking unit 139 may receive an object ID of a specific pedestrian determined to be a patient with dementia (S581). In other words, when a patient with dementia is identified in the video 20 of a specific camera 10 , identification information ID of the patient with dementia may be propagated to the plurality of tracking units 139 .

The tracking unit 139 may receive the image of the object, the detection time, and the ID of the video 20 from the object database (S582).

The tracking unit 139 may search the storage unit for an object included in a search range predefined by the detection time of the object and the ID of the video 20, and may receive the object ID, image, and stride information (S582).

The tracking unit 139 may input an object image into the re-identification model and receive a feature value returned (S583).

The tracking unit 139 may generate a candidate group after selecting only N approximate object images by comparing the feature values of the object using a similarity determination algorithm (S584).

The tracking unit 139 may receive the spectrogram image and stride characteristics of the subject and candidate group from the pose database (S585).

The tracking unit 139 may return feature values by inputting the spectrogram image, object image, and stride characteristics to the analysis model (S586).

The tracking unit 139 may rearrange the ranks of the candidates based on the degree of similarity by comparing feature values of the object and the candidate group (S587).

The tracking unit 139 may deliver object IDs, images, and rank information of the target and candidate groups to the tracking model that tracks dynamics (S588). The disadvantages of the analysis model, which has high recognition accuracy but is slow, can be supplemented by the object recognition model. Since the analysis model requires a lot of resources and time, it can be applied only to selected candidates. Work efficiency can be improved by allowing only objects with high probability to perform element analysis.

13 is a schematic diagram showing another operation of the tracking unit 139.

The tracking model loaded in the tracking unit 139 may display a movement path of a specific pedestrian corresponding to a dementia object on a map. The candidate group is transmitted to the manager, and the manager can select a candidate object determined to be a dementia object from among candidates detected for each photographing device. When the selection is completed, the tracker 139 may track the movement line by displaying the movement path of the dementia object on the map.

The tracking model of the tracking unit 139 may receive object IDs, images, and rank information of the dementia object and candidate group (S591 and S592). When the analysis model and the tracking model are loaded into the tracking unit 139 together, this process is omitted and the output value of the analysis model can be directly input to the tracking model.

The re-identification result is provided to the manager's terminal by the tracking unit 139 (S593), and at this time, images of candidate objects for each photographing device may be displayed in order of rank.

The administrator may select a candidate object image determined to be the same as the dementia object from among the images displayed for each channel (S594).

When the manager's selection is completed, the tracking unit 139 may receive detection time and photographing location information of the selected candidate object from the storage unit.

The tracking unit 139 may mark the movement path of the selected candidate group object on the map with the received detection time and photographing location (S595). The administrator can finally determine whether the object is the same as the dementia object and track the movement path. Tens of thousands of objects can be selected into dozens of candidate groups for each channel using a deep learning model and provided to the manager. This allows administrators to quickly and accurately track the movements of dementia patients.

14 is a diagram illustrating a computing device according to an embodiment of the present invention. The computing device TN100 of FIG. 14 may be a device (eg, a tracking device, etc.) described in this specification.

In the embodiment of FIG. 14 , the computing device TN100 may include at least one processor TN110, a transceiver TN120, and a memory TN130. In addition, the computing device TN100 may further include a storage device TN140, an input interface device TN150, and an output interface device TN160. Elements included in the computing device TN100 may communicate with each other by being connected by a bus TN170.

The processor TN110 may execute program commands stored in at least one of the memory TN130 and the storage device TN140. The processor TN110 may mean a central processing unit (CPU), a graphics processing unit (GPU), or a dedicated processor on which methods according to embodiments of the present invention are performed. Processor TN110 may be configured to implement procedures, functions, methods, and the like described in relation to embodiments of the present invention. The processor TN110 may control each component of the computing device TN100.

Each of the memory TN130 and the storage device TN140 may store various information related to the operation of the processor TN110. Each of the memory TN130 and the storage device TN140 may include at least one of a volatile storage medium and a non-volatile storage medium. For example, the memory TN130 may include at least one of read only memory (ROM) and random access memory (RAM).

The transmitting/receiving device TN120 may transmit or receive a wired signal or a wireless signal. The transmitting/receiving device TN120 may perform communication by being connected to a network.

Meanwhile, the embodiments of the present invention are not implemented only through the devices and/or methods described so far, and may be implemented through a program that realizes functions corresponding to the configuration of the embodiments of the present invention or a recording medium in which the program is recorded. And, such implementation can be easily implemented by those skilled in the art from the description of the above-described embodiment.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements of those skilled in the art using the basic concept of the present invention defined in the following claims are also provided. belong to the scope of the invention.

10...Camera 2 0...Movie
31 ... extraction model 32 ... classification model
33...Discrimination model 34...Feature model
80...Pedestrian road 90...Pedestrian
110 ... acquisition unit 130 ... tracking unit
131 ... detecting unit 133 ... calculating unit
135 ... conversion unit 137 ... determination unit
139 ... tracking unit

Claims (10)

an acquisition unit that obtains a motion picture of a pedestrian;
a tracking unit that determines whether the pedestrian is a dementia patient or tracks a specific pedestrian through analysis of the video;
Tracking device comprising a.
According to claim 1,
The tracking unit analyzes the gait of the pedestrian included in the video and determines whether the pedestrian has dementia using the analysis result of the gait.
According to claim 1,
A camera for photographing the pedestrian and generating the video is provided,
The tracking unit detects patients with dementia using at least one of first location information including a location where the camera is installed, second location information indicating a location photographed by the camera, and information on a direction in which the pedestrian moves in the video. Tracking device to track.
According to claim 1,
The tracking unit is provided with an extraction model, a classification model, and a discrimination model generated through machine learning,
The extraction model extracts pedestrians included in the video when the video is input,
The classification model analyzes the joint keypoints of the pedestrian extracted through the extraction model, and forms a skeleton of the pedestrian using the keypoints,
The discrimination model is a tracking device for determining whether the pedestrian has dementia using gait characteristics including the stride length and walking speed of the pedestrian calculated through the analysis of the skeleton.
According to claim 1,
The tracking unit measures the distance between the feet of the pedestrian using the joint keypoint of the pedestrian included in the video;
The tracking unit calculates the step length and walking speed of the pedestrian using the interval,
The tracking unit extracts joint motion including the amount of change in arm joints and the amount of change in leg joints of the pedestrian when the pedestrian walks;
The tracking unit generates one test signal by summing each signal indicating the step length, the walking speed, and the joint motion obtained after the pedestrian's gait continuously lasts for a set time or more;
The tracking unit uses a short-time Fourier transform (STFT) algorithm to convert the test signal into a processing signal representing a step length change amount according to time change,
The tracking unit converts the processing signal into a spectrogram image including the stride length and the walking speed;
The tracking unit calculates a probability that the pedestrian has dementia through analysis of the spectrogram image;
wherein the tracking unit determines that the pedestrian is a dementia patient if the probability of dementia satisfies a set value, and determines the pedestrian as a normal person if the probability of dementia does not satisfy the set value.
According to claim 5,
The tracking unit inputs the spectrogram image to a discriminant model generated using a convolutional neural network;
The discrimination model divides the spectrogram image into set intervals in the flow of time and analyzes the gait characteristics according to the change in time,
The discrimination model determines whether the pedestrian has dementia by analyzing the amount of change in the step length and the amount of change in the walking speed of the pedestrian through the analysis of the gait characteristics.
According to claim 1,
The tracking unit measures the distance between the feet of the pedestrian using the joint keypoint of the pedestrian included in the video;
The tracking unit calculates the step length and walking speed of the pedestrian using the interval,
The tracking unit extracts joint motion including the amount of change in arm joints and the amount of change in leg joints of the pedestrian when the pedestrian walks;
The tracking unit generates one test signal by summing each signal representing the step length, the walking speed, and the joint motion obtained after the pedestrian's gait continuously lasts for a set time or more;
The tracking unit is converted into a processing signal representing a change in step length with time using a short-time Fourier transform (STFT) algorithm,
The tracking unit converts the processing signal into a spectrogram image including the stride length and the walking speed;
The tracking unit determines whether the pedestrian is the specific pedestrian using the walking characteristics of the pedestrian including the step length and the walking speed and the spectrogram image;
The tracking unit tracks the movement of the pedestrian when the pedestrian is determined as the specific pedestrian.
According to claim 1,
A detection unit, a calculation unit, a conversion unit, a determination unit, and a tracking unit are provided,
The detection unit measures the distance between the feet of the pedestrian using the joint keypoint of the pedestrian included in the video,
The calculation unit calculates the step length and walking speed of the pedestrian using the interval,
The conversion unit generates a spectrogram image including the stride length and the walking speed,
The determination unit determines whether the pedestrian has dementia through analysis of the spectrogram image,
The tracking unit determines whether the pedestrian and the specific pedestrian match through analysis of the spectrogram image, and tracks the movement of the pedestrian determined to be the specific pedestrian tracking device.
In the tracking method performed by the tracking device,
Obtaining a video recording a set area;
extracting pedestrians included in the video;
forming a skeleton of the pedestrian;
measuring a step length and walking speed of the pedestrian through analysis of the skeleton;
generating a spectrogram image including the stride length and the walking speed;
analyzing the spectrogram image using a discrimination model generated through a deep learning algorithm and determining whether the pedestrian has dementia;
A tracking method that includes.
In the tracking method performed by the tracking device,
Obtaining a video recording a set area;
extracting an object image representing a pedestrian included in the video;
extracting a gait characteristic including a step length and a gait speed of the pedestrian through analysis of the object image;
generating a spectrogram image including the stride length and the walking speed;
determining whether or not the pedestrian matches a specific pedestrian entered using the object image, the gait characteristics, and the spectrogram image;
displaying a movement path of a candidate object corresponding to the pedestrian determined to match the specific pedestrian on a map;
A tracking method that includes.

Images