KR102566188B1 - System for providing missing person management service based on artificial intelligence - Google Patents

Abstract

Provided is an AI-based missing person management service provision system which comprises: a user terminal registering a facial image, body image, and information of a protected person, receiving a result of tracking the real-time movement of the protected person, and outputting the same on a map; and a management service provision server including a storage unit registering and storing the facial images, body images, and information of the protected person from the user terminal, a prohibited zone setting unit setting at least one prohibited zone and setting an alarm to sound when the protected person goes into the at least one prohibited zone, a prohibited behavior setting unit setting at least one prohibited behavior and setting the alarm to sound when the protected person performs the at least one prohibited behavior, and an alarm unit guiding the real-time movement of the protected person to the user terminal and transmitting the alarm to the user terminal when entering the prohibited zone or performing the prohibited behavior. According to the present invention, not only elderly people with dementia and missing people but also cases where people attempt suicide or are drunk and are homeless can be identified through object detection.

Description

AI-based missing person management service provision system {SYSTEM FOR PROVIDING MISSING PERSON MANAGEMENT SERVICE BASED ON ARTIFICIAL INTELLIGENCE}

The present invention relates to a system for providing an AI-based missing person management service, and provides a system that maps the location of a missing person in real time based on AI and immediately sounds an alarm when entering a prohibited area or performing a prohibited action.

Korea is rapidly aging, and as the elderly population increases, the number of people with dementia is gradually increasing. Dementia gradually deteriorates cognitive function, resulting in memory impairment and language impairment, and after the middle period, it is accompanied by mental symptoms including depression, physical and verbal aggression such as violence, suspicion, and swearing, and behavioral symptoms such as wandering and sunset syndrome. do. Wandering behavior in patients with dementia is caused by a decrease in the ability to find one's way due to impaired spatial memory. Families caring for the elderly with dementia reported that the most difficult problem behavior in nursing patients with dementia was wandering behavior, and conflicts between the elderly with dementia and their families intensified due to forcibly stopping the wandering behavior of the elderly with dementia. The wandering behavior of the elderly with dementia is a major stress factor for the family, and the main cause of the elderly with dementia entering the elderly care facility is the wandering behavior. It can lead to fatal consequences ranging from simple accidents to death.

At this time, in order to determine the location of the ward, a method of using smart tags and face recognition in conjunction with CCTV has been researched and developed. Korean Patent Registration No. 10-2039277 (published on October 31, 2019) includes a smart tag possessed by a ward and a reader capable of tagging a smart tag in CCTV, and when the smart tag is tagged in the CCTV reader A configuration for transmitting a CCTV screen to a control center and a configuration for obtaining a face of a pedestrian passing through a pedestrian path, comparing it with the face of a registered subject, determining whether an event has occurred, and uploading the result are disclosed.

However, in the case of the former case, the ward must possess the smart tag, but the probability of possessing the smart tag is slim in a situation where the ward cannot visit his or her house. Even in the latter case, face identification is identified only when it is located within a preset distance or when quality or brightness is guaranteed. However, it is very difficult to identify the face itself in a dark evening, poor picture quality, or a screen smeared by light. it falls Looking at the recent e-Nara index, the National Police Agency's internal administrative data, the status of reports and processing of missing patients with dementia, the number of missing people due to dementia has increased by more than 72% over the past 5 years, and social services to prevent these disappearances have led to identification tags for elderly people who can roam around. Distribute business, dementia check app wandering detection service, long-term care insurance for the elderly, loitering detector payment, National Police Agency missing alert text messages, etc. are being implemented, but it is still not easy to find an elderly person with dementia. Accordingly, research and development of a system that detects the real-time location of a missing person, including an elderly person with dementia, and sounds an alarm when entering a prohibited area or performing a prohibited action, is required.

An embodiment of the present invention, when the user terminal registers the ward's face image, body image, and ward information, maps and guides the ward's real-time location, provides an alarm when entering a prohibited area or performs a prohibited action, and provides location By providing together, it helps users to focus on livelihood activities without protecting their wards 24/7, and even if they are unregistered wards, they can track their location in real time with the same process in the event of a missing person, elderly people with dementia And it is possible to provide an AI-based missing person management service providing system that can identify not only the missing person, but also a case of attempting suicide or being drunk and staying homeless through object detection. However, the technical problem to be achieved by the present embodiment is not limited to the technical problem as described above, and other technical problems may exist.

As a technical means for achieving the above-described technical problem, an embodiment of the present invention registers the ward's face image, body image, and ward information, receives the result of tracking the ward's real-time movement, and outputs it on a map A storage unit that receives and stores face images, body images, and ward information from user terminals and user terminals, and sets up at least one prohibited area to set an alarm to sound when a ward enters at least one prohibited area Part, Prohibited action setting unit that sets at least one prohibited action to set an alarm to sound when the ward performs at least one prohibited action, guides the ward's real-time movement to the user terminal, In this case, a management service providing server including an alarm unit for transmitting an alarm to a user terminal is included.

According to any one of the above-described problem solving means of the present invention, when the user terminal registers the ward's face image, body image, and ward information, mapping and guiding the ward's real-time location, entering a prohibited area or performing a prohibited action By providing an alarm and providing location together, it helps users to focus on livelihood activities without protecting 24/7 wards, and even unregistered wards can track their real-time location in the same process if a missing event occurs. It is possible to identify not only elderly people with dementia and missing people, but also cases of suicide attempt or drunken homelessness by object detection.

1 is a diagram for explaining an AI-based missing person management service providing system according to an embodiment of the present invention.
FIG. 2 is a block diagram illustrating a management service providing server included in the system of FIG. 1 .
3 and 4 are diagrams for explaining an embodiment in which an AI-based missing person management service is implemented according to an embodiment of the present invention.
5 is an operational flowchart illustrating a method for providing an AI-based missing person management service according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail so that those skilled in the art can easily practice the present invention with reference to the accompanying drawings. However, the present invention may be embodied in many different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part is said to be "connected" to another part, this includes not only the case where it is "directly connected" but also the case where it is "electrically connected" with another element interposed therebetween. . In addition, when a part "includes" a certain component, this means that it may further include other components, not excluding other components, unless otherwise stated, and one or more other characteristics. However, it should be understood that it does not preclude the possibility of existence or addition of numbers, steps, operations, components, parts, or combinations thereof.

As used throughout the specification, the terms "about", "substantially", etc., are used at or approximating that value when manufacturing and material tolerances inherent in the stated meaning are given, and do not convey an understanding of the present invention. Accurate or absolute figures are used to help prevent exploitation by unscrupulous infringers of the disclosed disclosure. The term "step of (doing)" or "step of" as used throughout the specification of the present invention does not mean "step for".

In this specification, a "unit" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. Further, one unit may be realized using two or more hardware, and two or more units may be realized by one hardware. On the other hand, '~ unit' is not limited to software or hardware, and '~ unit' may be configured to be in an addressable storage medium or configured to reproduce one or more processors. Thus, as an example, '~unit' refers to components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, and procedures. , subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays and variables. Functions provided within components and '~units' may be combined into smaller numbers of components and '~units' or further separated into additional components and '~units'. In addition, components and '~units' may be implemented to play one or more CPUs in a device or a secure multimedia card.

In this specification, some of the operations or functions described as being performed by a terminal, device, or device may be performed instead by a server connected to the terminal, device, or device. Likewise, some of the operations or functions described as being performed by the server may also be performed by a terminal, apparatus, or device connected to the server.

In this specification, some of the operations or functions described as mapping or matching with the terminal mean mapping or matching the terminal's unique number or personal identification information, which is the terminal's identifying data. can be interpreted as

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram for explaining an AI-based missing person management service providing system according to an embodiment of the present invention. Referring to FIG. 1 , an AI-based missing person management service providing system 1 may include at least one user terminal 100, a management service providing server 300, and at least one CCTV 400. However, since the AI-based missing person management service providing system 1 of FIG. 1 is only one embodiment of the present invention, the present invention is not limitedly interpreted through FIG. 1 .

At this time, each component of FIG. 1 is generally connected through a network (Network, 200). For example, as shown in FIG. 1 , at least one user terminal 100 may be connected to the management service providing server 300 through the network 200 . In addition, the management service providing server 300 may be connected to at least one user terminal 100 and at least one CCTV 400 through the network 200 . In addition, at least one CCTV 400 may be connected to the management service providing server 300 through the network 200 .

Here, the network refers to a connection structure capable of exchanging information between nodes such as a plurality of terminals and servers, and examples of such networks include a local area network (LAN) and a wide area network (WAN: Wide Area Network), the Internet (WWW: World Wide Web), wired and wireless data communications networks, telephone networks, and wired and wireless television communications networks. Examples of wireless data communication networks include 3G, 4G, 5G, 3rd Generation Partnership Project (3GPP), 5th Generation Partnership Project (5GPP), Long Term Evolution (LTE), World Interoperability for Microwave Access (WIMAX), Wi-Fi , Internet (Internet), LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), RF (Radio Frequency), Bluetooth (Bluetooth) network, NFC ( A Near-Field Communication (Near-Field Communication) network, a satellite broadcasting network, an analog broadcasting network, a Digital Multimedia Broadcasting (DMB) network, etc. are included, but not limited thereto.

In the following, the term at least one is defined as a term including singular and plural, and even if at least one term does not exist, each component may exist in singular or plural, and may mean singular or plural. It will be self-evident. In addition, the singular or plural number of each component may be changed according to embodiments.

At least one user terminal 100 registers the ward's face image, body image, and ward information using an AI-based missing person management service-related web page, app page, program, or application, and displays the ward's real-time movement on a map. It may be an output terminal. In addition, the user terminal 100 may be a terminal that receives an alarm from the management service providing server 300 and shares a result of tracking a real-time location when a ward enters a prohibited area or performs a prohibited action.

Here, at least one user terminal 100 may be implemented as a computer capable of accessing a remote server or terminal through a network. Here, the computer may include, for example, a laptop, a desktop, a laptop, and the like equipped with a navigation system and a web browser. In this case, at least one user terminal 100 may be implemented as a terminal capable of accessing a remote server or terminal through a network. At least one user terminal 100 is, for example, a wireless communication device that ensures portability and mobility, navigation, PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet ) may include all types of handheld-based wireless communication devices such as terminals, smartphones, smart pads, tablet PCs, and the like.

The management service providing server 300 may be a server that provides an AI-based missing person management service web page, app page, program, or application. And, the management service providing server 300 may be a server that registers at least one CCTV 400 and registers the face image, body image, and ward information of the ward registered in the user terminal 100 . In addition, the management service providing server 300 sets at least one prohibited area and prohibited action, and when a ward enters the prohibited area or a ward or any object performs a prohibited action, the user terminal 100 or the control center It may be a server that sends an alarm. In addition, when the management service providing server 300 outputs a missing event in a state in which the ward is not registered in the user terminal 100, the ward is tracked through the same process as described above and whether the ward has not entered the prohibited area. Alternatively, it may be a server that determines whether or not a prohibited action is being performed.

Here, the management service providing server 300 may be implemented as a computer capable of accessing a remote server or terminal through a network. Here, the computer may include, for example, a laptop, a desktop, a laptop, and the like equipped with a navigation system and a web browser.

At least one CCTV 400 may be an intelligent CCTV using an AI-based missing person management service-related web page, app page, program, or application.

Here, at least one CCTV (400) may be implemented as a computer capable of accessing a remote server or terminal through a network. Here, the computer may include, for example, a laptop, a desktop, a laptop, and the like equipped with a navigation system and a web browser. In this case, at least one CCTV 400 may be implemented as a terminal capable of accessing a remote server or terminal through a network. At least one CCTV (400), for example, as a wireless communication device that ensures portability and mobility, navigation, PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet) It may include all types of handheld-based wireless communication devices such as terminals, smartphones, smart pads, tablet PCs, and the like.

2 is a block diagram illustrating a management service providing server included in the system of FIG. 1, and FIGS. 3 and 4 describe an embodiment in which an AI-based missing person management service is implemented according to an embodiment of the present invention. It is a drawing for

Referring to FIG. 2, the management service providing server 300 includes a storage unit 310, a prohibited area setting unit 320, a prohibited action setting unit 330, an alarm unit 340, an unregistered management unit 350, a face It may include a recognition unit 360, an action recognition unit 370 and a clothing recognition unit 380.

The management service providing server 300 according to an embodiment of the present invention or another server (not shown) operating in conjunction with at least one user terminal 100 and at least one CCTV 400 is an AI-based missing person management service application. , When transmitting a program, app page, web page, etc., at least one user terminal 100 and at least one CCTV 400 install an AI-based missing person management service application, program, app page, web page, etc. can be opened In addition, a service program may be driven in at least one user terminal 100 and at least one CCTV 400 using a script executed in a web browser. Here, the web browser is a program that allows users to use the web (WWW: World Wide Web) service, and means a program that receives and displays hypertext described in HTML (Hyper Text Mark-up Language). For example, Netscape , Explorer, Chrome, and the like. In addition, an application means an application on a terminal, and includes, for example, an app running on a mobile terminal (smart phone).

Referring to FIG. 2 , the storage unit 310 may register and store face images, body images, and ward information from the user terminal 100 . The user terminal 100 may register the ward's face image, body image, and ward information, receive a result of tracking the ward's real-time movement, and output it on a map. At this time, the reason for receiving the body image of the ward is that in the case of the video taken by the CCTV (400), it is difficult to check the feature points of the face unless a certain distance is secured, and face identification and recognition to find feature points for the faces of all subjects This is because when running the algorithm for , enormous networking and computing resources are consumed and excessive time is consumed. Accordingly, when a certain distance is not secured, for example, when the distance between CCTV and the subject is not narrowed to less than X distance, the body image is received to determine or identify the ward by body proportion, color, or clothing. .

To this end, an appearance analysis technique using deep learning image analysis may be used. Animal appearance analysis technology using deep learning image analysis can analyze the appearance of an object by using CNN (Convolution Neural Network) to find feature points in body images and TensorFlow, a deep learning framework provided by Google. It is possible to obtain the unique appearance information of each subject by finding feature points in the external skeleton through CNN and analyzing the body proportion information with Tensor Flow. In this way, unique information can be obtained only with the appearance of an object, and through this, each object can be specified and identified.

At this time, if the distance between the CCTV (400) and the subject is long, the facial feature points cannot be accurately grasped, so in the first step, the external skeleton or proportions, for example, the ratio of the torso and legs, the size of the face It is identified using the ratio or length between the body and the length of the torso, and in the second step, in the case of close-up, it is identified using the feature points of the face. Here, in order to obtain the external skeleton, in addition to the above-mentioned Tensor Flow, after extracting the skeleton information using the OpenPose algorithm, it is possible to check whether it is the same object (protected person) by using the ratio or length of each part.

<Image Correction>

To brighten a dark image, simple image processing and deep learning are used. In the case of using deep learning, there is a technology that learns indoor and outdoor photos without distinction and restores the internal colors of dark images to similar colors among the learned data. As simple image processing, there is also a method of constantly adjusting the overall contrast or applying an intermediate brightness obtained by calculating the difference between a dark area and a bright area using brightness values for RGB colors. In an embodiment of the present invention, an image may be corrected using histogram equalization. When histogram equalization is applied to a black and white image, the brightness distribution of the image is uniformly changed and dark areas can be improved. This difference in contrast enhances the deep learning recognition rate by compensating for areas that are dark and difficult to see in black and white images or areas that are bright compared to the surroundings due to light.

In order to find a moving object, it is necessary to separate the background and the moving object. A technique that can do that is Background Subtraction. Foreground separation finds a moving object with the difference between the previous frame and the current frame compared to the previous frame when a new frame enters. With foreground separation, moving objects can be found, so moving areas can be extracted and used for deep learning recognition. If the ward is found, but the entire area is unclearly photographed, it is not good because the body is cut off when the area of interest is extracted for deep learning. To solve this problem, erode and dilate of morphology can be used. By using this, you can also remove small noises that can occur when leaves shake or insects appear. After morphology is applied, if morphology is applied less than necessary, a part of the ward's body may be cut off or noise may not be removed. In this case, deep learning recognition may fail because the body is not perfect, and the time for recognizing the ward may be delayed due to noise recognition. Considering this situation, it is possible to set the deep learning recognition to be moderate by applying the morphology slightly larger than the size of the ward.

When performing deep learning recognition, deep learning accuracy is improved when there are only objects to be found rather than images with many backgrounds. This is because the process of calculating other areas included in the image other than the object to be found is reduced. If an image that has undergone foreground separation and morphology is overlapped with an image that has undergone histogram smoothing, the location of the ward can be displayed in the histogram smoothed image. A box can be used for region of interest extraction, and a ward's region can be extracted by using the coordinates of the box. Deep learning recognition results can be improved by using an image that clearly shows the object to be found.

The prohibited area setting unit 320 may set at least one prohibited area to sound an alarm when a ward enters the at least one prohibited area. At this time, since the ward is registered, it is to guide the user to enter the prohibited area, but if the prohibited area is dangerous to everyone, for example, if there is a risk of collapse of a building or a river with a risk of flooding, all people Since it is dangerous to the ward, it can inform the control center that there is a person entering the prohibited area regardless of whether the person is protected, and it can recommend sending rescue personnel. At this time, the prohibited area may include an abandoned house, an empty house, a riverside, and a forest entrance, but is not limited to those listed and is not excluded for reasons not listed.

The prohibited behavior setting unit 330 may set at least one prohibited behavior so that an alarm sounds when a ward performs at least one prohibited behavior. At this time, the prohibited behavior may include a behavior of lying on the road without movement for more than a predetermined time, a behavior of jumping off a leg, and a wandering behavior of an elderly or dementia patient, but is not limited to the listed ones and is not excluded for reasons not listed. .

At this time, Darknet-based YOLO (You Only Look Once) may be used for object detection. The first thing to be implemented is the object detection technology that first finds out whether the object in the video received from the CCTV (400) is a person, whether a person is a ward or not, and if it is a ward, where the location is, Darknet-based YOLO can be implemented using In general, a classification problem is a problem of classifying an object class for an image having one object. On the other hand, the object detection problem is a problem that combines classification and localization, and means classifying an object and finding the location of the object with a bounding box. In the field of object detection, various models based on CNN (Convolutional Neural Network) may be used, and R-CNN, Fast R-CNN, Faster R-CNN, and the like may be used.

<Behavioral Classification>

In order to recognize a person's prohibited behavior, it can be defined by dividing it into a case where an alarm is sounded unconditionally when a person takes this action, and a case where an alarm is sounded to a user and related institutions when a ward takes this action. For example, when a flood occurs, the act of entering a river is dangerous to everyone including the ward, so it corresponds to the former. The act of walking on a sidewalk is not dangerous for everyone, but it can be dangerous for the ward, so it falls under the latter. At this time, a classifier that classifies what kind of action it is can be trained with, for example, a teachable machine serviced by Google. Google has unveiled Teachable Machine, a web-based learning tool that can understand and learn the machine learning process. Through three projects: image, audio, and posture, you can learn machine learning and create a learning model, then save and use it. . It is structured so that anyone can easily use it by visiting the website, and the training model for the image can be compared with other images using the Preview function, or exported or downloaded to Tensorflow.js using the Export Model menu. can receive Of course, it will be self-evident that behaviors can be recognized and patterns can be classified by various algorithms in addition to the above-described algorithms.

The alarm unit 340 may guide the ward's real-time movement to the user terminal 100 and transmit an alarm to the user terminal 100 when entering a prohibited area or performing a prohibited action.

The non-registration management unit 350 receives the face image, body image, and ward information of the ward from the user terminal 100 when a missing event of the ward is output even if the ward is not previously registered in the user terminal 100. After that, the real-time location of the ward can be tracked, and an alarm can be transmitted to the user terminal 100 when located in a prohibited area or performing a prohibited action. In other words, even if registration is not performed, the warded person can be found by starting the above-described process from the beginning.

When a face image of a ward is registered, the face recognition unit 360 may extract and track an object having the same face among objects in an image received from at least one CCTV using a previously registered face recognition algorithm. Face identification technology can be largely divided into face recognition and face verification problems. Among them, identity authentication is a one-to-one matching problem that registers one's face image as an anchor image and determines whether or not the person is the same as the anchor image whenever recognition is required. Face Verification corresponds to Existing same-person lookup models show high performance of over 99% in various data sets, but recognition obstacles such as hats and glasses still affect performance. In particular, when wearing a mask, face identification is very difficult as most of the coordinates of facial feature points such as the chin, nose, and mouth are covered. DeepFace, an open-source face recognition application, also cannot recognize faces when wearing masks.

Accordingly, the registration process of the present invention may consist of two steps. The first step is to use the facial image of Koreans, which is to augment the mask-wearing face image with the face image without a mask. This is the stage of learning and testing.

<Mask wearing data augmentation>

In order to improve the performance of Face Verification, it is necessary to use face images wearing masks for learning. To this end, it is not easy to secure a data set of actual mask wearing images and assign labels. However, a mask-wearing image may be generated through simulation of a method of combining a mask with a non-mask-wearing image. The Korean face image is facial image data taken by considering various ages and genders of Koreans. The data consists of 20 types of shooting angles, 30 types of lighting, 6 types of accessories, and 3 types of facial expressions for a total of 400 people. The mask wearing simulation may generate a face image wearing a mask by performing perspective transformation based on N points in the coordinates of the mask and facial feature points, respectively.

<Search for the same person wearing a mask>

For face verification model training, training and verification data are separated, and for performance evaluation, an image without a mask (Non_mask) and an image wearing a mask (Mask) are made into a pair, and a total of M pairs are created. can be configured. The learning model can use ArcFace and MobileFaceNets respectively to consider both the accuracy and speed of the model, and the results can be compared after learning by dividing according to whether the augmented mask image is used for learning.

When the body image of the ward is registered, the behavior recognition unit 370 may extract and track an object having the same behavior among objects in an image received from at least one CCTV using a previously registered behavior recognition algorithm. For example, a situation that is not dangerous to everyone but is dangerous to a ward would go through a two-step process of extracting the ward and figuring out whether the ward is doing the action. At this time, as described above, there are few cases where the distance between the CCTV (400) and the ward is close enough to distinguish the feature points of the face, and even in most public institution CCTVs (400) or private CCTVs, in order to secure a wide viewing distance It is always necessary to maintain a height of at least 2 meters above the ground. In this case, since facial feature points cannot be accurately grasped, as described above, a ward is specified through body proportions or clothes through a body image, and after figuring out whether the specified object (protected person) is performing the corresponding action, it is sent to the control center or user It can be notified to the terminal 100. Even if filtering is performed in this way, the number of images that a human needs to select each object can be significantly reduced, and a ward can be found in a short time.

The clothing recognition unit 380, when a missing event is output from the user terminal 100 and the color and design of the clothing of the ward are specified in the user terminal 100, color and design among objects collected from at least one CCTV It is possible to extract an object wearing clothes exceeding a preset similarity.

Hereinafter, an operation process according to the configuration of the above-described management service providing server of FIG. 2 will be described in detail with reference to FIGS. 3 and 4 as examples. However, it will be apparent that the embodiment is only any one of various embodiments of the present invention, and is not limited thereto.

Referring to FIG. 3, the management service providing server 300 receives ward information, face image, and body image from the user terminal 100, and (b) performs real-time movement mapping to map where they are located when wandering around. Provides a navigation that guides the route up to this point, and provides real-time location and navigation to the user terminal 100 when (c) a ward approaches a prohibited area or when a prohibited action is performed as in (d) Helps you find your patrons. As shown in FIG. 4, by using the pre-built CCTV 400, real-time tracking and mapping of the ward, as well as identifying the case where the ward is trying to access or has already approached and entered the prohibited area, or if the prohibited act is performed, The range and types of services can be diversified through linkage and expansion with the established infrastructure. The differences between an embodiment of the present invention and the prior art can be summarized as shown in Table 1.

division prior art the present invention missing person search
method - Missing person notification text service
- Direct search by police and community personnel
- In the integrated control center
CCTV search
- The missing persons registered on the platform (elderly with dementia) automatically provide movement mapping service through the AI platform (APP)
- AI platform (APP) provides movement mapping service when a missing person (elderly with dementia) who is not registered on the platform is reported missing
- Provides automatic alarming service on the AI platform when registered missing persons (seniors with dementia) wander in prohibited areas (abandoned houses, empty houses, riversides, forest entrances, etc.) information
uses - Existing missing person facial recognition information is in the National Police Agency and is not provided to local governments, so facial recognition services are not being implemented.
- The police can also submit and use a complaint to the local government to view the CCTV for searching for missing people, so the utilization of CCTV and the integrated platform is low.
- Public health centers have important information for the elderly with dementia - In the form of an APP, the guardian of the missing person (elderly with dementia), etc. joins the APP, directly enters the information of the subject, and submits consent related to personal information through the APP. no need to get
- The search time for missing persons can be drastically reduced by simplifying the process of checking the movement information of the missing persons.
service scope - Centered on facial recognition of missing persons
- loitering center
- Prevention of accidents of the elderly with dementia who die in empty houses, abandoned houses, riversides, and hills, not wandering, through the alarming service for trespassing into prohibited areas
→ Possible to respond to non-roaming missing persons (seniors with dementia) service implementation - Problem solving approach with simple devices and smart technology
- Realization of a solution in connection with existing police search methods for missing persons, results of previous studies on wandering behavior of dementia patients, and public health center services for missing persons (dementia patients)

Matters not described for the AI-based missing person management service provision method of FIGS. 2 to 4 are the same as or easily inferred from the description of the AI-based missing person management service provision method through FIG. 1 above. Therefore, the following description is omitted.

FIG. 5 is a diagram illustrating a process of transmitting and receiving data between components included in the AI-based missing person management service providing system of FIG. 1 according to an embodiment of the present invention. Hereinafter, an example of a process of transmitting and receiving data between each component will be described through FIG. 5, but the present application is not limited to such an embodiment, and according to various embodiments described above, It is obvious to those skilled in the art that a process of transmitting and receiving data may be changed.

Referring to FIG. 5 , the management service providing server receives and stores face image, body image, and ward information registered from the user terminal (S5100).

Then, the management service providing server sets at least one prohibited area, sets an alarm to sound when a ward enters the at least one prohibited area (S5200), and sets at least one prohibited action to set at least one prohibited action. When the ward does, an alarm is set to sound (S5300).

In addition, the management service providing server guides the ward's real-time movement to the user terminal and transmits an alarm to the user terminal when entering a prohibited area or performing a prohibited action (S5400).

The order between the above-described steps (S5100 to S5400) is only an example, and is not limited thereto. That is, the order of the above-described steps (S5100 to S5400) may be mutually changed, and some of the steps may be simultaneously executed or deleted.

Matters not explained about the AI-based missing person management service provision method of FIG. 5 are the same as or easily inferred from the description of the AI-based missing person management service provision method through FIGS. 1 to 4 above. Therefore, the following description is omitted.

The AI-based missing person management service providing method according to an embodiment described with reference to FIG. 5 may be implemented in the form of a recording medium including instructions executable by a computer, such as an application or program module executed by a computer. . Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. Also, computer readable media may include all computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

The above-described method for providing missing person management service based on AI according to an embodiment of the present invention may be executed by an application basically installed in a terminal (this may include a program included in a platform or an operating system basically installed in the terminal). and may be executed by an application (that is, a program) directly installed in the master terminal by a user through an application providing server such as an application store server, an application or a web server related to the corresponding service. In this sense, the above-described AI-based missing person management service providing method according to an embodiment of the present invention is implemented as an application (ie, a program) that is basically installed in a terminal or directly installed by a user and can be read by a computer such as a terminal. can be recorded on a recording medium.

The above description of the present invention is for illustrative purposes, and those skilled in the art can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, the embodiments described above should be understood as illustrative in all respects and not limiting. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and equivalent concepts should be construed as being included in the scope of the present invention. do.

Claims (6)

A user terminal for registering the ward's face image, body image, and ward information, and receiving and outputting a result of tracking the ward's real-time movement on a map; and
A storage unit for receiving and storing face image, body image, and ward information from the user terminal, and setting at least one prohibited area to set an alarm to sound when a ward enters the at least one prohibited area. A prohibited action setting unit configured to set at least one prohibited behavior and set an alarm to sound when the ward performs at least one prohibited behavior, and guides the real-time movement of the ward to the user terminal and displays the at least one prohibited behavior. When entering the zone and performing the at least one prohibited action, an alarm unit for transmitting an alarm to the user terminal, and when the body image of the ward is registered, at least one action recognition algorithm based on deep learning is registered. Including a management service providing server including a behavior recognition unit for extracting and tracking whether the behavior of the object in the image received from the CCTV of the ward is the same behavior as the ward,
The management service providing server,
Using CNN (Convolution Neural Network) and TensorFlow, a deep learning framework provided by Google, extract (find) the feature points of the object from the stored body image, and use the extracted feature points to capture the CCTV nearby. identify the ward (object);
The prohibited behavior includes the behavior of lying on the road without movement for more than a predetermined time, the behavior of jumping off one's legs, and the wandering behavior of elderly or dementia patients, and the prohibited area includes abandoned houses, empty houses, riversides, and forest entrances. AI-based missing person management service providing system, characterized in that. delete delete delete delete delete

Images