WO2021024691A1 - Image processing system, image processing program, and image processing method - Google Patents

Image processing system, image processing program, and image processing method Download PDF

Info

Publication number
WO2021024691A1
WO2021024691A1 PCT/JP2020/026880 JP2020026880W WO2021024691A1 WO 2021024691 A1 WO2021024691 A1 WO 2021024691A1 JP 2020026880 W JP2020026880 W JP 2020026880W WO 2021024691 A1 WO2021024691 A1 WO 2021024691A1
Authority
WO
WIPO (PCT)
Prior art keywords
target person
image
predetermined
behavior
subject
Prior art date
Application number
PCT/JP2020/026880
Other languages
French (fr)
Japanese (ja)
Inventor
希武 田中
池田　直樹
Original Assignee
コニカミノルタ株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by コニカミノルタ株式会社 filed Critical コニカミノルタ株式会社
Priority to JP2021537637A priority Critical patent/JP7435609B2/en
Publication of WO2021024691A1 publication Critical patent/WO2021024691A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion

Definitions

  • the present invention relates to an image processing system, an image processing program, and an image processing method.
  • Patent Document 1 The following prior art is disclosed in Patent Document 1 below.
  • the monitoring function by the detection unit that detects the predetermined action of the monitored person and gives a notification or the like is stopped based on the information or the like received from the terminal unit. As a result, the monitoring function can be stopped as needed, so that false detections for persons other than the monitored person can be reduced.
  • Patent Document 1 can prevent erroneous detection of the behavior of a person other than the monitored person as the behavior of the monitored person, but cannot improve the detection accuracy of the behavior of the monitored person. There's a problem.
  • the present invention has been made to solve such a problem. That is, it is an object of the present invention to provide an image processing system, an image processing program, and an image processing method capable of improving the estimation accuracy of a person's behavior based on a captured image.
  • a feature point detection unit that detects feature points related to the target person's body based on an image including the target person taken by an imaging device, and the target from the central region of the image based on the image.
  • the behavior of the target person is determined based on the calculation unit that calculates the direction toward the person, the arrangement direction of the predetermined feature points among the detected feature points, and the calculated direction toward the target person.
  • the determination unit that determines whether or not the behavior is included in the behavior of the subject and the determination unit determine that the behavior of the target person is an behavior included in the predetermined behavior
  • information on the behavior of the target person is obtained.
  • An image processing system having an output unit for output.
  • the determination unit is an action in which the action of the target person is included in the predetermined action when the arrangement direction of the predetermined feature points and the direction toward the target person have a predetermined relationship.
  • the photographing device is a wide-angle camera, and the image is an image including the predetermined area taken by the wide-angle camera installed at a position overlooking a predetermined area.
  • the image processing system according to any one of 7).
  • the direction toward the target person is any of the above (1) to (8), which is a direction calculated based on the points included in the central region of the image and the predetermined feature points.
  • the behavior of the target person based on the procedure (b) for calculating the direction toward the person, the arrangement direction of the predetermined feature points among the detected feature points, and the calculated direction toward the target person.
  • the procedure (c) for determining whether or not is an action included in the predetermined action and when it is determined in the procedure (c) that the action of the target person is an action included in the predetermined action, the target An image processing program for causing a computer to execute a process having a procedure (d) for outputting information on a person's behavior.
  • the feature points related to the body of the subject are detected based on the captured image, and the behavior of the subject is a predetermined behavior based on the arrangement direction of the predetermined feature points and the direction from the central region of the image toward the subject. It is determined whether or not it is included in, and when it is determined that it is included, information on the behavior of the target person is output. As a result, it is possible to improve the estimation accuracy of the behavior of the person based on the captured image.
  • the image processing system, the image processing program, and the image processing method according to the embodiment of the present invention will be described with reference to the drawings.
  • the same elements are designated by the same reference numerals, and duplicate description will be omitted.
  • the dimensional ratios in the drawings are exaggerated for convenience of explanation and may differ from the actual ratios.
  • the angle formed by the two directions can be considered as two angles such as 60 degrees and 300 degrees (360 degrees -60 degrees), but in the present specification, the smaller of the two angles It means an angle.
  • FIG. 1 is a diagram showing a schematic configuration of the image recognition system 10.
  • the image recognition system 10 includes a detection unit 100, a server 200, a communication network 300, and a mobile terminal 400.
  • the detection unit 100 is communicably connected to the server 200 and the mobile terminal 400 by the communication network 300.
  • the mobile terminal 400 may be connected to the communication network 300 via the access point 310.
  • the detection unit 100 constitutes an image processing system.
  • the detection unit 100 may be one integrated device or a plurality of devices separately arranged.
  • the server 200 may perform a part of the functions of the detection unit 100.
  • FIG. 2 is a block diagram showing the configuration of the detection unit 100.
  • the detection unit 100 includes a control unit 110, a communication unit 120, a camera 130, and a body motion sensor 140, which are connected to each other by a bus.
  • the camera 130 constitutes a photographing device.
  • the control unit 110 is composed of a CPU (Central Processing Unit) and a memory such as a RAM (Random Access Memory) and a ROM (Read Only Memory), and controls and performs arithmetic processing of each part of the detection unit 100 according to a program.
  • the control unit 110 constitutes a feature point detection unit, a calculation unit, and a determination unit.
  • the control unit 110 constitutes an output unit together with the communication unit 120. The details of the operation of the control unit 110 will be described later.
  • the communication unit 120 is an interface circuit (for example, a LAN card or the like) for communicating with the mobile terminal 400 or the like via the communication network 300.
  • an interface circuit for example, a LAN card or the like
  • the camera 130 is, for example, a wide-angle camera.
  • the camera 130 is installed at a position where the detection unit 100 is installed on the ceiling or the like of the living room of the target person 500 to overlook a predetermined area, and an image including the predetermined area (hereinafter, also simply referred to as “image 600”). Take a picture of).
  • the target person 500 is a person who needs long-term care or nursing by, for example, a staff member.
  • the predetermined area may be a three-dimensional area including the entire floor surface of the living room of the subject 500.
  • the camera 130 may be a standard camera having a narrower angle of view than a wide-angle camera. Hereinafter, for the sake of simplicity, the camera 130 will be described as a wide-angle camera.
  • the image 600 may include the subject 500 as an image.
  • Image 600 includes still images and moving images.
  • the camera 130 is a near-infrared camera, which irradiates near-infrared rays toward the photographing area by an LED (Light Emitting Device) and emits the reflected light of the near-infrared rays reflected by an object in the photographing area to a CMOS (Complemementary Metal Oxide Semiconductor) sensor. A predetermined area can be photographed by receiving light from the camera.
  • the image 600 can be a monochrome image having the reflectance of near infrared rays as each pixel.
  • a visible light camera may be used instead of the near infrared camera, or these may be used in combination.
  • the body movement sensor 140 is a doppler shift type sensor that transmits and receives microwaves to the bed 700 and detects the doppler shift of microwaves generated by the body movement (for example, respiratory movement) of the subject 500.
  • control unit 110 The operation of the control unit 110 will be described.
  • the control unit 110 detects the silhouette of a person's image (hereinafter referred to as "human silhouette") from the image 600.
  • the human silhouette can be detected, for example, by extracting a range of pixels having a relatively large difference by the time difference method for extracting the difference between images (frames) whose shooting times are before and after.
  • the human silhouette may be detected by the background subtraction method that extracts the difference between the photographed image and the background image.
  • the control unit 110 can detect a predetermined action of the subject 500 based on the silhouette of the person. Predetermined actions include, for example, falls and falls.
  • the control unit 110 may fall due to, for example, the center of gravity of the detected silhouette changing from a state in which it was moving in time series to a state in which it suddenly stopped, or a change in the aspect ratio of a rectangle corresponding to a human silhouette. Can be detected.
  • the control unit 110 for example, has changed from a state in which the human silhouette exists in the area of the bed 700 to a state in which the person silhouette suddenly exists outside the area of the bed 700, and a rectangular aspect ratio corresponding to the human silhouette.
  • the fall can be detected by the change of.
  • the area of the bed 700 in the image 600 is preset when the detection unit 100 is installed, and can be stored in the memory of the control unit 110 as data.
  • the control unit 110 Based on the image 600, the control unit 110 detects the person area 610 as an area including the target person 500, and from the person area 610, a feature point related to the human body (hereinafter, simply referred to as “feature point 620”) is obtained. To detect.
  • FIG. 3 is a diagram showing a person area 610 detected in the image 600.
  • the control unit 110 detects an area including the target person 500 who is a person as a person area 610 from the image 600. Specifically, the control unit 110 can detect the person area 610 by detecting the area where the object (object) exists on the image 600 and estimating the category of the object included in the detected area. The region where the object exists can be detected as a rectangle (candidate rectangle) including the object on the image 600. The detection unit 100 detects the person area 610 by detecting the candidate rectangles whose object category is presumed to be a person among the detected candidate rectangles. The person region 610 can be detected using a neural network (hereinafter referred to as "NN").
  • NN neural network
  • Examples of the method for detecting the person region 610 by the NN include known methods such as Faster R-CNN, Fast R-CNN, and R-CNN.
  • the NN for detecting the person area 610 from the image 600 detects (estimates) the person area 610 from the image 600 by using the teacher data of the combination of the image 600 and the person area 610 set as the correct answer for the image 600. ) Is learned in advance.
  • FIG. 4 is a diagram showing feature points 620.
  • the control unit 110 detects the feature point 620 based on the person area 610.
  • Feature points 620 may include joint points 621 and a pair of vertices 622 of the head (eg, head rectangle).
  • the feature point 620 may further include, for example, the center point 621c of the two joint points 621a and 621b at the tip of the foot.
  • the central point 621c is calculated based on the two joint points 620a and 621b at the tip of the foot.
  • the feature point 620 can be detected by a known technique using NN such as DeepPose.
  • the feature point 620 can be detected (and calculated) as the coordinates in the image 600. Details of DeepPose are described in publicly known literature (Alexander Toshev, et al.
  • the NN for detecting the feature point 620 from the person area 610 uses the teacher data of the combination of the person area 610 and the feature point 620 set as the correct answer for the person area 610, and uses the teacher data of the combination of the person area 610 to the feature point 620. Learning for detecting (estimating) is performed in advance.
  • the feature point 620 may be estimated directly from the image 600 by using the NN for detecting the feature point 620 from the image 600.
  • the NN for detecting the feature point 620 from the image 600 uses the teacher data of the combination of the image 600 and the feature point 620 set as the correct answer for the image 600 to obtain the feature point 620 from the image 600. Learning for detection (estimation) is performed in advance.
  • the control unit 110 calculates the direction from the central region of the image 600 toward the target person 500 on the image 600 (hereinafter, also referred to as “target person direction”).
  • the center region of the image 600 includes the center of the image 600.
  • the subject direction is assumed to be the direction from the center of the image 600 toward the center of gravity of the feature point 620 of the subject 500.
  • the target person direction may be a direction from the center of the image 600 toward the center of the person area 610.
  • the subject direction may be a direction from the center of the image 600 toward the joint point 621e at the center of the waist.
  • the control unit 110 determines the behavior of the target person 500 based on the arrangement direction of the predetermined feature points 620 in the detected feature points 620 (hereinafter, also referred to as “characteristic point arrangement direction”) and the target person direction. Determine if the action is included in a predetermined action.
  • the predetermined feature points 620 are a plurality of feature points 620 arranged in the height direction of the subject 500, and from the viewpoint of the determination accuracy of whether or not they are included in the predetermined behavior, among the feature points 620 detected by the experiment. , Can be selected appropriately.
  • the feature point arrangement direction may be a direction from one of the two predetermined feature points to the other (hereinafter, also referred to as a "specific direction").
  • the feature point arrangement direction may be a direction parallel to a straight line that minimizes the sum of squares of the distances from three or more predetermined feature points.
  • the predetermined action may be a plurality of actions or a single action. Predetermined actions can include falls and falls.
  • the control unit 110 determines the feature point arrangement direction and the target person when the action of the target person 500 is detected as any one of the predetermined actions (for example, a fall) based on the person silhouette or the like. Based on the direction, it is determined (re-determined) whether or not the detected behavior corresponds to the behavior included in the predetermined behavior (for example, a fall and a fall). Whether or not the action of the subject 500 is an action included in the predetermined action is determined for the image 600 in which any one of the predetermined actions is detected based on the human silhouette.
  • FIG. 5 is a diagram showing the direction in which the feature points are arranged.
  • the feature point arrangement direction is indicated by a solid arrow in FIG.
  • the predetermined feature points are the two feature points 620 of the two joint points 620a and 620b at the tip of the foot and the joint point 621d at the center of the shoulder, and 2 of the tip of the foot.
  • the direction from the center point 621c of the two joint points 620a and 620b to the joint point 621d at the center of the shoulder is defined as the feature point alignment direction.
  • the feature point alignment direction may be the direction from the center point 621c of the two joint points 620a and 620b at the tip of the foot to the joint point 621e at the center of the waist. Further, the feature point alignment direction may be a direction from the joint point 621e at the center of the waist to the joint point 621d at the center of the shoulder.
  • the control unit 110 determines that the action of the target person 500 is an action included in the predetermined action. For example, the control unit 110 determines that the action of the target person 500 is an action included in the predetermined action when the angle formed by the feature point arrangement direction and the target person direction is equal to or more than a predetermined threshold value.
  • the predetermined threshold value can be appropriately set by an experiment from the viewpoint of the determination accuracy of whether or not the behavior of the subject 500 is included in the predetermined behavior.
  • FIG. 6 is a diagram showing the feature point arrangement direction and the target person direction in the image 600 taken by the wide-angle camera.
  • FIG. 7 is a diagram showing the relationship between the feature point arrangement direction and the target person direction when the target person 500 is walking in the image 600 taken by the wide-angle camera.
  • FIG. 8 is a diagram showing the relationship between the feature point arrangement direction and the target person direction when the target person 500 falls or falls at least in the image 600 taken by the wide-angle camera.
  • the feature point arrangement direction is shown as a solid arrow on the image 600.
  • the target person direction is shown as a broken line arrow.
  • feature points 620 are further shown along with an image of subject 500.
  • the subject 500 When the subject 500 is walking, the subject 500 is in a standing position. When the subject 500 is in either a fall or a fall behavior, the subject 500 is in a lying position. With reference to FIGS. 7 and 8, when the subject 500 is in a standing posture, the feature point arrangement direction and the subject direction are close to parallel to each other, and the angle ⁇ formed by the two is relatively small. .. When the subject 500 is in the recumbent posture, the feature point alignment direction and the subject direction approach each other orthogonally, and the angle ⁇ formed by the two becomes relatively large.
  • a fall or a fall is distinguished in the detection of a predetermined behavior based on a human silhouette, it is sufficient if it can be determined that the fall or the fall is at least one of the falls.
  • the fall and the fall are further determined by determining that the fall or the fall is at least one of the fall and the fall based on the feature point alignment direction and the subject direction. The detection accuracy can be improved.
  • the threshold value can be set according to the distance from the camera 130 to the subject 500.
  • the distance from the camera 130 to the subject 500 corresponds to the distance from the center of the image 600 to the subject 500 in the image 600. Therefore, setting a predetermined threshold value according to the distance from the camera 130 to the target person 500 means that the predetermined threshold value is set according to the distance from the center of the image 600 to the target person 500 in the image 600. Corresponding to that.
  • the range of a relatively short distance from the center of the image 600 is the first range
  • the range of a distance relatively far from the center of the image 600 is the third range
  • the range between the first range and the third range is the second range.
  • the angle that is the threshold value can be set larger in the order of the predetermined first threshold value set in the first range, the predetermined second threshold value set in the second range, and the predetermined third threshold value set in the third range.
  • the first threshold is set to 80 degrees
  • the second threshold is set to 70 degrees
  • the third threshold is set to 60 degrees.
  • the subject 500 which is closer to the center of the image 600, tends to have a smaller difference in the angle ⁇ between the feature point alignment direction and the subject direction between the standing posture and the lying posture. This is because it is difficult to determine whether or not the action corresponds to a predetermined action. This tendency occurs when the distortion at the center of the image 600 is relatively small and the distortion increases toward the periphery due to the characteristics of the wide-angle lens, such as the image 600 taken by a wide-angle camera. Especially noticeable.
  • control unit 110 Whether or not the control unit 110 is included in a predetermined action based on the relationship between the plurality of specific directions and the target person direction using a plurality of specific directions (directions from one of the two predetermined feature points to the other). Can be judged. For example, the direction formed from the center point 621c of the two joint points 620a and 620b at the tip of the foot toward the joint point 621e at the center of the waist is set as the first specific direction, and the angle formed by the first specific direction and the subject direction is predetermined. Judge whether it is above the threshold value of.
  • the angle formed by the second specific direction and the subject direction is equal to or greater than a predetermined threshold value. Then, when it is determined that any of the angles is equal to or greater than a predetermined threshold value, it can be determined that the behavior is at least one of the predetermined actions (action included in the predetermined action).
  • the control unit 110 determines that the action of the target person 500 is at least one of the predetermined actions based on the feature point arrangement direction and the target person direction
  • the control unit 110 provides information on the action of the target person 500. It is output by transmitting it to the server 200 by the communication unit 120 or the like.
  • the information regarding the behavior of the subject 500 is the first information indicating that the behavior of the subject 500 is at least one of the predetermined behaviors, or the probability (probability) of the predetermined behavior detected based on the human silhouette is high. It can be the second information indicating that.
  • the first information is, for example, information that "the behavior of the subject 500 is at least one of a fall and a fall".
  • the second information is, for example, information that "the probability of being a detected action is high".
  • the control unit 110 may further transmit the behavior specific information indicating the predetermined behavior of the target person 500, which is detected based on the human silhouette, to the server 200 or the like in association with the information regarding the behavior of the target person 500.
  • the first information, the second information, and the action specific information can be associated with each other by including information that identifies the target person 500 such as the ID (number) of the target person 500, and the shooting time of the image 600.
  • the server 200 makes a final determination that the target person 500 has performed a predetermined action detected based on the human silhouette based on the action specific information and the information on the behavior of the target person 500. obtain.
  • control unit 110 detects any of the predetermined actions of the target person 500 based on the silhouette of the person and at least one of the predetermined actions based on the relationship between the specific direction and the target person direction. When it is determined, the final determination that the subject 500 has performed a predetermined action detected based on the silhouette of the person may be made. In this case, the control unit 110 may transmit (output) the third information indicating the final determination that the target person 500 has performed a predetermined action to the server 200 or the like as information regarding the action of the target person 500.
  • the action specific information does not need to be transmitted to the server 200 or the like.
  • the third information is, for example, information that "the subject 500 has fallen".
  • the third information includes information that identifies the target person 500, such as the name of the target person 500.
  • FIG. 9 is a block diagram showing the configuration of the server 200.
  • the server 200 includes a control unit 210, a communication unit 220, and a storage unit 230. The components are connected to each other by a bus.
  • control unit 210 and the communication unit 220 The basic configuration of the control unit 210 and the communication unit 220 is the same as that of the control unit 110 and the communication unit 120, which are the corresponding components of the detection unit 100.
  • the control unit 210 receives information on the behavior of the target person 500 from the detection unit 100 by the communication unit 220.
  • the control unit 210 may further receive the action specific information from the detection unit 100.
  • the control unit 21 determines that the target person 500 indicates the behavior specific information. Make the final decision that you have acted.
  • the control unit 21 also receives the target person 500. Make a final decision that the action specific information has taken the prescribed action.
  • control unit 21 When the control unit 21 makes a final determination that the predetermined action indicated by the action specific information has been performed, the control unit 21 sends an event notification for notifying the staff or the like that the target person 500 has performed the predetermined action (for example, a fall). It can be transmitted to a mobile terminal 400 or the like.
  • the control unit 21 When the information regarding the behavior of the target person 500 is the third information indicating the final determination that the target person 500 has performed the predetermined action, the control unit 21 notifies the staff or the like that the target person 500 has performed the predetermined action.
  • the event notification for the purpose can be transmitted to the mobile terminal 400 or the like.
  • the server 200 can be implemented by substituting a part of the functions of the detection unit 100.
  • the server 200 receives the image 600 from the detection unit 100, detects the human silhouette from the image 600, and detects a predetermined action of the target person 500 based on the human silhouette.
  • a predetermined action of the target person 500 is detected, the person area 610 is detected, and the feature point 620 is detected based on the person area 610.
  • the target person direction is calculated based on the image 600 or the like, and the predetermined action of the target person 500 detected based on the human silhouette based on the feature point arrangement direction and the target person direction is included in the predetermined action. It can be determined whether or not the behavior is
  • FIG. 10 is a block diagram showing the configuration of the mobile terminal 400.
  • the mobile terminal 400 includes a control unit 410, a wireless communication unit 420, a display unit 430, an input unit 440, and a voice input / output unit 450.
  • the components are connected to each other by a bus.
  • the mobile terminal 400 may be composed of, for example, a communication terminal device such as a tablet computer, a smartphone, or a mobile phone.
  • the control unit 410 has a basic configuration such as a CPU, RAM, and ROM, similar to the configuration of the control unit 110 of the detection unit 100.
  • the wireless communication unit 420 has a function of performing wireless communication according to standards such as Wi-Fi and Bluetooth (registered trademark), and wirelessly communicates with each device via an access point or directly.
  • the wireless communication unit 420 receives the event notification from the server 200.
  • the display unit 430 and the input unit 440 are touch panels, and a touch sensor as the input unit 440 is provided on the display surface of the display unit 430 composed of a liquid crystal or the like.
  • the event notification is displayed by the display unit 430 and the input unit 440. Then, an input screen for prompting the response to the target person 500 regarding the event notification is displayed, and the staff's intention to respond to the event notification input on the input screen is received.
  • the voice input / output unit 450 is, for example, a speaker and a microphone, and enables voice communication between staff members with another mobile terminal 400 via the wireless communication unit 420. Further, the voice input / output unit 450 may have a function of enabling a voice call with the detection unit 100 via the wireless communication unit 420.
  • FIG. 11 is a flowchart showing the operation of the image recognition system 10. This flowchart is executed by the control unit 110 according to the program.
  • the control unit 110 detects the feature point 620 of the target person 500 based on the image 600 when the predetermined action of the target person 500 is detected based on the person silhouette detected from the image 600 (S101). ..
  • the control unit 110 calculates the feature point arrangement direction based on the detected feature point 620 (S102).
  • the control unit 110 calculates the target person direction based on the image 600 and the feature point 620 (S103).
  • the control unit 110 determines whether the action of the target person 500 is at least one of a fall and a fall (whether it is included in the fall and the fall). Whether or not) is determined (S104).
  • control unit 110 determines that the action of the target person 500 is neither a fall nor a fall (S105: NO), the process ends.
  • control unit 110 determines that the behavior of the target person 500 is at least one of a predetermined behavior, a fall and a fall (S105: YES)
  • the control unit 110 transmits information on the behavior of the target person to the server 200. Is output (S106).
  • the embodiment has the following effects.
  • the feature points of the subject were detected based on the captured image, and it was determined whether the behavior of the subject was included in the predetermined behavior based on the direction in which the feature points were arranged and the direction of the subject, and it was determined that the behavior was included. Sometimes it outputs information about the behavior of the subject. As a result, it is possible to improve the estimation accuracy of the behavior of the person based on the captured image.
  • the target person's action is a predetermined action. As a result, the accuracy of estimating the behavior of the person based on the captured image can be further improved.
  • the above-mentioned predetermined relationship is set according to the distance from the photographing device to the target person.
  • the behavior of the person can be estimated with high accuracy based on the image regardless of the position of the target person.
  • the direction in which the feature points are arranged is set to a specific direction from one of the two feature points to the other. As a result, it is possible to more easily improve the estimation accuracy of the behavior of the person based on the captured image.
  • the behavior of the target person is included in the predetermined behavior by using a plurality of specific directions.
  • the behavior of the person can be estimated with high accuracy based on the image for various postures belonging to the same posture of the subject.
  • the predetermined relationship is such that the angle formed by the feature point arrangement direction and the target person direction is equal to or greater than the predetermined threshold value.
  • a predetermined action is fallen and dropped, and it is determined whether or not the subject's action is at least one of a fall or a fall.
  • the accuracy of estimating the behavior of the person based on the captured image can be further improved.
  • the image is an image including a predetermined area taken by a wide-angle camera installed at a position overlooking the predetermined area. This makes it possible to more effectively improve the estimation accuracy of the behavior of the person based on the captured image.
  • the target person direction is the direction calculated based on the points included in the central region of the image and the feature points.
  • the calculation result of the feature point can be used for the calculation of the target person direction.
  • the configuration of the image recognition system 10 described above has been described as a main configuration in explaining the features of the above-described embodiment, and is not limited to the above-mentioned configuration and may be variously modified within the scope of claims. it can. Further, the configuration provided in a general image recognition system is not excluded.
  • the target person's action when the angle formed by the feature point arrangement direction and the target person direction becomes equal to or more than a predetermined threshold value, it is determined that the target person's action is included in the predetermined action.
  • the sine value of the angle formed by the feature point arrangement direction and the target person direction is calculated and the calculated sine value is equal to or more than a predetermined threshold value, the target person's action is included in the predetermined action. You may judge.
  • the detection unit 100, the server 200, and the mobile terminal 400 may each be configured by a plurality of devices, or any plurality of devices may be configured as a single device.
  • the means and methods for performing various processes in the image recognition system 10 described above can be realized by either a dedicated hardware circuit or a programmed computer.
  • the program may be provided by a computer-readable recording medium such as a USB memory or a DVD (Digital definitely Disc) -ROM, or may be provided online via a network such as the Internet.
  • the program recorded on the computer-readable recording medium is usually transferred to and stored in a storage unit such as a hard disk.
  • the above program may be provided as a single application software, or may be incorporated into the software of a device such as a detection unit as one function.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

[Problem] To provide an image processing system capable of improving deduction accuracy of a human motion based on a photographed image. [Solution] An image processing system has: a detection unit that detects feature points regarding the body of a subject on the basis of an image which has been photographed and which includes the subject; a calculation unit that calculates a direction from the center region of the image toward the subject on the basis of the image; a determination unit that determines whether a subject's motion is included in prescribed motions on the basis of the arraying direction of prescribed feature points among the feature points and the direction toward the subject; and an output unit that outputs information about the subject's motion when the subject's motion is determined to be included in the prescribed motions.

Description

画像処理システム、画像処理プログラム、および画像処理方法Image processing system, image processing program, and image processing method
　本発明は、画像処理システム、画像処理プログラム、および画像処理方法に関する。 The present invention relates to an image processing system, an image processing program, and an image processing method.
　我が国は、戦後の高度経済成長に伴う生活水準の向上、衛生環境の改善、および医療水準の向上等により、長寿命化が顕著となっている。このため、出生率の低下と相まって、高齢化率が高い高齢化社会になっている。このような高齢化社会では、病気、怪我、および加齢などにより、介護等の対応を必要とする要介護者等の増加が想定される。 In Japan, the longevity of life has become remarkable due to the improvement of living standards, the improvement of sanitary environment, and the improvement of medical standards due to the high economic growth after the war. For this reason, coupled with the decline in the birth rate, the aging society has a high aging rate. In such an aging society, it is expected that the number of people requiring long-term care will increase due to illness, injury, and aging.
　要介護者等は、病院や老人福祉施設などの施設において、歩行中に転倒したり、ベッドから転落して怪我をするおそれがある。そのため、要介護者等がこのような状態になったときに介護士や看護師等のスタッフがすぐに駆けつけられるようにするために、撮影された画像から要介護者等の状態を検出するためのシステムの開発が進められている。このようなシステムで要介護者等の状態を検出するためには、画像から検知対象である人物の姿勢や行動を高精度で検出する必要がある。 People requiring long-term care may fall while walking or fall out of bed and get injured in facilities such as hospitals and welfare facilities for the elderly. Therefore, in order to detect the condition of the care recipient from the captured image so that the staff such as the caregiver and the nurse can immediately rush to the care recipient when the care recipient becomes in such a state. System development is underway. In order to detect the state of a person requiring long-term care or the like with such a system, it is necessary to detect the posture and behavior of the person to be detected from the image with high accuracy.
　下記特許文献１には、次の先行技術が開示されている。被監視者の所定の行動を検知して通知等をする検知ユニットによる監視機能を、端末ユニットから受信した情報等に基づいて停止する。これにより、必要に応じて監視機能を停止できるため、被監視者以外の者に対する誤検知を低減できる。 The following prior art is disclosed in Patent Document 1 below. The monitoring function by the detection unit that detects the predetermined action of the monitored person and gives a notification or the like is stopped based on the information or the like received from the terminal unit. As a result, the monitoring function can be stopped as needed, so that false detections for persons other than the monitored person can be reduced.
国際公開第２０１６／１５２４２８号International Publication No. 2016/152428
　しかし、上記特許文献１に開示された先行技術は、被監視者以外の者の行動を被監視者の行動として誤検知することを防止できるが、被監視者の行動の検知精度を向上できないという問題がある。 However, the prior art disclosed in Patent Document 1 can prevent erroneous detection of the behavior of a person other than the monitored person as the behavior of the monitored person, but cannot improve the detection accuracy of the behavior of the monitored person. There's a problem.
　本発明は、このような問題を解決するためになされたものである。すなわち、撮影された画像に基づく人物の行動の推定精度を向上できる、画像処理システム、画像処理プログラム、および画像処理方法を提供することを目的とする。 The present invention has been made to solve such a problem. That is, it is an object of the present invention to provide an image processing system, an image processing program, and an image processing method capable of improving the estimation accuracy of a person's behavior based on a captured image.
　本発明の上記課題は、以下の手段によって解決される。 The above-mentioned problems of the present invention are solved by the following means.
　（１）撮影装置により撮影された、対象者を含む画像に基づいて、前記対象者の体に関する特徴点を検出する特徴点検出部と、前記画像に基づいて、前記画像の中心領域から前記対象者へ向かう方向を算出する算出部と、検出された前記特徴点の中の所定の特徴点の並び方向と、算出された前記対象者へ向かう方向とに基づいて、前記対象者の行動が所定の行動に含まれる行動かどうか判定する判定部と、前記判定部により、前記対象者の行動が前記所定の行動に含まれる行動であると判定された場合に、前記対象者の行動に関する情報を出力する出力部と、を有する画像処理システム。 (1) A feature point detection unit that detects feature points related to the target person's body based on an image including the target person taken by an imaging device, and the target from the central region of the image based on the image. The behavior of the target person is determined based on the calculation unit that calculates the direction toward the person, the arrangement direction of the predetermined feature points among the detected feature points, and the calculated direction toward the target person. When the determination unit that determines whether or not the behavior is included in the behavior of the subject and the determination unit determine that the behavior of the target person is an behavior included in the predetermined behavior, information on the behavior of the target person is obtained. An image processing system having an output unit for output.
　（２）前記判定部は、前記所定の特徴点の並び方向と、前記対象者へ向かう方向とが所定の関係にある場合に、前記対象者の行動が前記所定の行動に含まれる行動であると判定する、上記（１）に記載の画像処理システム。 (2) The determination unit is an action in which the action of the target person is included in the predetermined action when the arrangement direction of the predetermined feature points and the direction toward the target person have a predetermined relationship. The image processing system according to (1) above.
　（３）前記所定の関係は、前記撮影装置から前記対象者までの距離に応じて設定される、上記（２）に記載の画像処理システム。 (3) The image processing system according to (2) above, wherein the predetermined relationship is set according to the distance from the photographing device to the target person.
　（４）前記所定の特徴点の並び方向は、２つの前記特徴点の一方から他方へ向かう特定方向である、上記（１）～（３）のいずれかに記載の画像処理システム。 (4) The image processing system according to any one of (1) to (3) above, wherein the arrangement direction of the predetermined feature points is a specific direction from one of the two feature points to the other.
　（５）前記判定部は、複数の前記特定方向を用いて、前記対象者の行動が前記所定の行動に含まれる行動かどうか判定する、上記（４）に記載の画像処理システム。 (5) The image processing system according to (4) above, wherein the determination unit determines whether or not the behavior of the target person is included in the predetermined behavior by using the plurality of specific directions.
　（６）前記所定の関係は、前記所定の特徴点の並び方向と、前記対象者へ向かう方向とがなす角度が所定の閾値以上である、上記（２）または（３）に記載の画像処理システム。 (6) The image processing according to (2) or (3) above, wherein the predetermined relationship is such that the angle formed by the arrangement direction of the predetermined feature points and the direction toward the target person is equal to or more than a predetermined threshold value. system.
　（７）前記所定の行動は転倒および転落であり、前記判定部は、前記対象者の行動が、転倒および転落の少なくともいずれかであるかどうか判定する、上記（１）～（６）のいずれかに記載の画像処理システム。 (7) Any of the above (1) to (6), wherein the predetermined action is a fall and a fall, and the determination unit determines whether or not the action of the subject is at least one of a fall and a fall. Image processing system described in Crab.
　（８）前記撮影装置は広角カメラであり、前記画像は、所定の領域を俯瞰する位置に設置された前記広角カメラにより撮影された前記所定の領域を含む画像である、上記（１）～（７）のいずれかに記載の画像処理システム。 (8) The photographing device is a wide-angle camera, and the image is an image including the predetermined area taken by the wide-angle camera installed at a position overlooking a predetermined area. The image processing system according to any one of 7).
　（９）前記対象者へ向かう方向は、前記画像の中心領域に含まれる点と、前記所定の特徴点と、に基づいて算出される方向である、上記（１）～（８）のいずれかに記載の画像処理システム。 (9) The direction toward the target person is any of the above (1) to (8), which is a direction calculated based on the points included in the central region of the image and the predetermined feature points. The image processing system described in.
　（１０）撮影装置により撮影された、対象者を含む画像に基づいて、前記対象者の体に関する特徴点を検出する手順（ａ）と、前記画像に基づいて、前記画像の中心領域から前記対象者へ向かう方向を算出する手順（ｂ）と、検出された前記特徴点の中の所定の特徴点の並び方向と、算出された前記対象者へ向かう方向とに基づいて、前記対象者の行動が所定の行動に含まれる行動かどうか判定する手順（ｃ）と、前記手順（ｃ）において、前記対象者の行動が前記所定の行動に含まれる行動であると判定された場合に、前記対象者の行動に関する情報を出力する手順（ｄ）と、を有する処理をコンピューターに実行させるための画像処理プログラム。 (10) A procedure (a) for detecting a feature point related to the subject's body based on an image including the subject taken by an imaging device, and the subject from the central region of the image based on the image. The behavior of the target person based on the procedure (b) for calculating the direction toward the person, the arrangement direction of the predetermined feature points among the detected feature points, and the calculated direction toward the target person. In the procedure (c) for determining whether or not is an action included in the predetermined action, and when it is determined in the procedure (c) that the action of the target person is an action included in the predetermined action, the target An image processing program for causing a computer to execute a process having a procedure (d) for outputting information on a person's behavior.
　（１１）画像処理システムに実行させる方法であって、撮影装置により撮影された、対象者を含む画像に基づいて、前記対象者の体に関する特徴点を検出する段階（ａ）と、前記画像に基づいて、前記画像の中心領域から前記対象者へ向かう方向を算出する段階（ｂ）と、検出された前記特徴点の中の所定の特徴点の並び方向と、算出された前記対象者へ向かう方向とに基づいて、前記対象者の行動が所定の行動に含まれる行動かどうか判定する段階（ｃ）と、前記段階（ｃ）において、前記対象者の行動が前記所定の行動に含まれる行動であると判定された場合に、前記対象者の行動に関する情報を出力する段階（ｄ）と、を有する画像処理方法。 (11) A method of causing an image processing system to execute a step (a) of detecting a feature point related to the subject's body based on an image including the subject taken by an imaging device, and the image. Based on the step (b) of calculating the direction from the central region of the image toward the target person, the arrangement direction of the predetermined feature points in the detected feature points, and the calculated direction toward the target person. Based on the direction, the step (c) of determining whether the behavior of the target person is an action included in the predetermined action, and the action in which the behavior of the target person is included in the predetermined action in the step (c). An image processing method including a step (d) of outputting information regarding the behavior of the subject when it is determined to be.
　撮影された画像に基づいて対象者の体に関する特徴点を検出し、所定の特徴点の並び方向と、画像の中心領域から対象者へ向かう方向とに基づいて、対象者の行動が所定の行動に含まれるか判定し、含まれると判定したときに対象者の行動に関する情報を出力する。これにより、撮影された画像に基づく人物の行動の推定精度を向上できる。 The feature points related to the body of the subject are detected based on the captured image, and the behavior of the subject is a predetermined behavior based on the arrangement direction of the predetermined feature points and the direction from the central region of the image toward the subject. It is determined whether or not it is included in, and when it is determined that it is included, information on the behavior of the target person is output. As a result, it is possible to improve the estimation accuracy of the behavior of the person based on the captured image.
画像認識システムの概略構成を示す図である。It is a figure which shows the schematic structure of the image recognition system. 検出部の構成を示すブロック図である。It is a block diagram which shows the structure of the detection part. 画像において検出された人物領域を示す図である。It is a figure which shows the person area detected in the image. 特徴点を示す図である。It is a figure which shows the feature point. 特徴点並び方向を示す図である。It is a figure which shows the feature point arrangement direction. 広角カメラによる画像において、特徴点並び方向と対象者方向を示す図である。It is a figure which shows the feature point arrangement direction and the subject direction in the image by a wide-angle camera. 広角カメラによる画像において、対象者が歩行しているときの特徴点並び方向と対象者方向との関係を示す図である。It is a figure which shows the relationship between the feature point arrangement direction and the subject direction when the subject is walking in the image by a wide-angle camera. 広角カメラによる画像において、対象者が転倒および転落の少なくともいずれかをしているときの特徴点並び方向と対象者方向との関係を示す図である。It is a figure which shows the relationship between the feature point arrangement direction and the subject direction at the time of at least one of a fall and a fall in an image by a wide-angle camera. サーバーの構成を示すブロック図である。It is a block diagram which shows the configuration of a server. 携帯端末の構成を示すブロック図である。It is a block diagram which shows the structure of a mobile terminal. 画像認識システムの動作を示すフローチャートである。It is a flowchart which shows the operation of an image recognition system.
　以下、図面を参照して、本発明の実施形態に係る、画像処理システム、画像処理プログラム、および画像処理方法について説明する。なお、図面において、同一の要素には同一の符号を付し、重複する説明を省略する。図面の寸法比率は、説明の都合上誇張されており、実際の比率とは異なる場合がある。また、２つの方向がなす角度は、例えば、６０度と３００度（３６０度－６０度）というように２つの角度が観念できるが、本明細書においては、当該２つの角度のうち小さい方の角度のことを意味する。 Hereinafter, the image processing system, the image processing program, and the image processing method according to the embodiment of the present invention will be described with reference to the drawings. In the drawings, the same elements are designated by the same reference numerals, and duplicate description will be omitted. The dimensional ratios in the drawings are exaggerated for convenience of explanation and may differ from the actual ratios. Further, the angle formed by the two directions can be considered as two angles such as 60 degrees and 300 degrees (360 degrees -60 degrees), but in the present specification, the smaller of the two angles It means an angle.
　図１は、画像認識システム１０の概略構成を示す図である。 FIG. 1 is a diagram showing a schematic configuration of the image recognition system 10.
　画像認識システム１０は、検出部１００、サーバー２００、通信ネットワーク３００、および携帯端末４００を有する。検出部１００は、通信ネットワーク３００によりサーバー２００および携帯端末４００と相互に通信可能に接続される。携帯端末４００はアクセスポイント３１０を介して通信ネットワーク３００と接続され得る。検出部１００は、画像処理システムを構成する。検出部１００は、１つの一体化された装置でも、分離配置される複数の装置でもあり得る。なお、後述するように、検出部１００の機能の一部をサーバー２００が実施するようにしてもよい。 The image recognition system 10 includes a detection unit 100, a server 200, a communication network 300, and a mobile terminal 400. The detection unit 100 is communicably connected to the server 200 and the mobile terminal 400 by the communication network 300. The mobile terminal 400 may be connected to the communication network 300 via the access point 310. The detection unit 100 constitutes an image processing system. The detection unit 100 may be one integrated device or a plurality of devices separately arranged. As will be described later, the server 200 may perform a part of the functions of the detection unit 100.
　（検出部１００）
　図２は、検出部１００の構成を示すブロック図である。図２の例に示すように、検出部１００は、制御部１１０、通信部１２０、カメラ１３０、および体動センサー１４０を備え、これらはバスによって相互に接続されている。カメラ１３０は、撮影装置を構成する。 (Detection unit 100)
FIG. 2 is a block diagram showing the configuration of the detection unit 100. As shown in the example of FIG. 2, the detection unit 100 includes a control unit 110, a communication unit 120, a camera 130, and a body motion sensor 140, which are connected to each other by a bus. The camera 130 constitutes a photographing device.
　制御部１１０は、ＣＰＵ（Ｃｅｎｔｒａｌ　Ｐｒｏｃｅｓｓｉｎｇ　Ｕｎｉｔ）、およびＲＡＭ（Ｒａｎｄｏｍ　Ａｃｃｅｓｓ　Ｍｅｍｏｒｙ）、ＲＯＭ（Ｒｅａｄ　Ｏｎｌｙ　Ｍｅｍｏｒｙ）等のメモリにより構成され、プログラムに従って検出部１００の各部の制御および演算処理を行う。制御部１１０は、特徴点検出部、算出部、および判定部を構成する。制御部１１０は、通信部１２０とともに出力部を構成する。制御部１１０の作用の詳細については後述する。 The control unit 110 is composed of a CPU (Central Processing Unit) and a memory such as a RAM (Random Access Memory) and a ROM (Read Only Memory), and controls and performs arithmetic processing of each part of the detection unit 100 according to a program. The control unit 110 constitutes a feature point detection unit, a calculation unit, and a determination unit. The control unit 110 constitutes an output unit together with the communication unit 120. The details of the operation of the control unit 110 will be described later.
　通信部１２０は、通信ネットワーク３００を介して、携帯端末４００等と通信するためのインターフェース回路（例えばＬＡＮカード等）である。 The communication unit 120 is an interface circuit (for example, a LAN card or the like) for communicating with the mobile terminal 400 or the like via the communication network 300.
　カメラ１３０は、例えば広角カメラである。カメラ１３０は、検出部１００が対象者５００の居室の天井等に設置されることで、所定の領域を俯瞰する位置に設置され、当該所定の領域を含む画像（以下、単に「画像６００」とも称する）を撮影する。対象者５００は、例えばスタッフ等により介護または看護を必要とする者である。所定の領域は対象者５００の居室の床面全体を含む３次元の領域であり得る。カメラ１３０は、広角カメラより画角が狭い標準カメラであってもよい。以下、説明を簡単にするために、カメラ１３０は、広角カメラであるものとして説明する。画像６００には、対象者５００が画像として含まれ得る。画像６００には、静止画および動画が含まれる。カメラ１３０は近赤外線カメラであり、ＬＥＤ（Ｌｉｇｈｔ　Ｅｍｉｔｔｉｎｇ　Ｄｅｖｉｃｅ）により近赤外線を撮影領域に向けて照射し、撮影領域内の物体により反射される近赤外線の反射光をＣＭＯＳ（Ｃｏｍｐｌｅｍｅｍｔａｒｙ　Ｍｅｔａｌ　Ｏｘｉｄｅ　Ｓｅｍｉｃｏｎｄｕｃｔｏｒ）センサーにより受光することで所定の領域を撮影し得る。画像６００は近赤外線の反射率を各画素とするモノクロ画像であり得る。カメラ１３０は、近赤外線カメラに代替して可視光カメラを用いてもよく、これらを併用してもよい。 The camera 130 is, for example, a wide-angle camera. The camera 130 is installed at a position where the detection unit 100 is installed on the ceiling or the like of the living room of the target person 500 to overlook a predetermined area, and an image including the predetermined area (hereinafter, also simply referred to as “image 600”). Take a picture of). The target person 500 is a person who needs long-term care or nursing by, for example, a staff member. The predetermined area may be a three-dimensional area including the entire floor surface of the living room of the subject 500. The camera 130 may be a standard camera having a narrower angle of view than a wide-angle camera. Hereinafter, for the sake of simplicity, the camera 130 will be described as a wide-angle camera. The image 600 may include the subject 500 as an image. Image 600 includes still images and moving images. The camera 130 is a near-infrared camera, which irradiates near-infrared rays toward the photographing area by an LED (Light Emitting Device) and emits the reflected light of the near-infrared rays reflected by an object in the photographing area to a CMOS (Complemementary Metal Oxide Semiconductor) sensor. A predetermined area can be photographed by receiving light from the camera. The image 600 can be a monochrome image having the reflectance of near infrared rays as each pixel. As the camera 130, a visible light camera may be used instead of the near infrared camera, or these may be used in combination.
　体動センサー１４０は、ベッド７００に対してマイクロ波を送受信して対象者５００の体動（例えば呼吸動）によって生じたマイクロ波のドップラシフトを検出するドップラシフト方式のセンサーである。 The body movement sensor 140 is a doppler shift type sensor that transmits and receives microwaves to the bed 700 and detects the doppler shift of microwaves generated by the body movement (for example, respiratory movement) of the subject 500.
　制御部１１０の作用について説明する。 The operation of the control unit 110 will be described.
　制御部１１０は、画像６００から人の画像のシルエット（以下、「人シルエット」と称する）を検出する。人シルエットは、例えば、撮影時刻が前後する画像（フレーム）の差分を抽出する時間差分法により差分が相対的に大きい画素の範囲を抽出することで検出され得る。人シルエットは、撮影画像と背景画像との差分を抽出する背景差分法により検出されてもよい。制御部１１０は、人シルエットに基づいて、対象者５００の所定の行動を検出し得る。所定の行動には、例えば、転倒および転落が含まれる。制御部１１０は、例えば、検出されたシルエットの重心が、時系列で動いていた状態から急に停止した状態に変化したことや、人シルエットに対応する矩形のアスペクト比の変化等により、転倒を検出し得る。制御部１１０は、例えば、人シルエットがベッド７００の領域内に存在している状態から急にベッド７００の領域外に存在している状態に変化したことや、人シルエットに対応する矩形のアスペクト比の変化等により、転落を検出し得る。画像６００におけるベッド７００の領域は、検出部１００が設置される際に予め設定され、データとして制御部１１０のメモリに記憶され得る。 The control unit 110 detects the silhouette of a person's image (hereinafter referred to as "human silhouette") from the image 600. The human silhouette can be detected, for example, by extracting a range of pixels having a relatively large difference by the time difference method for extracting the difference between images (frames) whose shooting times are before and after. The human silhouette may be detected by the background subtraction method that extracts the difference between the photographed image and the background image. The control unit 110 can detect a predetermined action of the subject 500 based on the silhouette of the person. Predetermined actions include, for example, falls and falls. The control unit 110 may fall due to, for example, the center of gravity of the detected silhouette changing from a state in which it was moving in time series to a state in which it suddenly stopped, or a change in the aspect ratio of a rectangle corresponding to a human silhouette. Can be detected. The control unit 110, for example, has changed from a state in which the human silhouette exists in the area of the bed 700 to a state in which the person silhouette suddenly exists outside the area of the bed 700, and a rectangular aspect ratio corresponding to the human silhouette. The fall can be detected by the change of. The area of the bed 700 in the image 600 is preset when the detection unit 100 is installed, and can be stored in the memory of the control unit 110 as data.
　制御部１１０は、画像６００に基づいて、対象者５００を含む領域として、人物領域６１０を検出し、人物領域６１０から、人の体に関する特徴点（以下、単に「特徴点６２０」と称する）を検出する。 Based on the image 600, the control unit 110 detects the person area 610 as an area including the target person 500, and from the person area 610, a feature point related to the human body (hereinafter, simply referred to as “feature point 620”) is obtained. To detect.
　図３は、画像６００において検出された人物領域６１０を示す図である。 FIG. 3 is a diagram showing a person area 610 detected in the image 600.
　制御部１１０は、画像６００から、人物である対象者５００を含む領域を人物領域６１０として検出する。具体的には、制御部１１０は、画像６００上で物体（オブジェクト）が存在する領域を検出し、検出した領域に含まれる物体のカテゴリーを推定することで、人物領域６１０を検出し得る。物体が存在する領域は、画像６００上で物体が含まれる矩形（候補矩形）として検出され得る。検出部１００は、検出された候補矩形のうち、物体のカテゴリーが人物であると推定された候補矩形を検出することで、人物領域６１０を検出する。人物領域６１０は、ニューラルネットワーク（以下、「ＮＮ」と称する）を用いて検出され得る。ＮＮによる人物領域６１０の検出方法としては、例えば、Ｆａｓｔｅｒ　Ｒ－ＣＮＮ、Ｆａｓｔ　Ｒ－ＣＮＮ、およびＲ－ＣＮＮといった公知の方法が挙げられる。画像６００から人物領域６１０を検出するためのＮＮは、画像６００と、当該画像６００に対する正解として設定された人物領域６１０との組合せの教師データを用いて、画像６００から人物領域６１０を検出（推定）するための学習が予めされる。 The control unit 110 detects an area including the target person 500 who is a person as a person area 610 from the image 600. Specifically, the control unit 110 can detect the person area 610 by detecting the area where the object (object) exists on the image 600 and estimating the category of the object included in the detected area. The region where the object exists can be detected as a rectangle (candidate rectangle) including the object on the image 600. The detection unit 100 detects the person area 610 by detecting the candidate rectangles whose object category is presumed to be a person among the detected candidate rectangles. The person region 610 can be detected using a neural network (hereinafter referred to as "NN"). Examples of the method for detecting the person region 610 by the NN include known methods such as Faster R-CNN, Fast R-CNN, and R-CNN. The NN for detecting the person area 610 from the image 600 detects (estimates) the person area 610 from the image 600 by using the teacher data of the combination of the image 600 and the person area 610 set as the correct answer for the image 600. ) Is learned in advance.
　図４は、特徴点６２０を示す図である。 FIG. 4 is a diagram showing feature points 620.
　制御部１１０は、人物領域６１０に基づいて、特徴点６２０を検出する。特徴点６２０には、関節点６２１、および頭部（例えば、頭部矩形）の対頂点６２２が含まれ得る。特徴点６２０には、例えば、足の先端の２つの関節点６２１ａ、６２１ｂの中央点６２１ｃがさらに含まれ得る。当該中央点６２１ｃは、足の先端の２つの関節点６２０ａ、６２１ｂに基づいて算出される。特徴点６２０は、ＤｅｅｐＰｏｓｅ等のＮＮを用いた公知の技術により検出され得る。特徴点６２０は、画像６００における座標として検出（および算出）され得る。ＤｅｅｐＰｏｓｅについては、公知の文献（Alexander Toshev, et al. “DeepPose: Human Pose Estimation via Deep Neural Networks”, in CVPR, 2014）に詳細が記載されている。人物領域６１０から特徴点６２０を検出するためのＮＮは、人物領域６１０と、当該人物領域６１０に対する正解として設定された特徴点６２０との組合せの教師データを用いて、人物領域６１０から特徴点６２０を検出（推定）するための学習が予めされる。なお、特徴点６２０は、画像６００から特徴点６２０を検出するためのＮＮを用いて、画像６００から直接推定されてもよい。この場合、画像６００から特徴点６２０を検出するためのＮＮは、画像６００と、当該画像６００に対する正解として設定された特徴点６２０との組合せの教師データを用いて、画像６００から特徴点６２０を検出（推定）するための学習が予めされる。 The control unit 110 detects the feature point 620 based on the person area 610. Feature points 620 may include joint points 621 and a pair of vertices 622 of the head (eg, head rectangle). The feature point 620 may further include, for example, the center point 621c of the two joint points 621a and 621b at the tip of the foot. The central point 621c is calculated based on the two joint points 620a and 621b at the tip of the foot. The feature point 620 can be detected by a known technique using NN such as DeepPose. The feature point 620 can be detected (and calculated) as the coordinates in the image 600. Details of DeepPose are described in publicly known literature (Alexander Toshev, et al. “DeepPose: HumanPoseEstimation via DeepNeural Networks”, in CVPR, 2014). The NN for detecting the feature point 620 from the person area 610 uses the teacher data of the combination of the person area 610 and the feature point 620 set as the correct answer for the person area 610, and uses the teacher data of the combination of the person area 610 to the feature point 620. Learning for detecting (estimating) is performed in advance. The feature point 620 may be estimated directly from the image 600 by using the NN for detecting the feature point 620 from the image 600. In this case, the NN for detecting the feature point 620 from the image 600 uses the teacher data of the combination of the image 600 and the feature point 620 set as the correct answer for the image 600 to obtain the feature point 620 from the image 600. Learning for detection (estimation) is performed in advance.
　制御部１１０は、画像６００に基づいて、画像６００の中心領域から、画像６００上の対象者５００へ向かう方向（以下、「対象者方向」とも称する）を算出する。画像６００の中心領域には、画像６００の中心が含まれる。以下、説明を簡単にするために、対象者方向は、画像６００の中心から対象者５００の特徴点６２０の重心に向かう方向であるものとして説明する。なお、対象者方向は、画像６００の中心から人物領域６１０の中心に向かう方向であってもよい。また、対象者方向は、画像６００の中心から腰の中心の関節点６２１ｅに向かう方向等であってもよい。 Based on the image 600, the control unit 110 calculates the direction from the central region of the image 600 toward the target person 500 on the image 600 (hereinafter, also referred to as “target person direction”). The center region of the image 600 includes the center of the image 600. Hereinafter, for the sake of simplicity, the subject direction is assumed to be the direction from the center of the image 600 toward the center of gravity of the feature point 620 of the subject 500. The target person direction may be a direction from the center of the image 600 toward the center of the person area 610. Further, the subject direction may be a direction from the center of the image 600 toward the joint point 621e at the center of the waist.
　制御部１１０は、検出された特徴点６２０の中の所定の特徴点６２０の並び方向（以下、「特徴点並び方向」とも称する）と、対象者方向とに基づいて、対象者５００の行動が所定の行動に含まれる行動かどうかを判定する。所定の特徴点６２０は、対象者５００の身長方向に並ぶ複数の特徴点６２０であり、所定の行動に含まれるかどうかの判定精度の観点から、実験により、検出された特徴点６２０の中から、適当に選択され得る。特徴点並び方向は、２つの所定の特徴点の一方から他方へ向かう方向（以下、「特定方向」とも称する）であり得る。特徴点並び方向は、３つ以上の所定の特徴点からの距離の２乗和が最小となる直線と平行な方向であってもよい。所定の行動は、複数の行動であっても単一の行動であってもよい。所定の行動には、転倒および転落が含まれ得る。具体的には、制御部１１０は、人シルエット等に基づいて、対象者５００の行動が所定の行動の中のいずれか（例えば、転倒）として検出された場合に、特徴点並び方向と対象者方向とに基づいて、検出された行動が、所定の行動に含まれる行動（例えば、転倒および転落）に該当するかどうかを判定（再判断）する。対象者５００の行動が、所定の行動に含まれる行動かどうかの判定は、人シルエットに基づいて所定の行動のいずれか１つが検出された画像６００に対して行われる。 The control unit 110 determines the behavior of the target person 500 based on the arrangement direction of the predetermined feature points 620 in the detected feature points 620 (hereinafter, also referred to as “characteristic point arrangement direction”) and the target person direction. Determine if the action is included in a predetermined action. The predetermined feature points 620 are a plurality of feature points 620 arranged in the height direction of the subject 500, and from the viewpoint of the determination accuracy of whether or not they are included in the predetermined behavior, among the feature points 620 detected by the experiment. , Can be selected appropriately. The feature point arrangement direction may be a direction from one of the two predetermined feature points to the other (hereinafter, also referred to as a "specific direction"). The feature point arrangement direction may be a direction parallel to a straight line that minimizes the sum of squares of the distances from three or more predetermined feature points. The predetermined action may be a plurality of actions or a single action. Predetermined actions can include falls and falls. Specifically, the control unit 110 determines the feature point arrangement direction and the target person when the action of the target person 500 is detected as any one of the predetermined actions (for example, a fall) based on the person silhouette or the like. Based on the direction, it is determined (re-determined) whether or not the detected behavior corresponds to the behavior included in the predetermined behavior (for example, a fall and a fall). Whether or not the action of the subject 500 is an action included in the predetermined action is determined for the image 600 in which any one of the predetermined actions is detected based on the human silhouette.
　図５は、特徴点並び方向を示す図である。特徴点並び方向は、図５において、実線の矢印で示されている。 FIG. 5 is a diagram showing the direction in which the feature points are arranged. The feature point arrangement direction is indicated by a solid arrow in FIG.
　図５の例においては、所定の特徴点を、足の先端の２つの関節点６２０ａ、６２０ｂの中央点６２１ｃと、肩の中心の関節点６２１ｄの２つの特徴点６２０とし、足の先端の２つの関節点６２０ａ、６２０ｂの中央点６２１ｃから肩の中心の関節点６２１ｄへ向かう方向を特徴点並び方向としている。特徴点並び方向は、足の先端の２つの関節点６２０ａ、６２０ｂの中央点６２１ｃから腰の中心の関節点６２１ｅへ向かう方向であってもよい。また、特徴点並び方向は、腰の中心の関節点６２１ｅから肩の中心の関節点６２１ｄへ向かう方向等であってもよい。 In the example of FIG. 5, the predetermined feature points are the two feature points 620 of the two joint points 620a and 620b at the tip of the foot and the joint point 621d at the center of the shoulder, and 2 of the tip of the foot. The direction from the center point 621c of the two joint points 620a and 620b to the joint point 621d at the center of the shoulder is defined as the feature point alignment direction. The feature point alignment direction may be the direction from the center point 621c of the two joint points 620a and 620b at the tip of the foot to the joint point 621e at the center of the waist. Further, the feature point alignment direction may be a direction from the joint point 621e at the center of the waist to the joint point 621d at the center of the shoulder.
　特徴点並び方向と、対象者方向とに基づく、対象者５００の行動が所定の行動に含まれるかどうかの判定方法について、さらに詳細に説明する。 The method of determining whether or not the action of the target person 500 is included in the predetermined action based on the feature point arrangement direction and the target person direction will be described in more detail.
　制御部１１０は、特徴点並び方向と、対象者方向とが所定の関係にある場合に、対象者５００の行動が、所定の行動に含まれる行動であると判定する。例えば、制御部１１０は、特徴点並び方向と、対象者方向とがなす角度が所定の閾値以上である場合に、対象者５００の行動が、所定の行動に含まれる行動であると判定する。所定の閾値は、対象者５００の行動が、所定の行動に含まれるかどうかの判定精度の観点から、実験により、適当に設定し得る。 When the feature point arrangement direction and the target person direction have a predetermined relationship, the control unit 110 determines that the action of the target person 500 is an action included in the predetermined action. For example, the control unit 110 determines that the action of the target person 500 is an action included in the predetermined action when the angle formed by the feature point arrangement direction and the target person direction is equal to or more than a predetermined threshold value. The predetermined threshold value can be appropriately set by an experiment from the viewpoint of the determination accuracy of whether or not the behavior of the subject 500 is included in the predetermined behavior.
　図６は、広角カメラによる画像６００において、特徴点並び方向と対象者方向を示す図である。図７は、広角カメラによる画像６００において、対象者５００が歩行しているときの特徴点並び方向と対象者方向との関係を示す図である。図８は、広角カメラによる画像６００において、対象者５００が転倒および転落の少なくともいずれかをしているときの特徴点並び方向と対象者方向との関係を示す図である。これらの図においては、特徴点並び方向が、画像６００上の実線の矢印として示されている。また、対象者方向が、破線の矢印として示されている。図６においては、特徴点６２０が、対象者５００の画像とともにさらに示されている。 FIG. 6 is a diagram showing the feature point arrangement direction and the target person direction in the image 600 taken by the wide-angle camera. FIG. 7 is a diagram showing the relationship between the feature point arrangement direction and the target person direction when the target person 500 is walking in the image 600 taken by the wide-angle camera. FIG. 8 is a diagram showing the relationship between the feature point arrangement direction and the target person direction when the target person 500 falls or falls at least in the image 600 taken by the wide-angle camera. In these figures, the feature point arrangement direction is shown as a solid arrow on the image 600. In addition, the target person direction is shown as a broken line arrow. In FIG. 6, feature points 620 are further shown along with an image of subject 500.
　対象者５００が歩行という行動をしている場合は、対象者５００は立位の姿勢にある。対象者５００が転倒および転落のいずれかの行動をしている場合は、対象者５００は臥位の姿勢にある。図７および図８を参照すると、対象者５００が立位の姿勢である場合は、特徴点並び方向と対象者方向とは互いに平行に近い方向になり、両者のなす角度θは比較的小さくなる。対象者５００が臥位の姿勢である場合は、特徴点並び方向と対象者方向とは互いに直交に近づき、両者のなす角度θは比較的大きくなる。この傾向は、広角カメラによる画像６００では、広角カメラの広角レンズのもつ歪み特性に起因してより顕著になるが、画像６００が比較的広い領域を俯瞰して撮影された画像であれば、標準カメラによる画像６００であっても、同様な傾向が見られる。従って、特徴点並び方向と、対象者方向とがなす角度θが所定の閾値以上であれば、対象者５００の行動が、所定の行動である転倒および転落の少なくともいずれかであると判定できる。なお、特徴点並び方向と、対象者方向とに基づく判定では、転倒および転落を区別しない。しかし、転倒か転落かは、人シルエットに基づく所定の行動の検出において区別されているため、転倒および転落の少なくともいずれかであると判定できれば十分である。人シルエットに基づいて転倒または転落が検出されたときに、さらに、特徴点並び方向と、対象者方向とに基づいて、転倒および転落の少なくともいずれかであると判定することにより、転倒および転落の検出精度を向上できる。 When the subject 500 is walking, the subject 500 is in a standing position. When the subject 500 is in either a fall or a fall behavior, the subject 500 is in a lying position. With reference to FIGS. 7 and 8, when the subject 500 is in a standing posture, the feature point arrangement direction and the subject direction are close to parallel to each other, and the angle θ formed by the two is relatively small. .. When the subject 500 is in the recumbent posture, the feature point alignment direction and the subject direction approach each other orthogonally, and the angle θ formed by the two becomes relatively large. This tendency becomes more remarkable in the image 600 taken by the wide-angle camera due to the distortion characteristic of the wide-angle lens of the wide-angle camera, but if the image 600 is an image taken from a bird's-eye view of a relatively wide area, it is standard. The same tendency can be seen in the image 600 taken by the camera. Therefore, if the angle θ formed by the feature point arrangement direction and the target person direction is equal to or greater than a predetermined threshold value, it can be determined that the action of the target person 500 is at least one of the predetermined actions of falling and falling. It should be noted that the judgment based on the feature point arrangement direction and the subject direction does not distinguish between a fall and a fall. However, since a fall or a fall is distinguished in the detection of a predetermined behavior based on a human silhouette, it is sufficient if it can be determined that the fall or the fall is at least one of the falls. When a fall or fall is detected based on the silhouette of a person, the fall and the fall are further determined by determining that the fall or the fall is at least one of the fall and the fall based on the feature point alignment direction and the subject direction. The detection accuracy can be improved.
　上述した、対象者５００の行動が、所定の行動である、転倒および転落の少なくともいずれかであると判定するための、特徴点並び方向と、対象者方向とがなす角度θに設定される所定の閾値は、カメラ１３０から対象者５００までの距離に応じて設定され得る。カメラ１３０から対象者５００までの距離は、画像６００における、画像６００の中心から対象者５００までの距離に対応する。従って、所定の閾値が、カメラ１３０から対象者５００までの距離に応じて設定されることは、所定の閾値が、画像６００における、画像６００の中心から対象者５００までの距離に応じて設定されることに対応する。例えば、画像６００の中心から比較的短い距離の範囲内を第１範囲、画像６００の中心から比較的遠い距離の範囲内を第３範囲、第１範囲と第３範囲の間の範囲を第２範囲とする。そして、第１範囲に設定する所定の第１閾値、第２範囲に設定する所定の第２閾値、第３範囲に設定する所定の第３閾値の順に、閾値である角度を大きく設定し得る。例えば、第１閾値を８０度、第２閾値を７０度、第３閾値を６０度に設定する。これにより、画像６００の中心からの距離が短いほど所定の行動に含まれると判定する基準を厳しい基準にする。これは、画像６００の中心に近い対象者５００の方が、立位の姿勢と、臥位の姿勢とにおける、特徴点並び方向と、対象者方向とがなす角度θの差が小さくなる傾向があるため、所定の行動に該当するかどうかの判定が困難になるからである。この傾向は、広角カメラで撮影された画像６００のように、広角レンズの特性に起因して、画像６００の中心の歪みは比較的小さく、周辺に行くに従い歪みが大きくなっているような場合に特に顕著になる。 A predetermined angle set between the feature point alignment direction and the target person direction for determining that the action of the target person 500 described above is at least one of a predetermined action, a fall and a fall. The threshold value can be set according to the distance from the camera 130 to the subject 500. The distance from the camera 130 to the subject 500 corresponds to the distance from the center of the image 600 to the subject 500 in the image 600. Therefore, setting a predetermined threshold value according to the distance from the camera 130 to the target person 500 means that the predetermined threshold value is set according to the distance from the center of the image 600 to the target person 500 in the image 600. Corresponding to that. For example, the range of a relatively short distance from the center of the image 600 is the first range, the range of a distance relatively far from the center of the image 600 is the third range, and the range between the first range and the third range is the second range. The range. Then, the angle that is the threshold value can be set larger in the order of the predetermined first threshold value set in the first range, the predetermined second threshold value set in the second range, and the predetermined third threshold value set in the third range. For example, the first threshold is set to 80 degrees, the second threshold is set to 70 degrees, and the third threshold is set to 60 degrees. As a result, the shorter the distance from the center of the image 600, the stricter the criterion for determining that the image 600 is included in the predetermined action. This is because the subject 500, which is closer to the center of the image 600, tends to have a smaller difference in the angle θ between the feature point alignment direction and the subject direction between the standing posture and the lying posture. This is because it is difficult to determine whether or not the action corresponds to a predetermined action. This tendency occurs when the distortion at the center of the image 600 is relatively small and the distortion increases toward the periphery due to the characteristics of the wide-angle lens, such as the image 600 taken by a wide-angle camera. Especially noticeable.
　制御部１１０は、複数の特定方向（２つの所定の特徴点の一方から他方へ向かう方向）を用い、複数の特定方向と対象者方向との関係に基づいて、所定の行動に含まれるかどうか判定し得る。例えば、足の先端の２つの関節点６２０ａ、６２０ｂの中央点６２１ｃから腰の中心の関節点６２１ｅへ向かう方向を第１特定方向として、当該第１特定方向と対象者方向とのなす角度が所定の閾値以上であるかどうか判断する。腰の中心の関節点６２１ｅから肩の中心の関節点６２１ｄへ向かう方向を第２特定方向として、当該第２特定方向と対象者方向とのなす角度が所定の閾値以上であるかどうか判断する。そして、いずれかの角度が所定の閾値以上であると判断した場合に、所定の行動の少なくともいずれかの行動（所定の行動に含まれる行動）であると判定し得る。 Whether or not the control unit 110 is included in a predetermined action based on the relationship between the plurality of specific directions and the target person direction using a plurality of specific directions (directions from one of the two predetermined feature points to the other). Can be judged. For example, the direction formed from the center point 621c of the two joint points 620a and 620b at the tip of the foot toward the joint point 621e at the center of the waist is set as the first specific direction, and the angle formed by the first specific direction and the subject direction is predetermined. Judge whether it is above the threshold value of. With the direction from the joint point 621e at the center of the waist to the joint point 621d at the center of the shoulder as the second specific direction, it is determined whether or not the angle formed by the second specific direction and the subject direction is equal to or greater than a predetermined threshold value. Then, when it is determined that any of the angles is equal to or greater than a predetermined threshold value, it can be determined that the behavior is at least one of the predetermined actions (action included in the predetermined action).
　制御部１１０は、特徴点並び方向と、対象者方向とに基づいて、対象者５００の行動が所定の行動の少なくともいずれかの行動であると判定した場合、対象者５００の行動に関する情報を、通信部１２０によりサーバー２００に送信すること等により、出力する。対象者５００の行動に関する情報は、対象者５００の行動が所定の行動の少なくともいずれかであることを示す第１情報、または、人シルエットに基づき検出された所定の行動の確度（確率）が高いことを示す第２情報であり得る。第１情報は、例えば、「対象者５００の行動が、転倒および転落の少なくともいずれかである」という情報である。第２情報は、例えば、「検出された行動である確率が高い」という情報である。制御部１１０は、対象者５００の行動に関する情報と関連付けて、人シルエットに基づいて検出された、対象者５００の所定の行動を示す行動特定情報をさらにサーバー２００等に送信し得る。第１情報、第２情報、および行動特定情報には、対象者５００のＩＤ（番号）等の対象者５００を特定する情報、および画像６００の撮影時間等が含まれることで関連付けされ得る。後述するように、サーバー２００において、行動特定情報と、対象者５００の行動に関する情報と、に基づいて、対象者５００が、人シルエットに基づいて検出された所定の行動をしたという最終判断がされ得る。 When the control unit 110 determines that the action of the target person 500 is at least one of the predetermined actions based on the feature point arrangement direction and the target person direction, the control unit 110 provides information on the action of the target person 500. It is output by transmitting it to the server 200 by the communication unit 120 or the like. The information regarding the behavior of the subject 500 is the first information indicating that the behavior of the subject 500 is at least one of the predetermined behaviors, or the probability (probability) of the predetermined behavior detected based on the human silhouette is high. It can be the second information indicating that. The first information is, for example, information that "the behavior of the subject 500 is at least one of a fall and a fall". The second information is, for example, information that "the probability of being a detected action is high". The control unit 110 may further transmit the behavior specific information indicating the predetermined behavior of the target person 500, which is detected based on the human silhouette, to the server 200 or the like in association with the information regarding the behavior of the target person 500. The first information, the second information, and the action specific information can be associated with each other by including information that identifies the target person 500 such as the ID (number) of the target person 500, and the shooting time of the image 600. As will be described later, the server 200 makes a final determination that the target person 500 has performed a predetermined action detected based on the human silhouette based on the action specific information and the information on the behavior of the target person 500. obtain.
　一方、制御部１１０が、人シルエットに基づいて対象者５００の所定の行動のいずれかが検出され、かつ、特定方向と対象者方向との関係に基づいて所定の行動の少なくともいずれかであると判定された場合に、対象者５００が人シルエットに基づいて検出された所定の行動をしたという最終判断をしてもよい。この場合、制御部１１０は、対象者５００が所定の行動をしたという最終判断を示す第３情報を、対象者５００の行動に関する情報としてサーバー２００等に送信（出力）し得る。なお、行動特定情報はサーバー２００等に送信される必要はない。第３情報は、例えば、「対象者５００が転倒した」という情報である。第３情報には、対象者５００の氏名等の対象者５００を特定する情報が含まれる。 On the other hand, when the control unit 110 detects any of the predetermined actions of the target person 500 based on the silhouette of the person and at least one of the predetermined actions based on the relationship between the specific direction and the target person direction. When it is determined, the final determination that the subject 500 has performed a predetermined action detected based on the silhouette of the person may be made. In this case, the control unit 110 may transmit (output) the third information indicating the final determination that the target person 500 has performed a predetermined action to the server 200 or the like as information regarding the action of the target person 500. The action specific information does not need to be transmitted to the server 200 or the like. The third information is, for example, information that "the subject 500 has fallen". The third information includes information that identifies the target person 500, such as the name of the target person 500.
　（サーバー２００）
　図９は、サーバー２００の構成を示すブロック図である。サーバー２００は、制御部２１０、通信部２２０、および記憶部２３０を備える。各構成要素は、バスによって、相互に接続されている。 (Server 200)
FIG. 9 is a block diagram showing the configuration of the server 200. The server 200 includes a control unit 210, a communication unit 220, and a storage unit 230. The components are connected to each other by a bus.
　制御部２１０および通信部２２０の基本構成は、検出部１００の対応する構成要素である、制御部１１０および通信部１２０と同様である。 The basic configuration of the control unit 210 and the communication unit 220 is the same as that of the control unit 110 and the communication unit 120, which are the corresponding components of the detection unit 100.
　制御部２１０は、通信部２２０により、検出部１００から対象者５００の行動に関する情報を受信する。制御部２１０は、検出部１００から行動特定情報をさらに受信し得る。 The control unit 210 receives information on the behavior of the target person 500 from the detection unit 100 by the communication unit 220. The control unit 210 may further receive the action specific information from the detection unit 100.
　制御部２１は、対象者５００の行動に関する情報が、対象者５００の行動が所定の行動の少なくともいずれかであることを示す第１情報である場合、対象者５００が、行動特定情報が示す所定の行動をしたという最終判断をする。制御部２１は、対象者５００の行動に関する情報が、人シルエットに基づき検出された所定の行動の確度（確率）が高いことを示す第２情報である場合も、同様に、対象者５００が、行動特定情報が示す所定の行動をしたという最終判断をする。制御部２１は、行動特定情報が示す所定の行動をしたという最終判断をしたときに、対象者５００が所定の行動（例えば、転倒）をしたことをスタッフ等に通知するためのイベント通知を、携帯端末４００等に送信し得る。 When the information regarding the behavior of the target person 500 is the first information indicating that the behavior of the target person 500 is at least one of the predetermined behaviors, the control unit 21 determines that the target person 500 indicates the behavior specific information. Make the final decision that you have acted. Similarly, when the information regarding the behavior of the target person 500 is the second information indicating that the certainty (probability) of the predetermined behavior detected based on the human silhouette is high, the control unit 21 also receives the target person 500. Make a final decision that the action specific information has taken the prescribed action. When the control unit 21 makes a final determination that the predetermined action indicated by the action specific information has been performed, the control unit 21 sends an event notification for notifying the staff or the like that the target person 500 has performed the predetermined action (for example, a fall). It can be transmitted to a mobile terminal 400 or the like.
　制御部２１は、対象者５００の行動に関する情報が、対象者５００の所定の行動をしたという最終判断を示す第３情報である場合、対象者５００が所定の行動をしたことをスタッフ等に通知するためのイベント通知を、携帯端末４００等に送信し得る。 When the information regarding the behavior of the target person 500 is the third information indicating the final determination that the target person 500 has performed the predetermined action, the control unit 21 notifies the staff or the like that the target person 500 has performed the predetermined action. The event notification for the purpose can be transmitted to the mobile terminal 400 or the like.
　なお、サーバー２００は、検出部１００の機能の一部を代替して実施し得る。例えば、サーバー２００は、検出部１００から画像６００を受信し、画像６００から人シルエットを検出し、人シルエットに基づいて、対象者５００の所定の行動を検出する。対象者５００の所定の行動が検出されたときに、人物領域６１０を検出して、人物領域６１０に基づいて、特徴点６２０を検出する。そして、画像６００等に基づいて対象者方向を算出し、特徴点並び方向と対象者方向とに基づいて、人シルエットに基づいて検出された対象者５００の所定の行動が、所定の行動に含まれる行動かどうか判定し得る。 Note that the server 200 can be implemented by substituting a part of the functions of the detection unit 100. For example, the server 200 receives the image 600 from the detection unit 100, detects the human silhouette from the image 600, and detects a predetermined action of the target person 500 based on the human silhouette. When a predetermined action of the target person 500 is detected, the person area 610 is detected, and the feature point 620 is detected based on the person area 610. Then, the target person direction is calculated based on the image 600 or the like, and the predetermined action of the target person 500 detected based on the human silhouette based on the feature point arrangement direction and the target person direction is included in the predetermined action. It can be determined whether or not the behavior is
　（携帯端末４００）
　図１０は、携帯端末４００の構成を示すブロック図である。携帯端末４００は、制御部４１０、無線通信部４２０、表示部４３０、入力部４４０、および音声入出力部４５０を備える。各構成要素は、バスにより相互に接続されている。携帯端末４００は、例えば、タブレット型コンピューター、スマートフォン、または携帯電話等の通信端末機器によって構成され得る。 (Mobile terminal 400)
FIG. 10 is a block diagram showing the configuration of the mobile terminal 400. The mobile terminal 400 includes a control unit 410, a wireless communication unit 420, a display unit 430, an input unit 440, and a voice input / output unit 450. The components are connected to each other by a bus. The mobile terminal 400 may be composed of, for example, a communication terminal device such as a tablet computer, a smartphone, or a mobile phone.
　制御部４１０は、検出部１００の制御部１１０の構成と同様に、ＣＰＵ、ＲＡＭ、ＲＯＭなどの基本構成を備える。 The control unit 410 has a basic configuration such as a CPU, RAM, and ROM, similar to the configuration of the control unit 110 of the detection unit 100.
　無線通信部４２０は、Ｗｉ－Ｆｉ、Ｂｌｕｅｔｏｏｔｈ（登録商標）などの規格による無線通信を行う機能を有し、アクセスポイントを経由して、または直接に各装置と無線通信する。無線通信部４２０は、イベント通知をサーバー２００から受信する。 The wireless communication unit 420 has a function of performing wireless communication according to standards such as Wi-Fi and Bluetooth (registered trademark), and wirelessly communicates with each device via an access point or directly. The wireless communication unit 420 receives the event notification from the server 200.
　表示部４３０および入力部４４０は、タッチパネルであり、液晶などで構成される表示部４３０の表示面に、入力部４４０としてのタッチセンサーが設けられる。表示部４３０、入力部４４０によって、イベント通知を表示する。そして、イベント通知に関する対象者５００への対応を促す入力画面を表示するとともに、当該入力画面に入力された、スタッフによるイベント通知への対応の意思を受け付ける。 The display unit 430 and the input unit 440 are touch panels, and a touch sensor as the input unit 440 is provided on the display surface of the display unit 430 composed of a liquid crystal or the like. The event notification is displayed by the display unit 430 and the input unit 440. Then, an input screen for prompting the response to the target person 500 regarding the event notification is displayed, and the staff's intention to respond to the event notification input on the input screen is received.
　音声入出力部４５０は、例えばスピーカーとマイクであり、無線通信部４２０を介して他の携帯端末４００との間でスタッフ相互間の音声通話を可能にする。また、音声入出力部４５０は、無線通信部４２０を介して検出部１００との間で音声通話を可能にする機能を備え得る。 The voice input / output unit 450 is, for example, a speaker and a microphone, and enables voice communication between staff members with another mobile terminal 400 via the wireless communication unit 420. Further, the voice input / output unit 450 may have a function of enabling a voice call with the detection unit 100 via the wireless communication unit 420.
　画像認識システム１０の動作について説明する。 The operation of the image recognition system 10 will be described.
　図１１は、画像認識システム１０の動作を示すフローチャートである。本フローチャートは、プログラムに従い、制御部１１０により実行される。 FIG. 11 is a flowchart showing the operation of the image recognition system 10. This flowchart is executed by the control unit 110 according to the program.
　制御部１１０は、画像６００から検出した人シルエットに基づいて、対象者５００の所定の行動を検出したことを契機に、画像６００に基づいて、対象者５００の特徴点６２０を検出する（Ｓ１０１）。 The control unit 110 detects the feature point 620 of the target person 500 based on the image 600 when the predetermined action of the target person 500 is detected based on the person silhouette detected from the image 600 (S101). ..
　制御部１１０は、検出された特徴点６２０に基づいて、特徴点並び方向を算出する（Ｓ１０２）。 The control unit 110 calculates the feature point arrangement direction based on the detected feature point 620 (S102).
　制御部１１０は、画像６００および特徴点６２０に基づいて、対象者方向を算出する（Ｓ１０３）。 The control unit 110 calculates the target person direction based on the image 600 and the feature point 620 (S103).
　制御部１１０は、特徴点並び方向と、対象者方向とに基づいて、対象者５００の行動が所定の行動である、転倒および転落の少なくともいずれかであるかどうか（転倒および転落に含まれるかどうか）を判定する（Ｓ１０４）。 Based on the feature point alignment direction and the target person direction, the control unit 110 determines whether the action of the target person 500 is at least one of a fall and a fall (whether it is included in the fall and the fall). Whether or not) is determined (S104).
　制御部１１０は、対象者５００の行動が所定の行動である、転倒および転落のいずれでもないと判定した場合は（Ｓ１０５：ＮＯ）、処理を終了する。 When the control unit 110 determines that the action of the target person 500 is neither a fall nor a fall (S105: NO), the process ends.
　制御部１１０は、対象者５００の行動が所定の行動である、転倒および転落の少なくともいずれかであると判定した場合は（Ｓ１０５：ＹＥＳ）、対象者の行動に関する情報をサーバー２００へ送信することで出力する（Ｓ１０６）。 When the control unit 110 determines that the behavior of the target person 500 is at least one of a predetermined behavior, a fall and a fall (S105: YES), the control unit 110 transmits information on the behavior of the target person to the server 200. Is output (S106).
　実施形態は以下の効果を奏する。 The embodiment has the following effects.
　撮影された画像に基づいて対象者の特徴点を検出し、特徴点並び方向と、対象者方向とに基づいて、対象者の行動が所定の行動に含まれるか判定し、含まれると判定したときに対象者の行動に関する情報を出力する。これにより、撮影された画像に基づく人物の行動の推定精度を向上できる。 The feature points of the subject were detected based on the captured image, and it was determined whether the behavior of the subject was included in the predetermined behavior based on the direction in which the feature points were arranged and the direction of the subject, and it was determined that the behavior was included. Sometimes it outputs information about the behavior of the subject. As a result, it is possible to improve the estimation accuracy of the behavior of the person based on the captured image.
　さらに、特徴点並び方向と、対象者方向とが所定の関係にある場合に、対象者の行動が所定の行動であると判定する。これにより、撮影された画像に基づく人物の行動の推定精度をさらに向上できる。 Further, when the feature point arrangement direction and the target person direction have a predetermined relationship, it is determined that the target person's action is a predetermined action. As a result, the accuracy of estimating the behavior of the person based on the captured image can be further improved.
　さらに、上記所定の関係を、撮影装置から対象者までの距離に応じて設定する。これにより、対象者の位置によらず、画像に基づいて人物の行動を高精度に推定できる。 Furthermore, the above-mentioned predetermined relationship is set according to the distance from the photographing device to the target person. As a result, the behavior of the person can be estimated with high accuracy based on the image regardless of the position of the target person.
　さらに、特徴点並び方向を、２つの特徴点の一方から他方へ向かう特定方向とする。これにより、より簡単に、撮影された画像に基づく人物の行動の推定精度を向上できる。 Furthermore, the direction in which the feature points are arranged is set to a specific direction from one of the two feature points to the other. As a result, it is possible to more easily improve the estimation accuracy of the behavior of the person based on the captured image.
　さらに、複数の特定方向を用いて、対象者の行動が所定の行動に含まれるか判定する。これにより、対象者の同じ姿勢に属する様々な体勢について、画像に基づいて人物の行動を高精度に推定できる。 Furthermore, it is determined whether the behavior of the target person is included in the predetermined behavior by using a plurality of specific directions. As a result, the behavior of the person can be estimated with high accuracy based on the image for various postures belonging to the same posture of the subject.
　さらに、所定の関係を、特徴点並び方向と、対象者方向とがなす角度が所定の閾値以上とする。これにより、さらに簡単に、撮影された画像に基づく人物の行動の推定精度を向上できる。 Furthermore, the predetermined relationship is such that the angle formed by the feature point arrangement direction and the target person direction is equal to or greater than the predetermined threshold value. As a result, it is possible to more easily improve the estimation accuracy of the behavior of the person based on the captured image.
　さらに、所定の行動を転倒および転落とし、対象者の行動が、転倒および転落の少なくともいずれかであるかどうかを判定する。これにより、撮影された画像に基づく人物の行動の推定精度をさらに向上できる。 Furthermore, a predetermined action is fallen and dropped, and it is determined whether or not the subject's action is at least one of a fall or a fall. As a result, the accuracy of estimating the behavior of the person based on the captured image can be further improved.
　さらに、画像を、所定の領域を俯瞰する位置に設置された広角カメラにより撮影された所定の領域を含む画像とする。これにより、より効果的に、撮影された画像に基づく人物の行動の推定精度を向上できる。 Further, the image is an image including a predetermined area taken by a wide-angle camera installed at a position overlooking the predetermined area. This makes it possible to more effectively improve the estimation accuracy of the behavior of the person based on the captured image.
　さらに、対象者方向を、画像の中心領域に含まれる点と、特徴点とに基づいて算出される方向とする。これにより、対象者方向の算出に特徴点の算出結果を利用できる。 Furthermore, the target person direction is the direction calculated based on the points included in the central region of the image and the feature points. As a result, the calculation result of the feature point can be used for the calculation of the target person direction.
　以上に説明した画像認識システム１０の構成は、上述の実施形態の特徴を説明するにあたって主要構成を説明したのであって、上述の構成に限られず、特許請求の範囲内において、種々改変することができる。また、一般的な画像認識システムが備える構成を排除するものではない。 The configuration of the image recognition system 10 described above has been described as a main configuration in explaining the features of the above-described embodiment, and is not limited to the above-mentioned configuration and may be variously modified within the scope of claims. it can. Further, the configuration provided in a general image recognition system is not excluded.
　例えば、実施形態においては、特徴点並び方向と、対象者方向とがなす角度が所定の閾値以上となった場合に、対象者の行動が所定の行動に含まれると判定している。しかし、例えば、特徴点並び方向と、対象者方向とがなす角度の正弦値を算出し、算出した正弦値が所定の閾値以上である場合に、対象者の行動が所定の行動に含まれると判定してもよい。 For example, in the embodiment, when the angle formed by the feature point arrangement direction and the target person direction becomes equal to or more than a predetermined threshold value, it is determined that the target person's action is included in the predetermined action. However, for example, when the sine value of the angle formed by the feature point arrangement direction and the target person direction is calculated and the calculated sine value is equal to or more than a predetermined threshold value, the target person's action is included in the predetermined action. You may judge.
　また、検出部１００、サーバー２００、および携帯端末４００は、それぞれ複数の装置により構成されてもよく、いずれか複数の装置が単一の装置として構成されてもよい。 Further, the detection unit 100, the server 200, and the mobile terminal 400 may each be configured by a plurality of devices, or any plurality of devices may be configured as a single device.
　また、上述したフローチャートは、一部のステップを省略してもよく、他のステップが追加されてもよい。また各ステップの一部は同時に実行されてもよく、一つのステップが複数のステップに分割されて実行されてもよい。 Further, in the above-mentioned flowchart, some steps may be omitted or other steps may be added. Further, a part of each step may be executed at the same time, or one step may be divided into a plurality of steps and executed.
　また、上述した画像認識システム１０における各種処理を行う手段および方法は、専用のハードウェア回路、またはプログラムされたコンピューターのいずれによっても実現することが可能である。上記プログラムは、例えば、ＵＳＢメモリやＤＶＤ（Ｄｉｇｉｔａｌ　Ｖｅｒｓａｔｉｌｅ　Ｄｉｓｃ）－ＲＯＭ等のコンピューター読み取り可能な記録媒体によって提供されてもよいし、インターネット等のネットワークを介してオンラインで提供されてもよい。この場合、コンピューター読み取り可能な記録媒体に記録されたプログラムは、通常、ハードディスク等の記憶部に転送され記憶される。また、上記プログラムは、単独のアプリケーションソフトとして提供されてもよいし、一機能としてその検出部等の装置のソフトウエアに組み込まれてもよい。 Further, the means and methods for performing various processes in the image recognition system 10 described above can be realized by either a dedicated hardware circuit or a programmed computer. The program may be provided by a computer-readable recording medium such as a USB memory or a DVD (Digital Versailles Disc) -ROM, or may be provided online via a network such as the Internet. In this case, the program recorded on the computer-readable recording medium is usually transferred to and stored in a storage unit such as a hard disk. Further, the above program may be provided as a single application software, or may be incorporated into the software of a device such as a detection unit as one function.
　本出願は、２０１９年８月７日に出願された日本特許出願（特願２０１９－１４５５１１号）に基づいており、その開示内容は、参照され、全体として、組み入れられている。 This application is based on a Japanese patent application (Japanese Patent Application No. 2019-145511) filed on August 7, 2019, the disclosure of which is referenced and incorporated as a whole.

Claims (11)

  1. 　撮影装置により撮影された、対象者を含む画像に基づいて、前記対象者の体に関する特徴点を検出する特徴点検出部と、
    　前記画像に基づいて、前記画像の中心領域から前記対象者へ向かう方向を算出する算出部と、
    　検出された前記特徴点の中の所定の特徴点の並び方向と、算出された前記対象者へ向かう方向とに基づいて、前記対象者の行動が所定の行動に含まれる行動かどうか判定する判定部と、
    　前記判定部により、前記対象者の行動が前記所定の行動に含まれる行動であると判定された場合に、前記対象者の行動に関する情報を出力する出力部と、
    　を有する画像処理システム。     A feature point detection unit that detects feature points related to the target person's body based on an image including the target person taken by the photographing device.
    A calculation unit that calculates the direction from the central region of the image toward the target person based on the image.
    Judgment as to whether or not the action of the target person is included in the predetermined action based on the direction in which the predetermined feature points are arranged in the detected feature points and the calculated direction toward the target person. Department and
    When the determination unit determines that the behavior of the target person is an action included in the predetermined behavior, an output unit that outputs information about the behavior of the target person and
    Image processing system with.
  2. 　前記判定部は、前記所定の特徴点の並び方向と、前記対象者へ向かう方向とが所定の関係にある場合に、前記対象者の行動が前記所定の行動に含まれる行動であると判定する、請求項１に記載の画像処理システム。 When the predetermined feature point arrangement direction and the direction toward the target person have a predetermined relationship, the determination unit determines that the target person's action is included in the predetermined action. , The image processing system according to claim 1.
  3. 　前記所定の関係は、前記撮影装置から前記対象者までの距離に応じて設定される、請求項２に記載の画像処理システム。 The image processing system according to claim 2, wherein the predetermined relationship is set according to the distance from the photographing device to the target person.
  4. 　前記所定の特徴点の並び方向は、２つの前記特徴点の一方から他方へ向かう特定方向である、請求項１～３のいずれか一項に記載の画像処理システム。 The image processing system according to any one of claims 1 to 3, wherein the arrangement direction of the predetermined feature points is a specific direction from one of the two feature points to the other.
  5. 　前記判定部は、複数の前記特定方向を用いて、前記対象者の行動が前記所定の行動に含まれる行動かどうか判定する、請求項４に記載の画像処理システム。 The image processing system according to claim 4, wherein the determination unit determines whether or not the action of the target person is an action included in the predetermined action by using the plurality of the specific directions.
  6. 　前記所定の関係は、前記所定の特徴点の並び方向と、前記対象者へ向かう方向とがなす角度が所定の閾値以上である、請求項２または３に記載の画像処理システム。 The image processing system according to claim 2 or 3, wherein the predetermined relationship is such that the angle formed by the arrangement direction of the predetermined feature points and the direction toward the target person is equal to or more than a predetermined threshold value.
  7. 　前記所定の行動は転倒および転落であり、
    　前記判定部は、前記対象者の行動が、転倒および転落の少なくともいずれかであるかどうか判定する、請求項１～６のいずれか一項に記載の画像処理システム。     The prescribed actions are falls and falls,
    The image processing system according to any one of claims 1 to 6, wherein the determination unit determines whether or not the subject's behavior is at least one of a fall and a fall.
  8. 　前記撮影装置は広角カメラであり、前記画像は、所定の領域を俯瞰する位置に設置された前記広角カメラにより撮影された前記所定の領域を含む画像である、請求項１～７のいずれか一項に記載の画像処理システム。 The photographing device is a wide-angle camera, and the image is an image including the predetermined area taken by the wide-angle camera installed at a position overlooking a predetermined area, any one of claims 1 to 7. The image processing system described in the section.
  9. 　前記対象者へ向かう方向は、前記画像の中心領域に含まれる点と、前記所定の特徴点と、に基づいて算出される方向である、請求項１～８のいずれか一項に記載の画像処理システム。 The image according to any one of claims 1 to 8, wherein the direction toward the target person is a direction calculated based on the points included in the central region of the image and the predetermined feature points. Processing system.
  10. 　撮影装置により撮影された、対象者を含む画像に基づいて、前記対象者の体に関する特徴点を検出する手順（ａ）と、
    　前記画像に基づいて、前記画像の中心領域から前記対象者へ向かう方向を算出する手順（ｂ）と、
    　検出された前記特徴点の中の所定の特徴点の並び方向と、算出された前記対象者へ向かう方向とに基づいて、前記対象者の行動が所定の行動に含まれる行動かどうか判定する手順（ｃ）と、
    　前記手順（ｃ）において、前記対象者の行動が前記所定の行動に含まれる行動であると判定された場合に、前記対象者の行動に関する情報を出力する手順（ｄ）と、
    　を有する処理をコンピューターに実行させるための画像処理プログラム。     The procedure (a) of detecting the feature points related to the body of the subject based on the image including the subject taken by the photographing device, and
    The procedure (b) of calculating the direction from the central region of the image toward the target person based on the image, and
    A procedure for determining whether or not the behavior of the target person is included in the predetermined behavior based on the direction in which the predetermined feature points are arranged in the detected feature points and the calculated direction toward the target person. (C) and
    In the procedure (c), when it is determined that the behavior of the target person is an action included in the predetermined behavior, the procedure (d) of outputting information on the behavior of the target person and the procedure (d).
    An image processing program for causing a computer to execute a process having the above.
  11. 　画像処理システムに実行させる方法であって、
    　撮影装置により撮影された、対象者を含む画像に基づいて、前記対象者の体に関する特徴点を検出する段階（ａ）と、
    　前記画像に基づいて、前記画像の中心領域から前記対象者へ向かう方向を算出する段階（ｂ）と、
    　検出された前記特徴点の中の所定の特徴点の並び方向と、算出された前記対象者へ向かう方向とに基づいて、前記対象者の行動が所定の行動に含まれる行動かどうか判定する段階（ｃ）と、
    　前記段階（ｃ）において、前記対象者の行動が前記所定の行動に含まれる行動であると判定された場合に、前記対象者の行動に関する情報を出力する段階（ｄ）と、
    　を有する画像処理方法。     It ’s a method to let the image processing system execute.
    The step (a) of detecting the feature points related to the body of the subject based on the image including the subject taken by the photographing device, and
    A step (b) of calculating the direction from the central region of the image toward the target person based on the image, and
    A step of determining whether or not the behavior of the target person is included in the predetermined behavior based on the direction in which the predetermined feature points are arranged in the detected feature points and the calculated direction toward the target person. (C) and
    In the step (c), when it is determined that the behavior of the subject is an action included in the predetermined behavior, the step (d) of outputting information on the behavior of the subject,
    Image processing method having.
PCT/JP2020/026880 2019-08-07 2020-07-09 Image processing system, image processing program, and image processing method WO2021024691A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2021537637A JP7435609B2 (en) 2019-08-07 2020-07-09 Image processing system, image processing program, and image processing method

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2019-145511 2019-08-07
JP2019145511 2019-08-07

Publications (1)

Publication Number Publication Date
WO2021024691A1 true WO2021024691A1 (en) 2021-02-11

Family

ID=74502961

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/026880 WO2021024691A1 (en) 2019-08-07 2020-07-09 Image processing system, image processing program, and image processing method

Country Status (2)

Country Link
JP (1) JP7435609B2 (en)
WO (1) WO2021024691A1 (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014010203A1 (en) * 2012-07-13 2014-01-16 日本電気株式会社 Fall detection device, fall detection method, fall detection camera, and computer program
WO2016199749A1 (en) * 2015-06-10 2016-12-15 コニカミノルタ株式会社 Image processing system, image processing device, image processing method, and image processing program
JP2019121045A (en) * 2017-12-28 2019-07-22 コニカミノルタ株式会社 Posture estimation system, behavior estimation system, and posture estimation program

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014010203A1 (en) * 2012-07-13 2014-01-16 日本電気株式会社 Fall detection device, fall detection method, fall detection camera, and computer program
WO2016199749A1 (en) * 2015-06-10 2016-12-15 コニカミノルタ株式会社 Image processing system, image processing device, image processing method, and image processing program
JP2019121045A (en) * 2017-12-28 2019-07-22 コニカミノルタ株式会社 Posture estimation system, behavior estimation system, and posture estimation program

Also Published As

Publication number Publication date
JP7435609B2 (en) 2024-02-21
JPWO2021024691A1 (en) 2021-02-11

Similar Documents

Publication Publication Date Title
JP6137425B2 (en) Image processing system, image processing apparatus, image processing method, and image processing program
US10786183B2 (en) Monitoring assistance system, control method thereof, and program
KR102052883B1 (en) Prediction system using a thermal imagery camera and fall prediction method using a thermal imagery camera
JP7271915B2 (en) Image processing program and image processing device
JP6984712B2 (en) Program of monitored person monitoring system and monitored person monitoring system
JP2022165483A (en) Detecting device, detecting system, detecting method, and detecting program
JP6115692B1 (en) Behavior detecting device, method and program, and monitored person monitoring device
WO2016199504A1 (en) Behavior detection device, behavior detection method, and monitored-person monitoring device
US20180322334A1 (en) Person Monitoring Device And Method, And Person Monitoring System
WO2021033453A1 (en) Image processing system, image processing program, and image processing method
JP7347577B2 (en) Image processing system, image processing program, and image processing method
JP7500929B2 (en) IMAGE PROCESSING SYSTEM, IMAGE PROCESSING PROGRAM, AND IMAGE PROCESSING METHOD
WO2021024691A1 (en) Image processing system, image processing program, and image processing method
JP6870514B2 (en) Watching support system and its control method
WO2021033597A1 (en) Image processing system, image processing program, and image processing method
JP2020013185A (en) Watching device and watching device control program
JPWO2020008726A1 (en) Target object detection program and target object detection device
CN116243306A (en) Method and system for monitoring object
JP5207040B2 (en) Fall detection system
JPWO2016181731A1 (en) Fall detection device, fall detection method and monitored person monitoring device
WO2020008995A1 (en) Image recognition program, image recognition device, learning program, and learning device
JP2020077337A (en) Image monitoring system
JP2021065617A (en) Image processing device and image processing program
JP2024093174A (en) Self-injury detection device, self-injury detection system, self-injury detection program, and self-injury detection method
JP2022126071A (en) Image processing method

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20849820

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021537637

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20849820

Country of ref document: EP

Kind code of ref document: A1