JP2022055249A - System, electronic apparatus, method for controlling electronic apparatus, and program - Google Patents

Abstract

To provide a system, an electronic apparatus, a method for controlling an electronic apparatus, and a program contributing to the safety of the surroundings of a subject to be monitored.SOLUTION: A system 1 comprises: an imaging unit 20 that picks up images of a subject to be monitored and the other person; an extraction unit 11 that extracts the coordinates of a predetermined part of the subject to be monitored and the coordinates of a predetermined part of the other person from the images picked up by the imaging unit; and a controller 15 that, when the distance between at least part of the predetermined part of the subject to be monitored and at least part of the predetermined part of the other person becomes less than a first threshold, determines that the subject to be monitored comes closer to the other person. When the elbow of the subject to be monitored moves upward and the wrist of the subject to be monitored moves upward beyond the elbow, or when the wrist of the subject to be monitored moves upward beyond the elbow and subsequently moves downward at a predetermined speed or more, the controller 15 outputs a predetermined alert signal.SELECTED DRAWING: Figure 1

Description

The present disclosure relates to systems, electronic devices, control methods for electronic devices, and programs.

For example, in a field such as a long-term care facility, a device for monitoring the behavior of a monitored person such as a nurse or a care recipient has been proposed. For example, Patent Document 1 discloses a monitored person system that detects a predetermined behavior in a monitored person based on an image obtained by an image pickup device. Patent Document 2 discloses a fall prevention system that prevents a subject from falling while walking by attaching a detection device to the foot of the subject. Further, Patent Document 3 discloses a monitoring support device for determining the body position of a human body by detecting a temperature distribution. Cited Document 4 also discloses a medical system for monitoring elderly psychotic patients at home or in elderly housing with care or long-term care facilities.

Japanese Unexamined Patent Publication No. 2017-91552 Japanese Unexamined Patent Publication No. 2017-221502 Japanese Unexamined Patent Publication No. 2014-106636 Japanese Patent Application Laid-Open No. 2003-91790

It would be beneficial if the monitored object could be monitored to ensure the safety of the surroundings of the monitored object.

An object of the present disclosure is to provide a system, an electronic device, a control method for the electronic device, and a program that can be safely provided around the monitored object.

The system according to one embodiment includes an imaging unit that captures an image of a monitored object and another person, coordinates of a predetermined portion of the monitored object from an image captured by the imaging unit, and a predetermined portion of the other person. It includes an extraction unit that extracts coordinates and a controller. When the distance between at least a part of the predetermined part of the monitored object extracted by the extraction unit and at least a part of the predetermined part of the other person is less than the first threshold value, the controller may use the controller. It is determined that the monitored object has approached the other person. When the controller determines that the monitored object has approached the other person, the elbow of the monitored object moves upward, and the wrist of the monitored object moves above the elbow. Or, when the wrist to be monitored moves above the elbow and then moves down at a predetermined speed or higher, a predetermined warning signal is output.

Further, in the system according to one embodiment, an image pickup unit that captures an image of a monitored object and another person, coordinates of a predetermined portion of the monitored object from an image captured by the image pickup unit, and a predetermined position of the other person. It includes an extraction unit that extracts the coordinates of the part and a controller. When the distance between at least a part of the predetermined part of the monitored object extracted by the extraction unit and at least a part of the predetermined part of the other person is less than the first threshold value, the controller may use the controller. It is determined that the monitored object has approached the other person. When the controller determines that the monitored object approaches the other person, the wrist of the monitored object moves above the elbow, and the elbow of the monitored object does not move upward. Or, when the wrist of the monitored object moves above the elbow and then the wrist of the monitored object moves horizontally at a predetermined speed or higher, a predetermined warning signal is output.

Further, in the system according to one embodiment, an image pickup unit that captures an image of a monitored object and another person, coordinates of a predetermined portion of the monitored object from an image captured by the image pickup unit, and a predetermined position of the other person. It includes an extraction unit that extracts the coordinates of the part and a controller. When the distance between at least a part of the predetermined part of the monitored object extracted by the extraction unit and at least a part of the predetermined part of the other person is less than the first threshold value, the controller may use the controller. It is determined that the monitored object has approached the other person. When the controller determines that the monitored target approaches the other person, the monitored target's wrist moves above the elbow, and the monitored target's wrist is the other person's wrist. The wrist of the monitored object moves horizontally toward at least a part of the predetermined part, or after the wrist of the monitored object moves toward at least a part of the predetermined part of the other person. When the wrist moves at a predetermined speed or higher and the wrist of the monitored object is separated from the other person, a predetermined warning signal is output.

Further, in the system according to one embodiment, an image pickup unit that captures an image of a monitored object and another person, coordinates of a predetermined portion of the monitored object from an image captured by the image pickup unit, and a predetermined position of the other person. It includes an extraction unit that extracts the coordinates of the part and a controller. When the distance between at least a part of the predetermined part of the monitored object extracted by the extraction unit and at least a part of the predetermined part of the other person is less than the first threshold value, the controller may use the controller. It is determined that the monitored object has approached the other person. When the controller determines that the monitored object approaches the other person, the monitored knee and ankle move upward or horizontally, or the monitored knee and ankle move. After moving up or horizontally, if the knee and ankle of the monitored object move further up or horizontally, a predetermined warning signal is output.

Further, in the system according to one embodiment, an image pickup unit that captures an image of a monitored object and another person, coordinates of a predetermined portion of the monitored object from an image captured by the image pickup unit, and a predetermined position of the other person. It includes an extraction unit that extracts the coordinates of the part and a controller. When the distance between at least a part of the predetermined part of the monitored object extracted by the extraction unit and at least a part of the predetermined part of the other person is less than the first threshold value, the controller may use the controller. It is determined that the monitored object has approached the other person. When the controller determines that the monitored object approaches the other person, the shoulder of the monitored object moves toward the other person, or the shoulder of the monitored object is the other person. After moving toward the person, when the distance between the shoulder of the monitored object and at least a part of the predetermined portion of the other person is less than the second threshold value, a predetermined warning signal is output.

Further, in the system according to one embodiment, an image pickup unit that captures an image of a monitored object and another person, coordinates of a predetermined portion of the monitored object from an image captured by the image pickup unit, and a predetermined position of the other person. It includes an extraction unit that extracts the coordinates of the part and a controller. When the distance between at least a part of the predetermined part of the monitored object extracted by the extraction unit and at least a part of the predetermined part of the other person is less than the first threshold value, the controller may use the controller. It is determined that the monitored object has approached the other person. When the controller determines that the monitored object approaches the other person, the neck and shoulders of the monitored object move downward, and the wrist of the monitored object is the other person. Or after the neck and shoulders of the monitored object move downward and the wrist of the monitored object moves toward the other person, the monitored object When the neck and shoulders move further downward, a predetermined warning signal is output.

Further, the system according to the embodiment includes an image pickup unit that captures an image of a monitored object, another person, and an object related to the other person, and the monitored object from the image captured by the image pickup unit. It includes an extraction unit for extracting the coordinates of a predetermined portion and the coordinates of an object related to the other person, and a controller. When the distance between at least a part of a predetermined part of the monitored object extracted by the extraction unit and at least a part of an object related to the other person is less than the first threshold value. , It is determined that the monitored object has approached an object related to the other person. When the controller determines that the monitored object has approached an object related to the other person, the wrist of the monitored object moves toward the object related to the other person, or the said. After the wrist of the monitored object moves toward the object related to the other person, a predetermined warning signal is output when the wrist of the monitored object moves back and forth in the horizontal direction.

The electronic device according to one embodiment includes an extraction unit that extracts the coordinates of a predetermined part of the monitored object and the coordinates of a predetermined part of the other person from an image obtained by capturing an image of the monitored object and another person, and a controller. And. When the distance between at least a part of the predetermined part of the monitored object extracted by the extraction unit and at least a part of the predetermined part of the other person is less than the first threshold value, the controller may use the controller. It is determined that the monitored object has approached the other person. When the controller determines that the monitored object has approached the other person, the elbow of the monitored object moves upward, and the wrist of the monitored object moves above the elbow. Or, when the wrist to be monitored moves above the elbow and then moves down at a predetermined speed or higher, a predetermined warning signal is output.

Further, the electronic device according to the embodiment includes an extraction unit that extracts the coordinates of a predetermined part of the monitored object and the coordinates of the predetermined part of the other person from an image obtained by capturing an image of the monitored object and another person. , With a controller. When the distance between at least a part of the predetermined part of the monitored object extracted by the extraction unit and at least a part of the predetermined part of the other person is less than the first threshold value, the controller may use the controller. It is determined that the monitored object has approached the other person. When the controller determines that the monitored object approaches the other person, the wrist of the monitored object moves above the elbow, and the elbow of the monitored object does not move upward. Or, when the wrist of the monitored object moves above the elbow and then the wrist of the monitored object moves horizontally at a predetermined speed or higher, a predetermined warning signal is output.

Further, the electronic device according to the embodiment includes an extraction unit that extracts the coordinates of a predetermined part of the monitored object and the coordinates of the predetermined part of the other person from an image obtained by capturing an image of the monitored object and another person. , With a controller. When the distance between at least a part of the predetermined part of the monitored object extracted by the extraction unit and at least a part of the predetermined part of the other person is less than the first threshold value, the controller may use the controller. It is determined that the monitored object has approached the other person. When the controller determines that the monitored target approaches the other person, the monitored target's wrist moves above the elbow, and the monitored target's wrist is the other person's wrist. The wrist of the monitored object moves horizontally toward at least a part of the predetermined part, or after the wrist of the monitored object moves toward at least a part of the predetermined part of the other person. When the wrist moves at a predetermined speed or higher and the wrist of the monitored object is separated from the other person, a predetermined warning signal is output.

Further, the electronic device according to the embodiment includes an extraction unit that extracts the coordinates of a predetermined part of the monitored object and the coordinates of the predetermined part of the other person from an image obtained by capturing an image of the monitored object and another person. , With a controller. When the distance between at least a part of the predetermined part of the monitored object extracted by the extraction unit and at least a part of the predetermined part of the other person is less than the first threshold value, the controller may use the controller. It is determined that the monitored object has approached the other person. When the controller determines that the monitored object approaches the other person, the monitored knee and ankle move upward or horizontally, or the monitored knee and ankle move. After moving up or horizontally, if the knee and ankle of the monitored object move further up or horizontally, a predetermined warning signal is output.

Further, the electronic device according to the embodiment includes an extraction unit that extracts the coordinates of a predetermined part of the monitored object and the coordinates of the predetermined part of the other person from an image obtained by capturing an image of the monitored object and another person. , With a controller. When the distance between at least a part of the predetermined part of the monitored object extracted by the extraction unit and at least a part of the predetermined part of the other person is less than the first threshold value, the controller may use the controller. It is determined that the monitored object has approached the other person. When the controller determines that the monitored object approaches the other person, the shoulder of the monitored object moves toward the other person, or the shoulder of the monitored object is the other person. After moving toward the person, when the distance between the shoulder of the monitored object and at least a part of the predetermined portion of the other person is less than the second threshold value, a predetermined warning signal is output.

Further, the electronic device according to the embodiment includes an extraction unit that extracts the coordinates of a predetermined part of the monitored object and the coordinates of the predetermined part of the other person from an image obtained by capturing an image of the monitored object and another person. , With a controller. When the distance between at least a part of the predetermined part of the monitored object extracted by the extraction unit and at least a part of the predetermined part of the other person is less than the first threshold value, the controller may use the controller. It is determined that the monitored object has approached the other person. When the controller determines that the monitored object approaches the other person, the neck and shoulders of the monitored object move downward, and the wrist of the monitored object is the other person. Or after the neck and shoulders of the monitored object move downward and the wrist of the monitored object moves toward the other person, the monitored object When the neck and shoulders move further downward, a predetermined warning signal is output.

Further, the electronic device according to the embodiment includes the coordinates of a predetermined portion of the monitored object and the other person from an image obtained by capturing an image of the monitored object, another person, and an object related to the other person. It is provided with an extraction unit for extracting the coordinates of an object related to the above, and a controller. When the distance between at least a part of a predetermined part of the monitored object extracted by the extraction unit and at least a part of an object related to the other person is less than the first threshold value. , It is determined that the monitored object has approached an object related to the other person. When the controller determines that the monitored object has approached an object related to the other person, the wrist of the monitored object moves toward the object related to the other person, or the said. After the wrist of the monitored object moves toward the object related to the other person, a predetermined warning signal is output when the wrist of the monitored object moves back and forth in the horizontal direction.

The method for controlling an electronic device according to an embodiment is an extraction step of extracting the coordinates of a predetermined part of the monitored object and the coordinates of a predetermined part of the other person from an image obtained by capturing an image of the monitored object and another person. including. The control method is when the distance between at least a part of the predetermined part of the monitored object extracted in the extraction step and at least a part of the predetermined part of the other person is less than the first threshold value. , A determination step for determining that the monitored object has approached the other person. In the control method, when it is determined that the monitored object approaches the other person, the elbow of the monitored object moves upward, and the wrist of the monitored object moves above the elbow. This includes an output step that outputs a predetermined warning signal when the wrist of the monitored object moves above the elbow and then moves downward at a predetermined speed or higher.

Further, in the method for controlling an electronic device according to an embodiment, the coordinates of a predetermined part of the monitored object and the coordinates of a predetermined part of the other person are extracted from an image obtained by capturing an image of the monitored object and another person. Includes extraction steps. The control method is when the distance between at least a part of the predetermined part of the monitored object extracted in the extraction step and at least a part of the predetermined part of the other person is less than the first threshold value. , A determination step for determining that the monitored object has approached the other person. In the control method, when it is determined that the monitored object approaches the other person, the wrist of the monitored object moves above the elbow, and the elbow of the monitored object does not move upward. In the case, or when the wrist of the monitored object moves above the elbow and then the wrist of the monitored object moves horizontally at a predetermined speed or higher, an output step of outputting a predetermined warning signal is included. ..

Further, in the method for controlling an electronic device according to an embodiment, the coordinates of a predetermined part of the monitored object and the coordinates of a predetermined part of the other person are extracted from an image obtained by capturing an image of the monitored object and another person. Includes extraction steps. The control method is when the distance between at least a part of the predetermined part of the monitored object extracted in the extraction step and at least a part of the predetermined part of the other person is less than the first threshold value. , A determination step for determining that the monitored object has approached the other person. In the control method, when it is determined that the monitored object approaches the other person, the wrist of the monitored object moves above the elbow, and the wrist of the monitored object is the other person. When the wrist of the monitored object moves toward at least a part of the predetermined part of the person, or after the wrist of the monitored object moves toward at least a part of the predetermined part of the other person, the wrist of the monitored object moves in the horizontal direction. Includes an output step that outputs a predetermined warning signal when the wrist moves at a predetermined speed or higher and the wrist to be monitored is separated from the other person.

Further, in the method for controlling an electronic device according to an embodiment, the coordinates of a predetermined part of the monitored object and the coordinates of a predetermined part of the other person are extracted from an image obtained by capturing an image of the monitored object and another person. Includes extraction steps. The control method is when the distance between at least a part of the predetermined part of the monitored object extracted in the extraction step and at least a part of the predetermined part of the other person is less than the first threshold value. , A determination step for determining that the monitored object has approached the other person. The control method determines that the monitored object approaches the other person, the knee and ankle of the monitored object move upward or horizontally, or the knee and ankle of the monitored object. Includes an output step that outputs a predetermined warning signal if the monitored knee and ankle move further up or horizontally after the move up or horizontally.

Further, in the method for controlling an electronic device according to an embodiment, the coordinates of a predetermined part of the monitored object and the coordinates of a predetermined part of the other person are extracted from an image obtained by capturing an image of the monitored object and another person. Includes extraction steps. The control method is when the distance between at least a part of the predetermined part of the monitored object extracted in the extraction step and at least a part of the predetermined part of the other person is less than the first threshold value. , A determination step for determining that the monitored object has approached the other person. In the control method, when it is determined that the monitored object approaches the other person, the shoulder of the monitored object moves toward the other person, or the shoulder of the monitored object is the said. Output that outputs a predetermined warning signal when the distance between the shoulder of the monitored object and at least a part of a predetermined portion of the other person is less than the second threshold value after moving toward another person. Including steps.

Further, in the method for controlling an electronic device according to an embodiment, the coordinates of a predetermined part of the monitored object and the coordinates of a predetermined part of the other person are extracted from an image obtained by capturing an image of the monitored object and another person. Includes extraction steps. The control method is when the distance between at least a part of the predetermined part of the monitored object extracted in the extraction step and at least a part of the predetermined part of the other person is less than the first threshold value. , A determination step for determining that the monitored object has approached the other person. In the control method, when it is determined that the monitored object approaches the other person, the neck and shoulders of the monitored object move downward, and the wrist of the monitored object is the other person. When moving toward a person, or after the neck and shoulders of the monitored target move downward and the wrist of the monitored target moves toward the other person, the monitored target Includes an output step that outputs a given warning signal if the neck and shoulders of the person move further downwards.

Further, the control method of the electronic device according to one embodiment includes the coordinates of a predetermined portion of the monitored object from an image obtained by capturing an image of the monitored object, another person, and an object related to the other person, and the coordinates of the predetermined portion of the monitored object. Includes an extraction step to extract the coordinates of an object related to another person. The control method is when the distance between at least a part of the predetermined portion of the monitored object extracted in the extraction step and at least a part of an object related to the other person is less than the first threshold value. Includes a determination step of determining that the monitored object has approached an object related to the other person. The control method determines that the monitored object has approached an object related to the other person, and the wrist of the monitored object moves toward the object related to the other person, or Includes an output step that outputs a predetermined warning signal when the monitored wrist moves back and forth in the horizontal direction after the monitored wrist moves toward an object related to the other person. ..

The program according to one embodiment executes an extraction step on a computer to extract the coordinates of a predetermined part of the monitored object and the coordinates of a predetermined part of the other person from an image obtained by capturing an image of the monitored object and another person. Let me. The program is set when the distance between at least a part of the predetermined part of the monitored object extracted in the extraction step and at least a part of the predetermined part of the other person is less than the first threshold value. The computer is made to execute a determination step of determining that the monitored object has approached the other person. When the program determines that the monitored object has approached the other person, the elbow of the monitored object moves upward, and the wrist of the monitored object moves above the elbow. Or, when the wrist to be monitored moves above the elbow and then moves down at a predetermined speed or higher, the computer is made to perform an output step of outputting a predetermined warning signal.

Further, in the program according to the embodiment, a computer performs an extraction step of extracting the coordinates of a predetermined part of the monitored object and the coordinates of a predetermined part of the other person from an image obtained by capturing an image of the monitored object and another person. To execute. The program is set when the distance between at least a part of the predetermined part of the monitored object extracted in the extraction step and at least a part of the predetermined part of the other person is less than the first threshold value. The computer is made to execute a determination step of determining that the monitored object has approached the other person. When the program determines that the monitored object has approached the other person, the wrist of the monitored object moves above the elbow, and the elbow of the monitored object does not move upward. Or, after the wrist of the monitored object moves above the elbow, if the wrist of the monitored object moves horizontally at a predetermined speed or higher, an output step of outputting a predetermined warning signal is output to the computer. Let it run.

Further, in the program according to the embodiment, a computer performs an extraction step of extracting the coordinates of a predetermined part of the monitored object and the coordinates of a predetermined part of the other person from an image obtained by capturing an image of the monitored object and another person. To execute. The program is provided when the distance between at least a part of the predetermined part of the monitored object extracted in the extraction step and at least a part of the predetermined part of the other person is less than the first threshold value. , The computer is made to execute a determination step of determining that the monitored object has approached the other person. When the program determines that the monitored object approaches the other person, the monitored object's wrist moves above the elbow, and the monitored object's wrist is the other person's wrist. The wrist of the monitored object moves horizontally toward at least a part of the predetermined part, or after the wrist of the monitored object moves toward at least a part of the predetermined part of the other person. When the wrist of the monitored object moves away from the other person while moving at a predetermined speed or higher, the computer is made to perform an output step of outputting a predetermined warning signal.

Further, in the program according to the embodiment, a computer performs an extraction step of extracting the coordinates of a predetermined part of the monitored object and the coordinates of a predetermined part of the other person from an image obtained by capturing an image of the monitored object and another person. To execute. The program is set when the distance between at least a part of the predetermined part of the monitored object extracted in the extraction step and at least a part of the predetermined part of the other person is less than the first threshold value. The computer is made to execute a determination step of determining that the monitored object has approached the other person. When the program determines that the monitored object approaches the other person, the monitored knee and ankle move upward or horizontally, or the monitored knee and ankle move. After moving up or horizontally, the computer is made to perform an output step to output a predetermined warning signal when the knee and ankle to be monitored move further up or horizontally.

Further, in the program according to the embodiment, a computer performs an extraction step of extracting the coordinates of a predetermined part of the monitored object and the coordinates of a predetermined part of the other person from an image obtained by capturing an image of the monitored object and another person. To execute. The program is set when the distance between at least a part of the predetermined part of the monitored object extracted in the extraction step and at least a part of the predetermined part of the other person is less than the first threshold value. The computer is made to execute a determination step of determining that the monitored object has approached the other person. When the program determines that the monitored object has approached the other person, the shoulder of the monitored object moves toward the other person, or the shoulder of the monitored object is the other person. An output step that outputs a predetermined warning signal when the distance between the shoulder of the monitored object and at least a part of a predetermined portion of the other person is less than the second threshold value after moving toward the person. Let the computer run.

Further, in the program according to the embodiment, a computer performs an extraction step of extracting the coordinates of a predetermined part of the monitored object and the coordinates of a predetermined part of the other person from an image obtained by capturing an image of the monitored object and another person. To execute. The program is set when the distance between at least a part of the predetermined part of the monitored object extracted in the extraction step and at least a part of the predetermined part of the other person is less than the first threshold value. The computer is made to execute a determination step of determining that the monitored object has approached the other person. When the program determines that the monitored object approaches the other person, the neck and shoulders of the monitored object move downward, and the wrist of the monitored object is the other person. When moving toward a person, or after the neck and shoulders of the monitored target move downward and the wrist of the monitored target moves toward the other person, the monitored target Have the computer perform an output step that outputs a given warning signal if the neck and shoulders move further down.

Further, the program according to the embodiment is to obtain the coordinates of a predetermined part of the monitored object and the other person from an image obtained by capturing an image of the monitored object, another person, and an object related to the other person. Have the computer perform an extraction step to extract the coordinates of the relevant object. In the program, when the distance between at least a part of a predetermined part of the monitored object extracted in the extraction step and at least a part of an object related to the other person is less than the first threshold value. To make a computer execute a determination step of determining that the monitored object has approached an object related to the other person. The program determines that the monitored object has approached an object related to the other person, and the wrist of the monitored object moves toward the object related to the other person, or the said. After the monitored wrist moves toward an object related to the other person, the computer is provided with an output step that outputs a predetermined warning signal when the monitored wrist moves back and forth in the horizontal direction. Let it run.

According to one embodiment, it is possible to provide a system, an electronic device, a control method for the electronic device, and a program that can be safely provided around the monitored object.

It is a functional block diagram which shows the schematic structure of the system which concerns on one Embodiment. It is a flowchart explaining operation by the system which concerns on one Embodiment. It is a flowchart which shows the example of the image imaged by the system which concerns on one Embodiment. It is a flowchart which shows the example of the image imaged by the system which concerns on one Embodiment. It is a figure which shows the example of the predetermined part extracted by the system which concerns on one Embodiment. It is a figure which shows the example of the predetermined part extracted from the image imaged by the system which concerns on one Embodiment. It is a figure explaining the coordinates of the predetermined part extracted by the system which concerns on one Embodiment. It is a figure which shows the example of the predetermined part extracted from the image imaged by the system which concerns on one Embodiment. It is a figure which shows the example of the predetermined part extracted from the image imaged by the system which concerns on one Embodiment. It is a figure explaining the determination by the system which concerns on one Embodiment.

In the present disclosure, the "electronic device" may be a device driven by electric power. Further, the "system" may include a device driven by electric power. Further, the "user" may be a person (typically a human being) who uses the system and / or the electronic device according to the embodiment. The user may include a person who monitors the monitored object by using the system and / or the electronic device according to the embodiment. Further, the "monitored target" may be a person (for example, a human being or an animal) to be monitored by the system and / or the electronic device according to the embodiment. Further, the user may include a monitored object.

It is assumed that the system according to one embodiment is used in a specific situation used by a person engaged in social activities such as a company, a hospital, an elderly home, a school, a sports gym, and a long-term care facility. It may be a facility. For example, in the case of a company, it is extremely important to understand and / or manage the health status of employees. Similarly, it is extremely important to understand and / or manage the health status of patients and medical staff in hospitals, and residents and staff in elderly housing with care. The scene in which the system according to the embodiment is used is not limited to the above-mentioned facilities such as companies, hospitals, and elderly housings, but any facility where it is desired to grasp and / or manage the health condition of the monitored object. May be. Any facility may also include non-commercial facilities, such as the user's home. Further, the scene in which the system according to one embodiment is used may be, for example, a moving body such as a train, a bus, or an airplane, a station, a platform, or the like.

The system according to one embodiment may be used for monitoring the behavior of a monitored object such as a nurse-requiring person or a nursing-requiring person in a nursing care facility or the like. The system according to one embodiment monitors the behavior of a monitored object, such as a person requiring nursing care or a person requiring long-term care, to violently attack another person such as a staff member who cares for or cares for the monitored object. can do. The act of wielding violence may include the act of hitting, hitting, pushing, kicking, ramming, or biting another person. The act of wielding violence may include the act of pushing or rocking a chair or wheelchair on which another person is sitting. The violent movement is not limited to these, and may include various other movements.

In particular, in the system according to one embodiment, when a monitored target such as a nurse or a care recipient is approaching another person, the monitored target violently acts against the other person. A predetermined warning can be issued before the operation starts or ends. Therefore, according to the system according to one embodiment, the staff of a nursing care facility, for example, a monitored object such as a nurse or a person requiring long-term care violently or further violently acts against another person. Before continuing, it is possible to recognize that the monitored subject is or is trying to violence against another person.

Hereinafter, the system according to the embodiment will be described in detail with reference to the drawings.

FIG. 1 is a diagram showing a schematic configuration of a system according to an embodiment. As shown in FIG. 1, the system 1 according to an embodiment may be configured to include an electronic device 10 and an image pickup unit 20. The electronic device 10 and the image pickup unit 20 may be connected by wire or wireless, or a combination of wire and wireless. The system 1 according to one embodiment may not include a part of the functional unit shown in FIG. 1, or may include a functional unit other than that shown in FIG. For example, the system 1 according to the embodiment may not include at least one of the warning unit 17 and the communication unit 19. Further, for example, the system 1 according to the embodiment may include a display capable of displaying an image and / or a slot into which a storage such as a memory card can be inserted.

The image pickup unit 20 shown in FIG. 1 may include an image sensor that electronically captures an image, such as a digital camera. The image pickup unit 20 may be configured to include an image pickup element that performs photoelectric conversion, such as a CCD (Charge Coupled Device Image Sensor) or a CMOS (Complementary Metal Oxide Semiconductor) sensor. The image pickup unit 20 may take an image of the monitored object T, for example, as shown in FIG. Here, the monitored target T may be, for example, a human being. The image pickup unit 20 may convert the captured image into a signal and transmit it to the electronic device 10. For example, the image pickup unit 20 may transmit a signal based on the captured image to the extraction unit 11, the storage unit 13, and / or the controller 15 of the electronic device 10. The image pickup unit 20 is not limited to an image pickup device such as a digital camera as long as it captures the monitored target T, and may be any device. The image pickup unit 20 is connected to the electronic device 10 by wire, wirelessly, or any combination thereof.

In one embodiment, the image pickup unit 20 may take an image of the monitored target T, for example, as a still image at predetermined time intervals (for example, 15 frames per second). Further, in one embodiment, the imaging unit 20 may image, for example, the monitored target T as a continuous moving image.

As shown in FIG. 1, the electronic device 10 according to the embodiment may include an extraction unit 11, a storage unit 13, a controller 15, a warning unit 17, and a communication unit 19. The electronic device 10 according to the embodiment may not include a part of the functional unit shown in FIG. 1, or may include a functional unit other than that shown in FIG.

The extraction unit 11 may have a function of extracting a predetermined feature point from the image captured by the image pickup unit 20. For example, the extraction unit 11 may extract the coordinates of a feature point such as a predetermined portion of the body of the monitored object T from the image of the monitored object T captured by the imaging unit 20. Here, the feature points will be further described later. In one embodiment, the extraction unit 11 obtains the coordinates of each part such as the head, trunk, limbs, and / or joints of the monitored object T from the image of the monitored object T captured by the imaging unit 20. It may be extracted. The extraction unit 11 may be configured as dedicated hardware, may be configured to include software at least in part, or may be configured entirely by software. In this way, the extraction unit 11 may extract the coordinates of the predetermined portion of the monitored target T from the image captured by the image pickup unit 20. The extraction unit 11 may extract the coordinates of a feature point such as a predetermined part in the body of the other person W from the image of the other person W (see FIG. 3 or the like). The extraction unit 11 may extract the coordinates of a predetermined portion of another person W from the image captured by the image pickup unit 20. The extraction unit 11 may recognize a chair or a wheelchair on which another person W is sitting from the image captured by the image pickup unit 20. A chair or wheelchair on which another person W sits is also referred to as an object related to the other person W.

The storage unit 13 may have a function as a memory for storing various types of information. The storage unit 13 may store, for example, a program executed by the controller 15, the result of processing executed by the controller 15, and the like. In one embodiment, the storage unit 13 stores a predetermined condition as a reference for the controller 15 to determine whether or not the monitored target T has started an action of violently acting against another person W. May be good. The predetermined conditions will be further described later. Further, the storage unit 13 may function as a work memory of the controller 15. The storage unit 13 can be configured by, for example, a semiconductor memory or the like, but is not limited to this, and can be any storage device. For example, the storage unit 13 may be a storage medium such as a memory card inserted in the electronic device 10 according to the embodiment. Further, the storage unit 13 may be the internal memory of the CPU used as the controller 15 described later, or may be connected to the controller 15 as a separate body.

The controller 15 controls and / or manages the entire electronic device 10 including each functional unit constituting the electronic device 10. The controller 15 may include at least one processor, such as a CPU (Central Processing Unit), to provide control and processing power to perform various functions. The controller 15 may be realized collectively by one processor, by several processors, or by individual processors. The processor may be realized as a single integrated circuit. The integrated circuit is also referred to as an IC (Integrated Circuit). The processor may be realized as a plurality of communicably connected integrated circuits and discrete circuits. The processor may be implemented on the basis of various other known techniques.

In one embodiment, the controller 15 may be configured as, for example, a CPU and a program executed by the CPU. The program executed by the controller 15, the result of the process executed by the controller 15, and the like may be stored in, for example, the storage unit 13. The controller 15 may appropriately include a memory necessary for the operation of the controller 15.

In particular, in one embodiment, the controller 15 may perform a process of determining whether or not the monitored target T has started an operation of violently attacking another person W. Hereinafter, the action of the monitored target T to violently attack another person W may be referred to as "violent action". The controller 15 may determine whether or not a predetermined condition is satisfied in order to determine the start of a violent action. The controller 15 determines the start of the violent movement based on the coordinates of the predetermined parts of the monitored target T and the other person W extracted by the extraction unit 11 or the coordinates of the object related to the other person W. , It may be determined whether or not a predetermined condition is satisfied. Further, the controller 15 may output a predetermined warning signal when determining the start of the violent operation by satisfying the predetermined condition. The controller 15 may determine whether or not a predetermined condition is satisfied in order to determine a sign of violent behavior. The controller 15 is based on the coordinates of a predetermined part of the monitored target T and another person W extracted by the extraction unit 11 or the coordinates of an object related to the other person W in order to determine a sign of violent movement. It may be determined whether or not a predetermined condition is satisfied. Further, the controller 15 may output a predetermined warning signal when determining a sign of violent action by satisfying a predetermined condition. The predetermined conditions will be further described later. Further, the operation of the controller 15 of the electronic device 10 according to the embodiment will be further described later.

The warning unit 17 may issue a predetermined warning for calling attention to the user of the system 1 or the electronic device 10 based on the predetermined warning signal output from the controller 15. As a predetermined warning, the warning unit 17 may be any functional unit that stimulates at least one of the user's auditory sense, visual sense, and tactile sense, such as sound, voice, light, characters, video, and vibration. Specifically, the warning unit 17 may be at least one of, for example, an audio output unit such as a buzzer or a speaker, a light emitting unit such as an LED, a display unit such as an LCD, and a tactile presentation unit such as a vibrator. .. In this way, the warning unit 17 may issue a predetermined warning based on a predetermined warning signal output from the controller 15. In one embodiment, the warning unit 17 may issue a predetermined alarm as information acting on at least one of auditory, visual, and tactile sensations.

In one embodiment, in the warning unit 17, for example, before the monitored target T starts the action of violently acting against the other person W (at the time when there is a sign of the violent action), the monitored target T is the other. You may issue a warning that there is a risk of starting violence against the person W. Further, in one embodiment, the warning unit 17 continues to violence against the other person W before the monitored object T finishes the action of violently acting against the other person W. You may issue a warning that there is a risk. For example, in one embodiment, when the warning unit 17 that outputs visual information detects that the monitored target T has a risk of starting or continuing to violence against another person W, The user may be warned by light emission or a predetermined display to that effect. Further, in one embodiment, when the warning unit 17 that outputs auditory information detects that the monitored target T has a risk of starting or continuing to violence against another person W, The user may be warned to that effect by a predetermined sound or voice. In the present embodiment, the warning may be a combination of light emission or a predetermined display, and a predetermined sound or voice.

The electronic device 10 shown in FIG. 1 has a built-in warning unit 17. However, in one embodiment, in the system 1, the warning unit 17 may be provided outside the electronic device 10. In this case, the warning unit 17 and the electronic device 10 may be connected by wire or wireless, or a combination of wire and wireless.

The communication unit 19 has an interface function for communicating by wire or wirelessly. The communication method performed by the communication unit 19 of one embodiment may be a wireless communication standard. For example, wireless communication standards include communication standards for cellular phones such as 2G, 3G, 4G, and 5G. For example, the communication standards for cellular phones are LTE (Long Term Evolution), W-CDMA (Wideband Code Division Multiple Access), CDMA2000, PDC (Personal Digital Cellular), GSM (Registered Trademark) (Global System for Mobile communications), and PHS. (Personal Handy-phone System) etc. are included. For example, wireless communication standards include WiMAX (Worldwide Interoperability for Microwave Access), 802.11, WiFi, Bluetooth®, IrDA (Infrared Data Association), NFC (Near Field Communication) and the like. The communication unit 19 can support one or more of the above communication standards. The communication unit 19 may be configured to include, for example, an antenna for transmitting and receiving radio waves, an appropriate RF unit, and the like. Further, the communication unit 19 may be configured as an interface such as a connector for making a wired connection to the outside. Since the communication unit 19 can be configured by a known technique for performing wireless communication, more detailed description of hardware and the like will be omitted.

Various types of information received by the communication unit 19 may be supplied to, for example, the storage unit 13 and / or the controller 15. Various information received by the communication unit 19 may be stored in, for example, a memory built in the storage unit 13 and / or the controller 15. Further, the communication unit 19 may transmit, for example, the processing result by the controller 15, the extraction result by the extraction unit 11, and / or the information stored in the storage unit 13 to the outside.

As shown in FIG. 1, at least a part of each functional unit constituting the electronic device 10 according to the embodiment may be configured by a specific means in which software and hardware resources cooperate.

Next, the operation of the system 1 according to the embodiment will be described.

In the system 1 according to the embodiment, the position (coordinates) of each part of the body of the monitored target T in the action of the monitored target T violently acting against the other person W, and each part of the body of the other person W. The start of the violent movement may be determined from the position (coordinates) of the object (coordinates) or the position (coordinates) of the object related to the other person W.

FIG. 2 is a flowchart showing an example of the operation of the system 1 according to the embodiment. FIG. 2 may be a flowchart focusing on the operation of the electronic device 10 included in the system 1 according to the embodiment.

For example, a person suspected of developing dementia (for example, a nurse or a care recipient) may violence against another person W such as a nurse or a caregiver. In such a case, another person W may be injured. In addition, another person W feels uncomfortable or dangerous. That is, there is a risk that the safety of another person W will be threatened. Therefore, to warn another person that violence is about to be violent against another person W or that violence has been violent against another person W is to warn others around the monitored target T. Not only can it be used for the safety of the person W, but it can also be used for the safety of the surrounding environment of the monitored target T. In the system 1 according to the embodiment, as described above, the position (coordinates) of each part of the body of the monitored object T in the action of a human being such as the monitored object T violently acting against another person W. Further, the sign or start of the violent movement may be determined from the position (coordinates) of each part of the body of the other person W or the position (coordinates) of the object related to the other person W. Hereinafter, such an operation will be described in more detail.

At the time when the operation shown in FIG. 2 starts, the imaging unit 20 of the system 1 may start imaging a human such as the monitored target T. The time point at which the operation shown in FIG. 2 starts may be the time when the image pickup unit 20 starts image pickup of a human such as the monitored target T. Further, the time point at which the operation shown in FIG. 2 starts may be the time when a person such as the monitored target T enters the image pickup range of the image pickup unit 20 after the image pickup unit 20 starts image pickup.

When the operation shown in FIG. 2 starts, the controller 15 of the electronic device 10 acquires the image captured by the image pickup unit 20 (step S11).

FIG. 3 is a diagram schematically showing an example of an image acquired by the controller 15 in step S11 shown in FIG. 2, that is, an image captured by the imaging unit 20.

As shown in FIG. 3, the image pickup unit 20 may take an image of a state in which a person such as a monitored target T is approaching another person W. FIG. 3 schematically shows how a person such as the monitored target T is approaching another person W. In this case, in step S11, the controller 15 acquires an image in which a person such as the monitored target T is approaching another person W. As will be described later, the image pickup unit 20 may capture an image in a state other than the state in which a person such as the monitored target T is approaching another person W. In this case, in step S11, the controller 15 may acquire an image of a state other than the state in which a person such as the monitored target T is approaching another person W.

FIG. 4 is a diagram showing another example of the image acquired by the controller 15 in step S11 shown in FIG. 2, that is, the image captured by the imaging unit 20. As shown in FIG. 4, the image pickup unit 20 may take an image of a state in which a person such as a monitored target T is hitting another person W. FIG. 4 schematically shows how the monitored target T is hitting another person W. In FIG. 4, the monitored target T and / or another person W may be moving or stationary. In this case, in step S11, the controller 15 acquires an image including a human being such as the monitored target T and another person W. As will be described later, the image pickup unit 20 may capture an image in a state other than the state in which a person such as the monitored target T is hitting another person W. In this case, in step S11, the controller 15 may acquire an image of a state other than the state in which a person such as the monitored target T is hitting another person W.

The image pickup unit 20 may capture each image of a predetermined number of frames per second. Here, the image captured by the image pickup unit 20 may be a still image of continuous frames or a moving image. For example, the image pickup unit 20 may capture an image of 15 frames per second. In step S11, the controller 15 may acquire images of a predetermined number of frames per second captured by the image pickup unit 20.

As shown in FIG. 2, when the image captured in step S11 is acquired, the extraction unit 11 extracts the coordinates of a predetermined portion of the body of the monitored target T (step S12). The operation in step S12 may be performed by the controller 15 instead of the extraction unit 11. The extraction unit 11 further extracts not only the coordinates of the predetermined portion of the body of the monitored target T but also the coordinates of the predetermined portion of the body of the other person W. The extraction unit 11 may further extract the coordinates of the object related to the other person W.

FIG. 5 is a diagram showing an example of a predetermined portion of the body of the monitored target T extracted in step S12.

The extraction unit 11 may extract the coordinates of a predetermined portion of the body of the monitored target T as shown in FIG. 5, for example. As shown in FIG. 5, the predetermined portion from which the coordinates are extracted in step S12 may include, for example, the neck, left shoulder, left elbow, left wrist, right shoulder, right elbow, and right wrist in the body of the monitored object T. .. Further, as shown in FIG. 5, the predetermined portion for which the coordinates are extracted in step S12 further includes, for example, the left hip, left knee, left ankle, right hip, right knee, and right ankle in the body of the monitored target T. It's fine. As described above, the coordinates of the predetermined portion extracted in step S12 may be the coordinates of the predetermined joint point in the body of the monitored target T or the like.

FIG. 6 is a diagram showing an example of coordinates extracted as a predetermined part of the body of the monitored target T in the image shown in FIG. The image of the monitored object T shown in FIG. 6 is the same as the image of the monitored object T shown in FIG. FIG. 6 shows the coordinates extracted as a predetermined part of the body of the monitored object T shown in FIG. 5 in the image of the monitored object T shown in FIG.

The extraction unit 11 extracts the coordinates of the plurality of dots shown in FIG. 6 as predetermined parts of the body of the monitored target T shown in FIG. For example, the extraction unit 11 may two-dimensionally extract the coordinates of the plurality of dots shown in FIG. 6 according to the coordinate axes shown in FIG. That is, in the extraction unit 11, the lower left end portion of the image pickup range of the image captured by the image pickup unit 20 may indicate the origin of the coordinate axis shown in FIG. For example, the extraction unit 11 acquires the coordinates of the position of the neck of the monitored target T shown in FIG. 6 according to the coordinate axes shown in FIG.

Here, when the imaging unit 20 captures each image of a predetermined number of frames per second, the extraction unit 11 may extract the image as a predetermined portion of the body of the monitored target T in the predetermined number of frames per second. Further, even when the controller 15 acquires images of a predetermined number of frames per second, the extraction unit 11 may extract the images of the monitored target T as a predetermined part in the body in the predetermined number of frames per second. As an example, the extraction unit 11 may extract a predetermined portion of the body of the monitored target T at 15 frames per second.

FIG. 7 is a diagram showing the coordinates of predetermined parts extracted in the body of the monitored target T collectively, for example, in 15 frames per second. As shown in FIG. 7, in step S12, the controller 15 (or the extraction unit 11) may arrange the coordinates of the predetermined parts extracted in the body of the monitored target T side by side for each frame. As shown in FIG. 7, the extraction unit 11 may extract the coordinates of a predetermined portion two-dimensionally (as X and Y coordinates) in the body of the monitored target T for each frame. In the table shown in FIG. 7, each row schematically shows how a predetermined part of the body of the monitored target T is extracted as X and Y coordinates in each frame. Also, in the table shown in FIG. 7, the rows showing each frame are shown from top to bottom over time. The coordinates of the 15 frames shown in FIG. 7 may be, for example, the coordinates of one second in the image (or moving image) as shown in FIG. 6 tracked. Further, even after the 15 frames shown in FIG. 7, the coordinates of a predetermined portion in the body of the monitored target T may be sequentially extracted.

As described above, in one embodiment, the extraction unit 11 two-dimensionally obtains the coordinates of a predetermined number of joint points in the body of the monitored target T from the images of the predetermined number of frames per second captured by the image pickup unit 20. It may be extracted.

The coordinates of the predetermined portion in the body of the other person W may be extracted in the same manner as the coordinates of the predetermined portion in the body of the monitored target T. The coordinates of the object related to the other person W may be further extracted. Further, as shown in FIG. 6, not only the coordinates when the violence in the mode in which the monitored target T is hitting the other person W is being performed, but also the violence in the case where the violence in another mode is being performed. Coordinates may be extracted.

FIG. 8 is a diagram showing another example of the monitored target T and another person W recognized in step S12. FIG. 8 is a diagram schematically showing how the monitored target T is biting the other person W, and the feature points obtained as a result of the extraction unit 11 recognizing the monitored target T and the other person W. It is the figure which superposed and displayed.

FIG. 9 is a diagram showing another example of the wheelchair C in which the monitored target T and another person W recognized in step S12, and the other person W are sitting. FIG. 9 is a diagram schematically showing how the monitored target T strongly pushes the wheelchair C on which the other person W is sitting, and the extraction unit 11 shows the monitored target T, the other person W, and the wheelchair. It is a figure which superposed and displayed the feature points obtained as a result of recognizing C.

The extraction unit 11 may recognize not only FIG. 6 but also predetermined feature points in the monitored target T and another person W as shown in FIGS. 8 and 9. Further, the extraction unit 11 may two-dimensionally extract the monitored target T and another person W shown in FIGS. 8 and 9 according to the coordinate axes shown in FIGS. 8 and 9. Further, the extraction unit 11 may extract the positions (for example, coordinates) of the monitored target T and the other person W in the image when recognizing the monitored target T and the other person W. Further, the extraction unit 11 may extract the wheelchair C as an object related to the other person W, and extract the coordinates of the wheelchair C or the coordinates of the feature point of the wheelchair C.

The extraction unit 11 normalizes the coordinates according to the maximum and minimum values of the coordinates (X, Y) in the extracted predetermined number of frames (for example, 15 frames per second) (step S13).

It is assumed that the coordinates of the predetermined portion extracted in step S12 vary due to, for example, the size of the body of the monitored target T. Further, it is assumed that the coordinates of the predetermined portion extracted in step S12 vary due to, for example, the distance between the image pickup unit 20 and the monitored target T, the direction from the image pickup unit 20 toward the monitored target T, and the like. Will be done. Therefore, in one embodiment, by normalizing the X-direction component and the Y-direction component of the coordinates extracted in step S12, the extracted coordinates can be determined using a general-purpose standard. do.

In this case, for example, the extracted X and Y coordinates may be normalized based on the maximum and minimum values of the extracted X and Y coordinates in 15 frames per second. Here, the maximum value of the X coordinate extracted in step S12 is Xmax, and the minimum value of the X coordinate extracted in step S12 is Xmin. Further, the maximum value of the X coordinate after normalization is set to X'max. In this case, the X coordinate (X) before normalization can be converted into the X coordinate (X') after normalization by using the following equation (1).

X'= ((X-Xmin) / (Xmax-Xmin)) · X'max (1)

Similarly, the maximum value of the Y coordinate extracted in step S12 is Ymax, and the minimum value of the Y coordinate extracted in step S12 is Ymin. Further, the maximum value of the Y coordinate after normalization is set to Y'max. In this case, the Y coordinate (Y) before normalization can be converted into the Y coordinate (Y') after normalization by using the following equation (2).

Y'= ((Y-Ymin) / (Ymax-Ymin)) · Y'max (2)

By normalizing the X-direction component and the Y-direction component of the extracted coordinates according to the above equations (1) and (2), the individual difference of the monitored target T and the imaging unit 20 determine the monitored target T. It can be expected to reduce the influence of the imaged environment on the judgment of the coordinates.

As described above, in one embodiment, the extraction unit 11 sets each direction component of the coordinates of a predetermined number of joint points in the body of the monitored target T two-dimensionally extracted to the maximum value and the minimum value of each direction component. It may be normalized based on the value. Further, such an operation may be performed by the controller 15 instead of the extraction unit 11.

By normalizing the coordinates shown in step S13, each of the coordinates (X, Y) shown in FIG. 7 is normalized to the coordinates (X', Y'), respectively.

The coordinates of the predetermined part in the body of the other person W may be normalized in the same manner as the coordinates of the predetermined part in the body of the monitored object T. The coordinates of the object associated with the other person W may be further normalized.

The controller 15 determines the start of the violent movement based on the normalized coordinates (step S14). In step S14, the controller 15 may determine whether or not the violent movement has started based on the displacement of the normalized coordinates. In one embodiment, the controller 15 may determine in step S14 that the violent action has started when the normalized coordinates satisfy a predetermined condition. The controller 15 may determine that there is a sign of violent movement when the normalized coordinates satisfy a predetermined condition. Hereinafter, in step S14, the case where it is determined that there is a sign of violent movement and the case where it is determined that the violent movement has started will be further described.

The violent action is executed when the monitored target T approaches another person W. In the controller 15, when the distance between at least a part of the predetermined part of the monitored target T and at least a part of the predetermined part of the other person W is less than the first threshold value, the monitored target T is another. It may be determined that the person W has approached.

When the wrist of the monitored target T moves toward the chest, foot, hand or shoulder of another person W, the controller 15 is said to have a sign of violent movement using a hand such as hitting, hitting or pushing. You may judge. When the controller 15 determines that there is a sign of violent movement, the distance D0 between the wrist feature point Ta of the monitored target T and the elbow feature point Wa of another person W is further determined, for example, as shown in FIG. When it becomes less than the second threshold value, it may be determined that a violent action using a hand such as hitting, hitting or pushing has started.

The controller 15 violently operates in a manner in which the monitored target T hits another person W when the monitored target T's elbow moves upward and the monitored target T's wrist moves above the elbow. It may be determined that there is a sign of. The controller 15 said that when the wrist of the monitored target T moved above the elbow and then moved down at a predetermined speed or higher, the monitored target T started a violent action in which the monitored target T hit another person W. You may judge.

The controller 15 is a violent operation in which the monitored target T hits another person W when the monitored target T's wrist moves above the elbow and the monitored target T's elbow does not move upward. It may be determined that there is a sign of. In the controller 15, when the wrist of the monitored target T moves above the elbow and then the wrist of the monitored target T moves horizontally at a predetermined speed or higher, the monitored target T hits another person W. It may be determined that the violent movement of the embodiment has started.

The controller 15 is monitored when the wrist of the monitored target T moves above the elbow and the wrist of the monitored target T moves toward at least a part of a predetermined portion of another person W. It may be determined that there is a sign of a violent action in which T pushes another person W. In the controller 15, after the wrist of the monitored target T moves toward at least a part of a predetermined portion of the other person W, the wrist of the monitored target T moves horizontally at a predetermined speed or more, and is monitored. When the wrist of the target T is separated from the other person W, it may be determined that the violent movement in which the monitored target T pushes the other person W has started.

When the ankle of the monitored target T moves toward the foot of another person W or the like, the controller 15 may determine that there is a sign of a violent movement in which the foot is used, such as kicking. The controller 15 may determine that when the knee and ankle of the monitored target T move upward or horizontally, there is a sign of a violent movement in which the monitored target T kicks another person W. In the controller 15, when the knee and ankle of the monitored target T move upward or horizontally and then the knee and ankle of the monitored target T move further upward or horizontally, the monitored target T is another person. It may be determined that the violent action of kicking W has started.

When the shoulder of the monitored target T moves toward the shoulder or arm of another person W, the controller 15 may determine that there is a sign of violent movement in a ramming manner. When the shoulder of the monitored target T moves toward the other person W, the controller 15 may determine that there is a sign of a violent movement in which the monitored target T rams the other person W. In the controller 15, after the shoulder of the monitored target T moves toward the other person W, the distance between the shoulder of the monitored target T and at least a part of the predetermined portion of the other person W becomes less than the second threshold value. In this case, it may be determined that the violent movement in which the monitored target T hits another person W has started.

As shown in FIG. 8, for example, the controller 15 bites when the distance D0 between the feature point Ta of the neck of the monitored target T and the feature point Wa of the wrist of another person W becomes less than the second threshold value. It may be determined that the violent movement has started. In the controller 15, when the neck and shoulder of the monitored target T move downward and the wrist of the monitored target T moves toward the other person W, the monitored target T moves to the other person W. It may be determined that there is a sign of violent movement in a manner of biting. In the controller 15, after the neck and shoulder of the monitored target T move downward and the wrist of the monitored target T moves toward the other person W, the neck and shoulder of the monitored target T further move downward. When moving downward, it may be determined that the violent movement in which the monitored target T bites another person W has started.

The violent movement also includes, for example, as shown in FIG. 9, a mode in which the monitored object T violently acts on an object (such as a chair or a wheelchair) related to another person W. This aspect of violent action is performed when the monitored object T approaches an object related to another person W. When the distance between at least a part of the predetermined portion of the monitored target T and at least a part of the object related to the other person W is less than the first threshold value, the controller 15 determines the monitored target T. It may be determined that the person has approached an object related to the other person W.

As shown in FIG. 9, for example, when the distance between the feature point Ta of the wrist of the monitored target T and the handle Ca of the wheelchair C related to the other person W becomes less than the second threshold value, the controller 15 makes another. It may be determined that the violent movement in the form of shaking the wheelchair on which the person W is sitting has started. In the controller 15, when the wrist of the monitored target T moves toward an object related to the other person W (for example, the handle of a wheelchair or the backrest of a chair), the monitored target T sits on the other person W. It may be determined that there is a sign of violent movement in the form of shaking the wheelchair. In the controller 15, when the wrist of the monitored target T moves toward an object related to another person W and then the wrist of the monitored target T moves back and forth in the horizontal direction, the monitored target T moves to another person. It may be determined that the violent movement in the form of shaking the wheelchair on which the person W is sitting has started.

The first threshold value and the second threshold value may be appropriately determined based on the physique and the like of the monitored target T and another person W.

When the controller 15 determines that the violent operation has started (step S14: Y), the controller 15 outputs a predetermined warning signal (step S15). The controller 15 may output a predetermined warning signal when it is determined that there is a sign of violent movement. After executing the procedure in step S15, the controller 15 ends the execution of the flowchart of FIG.

When the controller 15 does not determine that the violent operation has started (step S14: N), the controller 15 ends the execution of the flowchart of FIG.

According to the system 1 according to the embodiment, for example, it is possible to detect a sign or start of an action in which the monitored target T violently acts against another person W. Therefore, according to the system 1 according to the embodiment, before the monitored target T begins to violence against another person W, or while the violence continues to be violent, before the violence is finished. A predetermined warning can be issued. Therefore, according to the system 1 according to the embodiment, the staff of a nursing care facility, for example, is trying to violence or is violent by the monitored target T such as a nurse or a care recipient. I can recognize that. As a result, the staff can protect another person W who is the target of violence. Therefore, according to the system 1 according to the embodiment, it is possible to provide safety around the monitored target T.

FIG. 10 is a diagram further explaining the determination process in step S14 of FIG. FIG. 10 shows the state of the monitored target T captured in the image data acquired in step S11 of FIG. 2, for example, in the left column. As shown in FIG. 10, in the captured image data, the monitored target T approaches the other person W after entering the room in which the image pickup unit 20 is installed, reaches for the other person W, and violence. Suppose that the operation is started. From the moment when the violent operation starts, the controller 15 shall perform the operation after step S11 shown in FIG. 2 as a process of 15 frames per second. That is, in the system 1, the image pickup unit 20 may capture an image of 15 frames per second. Further, in the system 1, the controller 15 may acquire an image of 15 frames per second. Further, in the system 1, the extraction unit 11 extracts the coordinates of the body joint point of the monitored target T, the coordinates of the body joint point of the other person W, or an object related to the other person W from the image of 15 frames per second. It may be used to extract the coordinates of. Further, in the system 1, the controller 15 (or the extraction unit 11) may normalize the coordinates extracted from the image of 15 frames per second. Further, in the system 1, the controller 15 may determine the start of the violent movement from the normalized coordinates of 15 frames per second.

In the central column of FIG. 10, a state in which the controller 15 continuously acquires frames of image data is conceptually shown. Here, the image data of each frame may be the coordinates extracted from the image data or may be the normalized coordinates. Further, in the central column of FIG. 10, the hatched image data represents a frame for one second after the violent movement starts.

In such a situation, the controller 15 may determine the start of the violent movement at that time based on 15 frames per second (frames 1 to 15) after the start of the violent movement (FIG. 10). Judgment shown 1). Next, the controller 15 may determine the start of the violent movement at that time based on 15 frames per second from frame 2 to frame 16 (determination 2 shown in FIG. 10). Further, the controller 15 may determine the start of the violent movement at that time based on 15 frames per second from the frame (N-14) to the frame N (determination 3 shown in FIG. 10). By repeating the above operation, the controller 15 determines the start of the violent operation 15 times per second. Therefore, according to the system 1 according to the embodiment, even if the start of the violent action that should be originally determined is not determined for some reason in the determination 1, the determination 2 and the determination 3 shown in FIG. 10, for example, 15 times per second. The risk of misreporting can be reduced by the judgment of.

In the above-described embodiment, for example, as shown in FIG. 5, the extraction unit 11 has described an example in which the extraction unit 11 extracts the coordinates of 13 sites as the joint points of the monitored target T. However, in one embodiment, the extraction unit 11 may extract the coordinates of the parts having more than 13 parts, or may extract the coordinates of the parts having less than 13. Further, in the above-described embodiment, the system 1 has described an example of processing 15 frames per second. However, in one embodiment, the system 1 or each functional unit constituting the system 1 may process more than 15 frames per second or may process less than 15 frames per second. In one embodiment, the number of joint points handled by the system 1 and / or the number of frames processed may be adjusted so that the determination of the start of the violent movement gives a reasonable result.

As described above, in one embodiment, the controller 15 may determine at least one of the number of the frames and the number of the joint points so that the validity of the determination of the start of the violent movement becomes more than a predetermined value.

In the above embodiment, since the captured image data is used, visible light is used as a detection target for monitoring. However, the present disclosure is not limited to such cases, and other detection targets such as arbitrary electromagnetic waves, sound waves, temperatures, and vibrations may be arbitrarily used.

Although the embodiments according to the present disclosure have been described based on the drawings and examples, it should be noted that those skilled in the art can make various modifications or modifications based on the present disclosure. It should be noted, therefore, that these modifications or modifications are within the scope of this disclosure. For example, the functions included in each component or each step can be rearranged so as not to be logically inconsistent, and a plurality of components or steps can be combined or divided into one. Is. Although the embodiment according to the present disclosure has been mainly described with respect to the apparatus, the embodiment according to the present disclosure can also be realized as a method including steps executed by each component of the apparatus. The embodiments according to the present disclosure can also be realized as a method, a program, or a storage medium on which a program is recorded, which is executed by a processor included in the apparatus. It should be understood that these are also included in the scope of this disclosure.

For example, the process of the system 1 and the electronic device 10 according to the embodiment may be simplified, and the process of normalizing the coordinates shown in step S13 of FIG. 2 may be omitted.

The above-described embodiment is not limited to the implementation as the system 1. For example, the above-described embodiment may be implemented as an electronic device 10 included in the system 1. Further, the above-described embodiment may be implemented as a monitoring method using a device such as an electronic device 10. Further, the above-described embodiment may be implemented as a program executed by, for example, a device such as an electronic device 10 or an information processing device (for example, a computer). Further, in the technique of the present disclosure, not all of the components of the electronic device 10 shown in FIG. 1 need be present in one housing or server. For example, each part such as a controller or a storage part of a component of an electronic device 10 is connected to each other by a network consisting of a wired, wireless, or a combination thereof, and is connected to different housings, servers, devices, rooms, buildings, regions, countries, and the like. It may be arranged arbitrarily.

1 System 10 Electronic equipment 11 Extraction unit 13 Storage unit 15 Controller 17 Warning unit 19 Communication unit 20 Imaging unit

Claims (33)

An imaging unit that captures images of the monitored object and other people,
An extraction unit that extracts the coordinates of the predetermined portion of the monitored object and the coordinates of the predetermined portion of the other person from the image captured by the imaging unit.
When the distance between at least a part of the predetermined part of the monitored object extracted by the extraction unit and at least a part of the predetermined part of the other person is less than the first threshold value, the monitored object Equipped with a controller that determines that the person has approached the other person.
When the controller determines that the monitored object has approached the other person, the elbow of the monitored object moves upward, and the wrist of the monitored object moves above the elbow. Or, a system that outputs a predetermined warning signal when the wrist to be monitored moves above the elbow and then moves downward at a predetermined speed or higher. An imaging unit that captures images of the monitored object and other people,
An extraction unit that extracts the coordinates of the predetermined portion of the monitored object and the coordinates of the predetermined portion of the other person from the image captured by the imaging unit.
When the distance between at least a part of the predetermined part of the monitored object extracted by the extraction unit and at least a part of the predetermined part of the other person is less than the first threshold value, the monitored object Equipped with a controller that determines that the person has approached the other person.
When the controller determines that the monitored object approaches the other person, the wrist of the monitored object moves above the elbow, and the elbow of the monitored object does not move upward. Or, a system that outputs a predetermined warning signal when the wrist of the monitored object moves above the elbow and then the wrist of the monitored object moves horizontally at a predetermined speed or higher. An imaging unit that captures images of the monitored object and other people,
An extraction unit that extracts the coordinates of the predetermined portion of the monitored object and the coordinates of the predetermined portion of the other person from the image captured by the imaging unit.
When the distance between at least a part of the predetermined part of the monitored object extracted by the extraction unit and at least a part of the predetermined part of the other person is less than the first threshold value, the monitored object Equipped with a controller that determines that the person has approached the other person.
When the controller determines that the monitored target approaches the other person, the monitored target's wrist moves above the elbow, and the monitored target's wrist is the other person's wrist. When the wrist of the monitored object moves toward at least a part of the predetermined part of the predetermined part, or after the wrist of the monitored object moves toward at least a part of the predetermined part of the other person, the wrist of the monitored object moves horizontally. A system that outputs a predetermined warning signal when the wrist of the monitored object moves away from the other person while moving at a predetermined speed or higher. An imaging unit that captures images of the monitored object and other people,
An extraction unit that extracts the coordinates of the predetermined portion of the monitored object and the coordinates of the predetermined portion of the other person from the image captured by the imaging unit.
When the distance between at least a part of the predetermined part of the monitored object extracted by the extraction unit and at least a part of the predetermined part of the other person is less than the first threshold value, the monitored object Equipped with a controller that determines that the person has approached the other person.
When the controller determines that the monitored object approaches the other person, the monitored knee and ankle move upward or horizontally, or the monitored knee and ankle move. A system that outputs a predetermined warning signal when the knee and ankle to be monitored move further upward or horizontally after moving upward or horizontally. An imaging unit that captures images of the monitored object and other people,
An extraction unit that extracts the coordinates of the predetermined portion of the monitored object and the coordinates of the predetermined portion of the other person from the image captured by the imaging unit.
When the distance between at least a part of the predetermined part of the monitored object extracted by the extraction unit and at least a part of the predetermined part of the other person is less than the first threshold value, the monitored object Equipped with a controller that determines that the person has approached the other person.
When the controller determines that the monitored object approaches the other person, the shoulder of the monitored object moves toward the other person, or the shoulder of the monitored object is the other person. A system that outputs a predetermined warning signal when the distance between the shoulder of the monitored object and at least a part of a predetermined portion of the other person is less than the second threshold value after moving toward the person. .. An imaging unit that captures images of the monitored object and other people,
An extraction unit that extracts the coordinates of the predetermined portion of the monitored object and the coordinates of the predetermined portion of the other person from the image captured by the imaging unit.
When the distance between at least a part of the predetermined part of the monitored object extracted by the extraction unit and at least a part of the predetermined part of the other person is less than the first threshold value, the monitored object Equipped with a controller that determines that the person has approached the other person.
When the controller determines that the monitored object approaches the other person, the neck and shoulders of the monitored object move downward, and the wrist of the monitored object is the other person. Or after the neck and shoulders of the monitored object move downward and the wrist of the monitored object moves toward the other person, the monitored object A system that outputs a predetermined warning signal when the neck and shoulders move further downwards. An image pickup unit that captures an image of a monitored object, another person, and an object related to the other person.
An extraction unit that extracts the coordinates of a predetermined portion of the monitored object and the coordinates of an object related to the other person from the image captured by the imaging unit.
When the distance between at least a part of the predetermined part of the monitored object extracted by the extraction unit and at least a part of the object related to the other person is less than the first threshold value, the monitored part is monitored. It is equipped with a controller that determines that the target has approached an object related to the other person.
When the controller determines that the monitored object has approached an object related to the other person, the wrist of the monitored object moves toward the object related to the other person, or the said. A system that outputs a predetermined warning signal when the wrist of the monitored object moves toward an object related to the other person and then the wrist of the monitored object moves back and forth in the horizontal direction. The extraction unit extracts the coordinates of the predetermined portion of the monitored object from the images of a predetermined number of frames per unit time captured by the imaging unit.
The system according to any one of claims 1 to 7, wherein the controller outputs the predetermined warning signal based on the displacement of the coordinates of the predetermined portion of the monitored object extracted by the extraction unit. .. From claim 1, the extraction unit two-dimensionally extracts the coordinates of a predetermined number of joint points in the body to be monitored based on each image of a predetermined number of frames per second captured by the imaging unit. The system according to any one of up to 8. The extraction unit normalizes each direction component of the coordinates of a predetermined number of joint points in the body to be monitored, which is two-dimensionally extracted, based on the maximum value and the minimum value of each direction component. The system according to any one of Items 1 to 9. The system according to claim 8 or 9, wherein the controller determines the number of frames so that the validity of the output of the predetermined warning signal is equal to or higher than a predetermined value. The system according to claim 9 or 10, wherein the controller determines the number of the joint points so that the validity of the output of the predetermined warning signal becomes equal to or higher than a predetermined value. An extraction unit that extracts the coordinates of a predetermined part of the monitored object and the coordinates of a predetermined part of the other person from an image obtained by capturing an image of the monitored object and another person.
When the distance between at least a part of the predetermined part of the monitored object extracted by the extraction unit and at least a part of the predetermined part of the other person is less than the first threshold value, the monitored object Equipped with a controller that determines that the person has approached the other person.
When the controller determines that the monitored object has approached the other person, the elbow of the monitored object moves upward, and the wrist of the monitored object moves above the elbow. Or, an electronic device that outputs a predetermined warning signal when the wrist to be monitored moves above the elbow and then moves downward at a predetermined speed or higher. An extraction unit that extracts the coordinates of a predetermined part of the monitored object and the coordinates of a predetermined part of the other person from an image obtained by capturing an image of the monitored object and another person.
When the distance between at least a part of the predetermined part of the monitored object extracted by the extraction unit and at least a part of the predetermined part of the other person is less than the first threshold value, the monitored object Equipped with a controller that determines that the person has approached the other person.
When the controller determines that the monitored object approaches the other person, the wrist of the monitored object moves above the elbow, and the elbow of the monitored object does not move upward. Or, an electronic device that outputs a predetermined warning signal when the wrist of the monitored object moves above the elbow and then the wrist of the monitored object moves horizontally at a predetermined speed or higher. An extraction unit that extracts the coordinates of a predetermined part of the monitored object and the coordinates of a predetermined part of the other person from an image obtained by capturing an image of the monitored object and another person.
When the distance between at least a part of the predetermined part of the monitored object extracted by the extraction unit and at least a part of the predetermined part of the other person is less than the first threshold value, the monitored object Equipped with a controller that determines that the person has approached the other person.
When the controller determines that the monitored target approaches the other person, the monitored target's wrist moves above the elbow, and the monitored target's wrist is the other person's wrist. The wrist of the monitored object moves horizontally toward at least a part of the predetermined part, or after the wrist of the monitored object moves toward at least a part of the predetermined part of the other person. An electronic device that outputs a predetermined warning signal when the wrist of the monitored object moves away from the other person while moving at a predetermined speed or higher. An extraction unit that extracts the coordinates of a predetermined part of the monitored object and the coordinates of a predetermined part of the other person from an image obtained by capturing an image of the monitored object and another person.
When the distance between at least a part of the predetermined part of the monitored object extracted by the extraction unit and at least a part of the predetermined part of the other person is less than the first threshold value, the monitored object Equipped with a controller that determines that the person has approached the other person.
When the controller determines that the monitored object approaches the other person, the monitored knee and ankle move upward or horizontally, or the monitored knee and ankle move. An electronic device that outputs a predetermined warning signal when the knee and ankle to be monitored move further upward or horizontally after moving upward or horizontally. An extraction unit that extracts the coordinates of a predetermined part of the monitored object and the coordinates of a predetermined part of the other person from an image obtained by capturing an image of the monitored object and another person.
When the distance between at least a part of the predetermined part of the monitored object extracted by the extraction unit and at least a part of the predetermined part of the other person is less than the first threshold value, the monitored object Equipped with a controller that determines that the person has approached the other person.
When the controller determines that the monitored object has approached the other person, the shoulder of the monitored object moves toward the other person, or the shoulder of the monitored object is the other person. An electronic device that outputs a predetermined warning signal when the distance between the shoulder of the monitored object and at least a part of a predetermined portion of the other person becomes less than the second threshold value after moving toward the person. machine. An extraction unit that extracts the coordinates of a predetermined part of the monitored object and the coordinates of a predetermined part of the other person from an image obtained by capturing an image of the monitored object and another person.
When the distance between at least a part of the predetermined part of the monitored object extracted by the extraction unit and at least a part of the predetermined part of the other person is less than the first threshold value, the monitored object Equipped with a controller that determines that the person has approached the other person.
When the controller determines that the monitored object approaches the other person, the neck and shoulders of the monitored object move downward, and the wrist of the monitored object is the other person. Or after the neck and shoulders of the monitored object move downward and the wrist of the monitored object moves toward the other person, the monitored object An electronic device that outputs a predetermined warning signal when the neck and shoulders move further downward. Extraction to extract the coordinates of the predetermined part of the monitored object and the coordinates of the object related to the other person from the image obtained by capturing the monitored object, the other person, and the object related to the other person. Department and
When the distance between at least a part of the predetermined part of the monitored object extracted by the extraction unit and at least a part of the object related to the other person is less than the first threshold value, the monitored part is monitored. It is equipped with a controller that determines that the target has approached an object related to the other person.
When the controller determines that the monitored object has approached an object related to the other person, the wrist of the monitored object moves toward the object related to the other person, or the said. An electronic device that outputs a predetermined warning signal when the wrist of the monitored object moves toward an object related to the other person and then the wrist of the monitored object moves back and forth in the horizontal direction. An extraction step of extracting the coordinates of a predetermined part of the monitored object and the coordinates of a predetermined part of the other person from an image obtained by capturing an image of the monitored object and another person.
When the distance between at least a part of the predetermined part of the monitored object extracted in the extraction step and at least a part of the predetermined part of the other person is less than the first threshold value, the monitored object The determination step of determining that the person has approached the other person,
When it is determined that the monitored object has approached the other person, the elbow of the monitored object moves upward, and the wrist of the monitored object moves above the elbow, or the above. A method of controlling an electronic device, including an output step that outputs a predetermined warning signal when the wrist to be monitored moves above the elbow and then moves downward at a predetermined speed or higher. An extraction step of extracting the coordinates of a predetermined part of the monitored object and the coordinates of a predetermined part of the other person from an image obtained by capturing an image of the monitored object and another person.
When the distance between at least a part of the predetermined part of the monitored object extracted in the extraction step and at least a part of the predetermined part of the other person is less than the first threshold value, the monitored object The determination step of determining that the person has approached the other person,
When it is determined that the monitored object has approached the other person, the wrist of the monitored object moves above the elbow, and the elbow of the monitored object does not move upward, or the above. An electronic device that includes an output step that outputs a predetermined warning signal when the monitored wrist moves above the elbow and then the monitored wrist moves horizontally at a predetermined speed or higher. Control method. An extraction step of extracting the coordinates of a predetermined part of the monitored object and the coordinates of a predetermined part of the other person from an image obtained by capturing an image of the monitored object and another person.
When the distance between at least a part of the predetermined part of the monitored object extracted in the extraction step and at least a part of the predetermined part of the other person is less than the first threshold value, the monitored object The determination step of determining that the person has approached the other person,
When it is determined that the monitored object has approached the other person, the wrist of the monitored object moves above the elbow, and the wrist of the monitored object is at least a predetermined portion of the other person. When moving toward a part, or after the wrist of the monitored object moves toward at least a part of a predetermined part of the other person, the wrist of the monitored object moves horizontally at a predetermined speed or higher. A method of controlling an electronic device, comprising an output step of moving and outputting a predetermined warning signal when the monitored wrist is separated from the other person. An extraction step of extracting the coordinates of a predetermined part of the monitored object and the coordinates of a predetermined part of the other person from an image obtained by capturing an image of the monitored object and another person.
When the distance between at least a part of the predetermined part of the monitored object extracted in the extraction step and at least a part of the predetermined part of the other person is less than the first threshold value, the monitored object The determination step of determining that the person has approached the other person,
When it is determined that the monitored object approaches the other person, the knee and ankle of the monitored object move upward or horizontally, or the knee and ankle of the monitored object move upward or horizontally. A method for controlling an electronic device, which comprises an output step of outputting a predetermined warning signal when the knee and ankle to be monitored move further upward or horizontally after moving to. An extraction step of extracting the coordinates of a predetermined part of the monitored object and the coordinates of a predetermined part of the other person from an image obtained by capturing an image of the monitored object and another person.
When the distance between at least a part of the predetermined part of the monitored object extracted in the extraction step and at least a part of the predetermined part of the other person is less than the first threshold value, the monitored object The determination step of determining that the person has approached the other person,
When it is determined that the monitored object approaches the other person, the shoulder of the monitored object moves toward the other person, or the shoulder of the monitored object faces the other person. A step of outputting a predetermined warning signal when the distance between the shoulder of the monitored object and at least a part of the predetermined portion of the other person becomes less than the second threshold value after moving. How to control electronic devices. An extraction step of extracting the coordinates of a predetermined part of the monitored object and the coordinates of a predetermined part of the other person from an image obtained by capturing an image of the monitored object and another person.
When the distance between at least a part of the predetermined part of the monitored object extracted in the extraction step and at least a part of the predetermined part of the other person is less than the first threshold value, the monitored object The determination step of determining that the person has approached the other person,
When it is determined that the monitored object approaches the other person, the neck and shoulders of the monitored object move downward, and the wrist of the monitored object moves toward the other person. Or, after the neck and shoulders of the monitored object move downward and the wrist of the monitored object moves toward the other person, the neck and shoulders of the monitored object move downward. A method of controlling an electronic device, including an output step that outputs a predetermined warning signal when further downwardly moved. Extraction to extract the coordinates of the predetermined part of the monitored object and the coordinates of the object related to the other person from the image obtained by capturing the monitored object, the other person, and the object related to the other person. Steps and
When the distance between at least a part of the predetermined portion of the monitored object extracted in the extraction step and at least a part of the object related to the other person is less than the first threshold value, the monitored object is monitored. A determination step for determining that the target has approached an object related to the other person, and
When it is determined that the monitored object has approached an object related to the other person, the wrist of the monitored object moves toward the object related to the other person, or the monitored object of the monitored object. An electronic device that includes an output step that outputs a predetermined warning signal when the monitored wrist moves in a horizontal reciprocation after the wrist has moved toward an object related to the other person. Control method. An extraction step of extracting the coordinates of a predetermined part of the monitored object and the coordinates of a predetermined part of the other person from an image obtained by capturing an image of the monitored object and another person.
When the distance between at least a part of the predetermined part of the monitored object extracted in the extraction step and at least a part of the predetermined part of the other person is less than the first threshold value, the monitored object The determination step of determining that the person has approached the other person,
When it is determined that the monitored object has approached the other person, the elbow of the monitored object moves upward, and the wrist of the monitored object moves above the elbow, or the above. A program that causes a computer to perform an output step that outputs a predetermined warning signal when the wrist to be monitored moves above the elbow and then moves down at a predetermined speed or higher. An extraction step of extracting the coordinates of a predetermined part of the monitored object and the coordinates of a predetermined part of the other person from an image obtained by capturing an image of the monitored object and another person.
When the distance between at least a part of the predetermined part of the monitored object extracted in the extraction step and at least a part of the predetermined part of the other person is less than the first threshold value, the monitored object The determination step of determining that the person has approached the other person,
When it is determined that the monitored object has approached the other person, the wrist of the monitored object moves above the elbow, and the elbow of the monitored object does not move upward, or the above. A program that causes the computer to execute an output step that outputs a predetermined warning signal when the monitored wrist moves above the elbow and then the monitored wrist moves horizontally at a predetermined speed or higher. .. An extraction step of extracting the coordinates of a predetermined part of the monitored object and the coordinates of a predetermined part of the other person from an image obtained by capturing an image of the monitored object and another person.
When the distance between at least a part of the predetermined part of the monitored object extracted in the extraction step and at least a part of the predetermined part of the other person is less than the first threshold value, the monitored object The determination step of determining that the person has approached the other person,
When it is determined that the monitored object has approached the other person, the wrist of the monitored object moves above the elbow, and the wrist of the monitored object is at least a predetermined portion of the other person. When moving toward a part, or after the wrist of the monitored object moves toward at least a part of a predetermined part of the other person, the wrist of the monitored object moves horizontally at a predetermined speed or higher. A program that causes a computer to perform an output step that outputs a predetermined warning signal when the wrist to be monitored moves away from the other person. An extraction step of extracting the coordinates of a predetermined part of the monitored object and the coordinates of a predetermined part of the other person from an image obtained by capturing an image of the monitored object and another person.
When the distance between at least a part of the predetermined part of the monitored object extracted in the extraction step and at least a part of the predetermined part of the other person is less than the first threshold value, the monitored object The determination step of determining that the person has approached the other person,
When it is determined that the monitored object approaches the other person, the knee and ankle of the monitored object move upward or horizontally, or the knee and ankle of the monitored object move upward or horizontally. A program that causes a computer to perform an output step that outputs a predetermined warning signal when the knee and ankle to be monitored move further upward or horizontally after moving to. An extraction step of extracting the coordinates of a predetermined part of the monitored object and the coordinates of a predetermined part of the other person from an image obtained by capturing an image of the monitored object and another person.
When the distance between at least a part of the predetermined part of the monitored object extracted in the extraction step and at least a part of the predetermined part of the other person is less than the first threshold value, the monitored object The determination step of determining that the person has approached the other person,
When it is determined that the monitored object approaches the other person, the shoulder of the monitored object moves toward the other person, or the shoulder of the monitored object faces the other person. After moving, the computer is provided with an output step that outputs a predetermined warning signal when the distance between the shoulder of the monitored object and at least a part of the predetermined portion of the other person is less than the second threshold value. The program to be executed. An extraction step of extracting the coordinates of a predetermined part of the monitored object and the coordinates of a predetermined part of the other person from an image obtained by capturing an image of the monitored object and another person.
When the distance between at least a part of the predetermined part of the monitored object extracted in the extraction step and at least a part of the predetermined part of the other person is less than the first threshold value, the monitored object The determination step of determining that the person has approached the other person,
When it is determined that the monitored object approaches the other person, the neck and shoulders of the monitored object move downward, and the wrist of the monitored object moves toward the other person. Or, after the neck and shoulders of the monitored object move downward and the wrist of the monitored object moves toward the other person, the neck and shoulders of the monitored object move downward. A program that causes a computer to perform an output step that outputs a predetermined warning signal when it moves further downward. Extraction to extract the coordinates of the predetermined part of the monitored object and the coordinates of the object related to the other person from the image obtained by capturing the monitored object, the other person, and the object related to the other person. Steps and
When the distance between at least a part of the predetermined portion of the monitored object extracted in the extraction step and at least a part of the object related to the other person is less than the first threshold value, the monitored object is monitored. A determination step for determining that the target has approached an object related to the other person, and
When it is determined that the monitored object has approached an object related to the other person, the wrist of the monitored object moves toward the object related to the other person, or the monitored object of the monitored object. A program that causes a computer to perform an output step that outputs a predetermined warning signal when the monitored wrist moves back and forth in the horizontal direction after the wrist moves toward an object related to the other person. ..

Images