JP2021043932A - Information process system, information processing device, server device, program, or method - Google Patents

Abstract

To provide an information process system that uses image data imaged by an imaging apparatus.SOLUTION: A system comprises: an imaging part; a display part that can display an image imaged by the imaging part; a region part which can set a region in the display part; and a determination part that can determine whether or not a person in the image imaged by the imaging part is in the region or out of the region. The region set by the region part is associated with a position in the display part. The region set by the region part is not associated with a target in the image imaged by the imaging part. The determination part determines whether or not the person wears a prescribed item when it is determined that the person is in the region.SELECTED DRAWING: Figure 1

Description

The technology disclosed in this application relates to an information processing system, an information processing device, a server device, a program, or a method.

The image captured by the image pickup device can provide objective information.

Japanese Unexamined Patent Publication No. 2012-203677 Japanese Unexamined Patent Publication No. 2003-105807 Japanese Unexamined Patent Publication No. 2019-101516

In the invention according to Patent Document 1, the target area is only set based on the heavy machinery and the worker who shift in the image. Further, in the invention according to Patent Document 2, the entry prohibited area is only determined according to the position and orientation of the work vehicle. Further, the invention according to Patent Document 3 only acquires the status of the worker. Therefore, in each case, the information in the captured image cannot be used appropriately. Therefore, various embodiments of the present invention provide an information processing system, an information processing device, a server device, a program, or a method in order to solve the above problems.

The first embodiment of the system according to the present application is
Imaging unit and
A display unit capable of displaying an image captured by the imaging unit, and a display unit capable of displaying the image.
In the display unit, an area unit in which an area can be set and an area unit
A determination unit capable of determining whether a person in an image captured by the imaging unit is within the region or outside the region.
System with.
A second embodiment of the system according to the present application is
The area set by the area unit is related to the position in the display unit.
The system of the first embodiment.
A third embodiment of the system according to the present application is
The region set by the region portion is not related to the object in the image captured by the imaging unit.
The system of the first or second embodiment.
A fourth embodiment of the system according to the present application is
When the determination unit determines that the person is in the area, the determination unit determines whether or not the person is wearing a predetermined item.
The system according to any one of the first to third embodiments.
A fifth embodiment of the system according to the present application is
The predetermined items include fall arrest devices or helmets, safety goggles, gloves, safety boots, smartphones, swords, and / or assault items.
The system of the fourth embodiment.
A sixth embodiment of the system according to the present application is
A first region and a second region can be set in the region portion.
The area is
The person in the image captured by the imaging unit is in the first region.
Or
The person in the image captured by the imaging unit is in the second region, and the person is not wearing the predetermined item.
The system according to any one of the first to fifth embodiments, which can be set to notify when the determination unit determines.
A seventh embodiment of the system according to the present application is
A portable information processing device having the imaging unit and the display unit integrated.
The system according to any one of the first to sixth embodiments.
The eighth embodiment of the system according to the present application is
A portable information processing device including the imaging unit and the display unit in an integrated manner.
The system according to any one of the first to seventh embodiments.
A ninth embodiment of the system according to the present application is
The determination unit has a machine-learned function that determines only whether or not a person is in the area and whether or not the person wears the predetermined item.
The system according to any one of the first to eighth embodiments.
A tenth embodiment of the system according to the present application is
The determination unit is determined by a machine-learned function that determines whether or not the person is in the area and whether or not the person is wearing the predetermined item, and acquires information on whether or not the notification status is reached. ,
The system according to any one of the first to ninth embodiments.
The eleventh embodiment of the system according to the present application is
The area can be set in the display screen,
When it is determined that there is a person in the area, another information processing device is notified.
A portable information processing device
The information processing device is an information processing device in which an imaging unit and a display unit are integrated.
The area is set at a position where the information processing apparatus is scheduled to be used.
Information processing device.
The twelfth embodiment of the information processing apparatus according to the present application is
The area can be set in the display screen,
When it is determined that there is a person in the area, sound, light, or vibration is generated.
A portable information processing device
The information processing device is an information processing device in which an imaging unit and a display unit are integrated.
The area is set at a position where the information processing apparatus is scheduled to be used.
Information processing device.
The thirteenth embodiment of the program according to the present application is
Computer,
A display means for displaying an image captured by the image pickup means,
Means for setting an area in the display,
A determination means for determining whether a person in an image captured by an imaging means is within the region or outside the region.
A program to function as.
The 14th embodiment of the method according to the present application is
The computer
A step of displaying an image captured by an imaging means,
Steps to set the area in the display,
A step of determining whether a person in an image captured by an imaging means is within the region or outside the region.
How to do.
The fifteenth embodiment of the program according to the present application is
A program for causing a computer to function as a system according to any one of the first to tenth embodiments or an information processing apparatus according to the eleventh or twelfth embodiment.

According to one embodiment of the present invention, the image can be utilized more appropriately.

FIG. 1 is a block diagram showing the functions of the system according to the embodiment. FIG. 2 is an example of the data structure of the system according to the embodiment. FIG. 3 is a diagram showing a display example of the system according to one embodiment. FIG. 4 shows an example of the flow of the system according to one embodiment. FIG. 5 shows an example of the flow of the system according to one embodiment. FIG. 6 shows an example of the flow of the system according to one embodiment. FIG. 7 is an example showing a situation in which the system according to the embodiment is imaging. FIG. 8 is another example showing a situation in which the system according to one embodiment is imaging. FIG. 9 is another example showing a situation in which the system according to one embodiment is imaging. FIG. 10 is a diagram showing a display example of the system according to one embodiment. FIG. 11 is a diagram showing a display example of the system according to one embodiment. FIG. 12 is an example of the data structure of the system according to the embodiment. FIG. 13 is another example showing a situation in which the system according to one embodiment is imaging. FIG. 14 is an example showing a usage mode of the system according to the embodiment. FIG. 15 is an example of the data structure of the system according to the embodiment. FIG. 16 is a diagram showing a display example of the system according to one embodiment. FIG. 17 is a diagram showing a display example of the system according to one embodiment. FIG. 18 is a diagram showing a display example of the system according to one embodiment. FIG. 19 is an example of the data structure of the system according to the embodiment. FIG. 20 is a diagram showing a display example of the system according to one embodiment. FIG. 21 is an example showing a usage mode of the system according to the embodiment. FIG. 22 is an example showing a usage mode of the system according to the embodiment. FIG. 23 is an example showing a usage mode of the system according to the embodiment. FIG. 24 is a block diagram showing a configuration of an information processing device according to an embodiment.

1. 1. Functions of the system according to the present invention An example of the functions in the system according to the present invention will be described with reference to FIG. FIG. 1 is a block diagram showing a specific example of the function related to the system of this example. The system according to the present invention may have all the functions described below, or may have only a part of the following functions having a predetermined technical significance alone or in combination.

1.1. Acquisition unit The acquisition unit has a function of acquiring information. The acquisition unit may acquire an image. The acquisition unit may acquire a moving image or a still image. Images are treated as superordinate concepts of moving images and still images in the documents of the present application.

In addition, the acquisition unit may acquire information related to the image. The information related to the image may include information related to the image pickup device that captured the image, information related to the position where the image was captured, and information related to the time when the image was captured.

The information related to the image pickup device may be information that identifies the image pickup device, such as the ID of the image pickup device, the affiliation information of the image pickup device, and the name of the image pickup device, information indicating the type of the image pickup device, and the function of the image pickup device. It may be information, etc.

The information related to the position where the image is captured may be the geographical position where the image was captured, the direction in which the image was captured, and the like. The geographical location may be GPS-measured information, base station-based information, and the like. The direction in which the image is captured may be information measured by a gyroscope, or the like.

The information related to the time when the image was captured may be the information related to the time when the image was captured and the duration during which the image was captured. The information related to the time when the image is captured may be the time when the imaging is started, the time when the imaging is finished, the time when the notification status is determined, and the like.

The acquisition unit may store the image and the information related to the image in association with each other. For example, the acquisition unit may store the image in association with information related to the image pickup device that captured the image, information related to the position where the image was captured, and information related to the time when the image was captured. .. FIG. 2 is an example of associating and storing an image and information related to the image acquired by the acquisition unit.

The acquisition unit may include an imaging unit for acquiring an image. The object imaged by the imaging unit may be recorded in the recording unit. The recording unit includes a ring buffer, and the captured image may be recorded in the ring buffer. By storing in the ring buffer, there is an advantage that the recording amount of the information processing apparatus having a recording unit can be suppressed within a certain range.

The storage amount of the ring buffer may be set to a predetermined amount. The predetermined amount of setting may be set manually or may be set in advance. Further, the storage amount of the ring buffer may be set by being calculated based on the time of the captured moving image and / or the information related to the quality of the captured image.

The time of the video to be captured is, for example, a setting for shooting a 3-minute video, a setting for setting a 5-minute video, a setting for shooting a 10-minute video, and the like. The storage amount of the ring buffer may be set according to the above.

The information relating to the quality of the captured image may be, for example, the resolution, frame rate, and / or bit rate of the image, and the storage amount of the ring buffer may be set based on this information. ..

The time of the captured moving image and / or the information relating to the quality of the captured image may be input and set by the information processing apparatus. For example, FIG. 3 is an example of a display screen that can be input in a touch panel format. In this figure, an example in which the recording time 301, the image resolution 302, the frame rate 303, and the bit rate 304 can be input is shown. These may be set to be selected from a list showing a plurality of candidates.

In addition, on the recording setting screen, from the viewpoint of recording the situation up to the notification status described later, the recording period when recording the video before the notification status time (hereinafter, also referred to as "notification status pre-recording period"). Yes), may be entered, set, and remembered. When the acquisition unit acquires the notification status pre-recording period, it may store it.

The above-mentioned ring buffer may be used as a storage function for storing the image captured by the imaging unit, but when the notification status is determined and the notification information is generated, the imaging information processing apparatus , A part or all of the notification information may be stored in the ring buffer, or may be stored in a storage format different from that of the ring buffer. Here, after notifying the notification information, the imaging information processing apparatus may store a part or all of the notification information in the ring buffer, or may store it in a storage format different from that of the ring buffer. By storing the notification information even after it has been notified, there is an advantage that the notification status can be understood later.

In the former, the system according to one embodiment has a first storage unit that stores an image captured by the imaging unit and a second storage unit that stores a part or all of the notification information determined to be the notification status. You may be prepared. The first storage unit and the second storage unit may have the same storage format or may have different storage formats. The second storage unit may have a larger storage capacity than the first storage unit. In this case, it is possible to store notification information regarding a plurality of notification statuses, and there is an advantage that past notification statuses can be saved. Further, the first storage unit may be the first ring buffer, and the second storage unit may be the second ring buffer. Since the second storage unit is a ring buffer, the old memory in the past can be automatically deleted, and there is an advantage that the latest notification information can be stored even when the storage amount of the second storage unit is reduced. .. Further, in this case, the storage capacities of the first ring buffer and the second ring buffer may be different. Further, the storage capacity of the second ring buffer may be larger than the storage capacity of the first ring buffer. In this case, there is an advantage that a plurality of information determined to be the notification status can be stored.

Further, since the notification information is notified to the notification destination information processing device and the management information processing device, the notification destination information processing device and / or the management information processing device also has the same storage function as described above. Good.

However, the storage format of the information stored in the image pickup information processing device, the notification destination information processing device, and the management information processing device is not limited to the ring buffer, such as when the storage capacity is larger than a certain size, and other storage formats. It may be.

3.2. Judgment unit The determination unit has a function of determining an image acquired by the acquisition unit and has a function of generating a determination result. The image may be a moving image or a still image.

The determination unit may have a function of determining the relationship between one or more objects and a predetermined area (sometimes referred to as "target area relationship" in the documents of the present application). The determination unit may determine whether one or more objects are within the range of the predetermined area or outside the range of the predetermined area.

The target may be a living thing such as a person, a pet, a domestic animal, or a vehicle such as a two-wheeled vehicle or a four-wheeled vehicle. Further, the target may be a part of a living thing such as a person, a pet, a domestic animal, or a predetermined item. Examples of predetermined items include helmets, fall-prevention devices (in the documents of the present application, so-called safety belts, belts with lifelines, harnesses, etc. may be collectively referred to as "fall-prevention devices"), gun swords. Examples include, but are not limited to, swords, smartphones, safety goggles, gloves, and / or safety boots specified by the Possession Control Law. In addition, the safety goggles may be dustproof and eye protection during welding.

When the target is a person, the determination unit identifies an estimated position of the person in the captured image, and whether the position of the person is within a predetermined area in the captured image or a predetermined position. The determination unit may determine whether or not it is outside the region of. The position of the person may be any information that can identify the place where the person is. The position of the person may be the center of both feet of the person, including, for example, the ankle. Further, the position of the person may be estimated using the center of the person's waist, the center of both shoulders of the person, the position of the nose of the person, the position of the throat of the person, and the position of the neck of the person. Estimates are, for example, the distance from the center of a person's waist to the center of a person's feet in an average person, the distance from the center of a person's shoulders to the center of a person's feet in an average person, in the average person. The distance from the position of a person's nose to the center of a person's feet, the distance from the position of a person's throat to the center of a person's feet in an average person, the position of a person's neck in an average person to the position of both feet of a person The distance to the center may be used to specify the position of the center of both feet of the target person. This is because, depending on the position and orientation of the image pickup device, the waist, shoulders, neck, throat, and nose other than the human feet, even if the target person's feet are not necessarily included in the captured image. , Etc. are included in the captured image, and there is an advantage that the centers of both feet of a person can be estimated by using these.

In addition, the determination unit determines the position of the person in the captured image among the center of the person's waist, the center of both shoulders of the person, the position of the nose of the person, the position of the throat of the person, and the position of the neck of the person. It may be estimated using the distance between the two points. For example, the distance between the position of the person's throat and the position of the person's nose may be used to estimate the position of the person in the captured image as a position that is a constant multiple of the distance. The constant multiple may be determined using the average distance of the person.

A predetermined region may be designated by designating at least a part of the region in the captured image by coordinates, functions, or the like. For example, a predetermined region may be defined by at least a part of it connecting points of a plurality of Cartesian coordinates with vertices in the captured image, or a function of one or more. The boundary may be specified by the straight line or curve specified by, the boundary may be specified by the edge of the captured image, or the boundary may be specified by a combination thereof. It may be what is done.

When the image is a moving image, the determination unit may determine a specific one or a plurality of still images constituting the moving image, and determine the target area relationship using the specific one or a plurality of the still images. Further, the determination unit may determine the target area relationship after determining a specific one or a plurality of still images.

The determination unit may determine the target area relationship for all the still images related to the moving image. In this case, in the still image constituting the moving image, there is an advantage that the relationship can be determined at the stage where the target first has a relationship with the predetermined area. It should be noted that there is a burden of determination processing for all still images.

In addition, the determination unit may determine the target area relationship for the still images at predetermined intervals among the still images related to the moving image. In this case, since it is not necessary to perform the determination process for all the still images related to the moving image, there is an advantage that the processing load is reduced.

As an example of the processing flow of the determination unit in this case, as shown in FIG. 4, first, among a plurality of still images constituting the moving image, the still images are determined at predetermined intervals along the time series of the moving image. (Step 1). Next, the target area relationship is determined for the still image (step 2). When the target area relationship is determined and it is determined that a person is in the area, the notification status is determined and notification is performed (step 3A). When the target area relationship is determined and it is determined that the person is not in the area, the process returns to step 1 without notifying (step 3B).

In addition, the determination unit may have a function of determining a predetermined relationship (sometimes referred to as "plurality object relationship" in the documents of the present application) with respect to a plurality of objects.

For example, it may have a function of determining the relationship between the head of a person who is the first target and whether or not the helmet, which is the second target and is a predetermined item, is worn or not. ..

In addition, it has a function of determining the relationship between the chest of the person who is the first target and whether or not the fall prevention device, which is the second target and is a predetermined item, is worn or not. You can.

In addition, it has a function of determining the relationship between the head of the person who is the first target and whether or not the safety goggles, which is the second target and a predetermined item, are worn or not. Good.

In addition, it has a function of determining the relationship between the upper limbs or hands of the first target person and whether or not the gloves, which are the second target and the predetermined item, are worn or not. Good.

In addition, it has a function of determining whether or not the safety boot, which is the second target and a predetermined item, is worn or not worn on the leg of the person who is the first target. Good.

In addition, the first target person's hand is the second target and is a predetermined item, such as swords, baseball bats, sticks longer than the specified length, etc. specified by the Firearms and Swords Possession Control Law. It may have a function of determining the relationship of whether or not the item is held. Here, the assault use item refers to an item that is not normally used in the place where the image pickup device is imaged, but is dangerous to a person in the place where the image is imaged when used for violent acts. The swords, baseball bats, and sticks longer than the specified length specified by the Firearms and Swords Possession Control Law are examples of assault items.

In addition, it may have a function of determining the relationship between wearing or not wearing a pistol, which is a second target and a predetermined item, on the waist of a person who is the first target. The waist may be the right waist of a person.

In addition, it may have a function of determining the relationship between wearing or not wearing a baton, which is a second target and a predetermined item, on the waist of a person who is the first target.

Further, it may have a function of determining the relationship between whether the hand of the person who is the first target holds or does not hold the smartphone which is the second target and is a predetermined item.

When the image is a moving image, the determination unit may determine a specific one or a plurality of still images constituting the moving image, and may determine a plurality of object relations using the specific one or a plurality of still images. The determination unit may determine a plurality of object relations after determining a specific one or a plurality of still images.

The determination unit may determine a plurality of object relations for all the still images related to the moving image. In this case, in the still image constituting the moving image, there is an advantage that the relationship can be determined at the stage when the plurality of objects first have a predetermined relationship. However, since it is necessary to perform determination processing for all still images, the processing load increases.

In addition, the determination unit may determine a plurality of object relations for the still images at predetermined intervals among the still images related to the moving image. In this case, since it is not necessary to perform the determination process for all the still images related to the moving image, there is an advantage that the processing load is reduced.

As an example of the processing flow of the determination unit in this case, as shown in FIG. 5, first, among a plurality of still images constituting the moving image, the still images are determined at predetermined intervals along the time series of the moving image. (Step 1). Next, the multiple object relations are determined for the still image (step 2). When it is determined that the plurality of targets have a predetermined relationship in the determination of the plurality of object relations, the notification is performed as the notification status (step 3A). If it is determined that the plurality of targets do not have a predetermined relationship in the determination of the plurality of target relationships, the process returns to step 1 without notifying (step 3B). When a plurality of predetermined relationships with a plurality of objects are required (for example, wearing a helmet and wearing a fall prevention device), in step 3A, regarding the plurality of required predetermined relationships. A plurality of object relations may be determined.

Further, the determination unit may determine the plurality of target relationships after determining the target area relationships. For example, the determination unit may determine the target area relationship and determine a plurality of target areas for the still image determined to have the target area relationship. For example, it may be determined whether or not a person wears a predetermined item on the assumption that the person is in a predetermined area. In this case, since the still image to be determined to have a plurality of target relationships is a still image determined to have a target area relationship, the notification status is that the target is in a predetermined area and the plurality of targets are present. Having a predetermined relationship has the advantage of being able to process efficiently. Further, the determination unit may automatically determine a plurality of target relationships after determining the target area relationship.

The determination unit may determine the target area relationship for all the still images related to the moving image related to the moving image, and determine the plurality of target relationships for the still images having a predetermined relationship. In this case, there is an advantage that a plurality of target relationships can be determined at an early stage for still images having a target area relationship.

In addition, the determination unit may determine the target area relationship for the still images at predetermined intervals among the still images related to the moving image, and determine the plurality of target relationships for the still images having the predetermined relationship. In this case, since it is not necessary to perform the determination process for all the still images related to the moving image, there is an advantage that the processing load is reduced.

As an example of the processing flow of the determination unit in this case, as shown in FIG. 6, first, among a plurality of still images constituting the moving image, the still images are determined at predetermined intervals along the time series of the moving image. (Step 1). Next, the target area relationship is determined for the still image (step 2). When the target area relationship is determined and it is determined that the target is in a predetermined area, a plurality of target area relationships are determined (step 3A). When the target area relationship is determined and it is determined that the target is not in the predetermined area, the process returns to step 1 without notifying (step 3B). When the plurality of target relationships are determined and it is determined that the plurality of targets have a predetermined relationship, notification is performed as the notification status (step 4A). If the plurality of object relationships are determined and it is determined that the plurality of objects do not have a predetermined relationship, the process returns to step 1 without notifying (step 4B).

As described above, when a plurality of predetermined relationships for a plurality of objects are set (for example, wearing a helmet and wearing a fall prevention device), in step 4A, the plurality of predetermined relationships are set. For each of the relationships, a plurality of object relations may be determined.

In the above-mentioned determination of the plurality of target areas, there may be an image suitable for determining the success or failure of the relationship between the plurality of objects. For example, even though a person actually wears a helmet, it may not be possible to determine whether or not the helmet can be worn depending on the angle and position of the image. In such a case, if the notification status is set to indicate that the person is wearing a helmet or not, it should not be notified because the person is actually wearing a helmet, but it is systematic. Since it cannot be determined that a person is wearing a helmet, it is possible to notify the situation.

Therefore, assuming such a case, the determination unit determines the multiple object relations as a premise for determining the notification status, and additionally conditions described later as a premise for determining the multiple object relations. The multiple object relations may be determined after determining the above, or the determination unit determines the multiple object relations a plurality of times for a predetermined time or a predetermined number of still images, and the notification status is not even once. It may be determined whether or not a determination result of a plurality of object relations is obtained.

In this case, the additional condition in the former has an advantage that the plurality of object relations can be determined based on a more appropriate image and the accuracy of the determination of the plurality of object relations can be improved.

Regarding the latter, for example, as described above, by setting the human area relationship, it cannot be determined that the multiple target areas are not in the notification status within a predetermined time from the time when the person is determined to be in the area. In case, you may notify. Further, from the time when it is determined that the person is in the area, if it cannot be determined that the notification status is not obtained in the determination of the plurality of target areas within a predetermined number of still images, the notification may be made.

It should be noted that such additional conditions, a predetermined time, a predetermined number of still images, etc. are related to how strict the judgment conditions are, and if the risk prediction is important, the notification conditions. If the conditions are not met even a little, you can set the notification status, or if the situation of frequent notifications is troublesome, relax the conditions and keep the fixed time, the number of still images, etc. After a certain amount of grace, it may be set to notify when it cannot be determined that the status is not the notification status. From this point of view, the above-mentioned predetermined time, predetermined number of still images, additional conditions, etc. may be input and set by the system user, or may be set as internal information by the system administrator. You may.

Further, as described above, the determination unit may determine the relationship between one or more persons and a predetermined area (sometimes referred to as "human area relationship" in the documents of the present application). Then, even when the image is a moving image, the notification status is related only to the plurality of object relations, and the notification status is not related to the human domain relationship, the determination unit determines that the still image has the human domain relationship. May be configured to determine. By determining the human area relationship by the determination unit, it is possible to reduce the number of still images to be determined for the plurality of object relationships, and there is an advantage that the amount of processing can be reduced. This point will be described in detail below. For this case, the system of one embodiment may be able to set an area that is not a condition of the notification status.

For example, depending on the position and direction of the imaging camera, a large number of people may be included in the imaging range. For example, FIG. 7 is an example showing what kind of person is included in the imaging field of view by the imaging device provided inside the facility entrance. The information processing device 002 provided in the facility 001 with the imaging range directed to the entrance 004 of the facility 001 may allow various people 005 to 010 to enter the imaging range depending on the imaging range 003. Persons 005 and 006 are included in the imaging range 003 as persons within facility 001. On the other hand, people 007 to 009 are included in the imaging range 003, but are not in facility 001. Person 010 is out of the range of the imaging range 003. This is an example in which the person 010 is out of the imaging range because the imaging device 002 is not provided at an angle parallel to the ground and is installed at a predetermined depression angle. The distance 011 indicates the distance from the image pickup apparatus 002 to the entrance 004 in a direction parallel to the image pickup direction. Further, the real area corresponding to the predetermined area in the image captured by the imaging device (in the document of the present application, the real area corresponding to the predetermined area in the image captured by the imaging device is referred to as the "real area". The range is illustrated by the shaded area 013. Here, the distance 011 is set as the distance from the image pickup device 002 to the facility 001 in the direction along the image pickup direction, but as another example, the distance 011 is the entrance 004 from the image pickup device 002 to the facility 001. It may be the distance to one of both ends of the entrance of.

The determination unit may determine whether one or more people are within the predetermined area or outside the predetermined area as a human area relationship. In addition, the determination unit may determine whether one or more people are within the predetermined range or outside the predetermined range by using the positions of the one or more people.

As described above, when the predetermined area and the boundary inside and outside the facility are set to have a predetermined relationship, there is an advantage that the information outside the facility can be blocked. That is, even if the imaging device is provided mainly for collecting information inside the facility, it is preferable not to collect information outside the facility unnecessarily when it is installed from the inside of the facility to the outside. , There is an advantage that privacy can be configured not to be collected unnecessarily by setting an area within the imaging range of the image and configuring it so as not to acquire information on the corresponding part outside the facility. The predetermined area is set so that, for example, at least a part of the boundary of the predetermined area and a part of the boundary inside and outside the facility have a certain distance, or at least a part of the boundary of the predetermined area. However, it may be set so that it touches a part of the boundary inside and outside the facility.

FIG. 8 shows an example in which the imaging device is installed at a predetermined depression angle. The depression angle may be an angle between the imaging direction of the imaging device and the horizontal direction of the ground. The imaging direction may be the direction of the center of the angle of view.

In this figure, the image pickup apparatus 001 is provided on the ceiling 002 at a predetermined depression angle, and the image pickup range 004 is set for a part between the ceiling 002 and the ground 003. The determination unit may determine that among the persons 005 to 008, the person 006 is within the predetermined area 009, and the persons 005, 007, 008 are outside the predetermined area 009. Although the face portion of the person 005 is in the imaging range, the position of the person 005 may be determined to be outside the first predetermined area because the position of the person 005 is outside the actual area 009 corresponding to the first predetermined area. When the real area corresponding to the second predetermined area is set to 010, it may be determined that the real area including the person 005 is within the second predetermined area. In this figure, the angle of view of the image pickup device 001 is narrow, but this is an example. Even if the angle of view is wide like a so-called smartphone, the image is imaged on the ceiling with a predetermined depression angle. It may be installed if possible.

Further, FIG. 9 shows an example in which the image pickup apparatus 001 is provided in the instrument 009. Further, this figure is an example in which the imaging direction of the imaging device 001 is installed substantially parallel to the ground. In this case, the image pickup device 001 will take an image of a person between the ceiling 003 and the ground 004 until it reaches an obstacle such as a wall or a pillar in the image pickup direction, and the image pickup device 001 will see from the image pickup device due to the overlap of people. Although the person in the back is not imaged, there is a possibility that a large number of people will be imaged. In such a case, even if there is no privacy problem as described above, the processing amount can be increased when all the people who are in the imaging range are processed. Therefore, for the purpose of limiting the persons to be processed among the persons in the captured image, the determination unit determines that the persons in the captured image are in or outside the predetermined area. May be determined.

In addition, the determination unit may determine whether or not one or more people are in a predetermined area by using the length of a part of the people in the image. For example, the judgment part is a part of a person such as the position of the base of both thighs in the waist of the person, the position of both shoulders of the person, the position of both ears of the person, the position of the nose of the person, the position of the throat of the person, etc. Whether it is within a predetermined area or outside a predetermined area is determined by using the distance between two points or the distance between three points (sometimes referred to as "partial human distance" in the documents of the present application). You can do it. For example, the distance between the position of the person's throat and the position of the person's nose may be used to determine whether the person is within or outside the predetermined area. Further, for example, when the partial human distance is equal to or greater than a predetermined distance, the determination unit determines that the distance is within the predetermined area, and when the partial distance between the humans is less than the predetermined length, the determination unit is not within the predetermined area. You may judge.

For example, if the distance between the position of a person's throat and the position of a person's nose is within a predetermined area if it is 5 pixels or more and less than 10 pixels in an image with a predetermined resolution, it is considered as within a predetermined area, and if it is less than 5 pixels or 10 pixels or more Is not within a predetermined area, and the determination unit is within a predetermined area or a predetermined area by using the distance between the position of the person's throat and the position of the person's nose in the obtained image. It may be determined whether or not it is inside.

In this case, although there may be some error depending on the angle of view and distortion, the determination unit may set the position of a person or the distance between the image pickup device and the person using a predetermined parameter. For example, the determination unit may have parameters related to the angle of view, distortion, and / or resolution in advance, and may be configured to be set by using the relationship between such parameters and the type of camera lens. That is, the determination unit according to the system of one example has a rule indicating the relationship between the camera lens and the parameters related to the angle of view, distortion and / or resolution, and the determination unit is the camera related to the image pickup apparatus for imaging. The position of a person or the distance between an imager and a person may be measured using parameters related to the angle of view, distortion and / or resolution corresponding to the type of lens.

Further, the determination unit may manually set the angle of view and the resolution. In this case, the determination unit calculates the angle of view per pixel in the image using the set angle of view and resolution, and calculates the angle corresponding to the partial distance of the person to correspond to the standard person. The distance may be calculated back using a partial distance. The angle of view may be vertical XX degrees, horizontal XX degrees, or the like. Further, the partial distance of the person in this case may be the size of a face, but may be another distance.

In this way, after the determination unit determines whether the position of the person in the image is within the predetermined area or not within the predetermined area, only the person who is determined to be within the predetermined area is targeted. There is an advantage that the processing amount can be reduced by performing the processing of. In particular, when the depression angle of the image pickup device is equal to or less than a predetermined angle, the image pickup device can take an image in parallel with the ground, and in an environment without an obstacle, a large number of people are likely to appear in the image and the processing amount increases. Therefore, there is an advantage that the processing amount can be reduced by processing only the person who is determined to be within the predetermined area. Examples of the predetermined angle include, but are not limited to, 10 degrees, 15 degrees, 20 degrees, 25 degrees, and 30 degrees.

In addition, in the setting of the area described later, it is a reference for what size of person in the area screen is determined to be in the area and what size of person is out of the area in the display screen. The display unit may display such a virtual person. This provides reference information on what kind of person is judged to be in the area and what kind of person is judged to be out of the area when the image pickup device is set in a direction that does not image the ground. It may be a thing. In this case, the function of setting the area may or may not be provided. Even if there is no function for setting the area, the viewer can obtain information that can be used as a reference for what kind of person is determined to be in the area and what kind of person is determined to be outside the area. For example, in FIG. 10, the size 002 of a virtual human face that is not actually imaged is displayed on the display unit 001 that displays the imaged object. Here, when the face size 002 is closer than the size of this person, the person becomes a target to be determined to be in the area in the determination of the human area relationship, or a plurality of object relationships are determined. It shows that it becomes a target. Further, in this example, only the face of one person is shown, but for example, the faces of a plurality of people are copied, and among the faces of the plurality of people, the face of a person of any size is described above. It may indicate whether it is a target of judgment. For example, as shown in FIG. 12, the sizes of human faces 002 to 005 may be displayed, and from which size the area is targeted may be displayed. For example, the human faces 004 and 005 may be displayed in a manner different from that of the human faces 002 and 003. Although only the human face is shown in these figures, it may be the entire human body or the upper body instead of the face. In short, it should be something that people can understand the size of a person.

Further, the display unit may have an auxiliary function of displaying such a virtual person and setting an area. In this case, it is an advantage that the viewer himself can set the size of the target person. For example, in FIG. 11 described above, the size of the human face can be changed, and the size of the target person may be set. Further, in FIG. 12 described above, a plurality of people having different face sizes may be displayed, such as selecting the size of the person to be targeted.

As described above, when the area is set using a virtual person, the determination unit determines the above-mentioned person part distance and the corresponding person part in the above-mentioned virtual person used for setting the area. The distance may be used in comparison to determine whether the person in the captured image is within or outside a predetermined area.

The appliance 009 provided with the image pickup apparatus 001 may be various, for example, a sales floor, a shelf, a signboard, a wall, and the like. Further, the image pickup apparatus 001 may be directly provided on a part of the facility such as a wall or a pillar of the facility. Further, the image pickup apparatus 001 may be detachably installed in the instrument 009 or a part of the facility.

Further, the image pickup apparatus 001 may be integrated with the instrument 009. For example, when the instrument 009 is digital signage, the image pickup apparatus 001 may be provided as a part of the digital signage. When the information object is displayed by digital signage, the image pickup apparatus 001 may also be able to acquire an image of a person who has viewed the advertisement. In this case, the statistical processing unit according to the system of the present embodiment acquires both information on the person in the image, that is, the presence or absence of the predetermined item by the person and the statistical information on the person in the image, which will be described later. It's okay. In this case, the statistical information relating to the person in the image may be the number of people, the attribute of the person, the number of people who are interested in the advertisement, the number of people who are interested in the advertisement, and the like. These may be the total number, the average number, etc. of a predetermined period.

The determination unit may determine a plurality of object relationships for all the still images determined to have a human domain relationship, or may determine a plurality of object relationships for some of the still images determined to have a human domain relationship. May be determined. In the former, so to speak, the determination that there is a human domain relationship is a condition for determining a plurality of object relations, while in the latter, in addition to the determination that there is a human domain relationship, an additional condition is added. When is satisfied, the multiple object relations are determined.

An additional condition in the latter may be, for example, the relationship between the imaging device and a person. More specifically, for example, the distance between the image pickup device and the person, the directionality between the image pickup device and the person, and / or the relative moving speed between the image pickup device and the person, and the like can be mentioned.

For example, using the distance between the image pickup device and a person, among the still images related to moving images, a still image related to a distance shorter than a predetermined distance or a still image related to the shortest distance may be used. In this case, if the distance between the image pickup device and the person is short, a large image is taken, so that there is an advantage that the determination can be made more accurately. Further, by using the directionality of the image pickup device and the person, even if the distance between the image pickup device and the person is the same, the still image in which the person faces the image pickup device can be determined more accurately. There are advantages. Further, by using the relative moving speed between the image pickup device and the person, a still image having a relative moving speed smaller than a predetermined speed or a still image having the smallest relative relative speed may be used. In this case, since the still image is more stable, there is an advantage that the determination can be made more accurately.

The distance between the image pickup device and the person may be one that can estimate the distance between the image pickup device and the person, or may be a relative indicator of the distance between the image pickup device and the person. For example, the size of a person in an image captured by an image pickup device is an example of a device that proportionally indicates the distance between the image pickup device and the person. The size of a person is the height of the person, the imaged area of the person, the length of the shoulder width of the person, the height of the person, the size of the face of the person, the area of the face of the person, the distance from the nose of the person, It may be the distance between a person's ears, and so on. Moreover, one or more of these may be used.

Even when a person is facing up or down, the positions of both ears are basically unchanged, so the distance between the two ears of a person is when the person is shaking his face up and down. However, there is an advantage that the possibility of measuring the distance is increased.

In addition, the distance from the nose of a person is set at a position where a person who invades from the entrance is imaged even when one ear is not imaged as in the case where the person is located on the side surface of the image pickup device. There is an advantage that the nose is more likely to be imaged and the distance can be measured in the image pickup device.

As the directionality between the image pickup device and the person, for example, the length of the width of both shoulders of the person may be used. For example, a still image in which the width of both shoulders of the person related to the moving image is longer than a predetermined value, or a still image in which the width of both shoulders of the person related to the moving image is the longest may be used.

Further, as the relative moving speed of the image pickup device and the person, for example, a still image having a low degree of blurring may be used.

It is not necessary for the determination unit to explicitly use this information, and other information intended for this information may be used.

The determination unit may generate a determination result. The determination result may be information that one or more objects are within the predetermined area or outside the predetermined area. Further, the determination result may be information that one or more objects have a predetermined relationship. The predetermined relationship may be the above-mentioned relationship such as wearing, wearing, holding, and holding. Further, the determination result may be one or more of the above. For example, the person may be in a predetermined area, the person may be wearing a helmet, and the person may not be wearing a fall arrest device.

FIG. 12 is an example of the determination result. As for this determination result, the result is displayed for each of person 1, person 2, and person 3 for a specific still image. Further, the prohibited area A and the area B are set as the area, and the person 1 and the person 2 are determined to be in the prohibited area A, and the person 3 is determined to be in the area B. Further, in area B, a helmet and a fall-prevention device are set as a condition, and the result is that person 3 wears a helmet but does not wear a fall-prevention device. There is.

The imaging device related to the determination unit may be installed on the ceiling as shown in FIG. 13, for example. In this case as well, there is an advantage that a security camera or the like can be used, and there is an advantage that a wide area can be judged by one camera.

In addition, the determination unit may determine whether or not the situation is set to notify in advance (in the documents of the present application, it may be referred to as "notification status"). The notification status may be set to notify when the determination result is a predetermined one. The determination unit may notify the notification unit, which will be described later, whether or not it is in the notification status.

Further, the determination unit may determine in real time whether or not the notification status is present, and may transmit to the notification unit in real time whether or not the notification status is present.

The determination unit may identify the above-mentioned target by using a machine-learned identification device. Machine learning may be using a neural network. Further, the neural network may be one using deep learning.

The identification unit having a machine-learned identification function may be a component of the system according to the present invention. In this case, the identification unit may operate in the same information processing device as the imaging unit, the identification unit may operate in the same information processing device as the display unit, and the identification unit may operate in the imaging unit or the display. It may operate in an information processing device different from the unit. When the identification unit operates in the same information processing device as the image pickup unit, it is not necessary to transmit the captured image to another information processing device via a network, and there is an advantage that the captured image can be identified in real time. .. Further, when the identification unit operates in the same information processing device as the display unit, it is not necessary to transmit the identified image to another information processing device via the network, and there is an advantage that the identified image can be identified and displayed in real time. Further, when the identification unit operates in an information processing device different from the image pickup unit or the display unit, the identification unit may be included in an information processing device such as a server device or a cloud that can be connected via a network, and the image is captured. Although images need to be communicated via a network, they have the advantage of being able to be identified using a large-scale machine-learned identification function that is different from the imaging unit and display unit, such as server devices and cloud devices. In this case, there is an advantage that the amount of communication can be reduced when a part of the images after a predetermined interval is sent instead of communicating all the captured images.

Further, the identification unit may be limited to a predetermined object that has been machine-learned and can be identified. For example, people, helmets, fall arrest devices, safety goggles, gloves, safety boots, smartphones, swords, and / or assault items. Further, the identifiable relationship of the object may be limited to a predetermined one. For example, a person's head is wearing a helmet, a person's chest is wearing a fall-prevention device, a person's head is wearing safety goggles, and a person's upper limbs are wearing gloves. Relationships that have been carried out, relationships in which safety boots are worn on the legs of a person, relationships in which a smartphone, swords, and / or items for assault are held in a person's hand. As described above, when the identifiable object or the identifiable relationship is limited to a predetermined object, the portable information processing device has a lower calculation function than the GPU or the like. However, there is an advantage that it can be identified with less burden of calculation amount. Such a portable information processing device may have a superscalar CPU. Further, the portable information processing device may have a CPU that is not a vector processor system. Further, the portable information processing device may not have a GPU (Graphic Processor Unit).

The identification unit may exist outside the system according to the present invention. In this case, the determination unit in the system may transmit the image to the identification unit outside the system via the network and inquire whether or not the notification status is related to the image. The identification unit outside the system may use the acquired image to generate whether or not the image is in the notification status, and transmit to the determination unit in the system whether or not the notification status is present. The determination unit in the system may acquire whether or not the notification status is obtained from the identification unit outside the system. In this case, the system according to the present invention has an advantage of obtaining whether or not the image is in the notification status by using the identification function based on machine learning in the system possessed by a third party such as a cloud or a server.

1.3. Notification unit The notification unit has a function of notifying an information processing device related to a notification destination (sometimes referred to as a "notification destination information processing device" in the documents of the present application).

The notification unit may generate information to be notified (sometimes referred to as “notification information” in the documents of the present application) using the determination result generated by the determination unit.

The notification unit may notify when it is in the notification status. In addition, the notification unit may notify when the notification status is reached. That is, the notification unit may notify after acquiring the information that the notification status has been reached from the determination unit.

1.4. Statistical processing unit The statistical processing unit has a function of performing statistical processing using the information determined by the determination unit. The information obtained by the statistical processing unit may be transmitted to the notification destination information processing device.

1.5. Storage unit The storage unit has a function of storing a program related to all or a part of the above-mentioned functions and / or corresponding data.

Explaining the flow in which the system of one embodiment uses each of the above-described parts, first, the acquisition unit related to the system of this embodiment acquires an image (step 1), and the determination unit related to the system of this embodiment obtains an image. , Judgment is made using an image, and a judgment result is generated (step 2). The notification unit according to the system of the present embodiment generates notification information using the judgment result, and notifies the notification destination of the notification information. (Step 3) may be the flow.

2. Embodiment
2.1. Embodiment 1
A system according to an embodiment of the present invention will be described. The system of the present embodiment has, for example, the system configuration as shown in FIG. 14, and will be described as being used at a construction site such as a construction site or a civil engineering site.

First, to explain the people involved at the construction site, there are people in various positions at the construction site. For example, the foreman who performs the actual work, the foreman who is the team leader of these workers, the foreman who organizes the foreman, the safety and health manager who manages safety documents, etc., and the construction site are all responsible. The foreman, a supervising engineer who assists the foreman, etc. (in the documents of the present application, these persons may be referred to as "construction personnel").

In this figure, a smartphone 002 related to the system of this embodiment is placed at the construction site 001. The place where the smartphone 002 is placed at the construction site may be any place. Since there are various materials, buildings, various devices, vehicles, etc. at the construction site, for example, they may be installed, detachably fixed, placed, or leaned against them.

Since various works are performed every day at the construction site, there are one or more dangerous areas, which will be described later, and the locations can be changed every day. For example, if it rains the day before, a wet place can be a dangerous area, the area around the place where unstable materials are temporarily stored can be a dangerous area, and the area around aerial work can be a dangerous area. sell. On the other hand, at the construction site, one or more heavy machines are used, a large number of materials are brought in, and structures are constructed, so an imaging device that can always image every part of the construction site without blind spots is being constructed. It is difficult to expect that it will be installed at the site, and it will be difficult to install it. When the system of the present embodiment uses an image pickup device in a portable information processing device, the portable information processing device is lightweight and compact, so that it can be easily carried according to daily danger areas. There is an advantage that the imaging range can be changed by changing the location and direction of the information processing device. In particular, when one portable information processing device has an imaging unit and a display unit integrated, there is an advantage that it is convenient because the one portable information processing device has an imaging function and a display function. .. The mode of integration may be various, and the imaging function and the display function may be physically connected in one information processing device. The imaging direction in the imaging function may be the same direction as the display direction in the display function, or may be the opposite direction. Further, there may be two or more imaging functions in one information processing device.

In addition, although entering a dangerous area directly affects human life, it is not always possible to easily mobilize the labor force in a social situation where there is a labor shortage. Therefore, when the system of the present embodiment uses an imaging device in a portable information processing device, it can be downloaded, installed, and used by using a portable information processing device that is already widespread, so that a notification system can be easily performed. There is an advantage that can be introduced.

The smartphone 002 is provided with an image pickup device and is capable of capturing an image pickup range 003. A region may be set in this imageable range 003. In this figure, the first region 004a and the second region 004b are set, but one or more regions may be set. The area may be set in association with a place that is determined when a person enters and is notified as described later. Since a plurality of areas can be set, there is an advantage that even when it is desired to set a plurality of areas within a range that can be imaged by one smartphone, it is not necessary to prepare a plurality of smartphones and one smartphone can handle it. In addition, there is an advantage that the area required for notification can be set in addition to the limitation of the imaging range due to the installation location of the smartphone.

Examples of areas include places where people are in danger or places where people are in danger (these are collectively referred to as "danger areas" in the documents of the present application), but other than that, heavy machinery is used. The place itself, such as the driver's cab to be controlled, is a place where dangerous things can be controlled even if it is not dangerous (sometimes referred to as "danger control area" in the documents of the present application), or only a predetermined person is allowed to access it. A place where monetary value is managed (sometimes referred to as "monetary value management area" in the documents of the present application) or a place where confidential information is managed so that only a predetermined person can access it (this application). In the document, it may be "confidential information management area"), etc. (in the document of the present application, as a superordinate concept including a danger area, a danger control area, a monetary value management area, and a confidential information management area). Sometimes referred to as a "prohibited area").

In addition, conditions may be set in the area. By setting conditions, it is not always notified when a person is in the area, but it is configured so that the person is in the area and can be notified according to a predetermined condition, so that it is useless notification. There is an advantage that it can be notified when it is necessary and sufficient. In particular, the dangerous area may be an area where not everyone is allowed to be present, or an area where anyone who wears a predetermined item is allowed to be present. More specifically, for example, if a helmet is worn as a predetermined item, it may invade, and if a helmet and a fall prevention device are worn as a predetermined item, it may invade. is there. Therefore, when the system of the present embodiment can set the condition for the area, there is an advantage that the notification can be set in consideration of the danger of each area and the equipment of the person who enters the area.

The conditions to be set may be set to notify when the conditions are met, or may be set to notify when the conditions are not met. Here, the first area is set to notify when it is determined that there is a person in the first area, assuming that there is no condition, and the second area is assumed to have a condition, and the helmet and the crash are made in the second area. Explain that it is set to notify when there is a person who is not wearing a restraint device. The details of this area will be described later.

In this figure, the person 005 who owns the smartphone 005a is in the first area 004a, and the person 006 who owns the smartphone 006a is in the second area without wearing a helmet or a fall prevention device. , Are illustrated. Since the first region and the second region are regions set within the imageable range 003, in this case, the smartphone 002 determines that the person 005 is in the first region, and wears a helmet or a fall prevention device. It may be determined that the person 006 who is not wearing the helmet is in the second area.

When the smartphone 002 determines that the notification status is determined, the smartphone 002 may notify the notification information to a predetermined notification destination information processing device.

The notification information is information related to an information processing device (sometimes referred to as "imaging information processing device" in the documents of the present application) including an image pickup unit that has captured an image determined to be the notification status, and information indicating the notification status. , And / or information related to the notification status, and the like.

The information related to the image pickup information processing device may be information that identifies the image pickup information processing device, information that identifies the group to which the image pickup information processing device belongs, and the like.

The information that identifies the image pickup information processing device may be the position of the image pickup information processing device, the name or ID of the image pickup information processing device, the user and / or the administrator of the image pickup information processing device, and the like.

The information that identifies the group to which the imaging information processing device belongs identifies the affiliation of the imaging information processing device, the user and / or manager of the imaging information processing device, and the construction site where the imaging information processing device was used during the notification status. It may be information, etc.

The information that identifies the construction site may be the project name of the construction, the nickname of the construction site, the location of the construction site, and the like.

The position of the image pickup information processing device or the position of the construction site may be a position specified by GPS, a position based on a base station related to the image pickup information processing device, or a position specified from these.

In addition, the user and / or the manager may be an individual or a corporation.

The notification information may be stored in advance in the management information processing device described below, and the notification information includes one piece of information that identifies the imaging information processing device (for example, the imaging information processing device ID, imaging). It may be configured to transmit (name of information processing device, etc.). In this case, by using the notification information in the management information processing device transmitted in advance and one information that identifies the image pickup information processing device, other information related to the image pickup information processing device can be transmitted to the management information processing device. , The notification information is transmitted to other information processing devices that access the management information processing device, and the other information processing devices can also acquire and display the notification information. Good.

When the information indicating the notification status is transmitted from the imaging information processing device between the imaging information processing device and the notification destination information processing device, the notification means is executed in advance in the notification destination information processing device. It may be defined information.

The information related to the notification status may be an image related to the notification status, an area related to the notification status, a condition related to the notification status, a time related to the notification status, and the like.

The image related to the notification status may be a moving image or a still image that caused the notification status. Further, the moving image or the still image may be a part or all of the period from the predetermined time before reaching the notification status to the predetermined time after the notification status. The predetermined time before reaching the notification status may be 5 seconds, 10 seconds, 15 seconds, 20 seconds, 30 seconds, 1 minute, 2 minutes, etc. before the notification status occurs. In particular, the predetermined time before reaching the notification status may be set to 0 seconds to 15 seconds before, 0 seconds to 30 seconds before, 0 seconds to 1 minute before, 0 seconds to 2 minutes before, and the like. .. The predetermined time after the notification status may be 1 minute, 3 minutes, 5 minutes, 7 minutes, 10 minutes, or the like.

The area related to the notification status may be information that identifies one or a plurality of areas that have generated the notification status.

The conditions relating to the notification status may be one or more conditions that have generated the notification status, the feasibility of such conditions, and the like.

The time related to the notification status may be the date and time when the notification status occurred, the date and time when the image related to the notification status was captured, and the like.

The imaging date and time of the image related to the notification status may be the start time of the moving image related to the notification status, the end time of the moving image, the imaging date and time of the still image related to the notification status, and the like.

As an example of the notification information stored in the information processing device 025a according to the system of the present embodiment, the information may be stored in a table as shown in FIG. 15, for example. An ID may be assigned each time a notification status occurs, and notification information may be stored in association with the ID. Further, in the table according to this figure, notification information may be added in real time each time a notification status occurs.

By recording such information, the information processing device that has accessed the information processing device 025a has an advantage that it can acquire the past notification status and the information related to the notification status. That is, the notification advance notification processing device or other information processing device may be able to display a part or all of the notification information by accessing the information processing device 025a. Here, the other information processing device is not limited to the notification destination information processing device, and is accessed from another information processing device via the WEB or the like by, for example, a user who has an ID and is permitted to access. It may be possible.

The method by which the imaging information processing apparatus transmits the notification information to the notification destination information processing apparatus may be various methods. For example, the image pickup information processing device transmits the notification information to the notification destination information processing device (for example, in this figure, smartphone 006a, smartphone 007a, smartphone 008a, smartphone 009a, information processing device 011a, smartphone 021a, information processing device 023a, information. It may be transmitted directly to the processing device (025a, etc.). In this case, there is an advantage that information can be directly transmitted to each information processing device.

Further, the imaging information processing device transmits the notification information to one or a plurality of predetermined notification destination information processing devices, and one or a plurality of the notification information may be transmitted from a part or all of the notification destination information processing devices. It may be transmitted to a notification destination information processing device other than a predetermined part of the above. For example, in this figure, the smartphone 002 transmits notification information to the information processing device 025a, and the information processing device 025a uses the smartphone 006a, the smartphone 007a, the smartphone 008a, the smartphone 009a, the information processing device 011a, the smartphone 021a, and information processing. It may be transmitted to the device 023a or the like. When the amount of notification information processed is large, there is an advantage that the burden of transmission processing by the smartphone 002 can be reduced.

Further, the imaging information processing device transmits a part of the notification information to the first predetermined notification destination information processing device, and transmits a part or all of the notification information to the second predetermined notification destination information processing device. May be good. For example, in this figure, the smartphone 002 processes information indicating that it is in the notification status by the first predetermined notification destination information processing device (for example, in this figure, the smartphone 006a, the smartphone 007a, the smartphone 008a, the smartphone 009a, and the information processing. It may be transmitted to the device 011a, the smartphone 021a, the information processing device 023a, etc.), and the notification information may be transmitted to the second predetermined notification destination information processing device (for example, the information processing device 025a in this figure). In this case, each construction-related person can acquire information indicating the notification status, understand that it is the notification status, and if necessary, from the information processing device related to the construction-related person, the information processing device 025a. There is an advantage that you can access specific notification information by accessing. In this way, one or more information processing devices that are accessible from the imaging information processing device and each notification destination information processing device and that store the images captured by the imaging information processing device are referred to as "managed information processing devices. (Or sometimes called "management system"). The management information processing device may be a notification destination information processing device or may not be a notification destination information processing device.

Further, the notification information may be according to the notification destination. For example, the foreman 007 and the site supervisor 011 at the site can understand the specific significance of the notification status by immediately moving to the set area, so that the information indicating that the notification status is the information processing device related to the notification destination. It may only be transmitted to. For example, the smartphone related to the foreman 007 and the on-site supervisor 011 acquires information indicating that the notification status is in effect, and executes a process of making a sound, emitting light, and / or trembling as a notification means. Good.

On the other hand, there are cases where even the foreman 007 and the site supervisor 011 want to know specific information, and those who are not at the construction site 021 and 023 may not know the specific information. The notification information may be notified to the information processing device (smartphone 007a, information processing device 011a, smartphone 021a, information processing device 023a, etc.) related to the person.

Alternatively, when only the information indicating the notification status is transmitted to the information processing device related to the construction related person and the construction related person who recognizes the notification status wants to know more specific information, the construction related The information processing device related to the person may be configured to acquire the notification information by accessing the separate information processing device 025a to which the notification information is transmitted. In this case, there is an advantage that the specific contents of the notification information can be accessed as needed.

In addition, the smartphone 002 may notify the information processing devices related to various construction personnel.

For example, the smartphone 002 may notify the smartphone 006a related to the worker 006. For example, there is an advantage that a person who has entered the area without noticing it can be made aware that it is in the area.

Further, the smartphone 002 may notify the smartphone 008a related to the worker 008. This has the advantage that it is possible to help a person in the area in a state where he / she cannot get out of the area by himself / herself. Further, when the area is a danger control area, there is an advantage that it is possible to notify a person who is in a place where such a controllable danger occurs that the danger may occur.

On the other hand, the smartphone 002 may not be set to notify everyone on the construction site. For example, when the worker 009 is far from the imaging range 003 and it is clear that no danger occurs, the smartphone 009a related to the worker 009 does not have to be notified. That is, in the system of the present embodiment, the image pickup information processing device notifies or does not notify the notification destination information processing device according to the distance between the image pickup information processing device and the notification destination information processing device. You may decide. The distance may be determined from the position of the imaging information processing device and the position of the notification destination information processing device. Further, the system of the present embodiment is set so as to notify the first predetermined one or more information processing devices, and not to notify the second predetermined one or more information processing devices. You can do it. Further, the notification destination information processing device may be determined based on a predetermined condition. Further, the notification destination information processing device may be dynamically determined based on a predetermined condition. For example, as a predetermined condition, the notification may be set when the position related to the notification destination information processing device and the position related to the smartphone 002 related to the system of the present embodiment are within a predetermined distance.

When the system of one embodiment has such a configuration, the smartphone 002 related to the system of the present embodiment notifies the smartphone 002 within a predetermined distance, and the smartphone 002 outside the predetermined distance is not notified. It has the advantage of enabling operations in which personnel are notified that there is a potential risk or rescue is possible, while distant workers are not notified as irrelevant.

Further, the smartphone 002 may notify the information processing device 012. The information processing device 012 may include a sound generator that generates sound and / or a light generator that generates light, and may be configured to generate sound when the notification content is received. The information processing device 012 does not have to be associated with an individual person involved in the construction work, but has the advantage of being able to notify that a notification status has occurred at the construction site.

Further, the smartphone 002 may notify the smartphone 021a related to the construction-related person 02 1 in the facility 020 outside the construction site and the information processing device 023a related to the construction-related person 023 in the facility 022 outside the construction site.

Further, the smartphone 002 may notify the information processing device 025a related to the system of the present embodiment.

The notified information processing device may use various notification means so that it can be notified to the persons involved in the construction work. For example, the notified information processing device generates a sound, vibrates, emits light, or displays a predetermined screen as a notification means for notifying the user of the notified information processing device of the notification status. You may perform information processing, display the content of the notification, and so on.

<Application example>
Next, an application example of the system according to this embodiment in the smartphone 002 will be described. FIG. 16 shows an example of a display screen in the smartphone 002. On the display screen, the terminal ID 001 indicating the ID of the smartphone 002, the area setting 002 that enables the area to be set, the notification history 003 that is the history of determining and notifying the person in the area, and the image captured by the imaging unit. Screen 004, which displays the above, is included.

FIG. 17 is a screen example of specific settings for the area setting 002. Although the first region, the region 1002a, and the second region, the region 002b, are set as an example, one or more regions may be set. Further, what kind of conditions are set in the area 1 and the area 2 may be simply displayed. This has the advantage that it is possible to easily understand whether or not there is a condition for each area, and what kind of condition is when there is a condition.

Screen 003 is an example of a screen for specifically setting an area. Area 003a may be set as the area 1, and area 003b may be set as the area 2. The setting may be set by various methods. For example, in a state where the object imaged by the imaging device is displayed in the area 003, a person who sets the area touches and surrounds the area to be set within the range in which the imaged object is displayed on the touch panel. , Each point of the polygon related to the area may be touched. The display unit including the touch panel may store a specific area designated by touch.

Further, the set area may be associated with the displayed specific image, but may not be associated with the displayed specific image.

The former is a case where an area is set in association with a specific image. Therefore, for example, when the position or direction of the image pickup unit is changed, the captured image in the display unit also changes. In accordance with this, the position of the area in the display unit also changes. In this case, as will be described later, the area is associated with the area based on the color, form, etc. of the specific image captured by the image processing technology, and the display position of the area in the display unit is set according to the fluctuating image. Need to fluctuate.

On the other hand, in the latter case, for example, when the imaging unit images the lower part of the crane vehicle and the display unit displays the lower part of the crane vehicle, the set area is the clay vehicle. Refers to the case where it is not related to the image displayed below. In this case, the set area may be associated with the coordinates within the display unit range. The set area is different from the former described above, and how the set area is changed by the movement of the imaging target due to the movement of the imaging device or the imaging direction and the movement or change of the displayed object of the imaged object. There is an advantage that there is no calculation burden to calculate. That is, in the system of one embodiment, the region set by the region unit is associated with the position in the display unit or not with the object in the image captured by the imaging unit. It may be there. Then, when the system of the present embodiment includes such a configuration and a determination unit capable of determining whether the person in the image is in the area or outside the area, it is set in advance in the display unit. It has a technical idea that focuses on whether or not people can enter the area. In this case, when a construction worker sets an area corresponding to a certain image, the image changes when the position or direction of the imaging unit is changed, and the area is set in the initially assumed image. The part of is also subject to change.

Further, in the area setting, the condition, the notification destination, and / or the notification means may be set in relation to one area. The conditions, the notification destination, and / or the notification means set in relation to the one area may be stored in the system of the present embodiment in relation to the one area.

The conditions may be a helmet, a fall-prevention device, and the like, as described above. These may be selected from predetermined ones. The predetermined one may be one that can be identified as a machine-learned one by the determination unit.

The notification destination may be the above-mentioned construction personnel. The construction personnel to be set may be individually designated by a person's name, ID, job title, etc., or may be set by selecting a predetermined group. In this case, there is an advantage that the notification destination can be easily selected. As a group, for example, all workers, workers within a predetermined distance, construction workers at the site, remote managers, all managers, all construction workers, etc. are hierarchized. May be set. The information processing device 025a that records information may always be set as the notification destination. This is because it is necessary to keep a record when a notification status occurs. The information processing device 025a may or may not be displayed as a notification destination.

The notification means may be set according to each notification destination. The notification means may be configured to be selectable from one or a plurality of notification means provided in the notification destination information processing device. As the notification means, as described above, light, sound, vibration, or the like may be used.

The notification means may be set from the smartphone 002, or the notification means may be set by each information processing device itself which is the notification destination. This has an advantage that the notification means can be set according to the convenience of the user of the person related to each information processing device to be notified. On the other hand, the notification means can be set only from the smartphone 002, and the system may be configured so that the notification means cannot be set by each information processing device itself which is the notification destination. This is because it is highly necessary to notify the person related to the information processing device, which is the notification destination, in the dangerous area, so that measures that reduce the effectiveness of the notification means cannot be easily taken.

The determination unit may store one or more areas set as described above in association with information. For example, the determination unit stores information for specifying the area such as the coordinates of the area, a condition for the notification status, a notification destination, a notification means, and the like in association with each area. The smartphone 002 may store such information as the area is set.

FIG. 18 is a screen example in which the notification history 003 can be confirmed. In this figure, the notification status occurs twice, and the history 1003a and the history 2003b are displayed, respectively. Further, in each history, the area where the notification status is generated, the condition where the notification status is generated, and the date and time when the notification status is generated may be displayed. Further, by selecting these by tapping or the like, the moving image taken at the time of the notification status may be displayed on the screen 004. Such a moving image may be configured so that the moving image can be played from a predetermined time before the occurrence of the notification status. Such a video has the advantage of being able to understand what the notification status was like. In addition, these histories may be displayed as unread until they are selected. This unread display has the advantage that it can be understood by displaying a list of unconfirmed items that have occurred in the past.

Next, a step in which the area is set, the smartphone 002 starts imaging, and the person is in the area and the condition is determined will be described.

The imaging unit according to the system of the present embodiment may always continue imaging in the imaging range after the start. The determination unit may determine whether or not the notification status is being obtained during imaging. Further, the imaged object may be stored in the ring buffer. By storing in the ring buffer, as described above, the time before the notification status reaches a predetermined time can be configured to be stored in the ring buffer. Therefore, such information is transmitted to the information processing device 025a or the like and displayed. You can.

As described above, the determination unit may determine whether or not there is a person in the area and whether or not there is a relationship between the person in the area and a predetermined condition, and determine whether or not the person is in the notification status.

In the present embodiment, smartphones 002, 005a, 006a, 007a, 008a, 009a have been described, but instead of smartphones, various mobile terminals such as smartphones, portable computers, tablet PCs, handhelds, tablets, and PDAs, respectively. It may be. In addition, these mobile terminals may include an imaging unit capable of capturing an imaging range and a display unit that displays an imaged object. Further, the display unit may be provided with a touch panel in which an area can be set. In addition, the mobile terminal may include a sound generator that generates sound, a light generator that generates light, and / or a vibration device that generates vibration in order to notify the owner of the mobile terminal of the notification status.

Further, the information processing device 011a, the information processing device 023a, and the information processing device 025a may be a server, a desktop PC, or a notebook computer PC.

Further, the system of the present embodiment may be one information processing device or a combination of a plurality of information processing devices.

Here, an example of the flow of the usage method using the system of the present embodiment will be described at the construction site.

Since the image pickup information processing device (for example, smartphone 002) has an image pickup function,
People can also take pictures. Therefore, first, a worker at a construction site may be used for imaging by using an imaging information processing device (for example, a smartphone 002). The image captured by the worker may be stored in the imaging information processing device (for example, smartphone 002). In addition, the image of the worker may be stored in association with information that identifies the construction site (for example, the date and time of construction, the project name of construction, the location of construction, the manager of construction, etc.). Further, such stored information may be transmitted to and stored in the information processing device 025a and used as a record of the construction site.

In addition, the image of the worker may be used for collating the worker to see if the worker is scheduled to work at the construction site. For example, the image pickup information processing device (for example, smartphone 002) may have information related to a worker who is scheduled to work at the construction site in advance. The information related to such a worker is information for identifying the worker (ID, name, address, etc.), qualification of the worker, affiliation of the worker, image of the worker (for example, whole photograph, face photograph, etc.). , Contract information related to the worker, experience of the worker, and the like. FIG. 19 is an example of data storing information related to such a worker. It may be constructed from such data, a relational database, or the like.

The image pickup information processing apparatus may collate the worker by comparing the image of the worker included in the information related to the worker with the captured image. By collating the workers, the presence or absence of each worker scheduled to work at the construction site may be determined and displayed on the display device related to the image pickup information processing device. At this time, the information related to the worker stored in advance and the captured image may be displayed in comparison with each other. For example, the information related to the worker including the image related to the worker stored in advance and the captured image may be compared and displayed. FIG. 20 is an example of such a display. In this case, since the collation rate is displayed, the identity verification may be displayed numerically by image processing.

The information related to the worker may be stored in advance in a management information processing device (for example, information processing device 025a) instead of being stored in the image pickup information processing device, and the image pickup information processing device may store such management information. By accessing the processing device, information related to the worker may be acquired and displayed in comparison with the above-mentioned collation or as shown in this figure. In this case, the same collation is possible even when the information related to the worker is stored in the management information processing device, such as when the information related to the worker is input in advance by an information processing device other than the imaging information processing device. There are advantages.

Further, the image pickup information processing apparatus transmits the captured image to the management information processing apparatus, performs the above-mentioned collation in the management information processing apparatus, and the image pickup information processing apparatus performs the above-mentioned comparison display as a result of the collation. The information required for the information processing may be acquired and displayed in comparison as shown in this figure. Similarly, in this case, the same applies when the information related to the worker is stored in the management information processing device, such as when the information related to the worker is input in advance by an information processing device other than the imaging information processing device. There is an advantage that matching is possible. In particular, since the collation work can be executed in the management information processing device, there is an advantage that the collation can be efficiently performed even when the image pickup information processing device is worried about battery consumption in the field such as a mobile terminal.

Further, the information processing device related to other construction personnel may access the management information processing device, similarly acquire the necessary information in the above-mentioned comparison table, and display the comparison as shown in this figure. .. As a result, there is an advantage that the workers can be collated even in the information processing device related to the construction personnel other than the imaging information processing device.

Next, at the construction site, the foreman or the site supervisor recognizes the dangerous area, sets the area in the system of the present embodiment, and sets the image in the imaging state. This may be done at a construction site, typically at an occasion such as a morning assembly, but is not limited to that situation. There is an advantage that it can be carried out while sharing information on places that may be dangerous for the parties concerned and places that should be noted about the intrusion of people.

Here, the area in the display unit may be set in the state in which the image pickup information processing apparatus is placed in an actual usage situation. This is because if the position and / or direction of the image pickup information processing apparatus is changed, the object to be imaged is changed, and the set area may be changed from the originally intended one. The area may be set at the position where the information processing device will be used. In addition, an information processing device including an imaging unit may be placed at a position and direction in which an image can be taken from the front of the person in consideration of the path of the person. This makes it possible to take an image from the entire surface when a person comes, and has an advantage of improving the accuracy of determination of a helmet or a fall prevention device, for example. In this case, for example, the plane of the imaging screen may be placed at an angle of 30 degrees in the vertical direction or from the vertical to the front and back, 45 degrees in the front and back from the vertical, and 50 degrees in the front and back from the vertical with respect to the plane of the ground.

Further, in the image pickup information processing device, the position and / or direction of the image pickup information processing device may be set so that the ground on which it is determined that a person is present is included in the image pickup range. In this case, since the position of the person can be directly specified from the image, there is an advantage that the possibility of accurately determining whether the person is inside the area or outside the area is increased.

Further, in the imaging information processing apparatus, the position and / or direction of the imaging information processing apparatus may be set so that the portion to be determined of the plurality of object relations is included in the imaging range. In this case, since the portion to be determined for the target area relationship can be directly specified from the image, there is an advantage that the possibility that the plurality of target relationships can be accurately determined is increased. The portion to be determined for the plurality of object relations may be, for example, the head of a person in the case of a helmet, and the chest or waist of a person in the case of a fall prevention device. When targeting both a helmet and a fall arrest device, the position and / or direction of the imaging information processing device may be set so as to cover the human head, chest, and lumbar region.

Further, when there are a plurality of routes where a person comes, when it is unknown where a person comes from, or when it is desired to improve the accuracy of determination even if there is only one route where a person comes, an image pickup unit is provided. A plurality of information processing devices may be used. For example, a location that is desired to be notified when there is a person can be imaged by a plurality of imaging information processing devices, and each of the plurality of imaging information processing devices can be located at a different position and / or from each other with respect to the location. It may be possible to take an image from the direction. As a result, even if one imaging information processing device cannot make a notification status about the presence of a person, the possibility that another imaging information processing device can make a notification status about the presence of a person increases, which is more desirable. There is an advantage that the possibility of realizing the notification status of is increased.

Further, as described above, when it is determined that one of the plurality of imaging information processing devices is in the notification status for a place where the notification status is desired when there is a person, the notification status is set. In addition to the case of notifying, when at least two imaging information processing devices among the plurality of imaging information processing devices determine that they are in the notification status, they are notified as being in the notification status, or the plurality of imaging devices are imaged. When it is determined that all the imaging information processing devices among the information processing devices are in the notification status, the notification may be made as the notification status. In this case, even in the imaging information processing devices placed in various positions and directions, the notification is sent only when it is determined that the notification status is present, so that the notification is given when there is a high possibility that the notification status is present. There are advantages that can be done.

In the above description, the case of determining the person area relationship which is the notification status of the presence of a person has been described, but similarly, the same may be applied to the case of determining a plurality of target relationships one or more times. ..

After installation, if a person invades the area set in the system of the present embodiment or a person invades without wearing the predetermined equipment, the information processing device set in real time is sent from the system. You may be notified. This has the advantage that the person involved in the notified information processing device can quickly respond to an emergency.

After the system is shut down, in some cases, the parties concerned may check the history to see if the maintenance is sufficient so that they do not enter the set area, or if appropriate measures are taken in the notification situation. You may.

<System for multiple construction sites>
The system of another embodiment may be a system that manages a plurality of construction sites. For example, FIG. 21 is an example of using the system of this embodiment at construction sites 001A, 001B, and 001C. At each construction site, it is assumed that areas 003A, 003B, and 003C are set and placed for smartphones 002A, 002B, and 002C, respectively. Although this figure is an example in which one imaging information processing device is installed at each construction site, a plurality of imaging information processing devices may be installed at each construction site. Further, in this figure, although only one area is set, a plurality of areas may be set, and one or more conditions may be set.

In this case, when the imaging information processing device 002B determines that the notification status has been reached, the imaging information processing device 002B may transmit the notification information to the notification destination information processing device 005B and the management information processing device 025a. The content of the notification information for the notification destination information processing device 005B and the management information processing device 025a may be different. For example, the notification destination information processing device 005B is transmitted with information indicating that it is in the notification status. Information related to the notification status may be transmitted to the management information processing device 025a. Further, the notification information may be transmitted to the information processing device 021a and the information processing device 023a. The contents may be various as described above.

The notification information transmitted from the image pickup information processing device 002B may include information for identifying the image pickup information processing device 002B. By displaying such information on the display device of the information processing device related to the construction personnel, it is possible to understand that the information is generated at the construction site 002 among the three construction sites 001A, 001B, and 001C. There is. Even when a plurality of imaging information processing devices are installed at each construction site, information that identifies only the construction site may be displayed on the information processing devices related to the construction personnel, and imaging information processing may be performed. Information that identifies the device may be displayed. For information processing devices such as information processing devices 021a and 023, which are installed in a place different from the site of the construction site, which image processing device is used to capture the image in one construction site. When it is necessary information that the information is generated at the construction site, there is an advantage that only the information that identifies the construction site, for example, 001B, is displayed. Even in this case, the information processing device related to the construction personnel may be able to access the management information processing device for specific information related to the notification status.

<System at a single site>
Further, the system of another embodiment may be a system for notifying in a small-scale inspection or construction. For example, FIG. 22 shows an inspector 003 who inspects and adjusts a mechanical system 002 such as a power transmission system and a telephone system on a utility pole 001. In this case, since the inspector 003 is above the utility pole, he / she cannot pay sufficient attention to the lower part, but since there is a possibility of dropping an object downward, it is desirable to warn the general public not to invade the lower part. Is done.

In this figure, the smartphone 004a and the smartphone 004b, which are the systems of the present embodiment, capture the imaging ranges 005a and 005b, respectively, so that the general public 006 enters the set region within the imaging range 005a or 005b. When a set notification status occurs, such as, the notification means may be configured to operate.

In the system of the present embodiment in this figure, as a notification means, the smartphone 004a and / or the smartphone 004b itself may generate sound, light, or the like to alert the general public 006. Further, the system of the present embodiment notifies the notification status to a device 007 including a separately provided speaker that generates sound and / or a light generator that generates light, and the device 007 emits sound and / or light. When it occurs, the general public 006 may be alerted. In this case, there is an advantage that the possibility that the general public 006 can be noticed is increased by using a device that can call attention to the outside rather than the sound or light built in the smartphone.

Further, in the system of the present embodiment, the notification means may be similarly operated on the smartphone 008 related to the inspector 003. That is, the system 004a and / or 004b of the present embodiment may be configured so that the notification status is notified to the smartphone 008 and the notification means related to the smartphone 008 operates. In this case, the inspector 003 itself has the advantage of being able to understand the notification status within the area.

It should be noted that each of the single smartphone 004a or smartphone 004b may be an independent system of the present embodiment, or the system of the present embodiment can cooperate with each other. It may be composed of two smartphones, a smartphone 004a and a smartphone 004b. Further, although two examples of the smartphones 004a and 004b having the imaging unit are shown in this figure, the number of smartphones may be one or three or more. In addition to the above, the system of the present embodiment may include an information processing device 025a capable of storing a part or all of the notification information. In this case, the image pickup information processing device (for example, smartphones 004a, 004b, etc.) may have a storage function by recording the imaged contents stored.

In the system according to the present application, the moving image imaging function by the imaging unit may include a sound recording function. Further, such a recorded sound may be recorded in the captured moving image in the notification information. In the case where the system according to the present application has such a configuration, when a notification situation is reached such as a person entering the area or a condition is satisfied, a sound is generated by the notification means to the person in the real area. Also has the advantage of being able to keep a record showing that sufficient alerting has been done.

Although this figure has described the state in which the inspector 003 is present, the system according to the above-described embodiment may be used in the field where the inspector 003 is not present.

Further, although the system in each of the above-described embodiments has described an example of determining the human domain relationship, the system may determine the multiple object relationships one or more times without determining the human domain relationship. In this case, the system according to one embodiment does not have to have a configuration for determining the human area relationship, and may have a configuration for setting conditions for an area or a configuration for setting only conditions without an area.

When setting conditions for regions, one or more regions can be set, and one or more conditions may be set for each region. In this case, the construction worker may set one or more areas and set one or more conditions for each area. In this case, when one area is set, the conditions for that area are set, and it may not be possible to set only the area. In this way, in the case of a configuration in which conditions are set for an area, in addition to the fact that a person is in the area, one or more conditions are determined to determine whether the notification status is present or not. If it is a notification status, the notification information is notified to the notification destination information processing device as described above.

Further, in the case of a configuration in which only conditions are set without an area, one or more conditions may be set. In this case, the construction worker may set one or more conditions for the screen without setting the area. In this case, the area itself may not be set, and the person who has entered the imaging target is determined for one or more conditions to determine whether the notification status is present or not. In the case of the notification status, the notification information is notified to the notification destination information processing device as described above.

The system in each embodiment may be composed of only one or a plurality of image pickup information processing devices, or may be composed of one or a plurality of image pickup information processing devices and one or a plurality of management information processing devices, or may be composed of one or a plurality of management information processing devices. It may be composed of a plurality of image pickup processing devices, a management information processing device, and one or a plurality of notification destination information processing devices.

In each of the above embodiments, the imaging information processing device (for example, smartphones 002, 002a, 002b, 002c, 004a, 004b) is portable such as a tablet, a laptop computer, a laptop computer, a PDA, etc. instead of the smartphone. It may be an information processing device. If it is a portable information processing device, there is an advantage that it can be placed at a place desired by a person involved in the construction work so that the desired direction can be imaged by the person concerned with the construction work.

Further, in each of the above-described embodiments, similarly, notification destination information such as smartphones 005A, 005B, 005C, 006a, 007a, 008a, 009a, 021a, information processing device 023a, and information processing device 025a (management information processing device) The processing device may be the above-mentioned portable information processing device or a fixed information processing device. The fixed information processing device may be a server device, a cloud device, or the like.

In addition, in the above, it was explained that it is difficult to install an image pickup device that can always image the entire construction site without blind spots. For example, hovering over an image pickup device with a security camera that can image even a part of the construction site or on the construction site. Even when the image pickup device in the drone flying by the aircraft is used, the image pickup device may function as an image pickup information processing device. In this case as well, the imaging information processing apparatus may function as described above, or the management information processing apparatus that has acquired information from the imaging information processing apparatus may function as described above.

Further, one or more image pickup information processing devices, management information processing devices, and notification destination information processing devices may have the same owner, user, and administrator, or may be different from each other.

2.2. Embodiment 2
Next, an embodiment of determining the property of a person in the facility will be described with reference to FIG. 23. The description of the system according to the second embodiment will not be described for the parts that overlap with the system according to the first embodiment. For example, the notification information is information related to information that needs to be notified of danger in the system according to the first embodiment, whereas the notification information also needs to be notified of danger or convenience in the second embodiment. It may be information related to a thing. Further, the notification destination information processing device or the like may be a portable information processing device or a fixed information processing device as described above. The omission of these explanations is the same in the third and fourth embodiments.

This figure illustrates the entrance portions 001A and 001B of the facility, and the site 010 shows the inside of the facility. The information processing devices 002a and 002b according to the system of the present embodiment have imaging ranges 003A and 003B, respectively. Person 004 is imaged within the imaging range 003A, and person 005 is excluded from the subject of imaging because it is outside the facility although it is on the extension line of the imaging range 002B.

The facilities may be police stations, financial institutions, medical facilities, accommodation facilities, government offices, educational facilities, cultural facilities, sports facilities, factories, aircraft facilities, vehicle facilities, and their affiliated facilities.

The entrance portion may be a boundary area between the outside of the facility and the inside of the facility as long as a person can enter the inside of the facility from the outside of the facility, or a passage leading from the outside of the facility to the inside of the facility. It may be.

The system may be provided at the entrance portion of the facility described above with the imaging range facing the entrance side. The location where the system is installed may be installed or fixed to the ceiling, walls, or equipment in the facility.

The system of this embodiment is
Imaging unit and
A determination unit capable of determining whether a person in an image captured by the imaging unit has a predetermined item or does not have a predetermined item.
It may be a system including. The predetermined item that can be determined by the determination unit may be a sword or an assault-use item.

The image pickup unit may be a built-in terminal device such as a smartphone, a mobile phone, or a tablet, or may be a security camera that can be fixed to a facility.

When the determination unit determines that the notification status is set, the notification unit may notify the information processing device related to the notification destination. The information processing device related to the notification destination may be various information processing devices. Further, the information processing device related to the notification destination may depend on the facility provided with the imaging unit according to the system of the present embodiment.

For example, the imaging unit related to the system of the present embodiment may be a facility of a police-related organization related to a country or prefecture including a police box, a police box, or a police box (sometimes referred to as a "police-related facility" in the documents of the present application). It is possible to be prepared.

When the imaging unit according to the system of the present embodiment is provided in one police-related facility, the information processing device related to the notification destination is the information processing device related to the staff working in the one police-related facility at the time of the notification status. It may be. In this case, there is an advantage that the staff can detect the risk quickly. Further, in this case, the information processing device related to the staff may use vibration as a notification means so that only the staff can be notified, or use sound as a notification means so that the information can be notified to the surrounding people. May be good.

Further, the information processing device related to the notification destination may be an information processing device installed in the above-mentioned police-related facility. In this case, as the notification means, the person in the police-related facility may be notified by voice, light, or the like. In this case, it is possible to inform not only the persons concerned in the one police-related facility but also the person who happened to be in the one police-related facility that a danger may occur, and to take measures. There are advantages.

Further, the information processing device related to the notification destination may be an information processing device related to an employee who is responsible for dealing with the risk in the above-mentioned police-related facility at the time of notification status. In this case, the staff responsible for dealing with the risk is more likely to safely eliminate the risk.

Further, the information processing device related to the notification destination may be an information processing device installed in another police-related facility within a predetermined range from the one police-related facility provided. In this case, there is an advantage that people in other police-related facilities within a predetermined range can understand the risk and can take measures such as rescue and support.

In addition, the information processing device related to the notification destination is related to the staff working at the time of notification status from the one police-related facility provided to other police-related facilities within the prescribed range or the staff responsible for dealing with risks. It may be an information processing device. In this case, there is an advantage that the staff of other police facilities within the predetermined range can understand the risk and can take measures such as rescue and support.

2.2.1. Embodiment 2.1
Further, it is conceivable that the imaging unit related to the system of the present embodiment is provided in a facility other than police-related facilities (sometimes referred to as a "general facility" in the documents of the present application).

In this case, the information processing device related to the notification destination may be the information processing device related to the security room having jurisdiction over the safety in the general facility. In this case, the risk information in the facility is transmitted to the security room, which increases the possibility of taking appropriate measures.

In addition, the information processing device related to the notification destination is the information processing device related to the staff who manages the responsibilities within a predetermined range from the place in the facility where the image pickup device from which the image of the notification status is acquired is installed, or the predetermined information processing device. It may be an information processing device within the range. Here, the staff may be persons who have duties in the facility according to the characteristics of the facility. In this case, since it is close to the place where the danger has occurred, there is an advantage that the possibility of taking an appropriate response is increased.

In addition, the information processing device related to the notification destination may be an information processing device related to an employee whose duty is to maintain safety in a general facility. Security guards are examples of employees whose duty is to maintain safety. In this case, the risk information in the facility is directly transmitted to the guards, which increases the possibility of taking appropriate measures.

Further, the information processing device related to the notification destination may be an information processing device installed in a police-related facility within a predetermined range from the facility or an information processing device related to a police-related person within a predetermined range from the facility. In this case, there is an advantage that police personnel within a predetermined range can understand the risk and can take measures such as rescue and support.

2.2.2. Embodiment 2.2
Further, instead of the swords and the assault use item in the second embodiment, other items may be used. For example, as another item, the facility visitor may be notified when he / she is using a stroller or has luggage larger than a predetermined size in order to promote services at the facility.

That is, the system of this embodiment is
Imaging unit and
A determination unit capable of determining whether a person in an image captured by the imaging unit has a predetermined item or does not have a predetermined item.
The predetermined item that can be determined by the determination unit may be a stroller or a baggage having a size equal to or larger than a predetermined value.

When the judgment unit determines that the notification status is in the notification status, the information processing device related to the notification destination takes responsibility within a predetermined range from the location in the facility where the image pickup device in which the image for which the notification status is determined is acquired is installed. It may be an information processing device related to the staff in charge or an information processing device within a predetermined range. This has the advantage that nearby staff can quickly understand that a visitor who is likely to need help is coming and are more likely to be able to help.

2.2.3. Embodiment 2.3
Further, instead of the swords and assault use items in the second embodiment, a helmet may be used. In the facility, anything that obstructs the face, such as a helmet, is not preferable for security reasons, so visitors wearing such a helmet may be notified to take off the helmet.

That is, the system of this embodiment is
Imaging unit and
A determination unit capable of determining whether or not a person in the image captured by the imaging unit is wearing a helmet, and a determination unit.
When it is determined that the person is wearing a helmet, a notification unit that notifies the information processing device related to the notification destination, and a notification unit.
It may be a system including.

The information processing device related to the notification destination is within a predetermined range from the location in the facility where the image pickup device from which the image for which the notification status is determined is installed is within the specified range or the information processing device related to the staff responsible for the responsibility. It may be an information processing device. In the former case, since the staff is contacted, there is an advantage that measures can be taken to take off the helmet quickly. Further, in the latter case, if the information processing device can generate a sound to the person wearing the helmet, there is an advantage that the person wearing the helmet can be alerted.

2.2.4. Embodiment 2.4
Further, instead of the swords and the assault use item in the second embodiment, a smartphone may be used. Since it is not preferable to use a smartphone in the kitchen area of the facility in terms of hygiene, if there is a person who uses such a smartphone, there is an advantage that the possibility of taking action is increased by notifying the designated notification destination. is there.

That is, the system of this embodiment is
Imaging unit and
A determination unit capable of determining whether or not a person in the image captured by the imaging unit owns or does not have a smartphone.
When it is determined that the person owns a smartphone, a notification unit that notifies the information processing device related to the notification destination, and a notification unit.
It may be a system including.

The information processing device related to the notification destination is within a predetermined range from the location in the facility where the image pickup device from which the image for which the notification status is determined is installed is within the specified range or the information processing device related to the staff responsible for the responsibility. It may be an information processing device. In the former case, since the staff is contacted, there is an advantage that measures can be taken to take off the helmet quickly. Further, in the latter case, if the information processing device can generate a sound to the person wearing the helmet, there is an advantage that the person wearing the helmet can be alerted.

2.3. Embodiment 3
The system according to the next embodiment is used for inspection of a predetermined item. That is, a person with a certain responsibility may be required to equip a predetermined item, and has an advantage of being able to detect forgetting to equip such an item.

That is, the system of this embodiment is
Imaging unit and
A determination unit capable of determining whether or not a person in an image captured by the imaging unit has or does not have a predetermined item.
When it is determined that the person does not have the predetermined item, the notification unit that notifies the information processing device related to the notification destination, and the notification unit.
It may be a system including. Also, the predetermined items may be a predetermined pistol, a predetermined handcuff, and a predetermined baton.

Further, the imaging unit may be installed in the above-mentioned police-related facility. Further, the imaging unit may be provided at a position where a person moving from the inside to the outside of the police-related facility can be imaged at the exit in the police-related facility. Further, the person who moves from the inside to the outside may be a police-related person, not a general person who enters a police-related facility. For example, a police officer may be on a passage that moves from the inside to the outside of a police facility. In addition, such a passage may be a passage dedicated to police personnel.

In addition, the determination unit capable of determining whether or not the person has a predetermined item determines whether or not the person is equipped with the predetermined item at a predetermined location. It may be a possible determination unit.

Further, when it is determined that the person does not have the predetermined item, the notification unit that notifies the information processing device related to the notification destination states that the person does not equip the predetermined item at the predetermined location. If it is determined, it may be a notification unit that notifies the information processing device related to the notification destination. It should be noted that having a predetermined item may be a superordinate concept of equipping a predetermined item at a predetermined place.

The information processing device related to the notification destination may be an information processing device related to the person. In this case, the person has the advantage of being able to notice that he / she is not equipped with the predetermined item and respond quickly.

In addition, the information processing device related to the notification destination may notify the person by voice and / or vibration. In this case, the person has the advantage of being more likely to notice that he is not equipped, even in a busy state just before departure, for example.

Further, the determination unit identifies unequipped items among predetermined items such as a predetermined pistol, a predetermined handcuff, and a predetermined baton, and the notification unit determines the unequipped item. The information processing device related to the notification destination may be notified. In this case, the person has the advantage of being able to notice what is not equipped among the predetermined items.

In addition, the information processing device related to the notification destination may notify the person of the information identifying the non-equipped item by sound. In this case, the person has an advantage that he / she can be specifically notified and noticed what is not equipped by sound.

Further, the information processing device related to the notification destination may be the information processing device related to the exit in the police-related facility. For example, it may be an information processing device related to the exit gate.

2.4. Embodiment 4
The system according to the next embodiment determines whether or not the automobile user is using a predetermined item. That is, since it is dangerous for a driving user to use a wireless communication device such as a smartphone or a mobile phone specified by the Road Traffic Act, there is an advantage that this can be determined.

The system according to one embodiment includes an imaging unit that captures an image of a driver in a vehicle and an imaging unit.
A determination unit that determines whether or not a person in an image captured by the imaging unit holds or does not hold a predetermined item.
When the determination unit determines that the person in the image holds the predetermined item, the notification unit that notifies the information processing device of the notification destination and the notification unit.
It may be a system including.

The predetermined item may be a wireless communication device specified by the Road Traffic Act.

The information processing device of the notification destination may be the information processing device related to the vehicle. In addition, the information processing device related to such a vehicle may have a warning function of issuing a warning to the driver. The warning function may be sound, light, action, or the like. The sound may be something that warns the driver, may be a simple alarm, or may be an expression that stops the use of the wireless communication device. The light may be an alarm. The movement may be an movement for directing the vehicle to slow down or stop.

Further, the information processing device of the notification destination may be an information processing device related to a police organization. In this case, it may be an information processing device related to a police organization having jurisdiction over a place where a vehicle or the like is located, or an information processing device related to a police person who is within a predetermined range from the place where the vehicle or the like is located. In the former case, information can be provided to police agencies within the jurisdiction of the traffic violation, and in the latter case, information can be provided to police personnel close to the scene of the traffic violation. The imaging unit in the system according to the embodiment may capture information that identifies the vehicle. For example, the imaging unit may image the vehicle including the license plate and / or the driver. In addition, the information processing device related to the police organization may identify the owner of the vehicle from the license plate. To identify the owner, an information processing device having a relationship between the license plate and the owner as a database may be used. Such an information processing device may be an information processing device related to a police organization, and may be, for example, a server device, a cloud device, or the like. In addition, the information processing device related to the police organization may have a display function for displaying the owner of the vehicle, the license plate of the vehicle, the captured image, the date and time of the capture, and / or the location where the image was captured. .. With such a display, the viewer can understand the information about the violator while understanding the violator by the image. Such a display may be transmitted and displayed to an information processing device related to a police officer who is within a predetermined range from the place where the vehicle or the like is imaged. For example, it may be a police person who is within a predetermined range from the imaged place, for example, a patrol car, a police motorcycle, or an information processing device related to a driver thereof. This could allow them to rush to the scene and deal with the offender.

Further, in the system according to one embodiment, in order to identify a driver in a vehicle, a person within a predetermined range from a steering device such as a steering wheel or a steering wheel in an image captured by an imaging unit is identified as a driver. You can do it. The determination unit in the system according to one embodiment may have the above-mentioned steering device and a function of identifying a person.

3. 3. Hardware Configuration of System The system according to the present invention may be composed of one or a plurality of information processing devices. As shown in FIG. 24, the information processing device 10 according to the present invention may include a bus 15, an arithmetic device 11, a storage device 12, and a communication device 16. Further, the information processing device 10 in one embodiment may include an input device 13 and a display device 14. It is also directly or indirectly connected to the network 17.

The bus 15 may have a function of transmitting information between the arithmetic unit 11, the storage device 12, the input device 13, the display device 14, and the communication device 16.

An example of the arithmetic unit 11 is a processor. This may be a CPU or an MPU. Further, the arithmetic unit in one embodiment may include a graphics processing unit, a digital signal processor, and the like. In short, the arithmetic unit 12 may be any device capable of executing program instructions.

The storage device 12 is a device that records information. This may be either an external memory or an internal memory, and may be either a main storage device or an auxiliary storage device. Further, a magnetic disk (hard disk), an optical disk, a magnetic tape, a semiconductor memory, or the like may be used. Further, it may have a storage device via a network or a storage device on the cloud via a network.

In this block diagram, the registers, L1 cache, L2 cache, etc. that store information at a position physically close to the arithmetic unit may be included in the arithmetic unit 11, but in the design of the computer architecture, As a device for recording information, the storage device 12 may include these. In short, the arithmetic unit 11, the storage device 12, and the bus 11 may be configured to cooperate with each other to execute information processing.

The storage device 12 may include a part or all of a program capable of executing the process according to the present invention. In addition, data necessary for executing the process according to the present invention can be appropriately recorded. Further, the storage device 12 in one embodiment may include a database.

Further, the above describes the case where the arithmetic unit 12 is executed based on the program provided in the storage device 13, but it is one of the forms in which the bus 11, the arithmetic unit 12 and the storage device 13 are combined. As a result, part or all of the information processing according to the present invention may be realized by a programmable logic device capable of changing the hardware circuit itself or a dedicated circuit in which the information processing to be executed is determined.

The input device 13 inputs information, but may have other functions. Examples of the input device 14 include an input device such as a keyboard, a mouse, a touch panel, or a pen-type instruction device.

The display device 14 has a function of displaying information. For example, a liquid crystal display, a plasma display, an organic EL display, and the like can be mentioned, but in short, any device capable of displaying information may be used. Further, the input device 13 may be partially provided like a touch panel.

The network 17 transmits information together with the communication device 16. That is, it has a function of enabling information of 10 information processing devices to be transmitted to other information terminals (not shown) via the network 17. The communication device 16 may use any connection type, such as IEEE1394, Ethernet (registered trademark), PCI, SCSI, USB, 2G, 3G, 4G, 5G, and the like. The connection to the network 17 may be either wired or wireless.

The information processing device according to the present invention may be a general-purpose type or a dedicated type. Further, the information processing device may be a workstation, a desktop personal computer, a laptop personal computer, a laptop computer, a PDA, a mobile phone, a smartphone or the like.

Although described as one information processing device 10 in this figure, the system according to the present invention may be composed of a plurality of information processing devices. The plurality of information processing devices may be internally connected or may be externally connected.

In addition, the system according to the present invention may be of various types of devices. For example, the system according to the present invention may be a stand-alone system, a server-client system, a peer-to-peer system, or a cloud system. The system according to the present invention may be a stand-alone information processing device, may be composed of a part or all of the information processing device of the server-client type, or may be a part or all of the information processing of the peer-to-peer type. It may be composed of devices, or may be composed of some or all information processing devices in the cloud format.

Further, when the system according to the present invention is composed of a plurality of information processing devices, the owner and the manager of each information processing device may be different.

Further, the information processing device 10 may be a physical existence or a virtual one. For example, the information processing device 10 may be virtually realized by using cloud computing.

In the above description, the configurations implemented by the system of this example have been described, but these may be configurations implemented by one or more information processing devices in the system. Further, the information processing device described above as the portable information processing device may be an information processing device that is appropriately installed and fixed.

It goes without saying that the invention examples described in the examples of the documents of the present application are not limited to those described in the documents of the present application, and can be applied to various examples within the scope of the technical idea. For example, in the embodiment of the document of the present application, the information presented on the screen of the information processing apparatus can be displayed on the screen of the other information processing apparatus and thus can be transmitted to the other information processing apparatus. The system may be configured.

Further, the processes and procedures described in the documents of the present application may be feasible not only by those explicitly described in the embodiments, but also by software, hardware or a combination thereof. Further, the processes and procedures described in the documents of the present application may be able to be implemented by various computers by implementing the processes and procedures as a computer program. Further, these computer programs may be stored in a storage medium. Also, these programs may be stored on a non-transient or temporary storage medium.

10 Information processing device 11 Arithmetic device 12 Storage device 13 Input device 14 Display device 15 Bus 16 Communication device 17 Network

Claims (15)

Imaging unit and
A display unit capable of displaying an image captured by the imaging unit, and a display unit capable of displaying the image.
In the display unit, an area unit in which an area can be set and an area unit
A determination unit capable of determining whether a person in an image captured by the imaging unit is within the region or outside the region.
System with. The area set by the area unit is related to the position in the display unit.
The system according to claim 1. The region set by the region portion is not related to the object in the image captured by the imaging unit.
The system according to claim 1 or 2. When the determination unit determines that the person is in the area, the determination unit determines whether or not the person is wearing a predetermined item.
The system according to any one of claims 1 to 3. The predetermined items include fall arrest devices, helmets, safety goggles, gloves, safety boots, smartphones, swords, and / or assault items.
The system according to claim 4. A first region and a second region can be set in the region portion.
The area is
The person in the image captured by the imaging unit is in the first region.
Or
The person in the image captured by the imaging unit is in the second region, and the person is not wearing the predetermined item.
The system according to any one of claims 1 to 5, which can be set to notify when the determination unit determines. The system is a portable information processing device having the imaging unit and the display unit integrated.
The system according to any one of claims 1 to 6. A portable information processing device including the imaging unit and the display unit in an integrated manner.
The system according to any one of claims 1 to 7. The determination unit has a machine-learned function that determines only whether or not a person is in the area and whether or not the person wears the predetermined item.
The system according to any one of claims 1 to 8. The determination unit is determined by a machine-learned function that determines whether or not the person is in the area and whether or not the person is wearing the predetermined item, and acquires information on whether or not the notification status is reached. ,
The system according to any one of claims 1 to 9. The area can be set in the display screen,
When it is determined that there is a person in the area, another information processing device is notified.
A portable information processing device
The information processing device is an information processing device in which an imaging unit and a display unit are integrated.
The area is set at a position where the information processing apparatus is scheduled to be used.
Information processing device. The area can be set in the display screen,
When it is determined that there is a person in the area, sound, light, or vibration is generated.
A portable information processing device
The information processing device is an information processing device in which an imaging unit and a display unit are integrated.
The area is set at a position where the information processing apparatus is scheduled to be used.
Information processing device. Computer,
A display means for displaying an image captured by the image pickup means,
Means for setting an area in the display,
A determination means for determining whether a person in an image captured by an imaging means is within the region or outside the region.
A program to function as. The computer
A step of displaying an image captured by an imaging means,
Steps to set the area in the display,
A step of determining whether a person in an image captured by an imaging means is within the region or outside the region.
How to do. A program for causing a computer to function as the system according to any one of claims 1 to 10 or the information processing device according to claim 11 or 12.

Images