JPH0759754A - Active state monitoring apparatus - Google Patents

Abstract

PURPOSE:To provide an active state monitoring apparatus which monitors the acting state of a worker by detecting infrared rays radiated from the body of the worker by specifying the worker accurately even when a plurality of men exist within a detection image area. CONSTITUTION:A part detection means 5 is provided to detect parts of a human body existing in an detection image area based on area information and or positional information in the area extracted by a processing of extracting an infrared image data detected by an infrared image detection means 2 using the upper limit and the lower limit of a skin temperature band as threshold. A worker detection means 8 is provided to specify part areas of a worker based on a high-frequency component of a luminance data of a visible image corresponding to part areas of the human body of the worker as detected by a part detection means 5 and a mean 3 to monitor the acting state of the worker based on coordinate path of parts of the worker specified by the worker detection means 8.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a device for detecting infrared rays emitted from a human body to monitor the activity state of an operator.

[0002]

2. Description of the Related Art In recent years, the role of a centralized control room in a large-scale automated system such as an unmanned manufacturing plant has become extremely important. Then, there is a possibility that a worker who manages the operation of the system may be fatigued and a work error due to awakening may cause a serious accident.

Therefore, it is an important subject to monitor the activity state of the worker and detect the degree of fatigue, the degree of awakening, etc. As a monitoring method, the applicant of the present invention uses infrared rays emitted from the human body. We propose a method to detect the human body part and extract the part of the human body to monitor the activity status of each part.

[0004]

However, in the above-mentioned prior art, an infrared camera captures an infrared image of a region including a place where a worker to be monitored is engaged, and the human body is detected based on the detected infrared image. Since the part extraction is performed, there is a possibility that only the part extraction of the human body cannot be surely performed when the infrared radiation substance other than the human body exists in the detected image area.

In addition to the infrared camera, when the CCD camera is used to extract the human body part based on the infrared image and the visible image, the human body part can be surely extracted. However, when another person is present in the detected image area, the worker may not be specified and may be erroneously detected.

The present invention has been made in view of the above point, and even when there are a plurality of persons in the detected image area, the operator can be reliably identified and the human body of the operator can be identified. It is an object of the present invention to provide an activity state monitoring device capable of detecting infrared rays emitted from a human body to extract a part of a human body, monitoring the activity state of each part, and detecting the fatigue level, awakening level, etc. of an operator.

[0007]

According to the present invention, a focus is adjusted to a place where a worker is engaged, and an image pickup means for picking up a visible image of the worker, and a red image in the same area as an image area detected by the image pickup means. Infrared image detecting means for detecting an outside image, and infrared image data detected by the infrared image detecting means is subjected to data extraction processing with the upper limit and the lower limit of the skin temperature band as thresholds, and each extracted region Based on the area information and / or position information of the human body, the body part detecting means for detecting each body part existing in the detected image area, and the visible part corresponding to each body part area detected by the body part detecting means. A worker detection unit that specifies each part region of the worker based on the high-frequency component of the brightness data of the image, and a worker based on the coordinate locus of each part of the worker specified by the worker detection unit Live It is active monitoring device according to claim which comprises a means for monitoring the condition, the.

[0008]

According to the present invention, even when a plurality of persons are present in the detected image area, the operator can be identified with certainty, and based on the infrared image data radiated from the human body of the operator. It is possible to monitor the site extraction of the worker and the activity status of each extracted site.

[0009]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below with reference to the drawings showing the embodiments thereof.

FIG. 1 is a schematic block diagram of an activity monitoring apparatus showing an embodiment of the present invention. In the figure, reference numeral 1 is an image pickup means for picking up a visible image of a place where a worker to be monitored is engaged, and is constituted by a CCD camera for outputting video data consisting of RGB signals.
The focus of the is adjusted to the place where the worker is engaged.

Reference numeral 2 denotes an infrared image detecting means for detecting an infrared image in the same area as the image area detected by the CCD camera 1 and outputting it as infrared image data, which is composed of an infrared camera. Reference numeral 3 denotes a main control unit that controls the driving of the CCD camera 1 and the infrared camera 2, and is supplied with the detected video data and infrared image data, and is composed of a CPU (microcomputer). In the present embodiment, the CPU 3 takes in the video data from the CCD camera 1 and the infrared camera 2 and the infrared image data every 0.1 seconds.

Reference numeral 4 is connected to the CPU 3 and, based on the image data from the CCD camera 1 and the infrared camera 2 and the infrared image data, extracts a part of the human body (in this embodiment, a face area in the present embodiment). , And the right-hand area and the left-hand area are extracted.), And a ROM storing a program for monitoring the activity state, 5 is image data from the CCD camera 1 and the infrared camera 2, and infrared image data. Based on the above, it is a part detecting means for extracting the part of each person existing in the detected image area and discriminating the kind of the extracted part. Specifically, in the part detecting means 5, from the difference between the image data from the CCD camera 1 and the image data when a person does not exist in the detected image area stored in the RAM 6 in advance,
The image data of each person existing in the detected image area is extracted, and the infrared image data corresponding to the extracted image data area is set to the upper limit value and the lower limit value of the skin temperature band (in the present embodiment, the skin temperature band). Upper limit of 38 ℃, lower limit of 33
℃ was made. ) Is used as a threshold value to perform data extraction processing to extract a human body part. As a result, the exposed skin site of each person existing in the detected image area is extracted. Then, the part detecting means 5 determines the type of the extracted part based on the area information and the position information of each part of the extracted part group of each person.

FIG. 2 shows a schematic view of the extraction site in this case. The stippled area in the figure indicates the extraction site, and the X mark indicates the center of gravity of each extraction site. Then, the type of the extraction site corresponding to each person existing in the detected image region is determined based on the area information and the position information of each extraction site, and for example, the site extraction result for each person is shown in FIG. In this case, the extracted part region A having the largest area is determined to be the face part, and the other extracted part regions B and C are determined to be the hand parts. From the positional relationship of the hand parts, the region B is the right hand and the region C is the left hand. It is determined.

Reference numeral 7 is a high frequency detecting means for calculating the luminance data based on the image data of each portion of the human body extracted by the portion detecting means 5 and subjecting the luminance data to frequency analysis to detect a high frequency component. , The level of the high frequency component of 200 kHz or more is detected in the brightness data of each part.

Reference numeral 8 denotes a worker detecting means for identifying a worker existing in the detected image area based on the detection data of the high frequency detecting means 7 and detecting the part of the human body. Specifically, based on the detection data of the high frequency detecting means 7, the high frequency components of the extracted region group of each person existing in the detected image area are integrated, and the extracted region having the largest value of the integrated high frequency component data is extracted. The group is defined as the part of the human body of the worker. This is set in advance so that the CCD camera 1 is focused on the place where the worker is engaged.
The fact that the high frequency component of the brightness data obtained from the CD camera 1 becomes large is used.

That is, as shown in FIG. 3, when a person b exists in addition to the worker a in the detected image area, CC
Since the focus of the D camera 1 is set so as to match the place where the worker a is engaged, the high frequency components of the brightness data of the extraction site groups 31 to 33 obtained from the CCD camera 1 are the extraction site groups 34 to 36. It becomes larger than the high-frequency component of the brightness data, and the extracted part groups 31 to 33 are determined to be the extracted parts of the human body of the worker a. Incidentally, the stippled area in the drawing indicates the extraction site, and the x mark indicates the center of gravity of each extraction site.

Reference numeral 9 denotes infrared image information having 512 frame memories for storing infrared image data, barycentric coordinate data, and measurement time data of the extracted region group of the worker region specified by the worker detecting means 8. It is a storage unit. Therefore,
The infrared image information storage unit 9 has a resolution of 5 seconds in units of 0.1 seconds.
Infrared image data, barycentric coordinate data, and measurement time data of the extracted region group of the worker area for 1.2 seconds are stored.

Reference numeral 10 denotes a frequency component calculation means for calculating the frequency spectrum of each extracted portion by performing a fast Fourier transform on the barycentric coordinate data stored in the infrared image information storage unit 9, and a fast Fourier translator (hereinafter referred to as an FFT circuit). Abbreviated.). Reference numeral 11 denotes a monitor that can input operation instructions of this device on the screen and displays the monitoring result.

The RAM 6 has a ROM in addition to the video data when a person does not exist in the detected image area described above.
The existence probability distribution data of the barycentric coordinates of each extracted portion during normal work, the frequency component data, and the moving speed data, which are calculated based on the program stored in 4, are stored.

Here, the CPU 3 receives the CCD camera 1, the infrared camera 2, and the infrared camera 2 based on the input signal input from the monitor 11.
By controlling the part detection unit 5, the high frequency detection unit 7, the worker detection unit 8, the infrared image information storage unit 9, the FFT circuit 10 and the like, the extraction of the part of the human body existing in the detected image area and the selection of the type of the extracted part Discrimination, identification of the extracted body parts of the human body of the worker,
And monitoring the activity status. Then, the activity state of the worker is monitored based on the coordinate locus of each of the extracted parts of the worker. Specifically, as will be described later, it is calculated from the existence probability of the barycentric coordinates of each extracted part and the barycentric coordinate track. It is performed based on the frequency component and the moving speed of the center of gravity.

Next, the operation of the activity monitoring apparatus of the present invention will be described with reference to the flow charts of FIGS. 4 and 5.

First, it is judged whether or not the initial state is detected when no human is present in the detected image area to be monitored (S10).
1), when detecting the initial state, drive the CCD camera 1,
The video data is stored in the RAM 6 as initial value data (S103), and the process returns to step S101.

If the initial state is not detected in step S101, that is, if the active state is detected,
The CCD camera 1 and the infrared camera 2 are driven to read the video data and the infrared image data (S105).

Then, the region detecting means 5 performs a region extracting process for each person existing in the detected image area to determine the type of the extracted region. Specifically, first, the RAM 6
The image data area of each person existing in the detected image area is extracted from the initial value data stored in step S105 and the image data read in step S105 (S1).
07). Then, the infrared image data corresponding to the video data area of each person extracted in step S107 is subjected to threshold value processing in the range of the brightness value corresponding to the temperature band of the skin to extract the site (S109). A number is assigned to each of the extracted region groups (hereinafter, abbreviated as labeling) (S111). Although the skin temperature band is set to 33 to 38 ° C. in the present embodiment, the optimum value may be determined depending on the conditions such as environment and clothes. Then, the labeled regions are sorted in descending order of area, and label numbers are allocated again in order (S113). In the next step S115, the face part of the human body, the right hand, and the left hand are based on the label area information and the position information. Determine the area. Specifically, the label 1 having the largest area in the extracted region group for each person is determined to be the face portion, and the right side of the labels 2 and 3 is assumed to be the right hand and the other is the left hand, and the predetermined time After the tracking process of 1., the right hand and the left hand are judged.

Next, the above steps S107 to S115.
The image data of the face portion, the right hand, and the left hand area for each person determined by the processing of 1 is supplied to the high frequency detecting means 7,
The level of the high frequency component is detected (S117), and the detection data is supplied to the worker detecting means 8 to detect the extracted region group of the worker (S119).

Then, the center of gravity of each part of the worker's face, right hand, and left hand detected in step S119 is calculated (S121), and infrared image data, barycentric coordinate data, and measurement of each extracted part are calculated. The time data is stored in the infrared image information storage unit 9
And stores it (S123). Then, it is determined whether or not the infrared image data, the barycentric coordinate data, and the measurement time data for each extracted portion of 512 sets of workers are stored in the infrared image information storage unit 9 (S125). Go to step. In this embodiment, infrared image data for 51.2 seconds with a resolution of 0.1 second is stored in the infrared image information storage unit 9, but this value needs to be changed appropriately depending on the work content. is there.

Next, 512 sets of barycentric coordinate data and measurement time data for each extracted part of the operator stored in the infrared image information storage section 9 are read (S127), and based on these data, each extracted part is extracted. The existence probability distribution of the center of gravity in the predetermined area is calculated for each (S129), and the process proceeds to the next step S131. For example, FIG. 6 shows the calculation result of the existence probability distribution of the center of gravity in the case where the horizontal movement work is performed with the left hand. The numbers in the figure represent the probability of existence of the center of gravity within that region.

In step S131, step S127
Is input to the FFT circuit 10, the frequency spectrum of each extracted part of the operator in the predetermined region calculated by the FFT circuit 10 is read (S133), the fundamental frequency of each extracted part, the harmonic pattern, etc. A characteristic frequency component is extracted (S135).

Then, in the next step S137, the moving speed of the center of gravity of each of the extracted parts of the worker is calculated based on the data of step S127, and the process proceeds to the next step S139.

In step S139, it is judged whether or not the normal working state is set as a reference, that is, whether the reference is set or not.
The existence probability distribution data of the center of gravity of each extracted part of the worker calculated in 129, S135, and S137, the frequency component data, and the moving speed data are stored in the RAM 6, and the reference setting operation is ended.

If it is determined in step S139 that the standard is not set, the flow advances to step S143, and the existence probability distribution data of the center of gravity of each extracted part of the operator calculated in steps S129, S135, and S137, The frequency component data and the moving speed data are compared with the stored data stored in the RAM 6 (S143), the change in the operation content is digitized based on the comparison data, and the value is displayed on the monitor 11 (S145). , End active activity monitoring. Here, the comparison of the existence probability distribution data of the center of gravity of each extracted part of the worker detects a sudden motion with respect to the normal motion, and the comparison of the frequency component data shows a change in the periodic motion or a change in the locus of the extracted part. It is detecting. In addition, the movement speed data of the center of gravity is compared to detect the agility and slowness of the movement.

Then, in step S145, the fatigue level and the awakening level of the worker are numerically divided into five stages using these comparison data. Level 1 is a normal state, Level 2 is a state where slight fatigue is seen but there is no problem in work, Level 3 is a state where a break is required within a predetermined time, Level 4 is fatigue and it is dangerous to continue work, immediately A state requiring a break, level 5 indicates a state in which work cannot be continued due to severe fatigue, and the worker needs to be replaced.

In the above embodiment, the case where the activity state of the human body is comprehensively monitored based on the existence probability distribution data of the center of gravity of each extracted portion of the worker, the frequency component data, and the moving speed data has been described. However, the activity state of the human body may be monitored based on the individual comparison data.

[0034]

As described above, according to the present invention, even when a plurality of people exist in the detected image area,
The worker can be identified with certainty, and based on the infrared image data radiated from the human body of the worker, site extraction of the worker and the activity state of each extracted site can be monitored.

Therefore, it is possible to detect the degree of fatigue of the worker, the degree of awakening, etc., and it is possible to prevent the occurrence of work mistakes due to the fatigue, awakening, etc. of the operator.

[Brief description of drawings]

FIG. 1 is a schematic block diagram of an activity state monitoring device showing an embodiment of the present invention.

FIG. 2 is a schematic diagram of an extracted part when the part of the human body is extracted according to the present invention.

FIG. 3 is a screen view of a detection image area by a CCD camera when a plurality of people are present.

FIG. 4 is a flowchart for explaining an activity state analysis operation of the present invention.

FIG. 5 is a flowchart for explaining an activity state analysis operation of the present invention.

FIG. 6 shows the existence probability distribution of the center of gravity when horizontal work is performed with the left hand.

[Explanation of symbols]

 1 CCD camera (imaging unit) 2 infrared camera (infrared image information storage unit) 3 CPU (main control unit) 5 site detection unit 7 high frequency detection unit 8 worker detection unit 10 FFT circuit (frequency component calculation unit)

Claims (2)

[Claims] 1. The focus is adjusted to the place where the worker is engaged,
Imaging means for picking up a visible image of the worker, infrared image detecting means for detecting an infrared image in the same area as the image area detected by the imaging means, and infrared image detected by the infrared image detecting means Data is extracted using the upper and lower limits of the skin temperature band as thresholds, and based on the area information and / or position information of each extracted region, each part of the human body present in the detected image region is detected. Part detection means for detecting, and worker detection means for specifying each part area of the worker based on the high frequency component of the brightness data of the visible image corresponding to each part area of the human body detected by the part detection means And a means for monitoring the activity status of the worker based on the coordinate trajectory of each part of the worker identified by the worker detection means. 2. The activity according to claim 1, wherein the monitoring means monitors the activity state of the worker based on the existence probability of the center of gravity of each part of the worker, the frequency component, and the moving speed. Condition monitoring device.