JP2003052039A - System and method for measuring work motion, and program for measurement of work motion - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make the work efficient by measuring the action of a worker and utilizing it for improvement of work environment. SOLUTION: This system is equipped with a camera 1 and a computer 2 which takes in an image and processes it. The camera 1 is equipped with a driver 11 which drives and controls the camera so that it may change the take-in range of the specified image through the control of the computer 2, and also the computer 2 is equipped with a following-data extracting function which extracts the following-data concerned with the worker 3 expressed in the images taken in at specified intervals, a camera control function which controls the drive of the driver of the camera following the worker 3 so that the position of the worker 3 may be positioned in the specified place on the image of the camera, based on this extracted following-data, and a measured data storage function which stores the time of take-in of the image and the quantity of drive control of the camera.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a work movement measuring system, method and program, and more particularly to a system, method and program for measuring the movement of a worker who works in a predetermined factory.

[0002]

2. Description of the Related Art At a work site such as a factory, from the viewpoint of reducing the burden on the worker and preventing accidents, the items and order of the work should be set so as to reduce the worker's movement within the area where he is in charge. We are taking measures such as ingenuity and improving the working environment.

As a data collection method for how much the worker moves during work, a pedometer (registered trademark) is generally used or a method in which another person measures the worker's movement is adopted. . Then, based on the data obtained in this way, the work process is reviewed, the layout, the environment, etc. are adjusted to improve the work efficiency in the factory.

[0004]

However, the above conventional example has the following inconveniences. When the pedometer is used as described above, the movement distance can be measured by the number of steps taken by the worker, but the movement direction cannot be obtained, and further, since the posture change cannot be recognized, the work There is a problem that the detailed behavior of the person cannot be grasped. In addition, when another person measures, there is a problem that the labor cost of the person who performs the measurement is required. Then, in such a case, it is considered that the worker is psychologically affected by being seen by the person who measures the worker, and the worker is prevented from performing the work as usual. In some cases, the reliability of the measurement data may deteriorate.

[0005]

An object of the present invention is to improve the inconvenience of the above-mentioned conventional example, and in particular, it is possible to measure the motion of the worker without giving excessive consciousness to the worker, and the measured data can be stored in the work environment. It is an object of the present invention to provide a work motion measuring system, method, and program that can be used for improvement and the like to improve work efficiency.

[0006]

Therefore, the present invention is a system for measuring the motion of a worker at a predetermined work site, and a camera for capturing an image of the work site at a predetermined time interval, and this camera. The camera includes a computer that controls an image capturing operation and processes the captured image, and the camera includes a drive unit that controls the camera to change a capture range of a predetermined image under the control of the computer. , A follow-up data extraction function for extracting follow-up data, which is data relating to the worker projected in the image captured at a predetermined time interval, and the position of the worker on the camera screen based on the extracted follow-up data. A camera control function that drives and controls the camera drive unit by following the worker so that they are located at a predetermined location, and capturing images And a measurement data storage function for storing a drive control amount of time the camera adopts a configuration that (claim 1).

With this structure, first of all,
Images of the work site are captured from the camera based on instructions from the computer. Then, this image is processed by a computer, and the worker imaged in the image is extracted. Then, to follow the extracted workers,
The camera is driven and controlled, for example, by a computer so that the orientation of the lens of the camera is changed. Then, the camera control amount is regarded as the movement amount of the operator, and the movement amount is stored in the storage unit of the computer together with the time when the image is captured. Then, after a predetermined time has elapsed, the image is captured again, and the above processing is repeated.

Therefore, since the movement status of the worker during the work can be grasped in chronological order from the data stored together with the image capturing time, the data such as the location and environment of the future work site can be referred to by referring to the data. Can be improved. Then, at the time of such a measurement, the movement of the worker is not measured by another person, but is measured through a device such as a camera, so that a psychological burden caused by being seen by another person is reduced, and a normal operation is performed. Data can be acquired.

Further, the computer has a basic data extraction function of previously capturing an image showing the worker and extracting basic data which is data relating to the worker from the image, and the camera control function is extracted by the basic data extraction function. It is desirable to control the camera by following the operator based on the basic data thus obtained and the follow-up data extracted by the follow-up data extraction function (claim 2).

As a result, before the measurement is started, the data regarding the worker (for example, the data regarding the body) is extracted by the computer, and the data is temporarily held.
Then, by comparing the basic data and the data at the time of the follow-up work, the current position of the worker is grasped and the follow-up is performed. Therefore, since the current operation status is grasped by comparing with the original data, the measurement can be performed easily and quickly.

Further, it is preferable that the follow-up data extracting function extracts a predetermined characteristic point of the worker (claim 3). It is desirable that the basic data extraction function and the follow-up data extraction function extract the same predetermined parameter of the operator (claim 4). In particular, the basic data extraction function and the follow-up data extraction function calculate the center of gravity of the body of the worker, and the camera control function controls the camera to follow the worker based on the calculated center of gravity. It is desirable to set (Claim 5). Furthermore, the basic data extraction function and the follow-up data extraction function calculate information about a predetermined size of the body and head of the worker, and the camera control unit follows the worker based on each calculated information. You may make it control a camera like this (Claim 6).

By doing so, the amount of information to be handled is reduced, and the measurement process can be speeded up and the measurement can be facilitated. In particular, by using the center of gravity of the body part and the length of the side of such a shape that approximates the body part as a follow-up data, in the former case it is more reliable by following the center of the worker. In this case, the movement in the depth (zoom) direction can also be measured.

Further, it is preferable that the driving unit provided in the camera drives the camera so as to move the range of the image captured by the camera in the pan direction, the tilt direction and the zoom direction (claim 7). Thereby, the work motion of the worker in the three-dimensional space can be measured.

Further, a predetermined work machine whose operation is controlled by a computer is provided at the work site, and the computer controls the operation of the work machine at the work site based on the measurement data. Good (Claim 8). As a result, it is possible to control the operation status of the work site during the work, and for example, when the computer recognizes that the work at a predetermined location is delayed due to the movement of the worker in the line, such a location can be controlled. The line can be controlled to operate slowly, the measured data can be used quickly, and the work efficiency at the work site can be improved quickly.

Further, according to the present invention, a camera having a driving unit for driving itself so as to capture an image of a predetermined work site at a predetermined time interval and change a capturing range of the predetermined image, and an image of the camera. A method for measuring the movement of a worker at a work site, which comprises a computer for controlling a capturing operation and for processing the captured image, wherein the computer captures an image of the worker in advance, and data regarding the worker from the image. A basic data extracting step of extracting basic data, a follow-up data extracting step of extracting follow-up data which is data relating to an operator projected on an image captured by a computer at a predetermined time interval, and the extracted follow-up data Follow the worker so that the position of the worker is located at a predetermined position on the screen of the camera. A camera control process for driving and controlling the camera, and a measurement data storage process for storing an image capture time and a camera drive control amount are provided, and the tracking data extraction process and the camera control process are performed at predetermined time intervals. It also provides a method of activating each time when is taken (Claim 9).

Furthermore, a camera having a driving unit for driving itself so as to capture an image of a predetermined work site at a predetermined time interval and changing a capturing range of the predetermined image, and an image capturing operation of this camera. A program that includes a computer that controls and processes the captured image, and is a program that controls the operation of the computer so as to measure the operation of the operator at the work site. Data extraction processing for extracting basic data that is data relating to the operator, tracking data extraction processing for extracting tracking data that is data related to the worker projected on the image captured at a predetermined time interval, and the extracted tracking data Follows the worker so that the position of the worker is located at a predetermined position on the screen of the camera based on A camera control process for driving and controlling the camera Te, and operation performance measuring program for controlling the operation of the computer to perform the measurement-data storing process of storing a drive control amount of capture time and camera images, and
A storage medium storing the program is also provided (claims 10 and 11). Even in this case, the same operation as described above can be performed and the above object can be achieved.

[0017]

BEST MODE FOR CARRYING OUT THE INVENTION <First Embodiment> A first embodiment of the present invention will be described below with reference to FIGS. FIG. 1 is a block diagram showing the configuration of the first exemplary embodiment of the present invention. FIG. 2 to FIG. 6 are explanatory diagrams showing a worker imaged on the camera and showing a state when data regarding the worker is extracted. 7 to 8
FIG. 6 is an explanatory diagram illustrating a calibration process when driving and controlling a camera. 9 to 13 are flowcharts showing the operation in the first embodiment.

(Overall Structure) As shown in FIG. 1, a work motion measuring system according to the present invention controls a camera 1 for capturing an image of a work site at predetermined time intervals, and an image capturing operation of this camera 1. And a computer 2 for processing the captured image. Then, the system is provided in the work site, and the operation of the worker 3 who works at a predetermined work site is measured by such a configuration. That is, it is measured how the worker 3 moves within the work site during the work and what posture the worker 3 takes. Hereinafter, this will be described in detail.

(Camera) The camera 1 is a CDD camera installed in a work site. For example, it can capture a color image and has a predetermined number of effective pixels. The camera 1 is provided with a drive unit 11 that changes the image capturing range. This drive unit 1
1 is the lens pan direction (left and right), tilt direction (up and down)
It drives in the direction, and further in the zoom direction.
Then, the pan / tilt / zoom operation of the drive unit 11 is drive-controlled by the computer 2. Thereby, as described later, the position of the worker on the two-dimensional plane can be tracked by the pan-tilt operation, and the position of the worker 3 in the front-back direction (depth) direction can be tracked by the zoom operation. You can Then, as a result, the position of the worker in the three-dimensional space can be tracked.

The camera 1 is connected to the computer 2 via a predetermined interface, captures an image according to a command from the computer 2, and transmits the captured image to the computer 2.

Here, the work site is a general factory or the like. For example, a line is formed in such a factory,
A plurality of workers 3 are arranged on this line. Then, these workers 3 perform various operations on the target object which can be a product flowing on the line.

The employee 3 is a person who is working at the work site. The worker 3 wears clothing such as predetermined work clothes (not shown). This garment is, for example, a blue garment that is a waist-length vest. The reason why the waist length is set to the waist and the sleeveless vest is used is to recognize the body portion B of the worker 3 from the image, as described later. And the blue one is
This is because the body portion B is characterized by color. Also,
In addition to the above-mentioned work clothes, the worker 3 may wear clothes different from blue, for example, red clothes on both arms. That is, the shoulders to the wrists are covered with the red clothing. Here, the shape and color of the work clothes worn by the worker 3 are not limited to those described above. For example, the work clothes may be long-sleeved clothing, and the body portion and the sleeve portions may be separated by different colors.

Further, the worker 3 wears a helmet (not shown) on the head H. This helmet is a safety helmet and is colored yellow. However,
The helmet is not limited to yellow, and may have other colors, but it is desirable that the helmet has a color different from that of the work clothes described above and that does not exist in the factory. This is also because, similarly to the above, the head H as well as the body portion of the worker 3 is later recognized, and therefore it is necessary to characterize the head H with a predetermined color. Here, the worker 3 may wear a hat colored in a predetermined color instead of the helmet.

(Computer) The computer 2 controls the operation of the camera 1 and processes the captured image as described above. Therefore, a general C which is a control unit
A computer such as a workstation provided with the PU 21, a server computer, or a personal computer is sufficient. However, since the image processing is performed, it is preferable that the computer has a CPU 21 having a relatively high calculation processing capability. Then, the computer 2 includes a frame buffer 22 (frame memory) which is a memory space for displaying and holding the captured image data, a RAM 23 which is a memory for temporarily storing a program incorporated in the CPU 21, and the program described above. A hard disk 24 or the like, which is a storage unit for storing the above, is provided. Each of these devices is connected to the CPU 21, and its operation is controlled by a command from the CPU 21. Since each of these devices is publicly known, description thereof will be omitted.

The functions of the CPU 21 will be described below. The CPU 21 of the computer 2 has a basic data extraction function of preliminarily capturing an image of the worker 3 and extracting basic data that is data relating to the worker 3 from the image, and displaying the captured image at a predetermined time interval. The follow-up data extraction function for extracting the follow-up data which is the data related to the worker 3, and the worker 3 so that the position of the worker is located at a predetermined position on the screen of the camera 1 based on the extracted follow-up data. A camera control function for following the drive control of the drive unit 11 of the camera 1 and a measurement data storage function for storing the image capture time and the camera drive control amount are provided.

The basic data extraction function will be described with reference to FIG. FIG. 2 is a diagram showing an image captured by the camera 1, and is an explanatory diagram showing how the worker extracts basic data. This basic data extraction function operates before the start of work. Specifically, for example, an operator makes a predetermined input to the computer 2 before starting the work, and operates after a lapse of a certain time after receiving the input. This is because the operator must stand in a predetermined position after inputting, as described later.

Then, the basic data extraction function firstly takes an image of a predetermined place in the work site to the camera 1.
Take in through. At this time, the worker wearing the predetermined work clothes or helmet as described above stands upright at the predetermined standing position with the front facing the camera 1. This allows the computer 2 to obtain an image of the worker 3 standing upright in the work site (see FIG. 2).

With respect to this image, the basic data extraction function has a size (height) H from the head to the shoulder (head H) of the worker 3.
h, size (height) Hb from shoulder to waist (body B), size Wh of width of head H, size W of shoulder width (width of body B)
b, the two-dimensional center-of-gravity position Gh (triangle mark) of the head and the two-dimensional center-of-gravity position Gb (square mark) of the body part are extracted.

Specifically, first, the captured image data is labeled for each color area. That is, each color region is separated into blocks. Then, the body portion B and the head portion H are extracted from each of the separated regions.
Since the worker 3 wears the blue vest as described above, the body portion B recognizes the area of such color,
The body part B can be extracted. Also, the head H
Searches for the color of the helmet (not shown) worn by the worker 3 and extracts the color region as the head H. However, since the helmet generally covers only the upper half of the head H, the lower half of the head H, that is, the face portion, has, for example, the upper end of the body B and the lower end of the head H. Assuming that a semi-elliptical (semi-circular) contour is formed by connecting both ends of a circular arc formed by the contour of the helmet (both ends of the chord) and the lower end with a curved line. Then, the upper half that is the helmet portion and the formed lower half are connected to each other to connect the head H
And that this is extracted. Here, the method of forming the contour of the lower half of the head H is not limited to this. For example, the upper half of the semicircular shape, which is the helmet portion, may be folded back toward the string and assumed to be the lower half, and may be the head H. Alternatively, extract the skin color part of the face part,
The contour of the head H may be formed by using such a portion as the lower half.

In this way, the body portion B and the head portion H are extracted, and the contours of the portions B and H are corrected. The body portion B has a substantially rectangular shape, but the curved portion is further approximated to a straight line.
That is, it is modeled as a rectangle having four sides. Also,
The head portion H is formed by connecting the chord portions having a semicircular shape or a semielliptical shape, and thus has a shape close to a circular shape or an elliptical shape. Therefore, by smoothing the curved portion of the curved portion, the curved portion can be approximated to a complete circular shape or elliptical shape. As the approximation method, a known calculation method is used and will not be described here.

Then, the above-mentioned respective sizes of the body portion B and the head portion H whose contours are corrected are calculated. First, for the heights Hb and Hh of the respective parts B and H, the lengths of the respective parts B and H in the vertical direction with respect to the captured image are calculated. And each part B,
For the widths Wb and Wh of H, the length in the horizontal direction is calculated similarly. However, when the posture of the worker is tilted, the height Hb ′ of each of the parts B and H in the vertical and horizontal directions with respect to the screen is retained while the approximated shape remains oblique to the screen. , Hh 'are measured. After that, the normal actual heights Hb and Hh are calculated by correcting such values (see FIG. 4).

Further, the two-dimensional center of gravity Gb, Gh of each part B, H
Is the center of gravity of a rectangle or oval (circle)
Calculate the center of gravity of. For example, in the body portion B, the intersection of the diagonal lines of the rectangle is the center of gravity, and when the head H is circular, the center is the center of gravity. A known method is used as the method of calculating the center of gravity.

Each of the obtained parameters is stored in the storage unit (hard disk) 24 of the computer 2 or the RAM 23.
Memorized in. The flow of processing by the basic data extraction function will be described later with reference to FIG. Then, after that, the timer is set, and the image of the work site is captured by the computer 2 via the camera 1 every time a predetermined time has elapsed. Follow-up data extraction is performed on this captured image,
Camera control is executed. Then, the image of the work site is captured again, and the same processing is repeated.

Next, the follow-up data extraction function will be described with reference to FIGS. 3 to 6 are diagrams showing images captured by the camera, and are explanatory diagrams showing how the operator's follow-up data is extracted. The tracking data extraction function first operates after the work is started and each time an image is captured at a predetermined time interval. And
After this function is activated, the camera control function to be described later is activated to drive and control the direction and focus of the camera 1.
That is, the characteristic points of the worker 3 are extracted at regular time intervals, and the camera 1 is drive-controlled based on the extracted characteristic points.

The following data extraction function is used by the worker 3 who is working.
Of the body B and the head H of each of the above, and further, the sizes Hb, Hh, Wb, Wh and the center of gravity G of each of the parts B and H described above.
Calculate b and Gh. The extraction method is basically the same as the extraction method using the basic data extraction function described above. However, when the body part B of the worker 3 and the like are extracted by the follow-up data extraction function, since the worker 3 is working, various postures can be considered. For example, when the arm overlaps the body B, the body B does not have a rectangular shape because of the overlapping portion, and therefore, such a portion is ignored and the shape is approximated to a rectangle. For example, as shown in FIG. 3A, when the worker 3 faces sideways, the color region of the body B is separated into two by the arm. At this time, it is assumed that the computer 2 has in advance information that the number of the body parts B is not two, and that the body parts B have other parts of the body (arms, legs, etc.) on the front side. Then, these are disregarded and it approximates to a rectangle. Therefore, FIG.
In the case of (a), since it is separated in the left-right direction, it is assumed that the two sides in the height direction do not change, and
Connect both ends of the side to make a rectangle.

At this time, as shown in FIG. 5A, there is another color area on the front surface of the body B, and that color area is the head H.
If so, information indicating that is stored in the storage unit 24 or the like. Such information is used in the camera control function described later. Then, as described above, the body portion B is approximated to a rectangle by assuming that there is nothing on the front surface as described above.

Here, whether or not another color area overlaps the front surface of the body B is determined as follows, for example. As one example, when the color region of the body B is divided into two or more regions, it is determined that the regions between them are overlapped. Further, as another example, when more than half of the outer edge of the predetermined area is surrounded by the color area of the body B, it is determined that the predetermined areas overlap. However,
The method for determining whether or not another region overlaps the front surface of the body B is not limited to the above.

When the color region of the body B or the head H is not extracted, that is, when the body B is hidden by the feet as shown in FIG.
When the head H is hidden by the body part B as shown in (b), these parts B and H are not present.

After that, the height H of each of the extracted parts B and H
b, Hh, widths Wb, Wh, and centers of gravity Gb, Gh are calculated. At this time, Gb and G which are the centers of gravity of the body B and the head H
When the straight line connecting h is not parallel to the vertical direction of the screen, the angle ψ formed by the GbGh straight line and the vertical direction of the screen is calculated as shown in FIG. Also, the above H at this time
b and Hh are values obtained by multiplying the heights Hb ′ and Hh ′ of the body B and the head H in the screen vertical direction by 1 / cos Ψ, respectively (see FIG. 4).

The camera control function operates with reference to the above-extracted and calculated values. The camera control function includes basic data of the worker 3 extracted by the basic data extraction function and temporarily stored, and tracking data (Hb, Hb) extracted by the tracking data extraction function.
h, Wb, Wh, Gb, Gh) to follow the worker 3 to drive and control the camera 1.

The camera control function is basically performed by the operator 3
The camera 1 is drive-controlled so that the center of gravity Gb of the body B of the camera is located at the center of the screen. That is, even if the worker 3 moves, when the center of gravity Gb of the body B is extracted, the camera 1 is driven so that the center of gravity Gb becomes the center of the captured image when capturing the next image. For example, in FIG.
As shown in (a), when the operator 3 moves to the left side from the initial position of the worker 3 shown by the dotted line to the position of the worker 3 in the lateral direction, the center of gravity of the body B shown by the square mark in the figure. G
The computer 1 drives and controls the camera 1 so that b is located at the center of the camera 1 (the intersection of the horizontal and vertical dotted lines). Specifically, a control signal is transmitted from the computer 2 to the driving unit 11 of the camera 1, and the driving unit 11 performs pan-tilt driving.

However, as described above, the center of gravity Gb of the body portion B is
The case where only the pan / tilt drive is performed based on the above means that the straight line GbGh is perpendicular to the screen, the ratio between Hb and Hh does not change from the basic data, and Hb and Hh or Wb and Wh are There are no changes to the basic data and when all the data are satisfied. Therefore, the camera control function first checks whether or not the straight line GbGh is perpendicular to the screen. At this time, when it is not vertical as shown in FIG. 4, the inclination ψ of the screen with respect to the vertical direction is calculated. Thereby, it can be recognized that the posture of the worker 3 is tilted. And
Information about the angle is stored in the hard disk 24 or the like as described later.

Further, in the case of being vertical, or after the inclination is calculated, whether or not the ratio of Hb and Hh subsequently changes with respect to that of the basic data measured and stored in advance. Can be examined. When there is no change, the position of the head H does not move back and forth with respect to the body part B,
That is, it can be said that the camera 1 is not approaching or moving away from the camera 1. Then, in such a case, it is subsequently checked whether or not the lengths of Hb and Hh have changed with respect to those of the basic data. As a result, the operator 3 itself can recognize whether the body B and the head H are generally approaching or moving away from the camera 1. And H
The camera 1 zooms so that b and Hh have the same length as those of the basic data (see FIG. 3B). Then, the zoom amount at this time is converted into the distance between the camera 1 and the worker 3, and is temporarily held.

When the ratio of Hb to Hh changes with respect to the basic data, it is checked whether or not the head H is in front of the body B. When the head H is in front of the body B (see FIGS. 5A and 5B), the parameters of Hh and Wh of the head H should be the same as the basic data, that is, Zooming is performed based on the head H. On the other hand, when the body B is in the front (see FIG.
(See (a) and (b)), the zoom is performed so that the parameters of Hb and Wb of the body B are the same as the basic data.
Further, as described above, in such a case, the worker 3 often tilts the posture with respect to the camera 1 largely, so that the heights Hb and Hh of the body B and the head H are compared with each other. By comparing the widths Wb and Wh of B and H, the distance to the worker 3 can be grasped more reliably. Therefore, when performing zooming as described above, Wb, W
Perform using h. Then, after that, the pan-tilt drive control is performed based on Gb.

Here, as described above, the camera 1 follows the extracted center of gravity of the operator, but at this time, the actual coordinate system and the camera coordinate system must be calibrated. The calibration process, which is the calibration process, will be described with reference to FIGS. 7 to 8. Figure 7
It is explanatory drawing explaining the procedure of a calibration process. FIG. 8 is an explanatory diagram showing an example of driving the camera.
8A shows the relationship between the virtual plane and the camera 1, and FIG.
(B) shows the position of the target point on the frame buffer. Note that, here, the calibration is based on the VVV system. The VVV system was developed by the Institute of Electrical Engineering 3
Since the three-dimensional object recognition system is well known, the description thereof will be omitted.

First, the point P _W in the world coordinate system is converted into the point P _{C in the} camera coordinate system. The conversion at this time is obtained from the origin O _C (focal point) of the camera coordinate system and the rotation conversion matrix R as follows.

[0047]

[Equation 1]

From the pinhole camera model, the camera coordinates
System point P_C= [X_C, Y_C, Z_C] ^TVirtual image against
The conversion to the point Pi on the plane I is performed with the focal length f and the extended vector.
Le [X, Y, 1]^T(Symbols are marked with ~ at the top
Can be expressed as the following equation.

[0049]

[Equation 2]

A conversion matrix in which the position of the virtual image center c on the frame buffer is [a ₁₃ , a ₂₃ ] can be defined and expressed as the following equation.

[0051]

[Equation 3]

Further, the conversion from the point P _i on the virtual image plane to the frame buffer P _f = [u, v] ^T can be performed by rearranging the equations (2) and (3). The conversion formula for converting to the point can be expressed as the following formula.

[0053]

[Equation 4]

Further, from the equation (1), a conversion equation for converting a point on the world coordinate system into a point on the frame buffer can be expressed as the following equation.

[0055]

[Equation 5]

Here, the internal parameter is a parameter indicating the performance or function of the camera and the relationship with the frame buffer, and a _ij = element a ₁₃ indicating the camera internal parameter is the center of the virtual image plane converted on the frame buffer. The X coordinate a _{23 of} is the Y coordinate f = focal length of the virtual image plane center converted on the frame buffer.

Further, the external parameter is a parameter representing the relation between the camera coordinate system and the world coordinate system, and r _ij = each component of the rotation conversion matrix from the world coordinate system to the camera coordinate system (camera attitude) t _ij = Xyz in the world coordinate system of the origin position of the camera coordinate system
Is the coordinate value of.

From the relationship of these equations, the object projected at the point P _f [u, v] on the frame buffer forms the point P _i [x _i , y _i ] on the virtual image plane with the lens focus of the camera. It can be seen that they exist on a straight line.

Where P_i[X_i, Y_i] In equation (3)
P by inverse transformation_f[U, v] to P _i= A^-1P_fAnd
It Also, [a_ij, F_ij, R_ij, T_ij]
Each camera parameter to be displayed is the box camera of the VVV system.
It can be obtained by recalibration. Where BO
X calibration is a pinhole camera model.
Take over the form and capture a known cube into the image
Perform calibration to associate the standard system with the camera coordinate system.
U

In addition, the power obtained from the calibration
Among the mel parameters, a in the internal parameters_Thirteen, A
₂₃That is, the virtual image converted on the frame buffer
The position of the center of the plane is due to camera error (such as image data noise
It is very unstable due to the influence)
Stable every time even when calibrated in standing position
It is difficult to get the camera parameters. Therefore, inside
Parameter a_13,a ₂₃Frame buffer center
C₁, C_TwoReplace with the value of. At this time, as shown in equation (5)
The relation is newly converted into the following equation.

[0061]

[Equation 6]

At this time, since the conservation rule of the camera parameters is established, the identity of equation (6) can be solved to derive r _ij 'and t _j ' in the equation. Further, r _ij ′ and t _j ′ represent the following elements. r _ij 'is each component (camera attitude) of the rotation transformation matrix from the world coordinate system to the camera coordinate system, and t _j ' is xyz in the world coordinate system of the origin position of the camera coordinate system.
By handling the coordinate values r _ij ′ and t _j ′ of the sigma as the true values of the external parameters of the camera, it is possible to obtain more stable and stable control.

Here, when the zoom is changed, the conservation rule of the camera parameter is not established, and therefore the above equation (6) is used.
The external parameter after conversion of is represented by the following formula by a rotational coordinate conversion matrix R (θ, φ, 0) that rotates the external parameter before conversion about θ around the x-axis and φ around the y-axis.

[0064]

[Equation 7]

At this time, the values of the rotation angles θ and φ can be obtained by performing the same processing as that described later for θ and φ (see equation 9). Since this is a conversion process for correcting the data coordinate system at the calibration position, the camera 1 keeps the calibration position stationary without moving. Specifically, when [a ₁₃ , a ₂₃ ] obtained from the calibration is converted into the virtual image plane, this point is converted into the virtual image plane center P _io .
Similarly, when [c ₁ , c ₂ ] is transformed on the virtual image plane and treated as the tracking target point, and expressed as P _{i (f)} ,
The above θ and φ are as follows.

[0066]

[Equation 8]

By using the rotational transformation coordinates R using θ and φ and performing the replacement with the above equation (6), the camera parameters obtained from the calibration can be stabilized. When [c ₁ , c ₂ ] is set to the original frame buffer center, P _io and P _{i (fo)}
And can be matched, and the calculation processing speed can be improved.

Here, the actual processing executed using the above method will be described with reference to FIG. The camera 1 has a center of gravity G of the body B, which is a feature point that follows the lens focus.
Rotational movement is performed in the direction of moving b to the center point of the frame buffer. At this time, it can be considered that the camera 1 does not move from the calibration position and only the camera posture changes. At this time, the rotation angle of the camera 1 is set to the pan direction θ,
It is indicated by the tilt direction φ. _Let v be a vector that moves an arbitrary tracking target point P _f set on the frame buffer F to the frame buffer center P _fo . v is decomposed into a pan direction component and a tilt direction component. The rotation angle (θ, φ) required for the tracking operation of the camera 1 can be obtained from the component vectors of the focal length f and the decomposed v in each direction.

First, the procedure for following the camera 1 will be described. First, each camera parameter of [a _ij , f _ij , r _ij , t _ij ] is obtained by the calibration method of the VVV system. Then, the target point P _f to follow on the frame buffer is set. Subsequently, the frame buffer center P converted from the frame buffer onto the virtual image plane by the inverse transformation (P _i = A ⁻¹ P _f ) of the equation (3).
_{i (fo)} , the conversion position P _{i (f)} of the tracking target point Pw, and the vector v are obtained. Then, the vector v is decomposed into the pan direction and the tilt direction, and the rotation angles θ and φ corresponding to the respective component vectors V _t and V _p are obtained.

[0070]

[Equation 9]

Then, by moving the camera by θ and φ, an operation in which a point designated on the frame buffer is projected at a position at the center of the frame buffer, that is, a follow-up operation is realized. The camera coordinate system after the tracking operation is newly updated by the position and orientation of the camera after the movement. At this time, the above equation (6) is converted as follows. By continuing to use this newly converted relational expression, it becomes possible to continuously follow up.

[0072]

[Equation 10]

Next, a method of changing the camera parameters when changing the zoom in the camera 1 will be described. When the camera 1 changes the zoom, of the camera parameters, f indicating the focal length is updated. Also, at this time, FOE
Due to the influence of (Forcus Of Expansion), the conversion measurement between the virtual image and the frame buffer changes. This correction will be dealt with by the following method.

First, camera parameters are obtained from the calibration. Then, the focal length f is updated by zooming and changes to f ′. Therefore, the above equation (6) is changed as follows.

[0075]

[Equation 11]

However, in the above case, the frame burst
Cuffa center C₁, C_TwoIs the change of f due to the influence of FOE
According to C₁’、 C_Two’ Where C₁’,
C_Two′ Indicates a specific value / change with respect to the focal length f,
C in advance for the focal length f '₁’、 C_Two’Day
By creating a database for calibration
Based on the position / orientation of the camera and the focal length f,
C corresponding to changes₁’、 C _TwoIt is possible to estimate the value of
come.

When changing the zoom, the conservation rule of the camera parameters is not satisfied. Therefore, based on the values of C ₁ ′ and C ₂ ′ estimated above, P _fo =
The pan-tilt angle θ · φ that follows the movement is obtained with (C ₁ , C ₂ ), P _f = (C ₁ ′, C ₂ ′). However, at this time, the camera 1 does not actually perform the pan-tilt operation but keeps its posture.

A camera parameter corresponding to a change in the focal length f can be obtained by obtaining a rotation transformation matrix R ′ that rotates the camera coordinate system based on the obtained θ · φ and executing the processing shown in the following equation. I can.

[0079]

[Equation 12]

By doing so, the tracking target point on the frame buffer can be made to correspond to the actual position of the target point, and the tracking can be realized by the pan / tilt / zoom operations of the camera drive unit 11. it can.

Here, the CPU of the computer 2 described above
The measurement data storage function of 21 will be described. This function stores the image capture time and the drive control amount of the camera 1 in the hard disk 24 of the computer 2, so that the drive control amount of the camera 1 becomes time-series data. That is, the operation of the camera 1 can be recognized in time series. At this time, the drive control amount of the camera 1 can be regarded as the movement amount of the worker. Therefore, the movement trajectory of the worker can be recognized by looking at the stored measurement data. Then, by referring to this movement locus, it is possible to consider the subsequent arrangement of the work site, the burden on the worker, etc., improve the work situation, and improve the work efficiency.

Each function of the computer 2 can be realized by the CPU 21 of the computer 2 executing the program for each function.
This program is read from a portable medium such as a CD-ROM, a hard disk, or the like, or downloaded from another computer on the network and incorporated into the computer 2. Therefore, the program is stored and provided in a predetermined storage medium or a storage medium included in another computer on the network.

Examples of the program include, for example, a basic data extraction process of previously capturing an image of a worker and extracting basic data that is data relating to the worker from the image, and a process of displaying the captured image on the captured image at a predetermined time interval. Tracking data extraction processing for extracting tracking data that is data related to the worker, and tracking the worker so that the position of the worker is located at a predetermined position on the screen of the camera based on the extracted tracking data. It is a work movement measurement program for controlling the operation of a computer so as to execute a camera control processing for driving and controlling a camera and a measurement data storage processing for storing an image capture time and a camera drive control amount.

(Operation) Next, the operation of this embodiment will be described with reference to FIGS. First, Fig. 9
The overall operation will be described with reference to.

(Overall Operation) First, the worker puts on a predetermined work clothes and a helmet, and inputs the work start to the computer 2. After that, the computer 2 receiving the input of the work start captures an image of a predetermined portion of the work site after a predetermined time elapses, for example, 10 seconds (step S
1). At this time, the worker must stand at a predetermined position in front of the camera 1. That is, after inputting the start of work, the worker must immediately stand upright at a predetermined image capturing position. As a result, the image of the worker himself is captured by the computer 2 via the camera 1.

Subsequently, basic data extraction processing is performed by the computer 2 on the image showing the upright worker (step S2, basic data extraction step). By this basic data extraction processing, the center of gravity Gb, Gh, the width Wb, Wh, and the height H of the body B and the head H of the worker are obtained.
Each size such as b and Hh is calculated. The details of the processing procedure will be described later. Then, the calculated basic data is stored in the storage unit 24 of the computer 2.

Then, the timer is set, and a standby state is set until a predetermined sampling time elapses (step S3). Here, the sampling time is, for example, 2
Seconds. And when the sampling time elapses,
An image of the work site is captured (step S4). Then, the operator in the captured image is followed by the camera,
After that, when the sampling time elapses, the image of the work site after the tracking is captured, and the same processing is performed. That is, the following processing is performed on the image captured every time the sampling time elapses.

Since the worker who is working is shown in the captured image, first, the head H and the body B of the worker are displayed.
Is extracted. At this time, the sizes Hh, Hb, Wh, Wb of the head H and the body B and their centers of gravity Gh, Gb
And are calculated (step S5, follow-up data extraction step). The procedure of each parameter extraction process will be described later.

Subsequently, the calculated parameters are compared with the basic data stored in the storage section 24 of the computer 2 (step S6), and the pan, tilt and zoom operations of the camera 1 are controlled based on the comparison. Is performed (step S7, camera control step). At this time, in the pan-tilt-zoom operation, the center of gravity G of the body of the worker is set at the center of the camera 1.
It is controlled so that b comes. As a result, the worker is always followed by the camera 1.

Then, the movement amount of the camera 1 is set as the movement amount of the operator, and this movement amount is stored in the storage unit 24 of the computer 2 together with the image capturing time (step S8, measurement data storing step). After that, until the work of the worker is completed (step S9), for example, until a predetermined time (specifically, a time such as one hour) has elapsed from the start of the work, or Each time the sampling time elapses, an image of the work site is captured until the computer 2 inputs the end of work to the computer 2, and the camera 1 follows the worker and measures the worker's motion as described above. It

By doing so, since the position of the worker who is working is stored together with the time, it is possible for the worker himself or the person in charge of the site to confirm the working state of the worker after the fact. It is possible to improve the environment of the work site by referring to such data. Specifically, when the amount of movement of the worker on the line is large, it can be dealt with by moving the position of the one causing the movement.

Further, as data representing the motion of the worker,
Since the amount of movement of the camera 1 is only stored at predetermined time intervals, the amount of data can be reduced. Therefore, even if the storage capacity of the computer 2 used is relatively small, the movement amount of the worker can be stored for a long time, and the motion of the worker that appears only after measuring for a long time can be grasped. You can

Further, the movement data of the worker is acquired by the image processing of the computer 2 using the camera 1 rather than grasping the motion of the worker by observing the worker by another person. During the work, the worker can obtain the worker's data without being seen by another person (for example, the site manager). Therefore, the psychological burden of being seen by another person during the work is reduced, and the data during the normal work can be acquired. That is, the peculiarity of the acquired data can be suppressed.

(Operation of Basic Data Extraction Process) Next, a specific processing procedure of the basic data extraction process in step S2 of FIG. 9 will be described with reference to the flowchart of FIG.

First, as described above, the image of the worker is captured when the work is started (step S11). continue,
An image of the same work site as this image, which is captured by the camera 1 in advance and is stored in the storage unit 24 of the computer 2.
Difference processing is performed with the image stored in (step S12). After that, an area having a predetermined threshold value or more is searched for in such an image (step S13), and the area having a predetermined threshold value or more can be estimated to be a place where the background of the work site has changed. Since it is predicted that the worker is standing in this area, the original image data corresponding to the area is left (step S14). On the other hand, since the area that does not reach the predetermined threshold value is the image data that is the same as the work site, the data of the original image of the area is discarded (step S15). That is, the background part of the worker is removed. This is executed for all pixels of the image (step S16). By such processing, the original image, that is, the image of the work site in which the worker is shown, is extracted and remains only for the worker.

As a result, the conditions for the subsequent image processing can be narrowed down, and the processing time can be shortened. However, even if such processing is not performed, the basic data can be extracted by performing the processing described below. Here, since the standing position of the worker at the start of work is predetermined, the image of the work site at the position is captured by the camera 1 in advance and stored in the computer 2.

Subsequently, a labeling process is performed on the original image for each color area (step S17). That is, the computer 2 recognizes a lump for each color area. Then, first, for each color region, a color region of the body B, that is, a color region corresponding to the color of the work clothes is searched (step S18). When the color region corresponding to the body B is searched, the contour of the body B is formed (step S19), and the height Hb, width Wb, and center of gravity Gb thereof are calculated (step S20).

After the respective parameters of the body B are calculated, the color region of the head H, that is, the color region corresponding to the color of the helmet or the like is similarly searched (step S2).
1). At this time, the information about the non-colored area such as the helmet is discarded (step S22). This is because the necessary data regarding the body B has already been calculated. When a color region such as a helmet is searched, the upper end of the body B is assumed to be the lower end of the head H (step S23), and the contour of the head H is formed (step S24). , Its height Hh, width Wh, and the center of gravity Gh are calculated (step S25).

Then, the pan / tilt control of the camera 1 is performed based on the calculated center of gravity Gb of the body B (step S).
26). Thus, the initial position of the center of capture of the camera 1 is set to the center of gravity Gb of the body B of the worker. Further, here, the size of the body portion B and the like, that is, the height Hb and the width Wb
Based on the above, the zoom operation of the camera 1 is not performed. This is because such basic data will be the reference after that. Then, the extracted basic data is stored in the storage unit 24 of the computer 2, the RAM 23, or the like (step S2).
7). Then, the timer is activated (step S28), and an image of the work site is captured every time a predetermined sampling time elapses.

(Operation of Follow-up Processing) Next, the operation when the camera 1 follows the operation of the operator after the acquisition of the basic data, that is, the operation from step 3 to step S8 in FIG. This will be described in detail with reference to the flowchart of FIG.

As described above, after the basic data is acquired,
For example, the timer inside the computer 2 is controlled, and while the worker is working, the worker continues to capture the camera 1 every time the sampling time elapses (step S31) (step S32). Then, each parameter of the body part B and the head part H of the worker is extracted for each image.

Each parameter extraction process is almost the same as the basic data extraction process procedure described above. However, since the image capturing range of the camera 1 during the work of the worker is determined according to the motion of the worker and the difference processing with the background is difficult, the difference processing is not performed. Therefore, the extraction process is performed from the step of labeling for each color region (step S33). Then, the extraction process of each parameter of the body part B and the head part H is performed (step S34). here,
The body part B and the head part H of the worker who is working are different in shape or overlap due to their motions, which makes the processing complicated. The details will be described later.

Subsequently, the heights Hb and H of the body B and the head H are
It is checked whether or not the size such as h matches that of the basic data (step S35). At this time, if the sizes do not match, the zoom operation is executed by the camera 1 so that the size becomes the same as the basic data (step S3).
6). Thereby, the movement of the worker in the depth direction can be recognized with respect to the camera 1. Further, when the center of gravity of the body is not in the center of the screen (step S37), the camera 1 executes the pan / tilt operation (step S3).
8) The worker is followed. Thereby, the position of the worker on the plane can be recognized. Therefore, it is possible to recognize the three-dimensional movement of the worker.

Thereafter, the camera movement amount (pan / tilt / zoom amount) is used as the movement amount of the operator, and the camera movement amount is stored in the storage unit 24 of the computer 2 together with the time when the image is captured (step S39). ). Then, the above procedure is repeated until the work of the worker is completed (step S40).

(Operation of Parameter Extraction Processing) Next, the calculation processing procedure of each parameter at the time of the following processing,
That is, the processing procedure in step S34 of FIG. 11 will be described with reference to the flowchart of FIG.

First, it is searched whether there is a color area of the body B (step S41). At this time, if the color region of the body B is not searched, it is considered that the body B is hidden by a part of the body of the operator, and that effect is stored in the storage unit of the computer 2. It is stored (not shown). On the other hand, when the color area of the body B is searched,
The color area is selected (step S42), and the contour of the area is formed (step S43). Then, the shape of the contour is calculated (step S44), but if it is a substantially rectangular shape, the size (height Hb, width Wb) and the center of gravity Gb of the body B are calculated (step S45).

When the shape of the body B is not substantially rectangular, whether there is another color area in front of the area, that is,
It is detected whether or not they overlap (step S46).
If it is determined that they overlap, it is checked whether or not the area on the front side is the color area of the head (step S47). Is stored in the storage unit (step S48). Then, thereafter, even if the overlapping region is not the head region, even if the region is once determined not to be substantially rectangular, it is approximated to a rectangle (step S49).

As described above, when it is determined that the color area of the body B is not present, or after the contour is formed into a rectangle and the size and the center of gravity are calculated, the color area of the head H is searched. It is performed (step S50). At this time, when it is determined that there is no color area of the head H, the fact is stored as in the case of the body B, and the process ends (not shown). On the other hand, when it is determined that there is a color area of the head H, the color area is selected (step S51). Then, the contour of the region is formed (step S52), and the size and the center of gravity are calculated (step S53). By doing so, it is possible to deal with the case where the worker makes a complicated motion.

(Camera Control Operation) Next, the operation for pan, tilt and zoom control of the camera 1 based on the parameters calculated as described above will be described with reference to FIG.
This will be described with reference to the flowchart of FIG. However, in the above process, when the body B and the head H are not detected, the process ends or the control of the camera 1 is not executed and the next image is captured at that position. Or the like is performed (not shown).

First, the centers of gravity Gb and Gh of the body B and the head H are measured.
It is checked whether or not the straight line connecting the lines is perpendicular to the screen, that is, whether or not the worker moves in parallel while standing upright (step S61). At this time, when the posture of the worker is tilted, that is, when the straight line GbGh forms a predetermined angle with respect to the vertical direction of the screen, the heights Hb and Hh of the respective parts B and H are corrected (step S62). ).

Subsequently, the ratio between Hb and Hh in such an image and Hb and H in the stored basic data.
It is compared with h (step S63). When there is a change in this ratio, it is considered that the head H is not on the same plane as the body B, but is positioned in front of and behind it. Therefore, it is checked whether or not the head H is in front of the body B (step S64). At this time, it can be checked by referring to the basic data. When it is determined that the head H is in front of the body B, it is considered that the worker 3 is tilting toward the camera 1, and the head H hides the body B. Therefore, the camera 1 is zoom-driven so that the width Wh of the head H becomes the same as the basic data (step S66). On the other hand, when it is determined that the head H is behind the body B, it is considered that the worker 3 is tilting the posture opposite to the camera 1, and in such a case, the head H
Is hidden in the body B, so that the zoom drive of the camera 1 is performed with reference to the width Wb of the body B (step S65).

Here, in the above step S63, HbH
When it is determined that the ratio of h does not change, it is checked whether the size itself (height, width) of Hb or Hh is different from the basic data (step S67). If the ratio does not change but the size changes, it is considered that the worker 3 has moved in the front-rear direction without changing the posture in the front-rear direction with respect to the camera 1. In such a case, Hb or Hh is compared with the basic data, and zoom drive is performed (step S68).

As a result, the tracking of the worker 3 in the depth direction is completed, and the tracking is finally performed on the two-dimensional plane. In this, the pan / tilt drive is performed while executing the calibration process so that the center of gravity Gb of the body portion B is located at the center of the screen (step S69).

By implementing the above operation, the camera 1 can follow the worker 2 in the work site.
At this time, since the camera 1 follows the worker 3 by pan / tilt / zoom operations and stores the control amount of the camera 1 at the time of tracking, the position of the worker 3 in the three-dimensional space can be recognized. it can. The work environment can be improved by referring to the measurement data.

Here, in the above embodiment, the case where the body B and the head H of the worker 3 are extracted has been illustrated, but the invention is not limited to this. Further, the point of interest when tracking is not limited to the center of gravity of the body B or the head H. Other parts, for example, joints such as the waist, knees, and elbows may be used as the characteristic points, and the tracking processing of the characteristic points by the optical flow that is a known technique may be executed.

<Second Embodiment> The second embodiment of the present invention will be described below. The second embodiment is the same as the first embodiment described above.
In the system of the embodiment of
A predetermined work machine whose operation is controlled by the computer 2 is provided (not shown). And computer 2
Has a function of controlling the operation of the work machine in the work site based on the measurement data.

For example, the line itself of the belt conveyor is a working machine whose operation is controlled by the computer 2,
The operation of the driving means of the belt conveyor is drive-controlled by the computer 2. Therefore, the moving speed of the product or the like flowing on the belt conveyor is changed to a low speed by the computer 2 even during the work.

The timing of shifting is, for example, when the moving amount of the worker 3 increases along the belt conveyor and the moving speed thereof also increases. In such a case, it is considered that the worker 3 could not perform a predetermined work on the work target flowing on the belt conveyor and chased the target. Then, at this time, the flow speed of the belt conveyor becomes low, so that the worker 3 can easily catch up with the delay in the work.

Therefore, the computer 2 constantly grasps the movement of the worker 3 in a time series, and when the movement of the worker 3 is abnormal as described above, the movement of the line is stopped. Measures can be taken, safety at the work site can be improved, and work efficiency can be improved.

[0120]

Since the present invention is constructed and functions as described above, according to this, the worker can be seen by another person by measuring the movement of the worker during work through the camera 1. Since it is possible to perform work without being affected by the psychological effects of this, normal data can be acquired, and the environment at the work site can be improved based on such data, improving work efficiency. It has an unprecedented excellent effect that can be achieved.

Further, since the camera always follows so as to focus on the operator, it is sufficient to process the vicinity of the center of the captured image, and the processing can be simplified. Then, at this time, the movement of the camera can be stored in time series as the movement amount of the worker, and it is not necessary to perform a predetermined conversion process, so that the process can be further speeded up.

Further, by extracting the data relating to the torso and head of the worker, the follow-up data extraction process can be executed by a relatively simple image process, so that the process can be speeded up. Since it is sufficient to store only the predetermined parameter for each time series, the storage capacity of the storage device can be saved.

Furthermore, at the same time when the movement of the worker is measured, the movement of the worker is recognized, and when a predetermined movement is made, the working machine or the like at the work site is stopped or the speed is reduced by a computer command. When controlled, the safety of the work site and the work efficiency can be improved.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of an embodiment of the present invention.

FIG. 2 is a diagram showing an image captured by a camera, and an explanatory diagram showing a state of extracting basic data of an operator.

FIG. 3 is a diagram showing an image captured by a camera and is an explanatory diagram showing a manner of extracting follow-up data of an operator. FIG. 3A is a diagram showing a state when the worker moves in the lateral direction, and FIG. 3B is a diagram showing a state when the worker moves in the depth direction.

FIG. 4 is a diagram showing an image captured by a camera and is an explanatory diagram showing a state of extraction of follow-up data of an operator. In particular, it is a diagram showing a state in which the posture of the worker is inclined with respect to the vertical direction of the screen.

FIG. 5 is a diagram showing an image captured by a camera and is an explanatory diagram showing a manner of extracting follow-up data of an operator. FIG. 5A and FIG. 5B are diagrams showing a state in which the worker projects his head forward (toward the camera).

FIG. 6 is a diagram showing an image captured by a camera, and is an explanatory diagram showing a manner of extracting follow-up data of an operator. FIG. 6A and FIG. 6B are diagrams showing a state in which the worker projects the head backward (in the depth direction).

FIG. 7 is an explanatory diagram illustrating a calibration processing procedure.

FIG. 8 is an explanatory diagram showing a driving state of the camera at the time when the calibration process is associated with the camera.
FIG. 8A shows the relationship between the camera and the virtual plane.
(B) shows the position of the target point on the frame buffer.

FIG. 9 is a flowchart showing the operation of the entire system in the first exemplary embodiment of the present invention.

FIG. 10 is a flowchart showing an operation of basic data extraction processing which is a part of the operation of the system.

FIG. 11 is a flowchart showing an operation of a tracking process which is a part of the operation of the system.

FIG. 12 is a flowchart showing an operation of a parameter extraction process which is a part of the operation of the system.

FIG. 13 is a flowchart showing an operation at the time of camera control, which is a part of the operation of the system.

[Explanation of symbols]

1 camera 2 computers 3 workers 11 Drive 21 CPU (control unit) 22 frame buffer (frame memory) 23 RAM 24 Hard disk (storage unit) B torso H head Hb Body height Hh head height Wb Body width Wh head width Center of gravity of Gb body Center of gravity of Gh head

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Sei Kubota             2-1, Sakura Namiki, Tsuzuki-ku, Yokohama-shi, Kanagawa             Zuki Co., Ltd. Yokohama Institute F-term (reference) 5B057 AA19 BA02 BA11 BA17 CA01                       CA08 CA12 CA16 CD01 DA07                       DA08 DB02 DB06 DB09 DC05                       DC06 DC14 DC16                 5C054 AA01 CA04 CC02 CG06 CH03                       CH09 EA01 EA07 FC07 FC13                       FC14 FC15 FF02 FF03 GA01                       GA04 GB00 HA03 HA05

Claims (11)

[Claims] 1. A system for measuring an operation of a worker at a predetermined work site, comprising: a camera for capturing an image of the work site at a predetermined time interval; and a control for capturing the image by the camera. A computer that processes an image, the camera includes a drive unit that drives and controls the camera to change a capture range of a predetermined image under the control of the computer, and the computer has the predetermined time interval. Tracking data extraction function that extracts tracking data that is data related to the worker projected in the image captured by, and the position of the worker at a predetermined position on the screen of the camera based on the extracted tracking data. A camera control function that drives and controls the drive unit of the camera by following the worker, and when capturing the image. A work motion measuring system having a measurement data storage function for storing the time and the drive control amount of the camera. 2. The computer has a basic data extraction function of previously capturing an image of the worker and extracting basic data that is data related to the worker from the image, and the camera control function includes the basic data extraction. The work operation according to claim 1, wherein the camera is driven and controlled by following the worker based on the basic data extracted by a function and the follow-up data extracted by the follow-up data extracting function. Measuring system. 3. The work motion measuring system according to claim 1, wherein the follow-up data extracting function extracts a predetermined characteristic point of the worker. 4. The work motion measuring system according to claim 2, wherein the basic data extracting function and the follow-up data extracting function extract the same predetermined parameter of the worker. 5. The basic data extraction function and the follow-up data extraction function each calculate the center of gravity of the body of the worker, and the camera control function follows the worker based on each calculated center of gravity. The work measurement system according to claim 2, wherein the camera is drive-controlled so as to: 6. The basic data extraction function and the follow-up data extraction function calculate information regarding a predetermined size of the body and head of the worker, and the camera control unit relates to the calculated predetermined size. The work measuring system according to claim 2, wherein the camera is controlled so as to follow the worker based on the information. 7. The driving unit included in the camera drives the camera so as to move a range of an image captured by the camera in a pan direction, a tilt direction, and a zoom direction. , 3, 4, 5 or 6 work motion measuring system. 8. A predetermined work machine whose operation is controlled by the computer is provided at the work site, and the computer controls the operation of the work machine in the work site based on the measurement data. The work motion measuring system according to claim 1, 2, 3, 4, 5 or 6. 9. A camera having a drive unit for driving itself so as to capture an image of a predetermined work site at a predetermined time interval and change a capture range of the predetermined image, and to control an image capture operation of this camera. And a computer for processing the captured image, and a method for measuring the motion of the worker at the work site, wherein the computer captures an image of the worker in advance and uses the data about the worker from the image. A basic data extracting step of extracting certain basic data, a follow-up data extracting step of extracting follow-up data which is data relating to an operator projected on an image captured by the computer at the predetermined time interval, and the extracted Based on the follow-up data, the operator's position should be adjusted so that it is located at a predetermined position on the screen of the camera. A follow-up data extraction step and a camera control step, which includes a camera control step of following the person to drive-control the camera, and a measurement data storage step of storing a capture time of the image and a drive control amount of the camera. And is activated each time an image is captured at the predetermined time interval. 10. A camera having a drive unit for driving itself so as to capture an image of a predetermined work site at a predetermined time interval and to change a capture range of the predetermined image,
A program for controlling the image capturing operation of the camera and for processing the captured image, which is a program for controlling the operation of the computer so as to measure the operation of the operator at the work site. A basic data extraction process that captures an image to be captured and extracts basic data that is data related to a worker from the image, and extracts follow-up data that is data related to the worker that is captured in the image captured at the predetermined time interval. Follow-up data extraction processing, and camera control processing that follows the worker so that the position of the worker is located at a predetermined position on the screen of the camera based on the extracted follow-up data, and controls the drive of the camera, Executes a measurement data storage process that stores the image capture time and the camera drive control amount A work movement measuring program for controlling the movement of the computer. 11. A camera having a drive unit for driving itself so as to capture an image of a predetermined work site at a predetermined time interval and change a capture range of the predetermined image,
A storage medium having a computer for controlling the image capturing operation of this camera and for processing the captured image, and storing a program for controlling the operation of the computer so as to measure the operation of the worker at the work site. Basic data extraction processing that captures an image of the worker in advance and extracts basic data that is data related to the worker from the image, and data related to the worker that is captured in the image captured at the predetermined time interval. Follow-up data extraction processing for extracting follow-up data, and based on the extracted follow-up data, follows the worker to drive and control the camera so that the position of the worker is located at a predetermined position on the screen of the camera. Camera control processing, measurement data that stores the image capture time and the camera drive control amount A storage medium that stores a work operation measurement program that controls the operation of the computer so as to execute a storage process.