JP5696482B2 - Information processing apparatus and program - Google Patents

Description

  The present invention relates to an information processing apparatus that processes an image.

  In general, a worker's work at a work site is photographed by a camera, and a signal of the photographed image is transmitted from the camera to a monitoring device and displayed on the display device of the monitoring device. You can monitor the work remotely.

  In a known data center / security management system, a person face image dispatched from the outside as a data center contractor and located in front of a server / rack door in the floor is photographed and authenticated by a Web camera installed on the rack. When the image collates with the registered face image and collation coincidence with the ID card is established at the same time, the image can be unlocked at the site. Alternatively, in that system, the captured video is transmitted in real time to a remote information terminal that can be operated by a person concerned with the person dispatched, and the person concerned directly recognizes the person's face on the monitor screen and confirms the identity. Is done. When the confirmation signal and the collation match by the ID card at the site are simultaneously established, the unlocking can be performed remotely.

  According to a known video surveillance system, an intelligent video surveillance system is formed by transplanting an intelligent video analysis system to a surveillance video system. When the video surveillance system detects a dangerous action that enters beyond the boundary of the surveillance area, the video surveillance system immediately displays the alarm and the video for which the judgment has been made. Sent to the mobile phone of the person at the same time to help prevent crimes early. The video surveillance system further provides a video surveillance system in which the safety of a vehicle such as a parking lot can be confirmed when the user leaves.

  One known safety monitoring system includes a photographing machine capable of monitoring all of a fixed work area separated from the outside, an image processing apparatus connected to the photographing machine, and a jig control panel connected to the image processing apparatus. And a robot control panel connected to the jig control panel. The photographing machine photographs all work areas, and images of the work areas are sent to the image processing apparatus. When an operator performs a work set on a jig in the work set area, the system stops the work robot. During operation of the work robot, when the image processing apparatus recognizes an entering object in the work area, the system shuts off all the power sources in the work area and causes the work robot to perform an emergency stop.

JP 2009-37573 A JP 2007-195130 A JP 2010-46735 A

  For example, installation, adjustment, adjustment, replacement, repair, removal, etc. of servers and / or communication devices, etc. in facilities such as the Internet Data Center (IDC) where servers and / or network devices are installed Done. Also, for the work, a releasable marking tape (e.g., a unique or identifiable color) is often used by an operator to distinguish between an operational area for work and other non-operated areas. In some cases, the operation area and the non-operation area are partitioned with a curing tape. In order to monitor the work, a monitor may be arranged at the work site.

  The inventor has recognized that it is difficult and strictly costly to constantly monitor the work of the worker by the monitor. Further, the inventor performs marking that represents the operation region in a form that can be identified by the image recognition technology, images the operation region and the marking with a camera, and recognizes the captured image using the image recognition technology. Recognized that workers' work or operations can be efficiently monitored.

  An object of the present invention is to make it possible to determine a state of a moving body image with respect to a boundary line of a non-operation area in an image.

According to one aspect of the embodiment, a storage unit that captures and stores an image signal via the input unit, a boundary line in the image of the first image signal stored in the storage unit, and the boundary line a determination unit for determining a non-operating region that, a detector for detecting a moving object image in the second image signal stored in the storage unit, is the detected moving object image in the image of the second image signal An information processing apparatus that includes a determination unit that determines a state of the moving image relative to the boundary line based on whether the boundary line is penetrated, and a signal generation unit that generates a signal based on the determined state Is provided.

  According to one aspect of the embodiment, it is possible to determine the state of the moving body image with respect to the non-operation area boundary line in the image.

FIG. 1 shows an example of a schematic configuration for monitoring an operation in a facility or apparatus that is a work target or a work target according to the embodiment. FIG. 2A illustrates an example of a schematic configuration of the information processing apparatus. FIG. 2B shows an example of a schematic configuration of a processor of the information processing apparatus. FIG. 3 shows an example of a schematic configuration of another information processing apparatus. FIG. 4 shows an example of a screen on the display device that includes an image taken by one of the cameras and a soft key for an operation monitoring function. FIG. 5 shows an example of a flowchart for generating an average image of a plurality of continuous latest images taken by one of the cameras, which is executed by the processor of the information processing apparatus. FIG. 6 is a diagram for explaining an example of a procedure for recognizing a marking and determining a non-operation area in an area on a memory storing data representing an image of the display device. FIG. 7 shows an example of a video image representing the non-operation area below the marking in the image of the display device. 8A and 8B show an example of a flowchart executed by the processor of the information processing apparatus for identifying the marking of the image on the display memory area on the memory and determining the non-operation area. (Explained in Figure 8A) FIG. 9A shows a moving image that remains in the operation area of the latest image and a moving image that has entered the non-operation area of the latest moving image in the average image or the latest image that is generated by being photographed by one of the cameras. The example of the image containing is shown. FIG. 9B is an enlarged view of a part of the image of FIG. 9A, and shows an example of a method for recognizing whether or not a moving object image passes through a marking image or a boundary line. FIGS. 10A and 10B show an example of a flowchart executed by the processor for determining whether the moving image has entered the non-operation area in the average image or the latest image captured and generated by each camera. Yes. (Explained in Fig. 10A) 11A to 11C show an example of a flowchart executed by the processor for confirming whether or not a moving body image is normally in the operation area in the normal operation confirmation mode. (Explained in Figure 11A) (Explained in Figure 11A) 12A and 12B show an example of updating the synthesized image.

The objects and advantages of the invention will be realized and attained by means of the elements and combinations particularly pointed out in the appended claims.
It is understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not intended to limit the invention.

  Non-limiting embodiments of the present invention will be described with reference to the drawings. In the drawings, similar components and elements are provided with the same reference numerals.

  FIG. 1 shows an example of a schematic configuration for monitoring operations in a work or construction target equipment or device 20, such as a server rack, according to an embodiment.

  In FIG. 1, for example, a server and / or a network device is installed in the facility or apparatus 20. In the facility or apparatus 20, a part of the operation area or work area 24 of the facility or apparatus 20 is used for work or construction such as installation, adjustment, replacement, repair and removal of equipment such as servers and / or communication equipment. On the other hand, an operator may perform an operation or work. For this purpose, the worker must define a boundary between the two in order to visually distinguish the operation area or work area 24 from the non-operation areas or non-work areas 26, 27, 28 and 29 that are not the target of operation or work. A marking 30 is applied to the surface. The marking 30 is applied so as to partition or surround the operation region 24 in a rectangular or polygonal shape, for example. The marking 30 is, for example, a peelable color marking tape (curing tape), a removable color marking rope fixed or bonded, or a erasable color marker (writing tool). ) May be formed by coloring. The color of the marking 30 may be distinguishable from surrounding colors, for example, green, yellow, and red. If the color marking tape or color marking rope is glued or attached to the side plate of the rack and is not directly touching the device 20 that is the work object placed in the rack, it is placed in the rack. There may be a space between the device 20 and the color marking tape or color marking rope.

  The worker performs an operation or work on the operation area 24 and takes care not to touch a hand or a part of the body with the devices or the like in the non-operation areas 26 to 29. When a hand or the like touches a device installed in the non-operation area 26 to 29, there is a risk that the device may cause a failure. However, even if the marking 30 is applied, the operator's hand may accidentally touch the equipment in the non-operation areas 26 to 29, causing a failure in the equipment of the facility or the apparatus 20 and / or a bus or network related thereto. There is sex.

  The inventor images the operation region 24 and the marking 30 with a camera, and recognizes the captured image with an image recognition technique to determine an erroneous entry into the non-operation regions 26 to 29 of the worker's body. Recognized that it was possible. Further, the inventor has recognized that the appropriateness or inappropriateness of the operation of the operator's operation area 24 can be determined by recognizing the captured image with an image recognition technique. The inventor has recognized that it can streamline the operation and monitoring of workers.

  In FIG. 1, a plurality of cameras 202, 204, 206, 208 and 210 are arranged in the facility or apparatus 20. The cameras 202-210 may be live cameras such as web cameras, for example. The cameras 202 to 208 are directed toward the operation area 24 on the front side of the facility or apparatus 20 and approach the operation area 24 so as to image the marking 30 that partitions the operation area 24 and the non-operation areas 26 to 29. And distributed. Thereby, it is possible to capture an image representing an erroneous entry into the non-operation areas 26 to 29 of the worker, or it is possible to capture an image of an appropriate operation in the operation area 24 of the worker. Another camera 210 is arranged so as to widely capture an image near the operation area 24 in order to monitor the entire operation of the operator in the facility or apparatus 20.

  The cameras 202 to 210 are coupled to the information processing apparatus 100 or 200 such as a personal computer by wired communication or wireless communication. The cameras 202 to 210 may be coupled to the information processing apparatus 100 via a cable and further via a USB hub 146 or a router or hub 148. The information processing apparatus 100 or 200 may be further connected to a remote communication network by wired communication or wireless communication. The information processing devices 100 and 200 may be monitoring devices or monitoring terminals.

FIG. 2A shows an example of a schematic configuration of the information processing apparatus 100.
The information processing apparatus 100 may be a computer or a device such as a personal computer including a processor 112, a memory 114, an input / output interface (I / O) 116, an internal bus, and the like. The processor 112 may be a CPU (Central Processing Unit) for a computer. The processor 112 and memory 114 are coupled to an input / output interface 116. The memory 114 includes, for example, a main storage device and a semiconductor memory. The information processing apparatus 100 further includes a display device (DSP) 122, a speaker (SPK) 124, an input unit or input device 126, a recording medium reading drive 128, coupled to an input / output interface (I / O) 116. And a hard disk drive (HDD) 130. The input unit 126 may include, for example, a keyboard, a pointing device such as a mouse or a touch pad, and a touch panel. The drive 128 is provided for reading a recording medium 129 such as an optical disk. The information processing apparatus 100 further includes a USB interface (USB I / F) 142 and a network interface (NW I / F) 144 coupled to an input / output interface (I / O) 116. In the information processing apparatus 100, the USB interface 142 and / or the network interface 144 may function as an input unit that inputs at least image signals from the cameras 202 to 210.

  In the information processing apparatus 100, the display device 122, the speaker 124, the input unit 126, and the drive 128 may be external to the information processing apparatus 100 and coupled to the information processing apparatus 100 as external elements.

The processor 112 may be a dedicated processor implemented as an integrated circuit including an operation monitoring function. The processor 112 may operate according to an application program having an operation monitoring function stored in the memory 114 and / or the hard disk drive 130. Application program is stored in the recording medium 129, it may be installed is read from the recording medium 129 to the information processing apparatus 100 by the drive 128.

  The cameras 202 to 210 are coupled to the information processing apparatus 100 via a USB interface 142 and optionally a USB hub 146 by a USB cable. The information processing apparatus 100 may be connected to the monitoring center apparatus 300 installed in the monitoring center via the network interface 144 and further via the external network 5. The monitoring center device 300 may be a computer such as a personal computer, for example, and receives image information captured by the camera 210 and optionally the cameras 202 to 208 from the information processing device 100 in accordance with the operation of the monitor. The image may be displayed on a display device.

FIG. 2B shows an example of a schematic configuration of the processor 112 of the information processing apparatus 100.
The processor 112 includes, for example, a control unit 1120, a setting unit 1122, an image generation unit 1124, an area determination unit 1126, a moving object detection unit 1128, a moving object stationary determination unit 1130, a state determination unit 1132, a signal generation unit 1134, and other processing units. 1136 may be included. The processing unit 1136 may include a communication processing unit, for example. The control unit 1120 supplies control signals to the setting unit 1122, the image generation unit 1124, the region determination unit 1126, the moving object detection unit 1128, the moving object stationary determination unit 1130, the state determination unit 1132, the signal generation unit 1134, and the processing unit 1136. The operation of these elements may be controlled.

  FIG. 3 shows an example of a schematic configuration of another information processing apparatus 102. In FIG. 3, the information processing apparatus 102 includes elements 112 to 144 of the information processing apparatus 100 in FIG. 2A. The processor 112 of the information processing apparatus 102 may include a configuration similar to that of the processor 112 of the information processing apparatus 100 illustrated in FIG. 2B.

  In FIG. 3, the information processing apparatus 102 is coupled to cameras 202 to 210 via a network interface 144 and a router or hub 148. The other configuration of the information processing apparatus 102 is the same as that of the information processing apparatus 100 in FIG. 2A. Network interface 144 and router or hub 148 may be for wired or wireless communication.

  FIG. 4 illustrates a screen on the display device 122 that includes an image 400 taken by one of the cameras 202-208 and soft keys 412, 414, 416, 418, 420, 422 and 428 for operation monitoring functions. An example is shown. The image 400 includes an image of the object of the marking 30 surrounding the operation area 24 and an image of each part of the adjacent non-operation areas 26 to 29. The marking 30 may include one or more white glowing portions 32 that reflect light from, for example, a ceiling luminaire.

  A soft key 412 is used to input a command for initializing the operation monitoring function of the information processing apparatus 100 or 102. A soft key 414 is used to input a command for setting a manual setting mode of the operation monitoring function. The soft key 416 is used to input an area confirmation command for displaying the non-operation areas 26 to 29 defined by the marking 30 and the area including the other operation area 24. A soft key 418 is used to input a command for starting a normal operation monitoring mode of the operation monitoring function. The soft key 420 is used to input a command for setting a normal operation confirmation mode for confirming the normal operation of the worker in the operation monitoring function. The plurality of soft keys 422 are used to input a command for selecting one of the monitoring cameras 202 to 210 that capture the image 400 displayed on the display device 122.

  The worker, after providing the markings 30 on the equipment 20 in the form as shown in FIG. 1, attaches the cameras 202 to 210 around the operation area 24 of the equipment 20, and connects the cameras 202 to 210 with the communication cable. 100 or 102. The worker may connect the information processing apparatus 100 or 102 to the external network 5 as necessary. Next, the worker activates an operation monitoring function or an operation monitoring application on the information processing apparatus 100 or 102. The worker may adjust the setting of the operation monitoring function before the initialization of the operation monitoring function or during the initialization process. The operator selects the cameras 202 to 208 and adjusts the field of view of the cameras 202 to 208 with respect to the marking 30 by adjusting the arrangement and angle of the cameras 202 to 208 while confirming with the image 400 of the display device 122. The worker may adjust, change, or edit the operation area 24 and the non-operation areas 26 to 29 of the image 400 by rotating or inverting the image 400.

  The processor 112 (or its operation monitoring function) sequentially performs initialization processing for each of the cameras 202 to 208. In the initialization process, the processor 112 recognizes a certain edge of the marking 30 and determines areas located in a certain direction from the edge as non-operation areas 26 to 29. The region located in a certain direction may be, for example, a region below, lower left, upper right, right, or left below the edge 34, 35 of the marking 30 on the side closer to the cameras 202-208. By selecting the edges 34 and 35 of the marking 30 on the side closer to each camera for each image 400 of the cameras 202 to 208 and determining the non-operation areas 26 to 29 and other areas as boundaries, the non-operation The determination of the areas 26 to 29 and the monitoring of operations in the non-operation areas 26 to 29 can be simplified. Here, for the sake of illustration, it is assumed that the region located in a certain direction is a region below the lower edges 34 and 35 of the marking 30. Further, the processor 112 may recognize the marking 30 and determine an area located in a direction opposite to a certain direction from the edge of the marking 30 as an area including the operation area 24. Here, the region located in the opposite direction is assumed to be a region above the lower edges 34 and 35 of the marking 30. As an alternative, the processor 112 also selects the edge 38, 39 of the marking 30 that is second closest to the cameras 202-208, and its edge 34, 35 and edge 38, 39 shown in broken lines. The non-operation areas 26 to 29 and other areas may be determined with the line 33 between them as a boundary. Alternatively, the processor 112 may determine the operation area 24 as an area surrounded by the outer edges 34 to 37 of the marking 30.

  The processor 112 determines the placement and angle of the cameras 202-208 that captured the image 400 if the determined edge or intersection point I of the two edges is not within a radius distance range of the center point C of the image 400. You may be instructed to adjust. The instruction is given acoustically and / or visually by the speaker 124 and / or the display device 122. By arranging the determined edge or the intersection or angle I of the two edges in the vicinity of the center point C of the image 400, the accuracy of image recognition of an operation error in the non-operation areas 26 to 29 by the operation monitoring function is high. Become.

  5 generates an average image A of a plurality of continuous images C taken by one of the cameras 202 to 208, which is executed by the processor 112 (or the image generation unit 1124 thereof) of the information processing apparatus 100 or 102. The example of the flowchart for doing is shown. The generation of the average image A may be performed on a plurality of images C captured by each of the cameras 202 to 208 in the initialization process of the operation monitoring function or during the operation according to the operation of the worker. . The processor 112 or an operation monitoring function (hereinafter also simply referred to as a processor) captures the latest image C taken by one of the cameras 202 to 208 and stores it in the memory 114.

  In step 500, the processor 112 initializes the average image A pixel value in the memory 114. The initialized pixel values are all 0 (zero) levels or gradations.

  In step 502, the processor 112 determines whether the total number n of the latest images C used to generate the average image A has reached a required number N (eg, N = 10 or 16). If it is determined in step 502 that the number n of images C is equal to the required number N, the procedure proceeds to step 512. If it is determined that the number n of images C is not equal to or less than the required number N, at step 504, the processor 112 determines that the latest image from the selected one of the cameras 202-208. C is acquired and stored in the memory 114.

  In step 506, the processor 112 multiplies the pixel value (level) for one field or frame of the image C by the coefficient 1 / N and adds it to the pixel value of the corresponding field or frame of the average image A. In step 508, the processor 112 adds 1 to the number n of images C cumulatively added to the average image A (n = n + 1).

  In step 510, the processor 112 introduces a certain time delay, such as 100 ms. Thereafter, the procedure returns to step 502. Steps 502-510 are repeated until an average image A of N consecutive latest images C is determined. By using the average image A, noise in the image A can be reduced or removed.

  In step 512, the processor 112 stores in the memory 114 one field or one frame worth of pixel values representing the determined average image A.

  The generation of the average image A in FIG. 5 may be performed periodically, for example, every hour. Thereby, a new average image A can be generated when the ambient brightness condition changes. When updating the image A during the operation of the worker, the processor 112 compares the first image A with the latest image C in step 504, recognizes the image, and the background portion (excluding the moving body image in the image C) ( The area C) may be combined with the background portion (area) at the corresponding position of the moving image in the image A to form the image C.

  FIG. 6 is a diagram for explaining an example of a procedure for recognizing the marking 30 and determining the non-operation areas 26 to 29 in the area on the memory 114 in which data representing the image 402 of the display device 122 is stored. is there. In this case, the image 402 represents the average image A.

In FIG. 6, a set of points p <b> 1 to p <b> 13 indicate possible start points for searching for the lower edge 34 of the marking 30 in the left half region of the image 402. A set of points q1 to q13 represents possible starting points for searching for the lower edge 35 of the marking 30 in the right half region of the image 402. The set of points p1 to p13 and the set of points q1 to q13 may have a mirror symmetry relationship with respect to the central vertical line.
With the lower left corner of the frame of the image 402 as the origin and the horizontal and vertical lengths of the frame as a reference (1), the coordinates of a pair of points p1 to p13 in the frame are, for example, as follows: is there.
Point p1 = (4/16, 1/5),
Point p2 = (5/16, 1/5),
Point p3 = (6/16, 1/5),
Point p4 = (7/16, 1/5),
Point p5 = (8/16, 1/5),
Point p6 = (4/16, 1/4),
Point p7 = (5/16, 1/4),
Point p8 = (6/16, 1/4),
Point p9 = (7/16, 1/4),
Point p10 = (8/16, 1/4),
Point p11 = (1/6, 1/4),
Point p12 = (1/6 + 1/16, 1/4),
Point p13 = (1/6 + 2/16, 1/4).
Here, the series of points p1 to p5 are separated from each other by 1/16 in the horizontal direction. The series of points p6 to p13 are also separated from each other by 1/16 in the horizontal direction. The series of points p11 to p13 are also separated from each other by 1/16 in the horizontal direction. The set of points q1 to q13 is symmetrical with the set of points p1 to p13 with the lower right corner of the frame of the image 402 as the origin.

  As indicated by an arrow, the processor 112 or the area determination unit 1126 moves each pixel upward along the vertical straight line starting from one of the points p1 to p13 toward the upper side of the frame of the image 402. Are checked for the lower edge 34 of the marking 30. If the processor 112 (region determination unit 1126) cannot identify or detect the lower edge 34 of the marking 30 even after starting the search from the first point (for example, p1, p2), the next point (for example, , P2, p3) to start the search again and attempt to identify the lower edge 34 of the marking 30. Thereby, an edge 302 of a certain length (eg, 30 pixels long) below the marking 30 is identified for any of the set of points p1-p13.

  After that, similarly, the processor 112 (region determination unit 1126) causes the color of each pixel (upward to the upper side of the frame of the image 402) along a vertical straight line starting from one of the points q1 to q13. Hue and gradation) are inspected to find and search for the lower edge 35 of the marking 30. Thereby, a certain length edge 306 under the marking 30 is identified for any of the set of points q1-q13. The edges 302 and 306 may be assumed to be straight.

  The processor 112 (region determination unit 1126) further extends the line segment of the lower edge 302, 306 of the marking 30 in the region on the memory 114 so as to extend the lower edge of the marking 30 or The boundary lines 304 and 308 are determined, respectively. The boundary lines 304 and 308 may be obtained by linearly extending the line segments of the end edges 302 and 306. In this case, the processor 112 (region determination unit 1126) determines the regions below the edges 304 and 308 that are darkly shaded in FIG. 7 as the non-operation regions 26 to 29, and sets the non-operation regions 26 to 29 as the non-operation regions 26 to 29. Data representing the pixel position of the image D to be represented is stored in an area on the memory 114. The image D may be one field or frame data.

  FIG. 7 shows an example of the image 404 of the image D representing the non-operation areas 26 to 29 below the marking 30 in the image 402 of the display device 122.

  In the initialization process, the processor 112 (area determination unit 1126) displays an image 404 for confirmation by the operator when the non-operation areas 26 to 29 are determined for the cameras 202 to 208, respectively. May be. The image 404 may be displayed until a soft key 422 that selects the next one of the camera images 404 is clicked or for a period of time. Further, the worker may confirm the lower non-operation areas 26 to 29 represented by dark shadows by clicking the area confirmation soft key 416 on the screen of the display device 122.

  In the initialization process, the edges 304 and 308 of the non-operation areas 26 to 29 may not be properly determined, or the determination of the edges 304 and 308 may fail. In such a case, the worker may click the manual setting soft key 414 to set the manual setting mode. In this case, the worker can input the straight lines representing the edges 304 and 308 with, for example, a pointing device of the input unit 126 to determine the lower non-operation areas 26 to 29. The processor 112 (image generation unit 1124) stores the image D representing the non-operation areas 26 to 29 in an area on the memory 114. In the initialization process, the worker may specify one or more pixels in the image of the marking 30 in the image 402 by, for example, a pointing device, and set the color (hue and gradation) of the marking 30. . As an alternative, the worker may select a color (hue and gradation) to be used as the color of the marking 30 from a certain color palette. Accordingly, the processor 112 (setting unit 1122) can store data representing a specific color of the marking 30 in an area on the memory 114. Such determination of the non-operation areas 26 to 29 is realized by the area determination unit 1126 of the processor 112 or a part of an operation monitoring application program.

  8A and 8B are flowcharts executed by the processor 112 of the information processing apparatus 100 or 102 for identifying the marking 30 of the image 402 on the display memory area on the memory 114 and determining the non-operation areas 26 to 29. An example is shown. The determination of the non-operation areas 26 to 29 is executed by the area determination unit 1126 of the processor 112 or a part of the operation monitoring application program. The non-operation areas 26 to 29 are determined for the latest image C taken by each of the cameras 202 to 208 after the generation of the average image A according to the flowchart of FIG. 5 in the initialization process of the operation monitoring function. Done.

  Referring to FIG. 8A, in step 522, the processor 112 determines that the data representing the average image A or the latest image C from the memory 114 and the first point p1 or the next point in the search start points p1 to 13 on the left side. Data representing p2 to p13 is read. In step 524, processor 112 searches upward for the boundary of marking 30 on image A or C. For this purpose, the processor 112 sequentially moves the pixels of the image A or C upward in pixel units from the search start points p1 to p13, and obtains a pixel value representing a specific color (hue and gradation) of the marking 30. To explore.

  In step 526, the processor 112 determines whether the marking 30 color (hue and tone) boundary has been detected. If it is determined in step 526 that no boundary has been detected, the procedure returns to step 522.

  If it is determined in step 526 that a boundary has been detected, then in step 528, processor 112 determines that the left, top, top left, and bottom left pixels in the adjacent positions of the first and next pixels having the color of marking 30 Inspect the color (hue and gradation). If the color of the pixel of the marking 30 is within the allowable error range, it may be determined as the pixel of the marking 30. The processor 112 traces and detects the boundary along the lower edge 302 of the marking 30 in the upward and left directions by repeatedly performing the inspection on a plurality of adjacent pixels.

  In step 530, the processor 112 determines whether the trace distance has exceeded a threshold distance (eg, 30 pixels long). If it is determined in step 530 that the trace distance has not exceeded the threshold distance, the procedure returns to step 522.

  If it is determined in step 530 that the trace distance has exceeded the threshold distance, in step 532, the processor 112 extends the lower edge 302 and determines the extension as the current boundary 304 to store the memory. 114. Thereafter, the procedure proceeds to Step 542 in FIG. 8B.

  Referring to FIG. 8B, in step 542, the processor 112 reads data representing the first point q1 or the next points q2 to q13 among the search start points q1 to q13 on the right side from the memory 114. In step 544, the processor 112 searches upward for the boundary of the marking 30 on the image A or C as in step 522.

  In step 546, the processor 112 determines whether a color boundary of the marking 30 has been detected. If it is determined in step 546 that no boundary has been detected, the procedure returns to step 542.

  If it is determined at step 546 that a boundary has been detected, then at step 548, the processor 112, similar to step 526, causes the right, top, top right to be adjacent to the first and next pixels having the marking 30 color. And inspect the color of the lower right pixels. The processor 112 traces and detects the boundary along the lower edge 306 of the marking 30 in the upward and right directions by repeatedly performing the inspection on a plurality of adjacent pixels.

  In step 550, the processor 112 determines whether the trace distance has exceeded a certain threshold distance. If it is determined in step 550 that the trace distance has not exceeded the threshold distance, the procedure returns to step 542.

  If it is determined in step 550 that the trace distance has exceeded the threshold distance, in step 552, the processor 112 extends the lower edge 306 and determines the extension as the current boundary 308 to store the memory. 114.

  In step 554, the processor 112 determines the non-operation areas (26 to 29) below the boundary lines 304 and 308 as the operation prohibition areas 26 to 29. The processor 112 stores data of the image D representing the boundary lines 304 and 308 of FIG. 7 and the non-operation area or the operation prohibition areas 26 to 29 in the memory 114.

  In step 522, the processor 112 further replaces the color of the pixel in the white shining portion 32 surrounded or sandwiched by pixels having the marking 30 color (eg, green) within a tolerance range with the marking 30 color. May be. Thereby, detection of the lower end edges 302 and 306 is facilitated.

  As an alternative or in addition, in the boundary traces of steps 528 and 548, a generalized Hough transform may be used to interpolate white space between multiple short boundaries to estimate long boundaries . The generalized Hough transform can extract a line segment having a start point and an end point, unlike a normal Hough transform that extracts a straight line based only on the distance and angle from the start point. By performing a generalized Hough transform on the coordinates of the lower edges 302 and 306 of the marking 30, a plurality of possible short line segments can be extracted as the lower edges 302 and 306. The longest of the plurality of extracted line segments may be selected as the lower edge 302, 306. As an alternative, the longest 3 to 6 line segments that are extracted and that are substantially collinear may be selected as the lower edges 302,306. In this case, the tentatively selected line segment is extended to generate a straight line, and the straight lines whose intersection points are close to the multiple line segments are the lower edges 302 and 306. It may be selected. Alternatively, a tentatively selected line segment may be extended to generate a straight line, and such a straight line having a small angle difference between the plurality of line segments may be selected as the lower end edges 302 and 306. Thereby, detection of the lower end edges 302 and 306 is facilitated. Generalized Hough transform techniques are described, for example, in Duda et al. “Use of the Hugh Transformation To Detect Lines and Curves in Pictures”, Graphics Image Processing, Communication of the ACM, Issue 1, November 1970, revised January 1971. .

  FIG. 9A shows a moving object image 40 that remains in the operation area 24 of the image D and the non-operation area 26 of the image D in the average image A or the latest image C that is captured and generated by one of the cameras 202 to 208. The example of the image 406 containing the moving body image | videos 42 and 44 which penetrated to -29 is shown. The moving body of the moving body images 40, 42, 44 may be, for example, a part of the worker's body, a hand or a finger, or a worn glove.

  Referring to FIG. 9A, when the processor 112 (state determination unit 1132) determines that the entire moving object image 40 is above or behind the boundary lines 304 and 308, the moving object remains in the operation region 24, that is, non-displayed. It is determined that the operation areas 26 to 29 have not been entered.

  When the processor 112 (the state determination unit 1132) determines that a part of the moving body images 42 and 44 is located in the non-operation areas 26 to 29, whether the moving body image 42 further passes across the boundary lines 304 and 308, That is, it is determined whether the boundary lines 304 and 308 are divided (through) or the boundary lines 304 and 308 are divided.

  When it is determined that the moving object image 42 passes through the boundary lines 304 and 308 and reaches the non-operation areas 26 to 29, the processor 112 (state determination unit 1132) does not operate the moving object in three dimensions. It is determined that the area 26-29 has been entered. On the other hand, when it is determined that the moving body image 44 is divided by the boundary lines 304 and 308 or is isolated from the non-operation areas 26 to 29 without crossing the boundary lines 304 and 308, the moving image 44 is viewed in three dimensions. It is determined that the moving object remains in the operation area 24, that is, has not entered the non-operation areas 26 to 29. In the moving body image 44, the moving body has only deeply penetrated deep inside the operation area 24, and remains in the operation area 24, and has not entered the non-operation areas 26 to 29, as viewed in three dimensions. Conceivable.

  FIGS. 10A and 10B determine whether the moving body images 40, 42, 44 have entered the non-operation areas 26-29 in the images A or C captured and generated by each camera 202-208, executed by the processor 112. The example of the flowchart for doing is shown. The operation monitoring for making this determination is executed in the normal operation monitoring mode by clicking the soft key 418 on the screen of the display device 122 of FIG. 9A.

  Referring to FIG. 10A, in step 702, the processor 112 (state determination unit 1132) reads from the memory 114 the data of the image D representing the operation prohibited area for each of the cameras 202 to 208. In step 704, the processor 112 (state determination unit 1132) acquires the average image A data regarding each of the cameras 202 to 208 from the memory 114. The image A is preferably a new image in order to reflect a change in brightness around the facility 20. In step 706, the processor 112 (state determination unit 1132) acquires the latest image C data from the memory 114 for each of the cameras 202 to 208. The image A corresponds to the background image of the moving object image M in the latest image C.

  In step 708, the processor 112 (moving object detection unit 1128) compares the data of the latest image C with the data of the average image A for each of the cameras 202 to 208, and compares the data of the average image A in the latest image C. Extract the pixel at the location that is different from the data. The processor 112 (moving object detection unit 1128), if the different part is not random noise and exists as one region exceeding a certain threshold area (for example, 100 pixels), the extracted pixel is used as the moving object image 40. , 42 and 44, the data of the moving object image M is generated. The moving body image M is data of one field or frame in which the pixel values other than the areas of the moving body images 40, 42, and 44 are 0 (zero).

  In step 848, the processor 112 (state determination unit 1132) determines whether there is valid moving body image M data. If it is determined that there is no valid moving body image M data, the procedure returns to step 706.

  If it is determined in step 848 that valid moving body image M data exists, the processor 112 (state determination unit 1132) determines that the moving body image M data and the image D data representing the operation-prohibited area in step 850. Compare.

  Referring to FIG. 10B, in step 852, the processor 112 (state determination unit 1132), for each of the one or more moving object images M of the cameras 202 to 208, displays a moving object image in the operation prohibited areas 26 to 29 of the corresponding image D. It is determined whether or not 40 to 44 exist. If it is determined that no moving body video 40 exists in the operation prohibited areas 26 to 29 of any image D, the procedure returns to step 706 in FIG. 10A.

  If it is determined in step 852 that the moving body images 42 and 44 exist in the operation prohibited areas 26 to 29 of any image D, the procedure proceeds to step 854. In step 854, the processor 112 (state determination unit 1132) recognizes an overlapping or continuity relationship (geometric relationship) between the moving body images 42 and 44 and the marking 30 images or boundary lines 304 and 308.

  FIG. 9B is an enlarged view of a portion of the image 406 of FIG. 9A. In step 854, it is recognized whether the moving object image 42 or 44 is continuous through the image 30 or the boundary lines 304 and 308 of the marking 30. An example of how to do that.

  In FIG. 9B, the processor 112 (state determination unit 1132) obtains a line segment of the boundary line 304 or 308 that passes through or touches the moving body image 42 or 44, and obtains a midpoint fc of the line segment. . Next, the processor 112 (state determination unit 1132) performs marking on the moving object image 42 or 44, the point fd on the moving object image 42 or 44 on the straight line passing through the middle point fc and below the boundary lines 304 and 308, and marking. A point fu on 30 is selected. The point fd is located at a position, for example, lower, lower left, or lower right from the midpoint fc, and is separated from the midpoint fc by a distance of a certain number of pixels (for example, 4 pixels). The point fu is at, for example, the upper, upper right, or upper left position from the midpoint fc, and is separated from the midpoint fc by a distance of a certain number of pixels (for example, 4 pixels).

  The processor 112 (state determination unit 1132) determines the color (hue and gradation) of the pixel between the point fd and the midpoint fc and the color of the pixel between the midpoint fc and the point fu. When the pixel color between the point fd and the midpoint fc is equal to the color of the pixel between the midpoint fc and the point fu within a tolerance range, the processor 112 (the state determination unit 1132) indicates that the moving object image 42 is marked. 30 is determined to pass through. This determination may be performed as an additional condition that the color of the pixel between the midpoint fc and the point fu is not the color of the marking 30. When the pixel color between the point fd and the midpoint fc exceeds the allowable error range and is different from the color of the pixel between the midpoint fc and the point fu, the processor 112 (state determination unit 1132) It is determined that 44 does not penetrate the marking 30. This determination may be performed as an additional condition that the color of the pixel between the midpoint fc and the point fu is the color of the marking 30 within a tolerance.

  Referring again to FIG. 10B, in step 856, the processor 112 (state determination unit 1132) determines that the moving body images 40 to 44 existing in the operation prohibited areas 26 to 29 pass through the images 30 or the boundary lines 304 and 308 of the marking 30. It is determined whether or not (stretching across). For any moving image M, if it is determined that the moving image 40, 44 does not penetrate the marking 30 image or the boundary lines 304, 308 or is divided by the boundary lines 304, 308, the procedure Returns to step 706 of FIG. 10A.

  If it is determined in step 856 that the moving body image 42 has penetrated the image 30 or the boundary lines 304 and 308 of the marking 30 for any moving body image M, the processor 112 (signal generation unit 1134) issues a warning in step 858. Generate a signal to represent. Accordingly, when any of the images C captured and generated by the cameras 202 to 208 enters the operation prohibited area and the condition for entering the non-operation areas 26 to 29 is satisfied in the form of logical sum (OR). In addition, a signal representing a warning is generated.

  The processor 112 (the signal generator 1134) may supply a signal representing a warning to the speaker 124 to cause the speaker 124 to generate a warning sound such as a buzzer sound for a certain length of time (for example, 2 seconds). Good. Alternatively or additionally, the processor 112 (signal generator 1134) may display a message representing the warning on the display device 122. Thereby, the worker can confirm or recognize an intrusion of an erroneous moving body in the operation prohibited areas 26 to 29. A signal or message representing the warning may be transmitted to the monitoring center apparatus 300 by the processor 112 (control unit 1120, processing unit 1136). Thereby, the operator's erroneous operation can be monitored by the monitoring center device 300. Thereafter, the procedure returns to step 706 of FIG. 10A.

  Instead of or in addition to generating a signal representing a warning in step 858, in step 857, the processor 112 (signal generation unit 1134) may generate a signal representing normal operation. That is, step 857 is executed when it is determined in step 852 that the moving body video 40 exists but does not exist in the operation prohibited areas 26 to 29 of any image D. In step 857, the moving body image 44 exists in any of the moving body images M in step 856, but none of the images of the marking 30 or the boundary lines 304 and 308 penetrates or is divided by the boundary lines 304 and 308. It is executed when it is determined that it is present. On the other hand, if step 858 is not executed, if it is determined in step 856 that the moving object image 42 has penetrated the image 30 or the boundary lines 304 and 308 of the marking 30 for any moving object image M, the procedure is as shown in FIG. Return to step 706.

  The processor 112 (signal generation unit 1134) generates a signal indicating normal operation, supplies a sound signal to the speaker 124, and generates a sound such as a chime sound for a certain length of time (for example, 1 second). You may let them. Alternatively or additionally, the processor 112 (signal generator 1134) may display a message representing normal operation on the display device 122. Thereby, the worker can confirm or recognize an appropriate operation in the operation area 24. A signal or message indicating the normal operation may be transmitted to the monitoring center apparatus 300 by the processor 112 (control unit 1120, processing unit 1136). Thereby, the operator's erroneous operation can be monitored by the monitoring center device 300. Thereafter, the procedure returns to step 706 of FIG. 10A.

  On the other hand, when the operation area 24 is difficult to understand in the facility or apparatus 20, for example, when a plurality of operation areas 24 are dispersed or at a very high position, the worker tries to operate before the actual operation. There is a case where it is desired to confirm in advance whether or not the position of the portion to be operated is within the appropriate operation area 24. In such a case, after the initialization process, the worker clicks the soft key 418 to set the operation monitoring function to the normal operation confirmation mode. The processor 112 (signal generation unit 1134) outputs a signal indicating the normal operation as described above when the moving body images 40 and 44 are three-dimensionally stopped in the operation region 24 for a certain time in the normal operation confirmation mode. It may occur. Further, the processor 112 (the signal generation unit 1134) is a signal indicating a warning as described above when the moving body image 42 is three-dimensionally stationary in the non-operation areas 26 to 29 in a normal operation confirmation mode. May be generated.

  FIGS. 11A to 11C show an example of a flowchart executed by the processor 112 for confirming whether or not the moving body images 42 to 44 are normally in the operation area 24 in the normal operation confirmation mode. In this case, the moving body images 42 to 44 are determined using the composite image B obtained by combining the image A and a plurality of consecutive N images C. The value of N may be a value different from N in FIG.

  Referring to FIG. 11A, steps 702-706 are similar to those of FIG. In this case, in step 706, the processor 112 (state determination unit 1132) may acquire data of one or a plurality of continuous latest images C for each of the cameras 202 to 208 from the memory 114. The number n of the plurality of images C may be a number ranging from 2 to a predetermined number N (for example, N = 10 or 16). The data of a plurality of continuous images C may have a certain time interval, for example, 100 ms interval.

  In step 808, the processor 112 (image generation unit 1124) compares the latest data of each image C with the data of the average image A for each of the cameras 202 to 208, and compares the average image A of the latest images C with each other. The pixel of the position of the part which is different from the data of and exceeding the tolerance range is extracted. The processor 112 (the image generation unit 1124) includes the extracted pixels as the moving body images 40, 42, and 44 if the different portion is not random noise but exists as one region exceeding a certain threshold area. Data of the moving object image M is generated.

  Further, the processor 112 (state determination unit 1132) determines that the moving body image M does not exist in the operation prohibited areas 26 to 29 based on the image D or based on the images A, C, and D, based on the boundary lines 304 and 308. When the moving body image 40 only included above is included, it may be determined as the moving body image M ′. The moving body image M ′ is used in place of the moving body image M in the alternative forms of FIGS. 11A to 11C described later.

  Referring to FIG. 11B, in step 820, the processor 112 (state determination unit 1132) determines whether or not the moving object image M exists. If it is determined in step 820 that the moving body image M does not exist, the processor 112 (state determination unit 1132) sets or turns on a flag for notifying normal operation in the flag storage area of the memory 114 in step 822. Determine if it is in a state. If it is determined that the flag is not set, the procedure returns to step 706 in FIG. 11A.

  If it is determined in step 822 that the flag is set, the processor 112 (state determination unit 1132) discards the current combined image B and cancels the normal operation notification flag in step 824. Or turn it off. Here, the synthesized image B is an image generated by cumulatively adding each moving body image M in the latest image C with a certain weight to the average image A as the background. The image B is generated in the form described later. Thereafter, the procedure returns to step 706.

  If it is determined in step 820 that the moving body image M exists, the processor 112 (state determination unit 1132) determines in step 826 whether the flag is cleared or in an off state. If it is determined that the flag has not been cleared or is in the on state, the procedure returns to step 706. Thereby, it is possible to prevent the signal representing the normal operation from being generated redundantly or redundantly during the period in which the flag representing the generation of the signal representing the normal operation is in the set state.

  When it is determined in step 826 that the flag is released or in the off state, the processor 112 (state determination unit 1132) determines in step 828 whether or not the synthesized image B exists. If it is determined that the synthesized image B does not exist, the processor 112 (image generation unit 1124) copies the image A in the memory 114 to generate a new initial image B in step 830 and stores it in the memory 114. Store. The processor 112 (image generation unit 1124) may further update the image B with the moving image M by adding each moving image M to the initial image B with a certain weight. Thereafter, the procedure returns to step 706.

  If it is determined in step 828 that the image B exists, the processor 112 (the image generation unit 1124) adds the moving image M to the image B with a certain weight in step 840, thereby adding the moving image M to the moving image M. Update. Next, the processor 112 (state determination unit 1132) compares the latest image C with the synthesized image B.

  In step 842, the processor 112 (moving object stationary determination unit 1130) determines whether or not the image C represents a stationary state. Therefore, for example, the processor 112 (moving object stationary determination unit 1130) determines whether there is a difference between the pixel value (hue and gradation) of the image C and the corresponding pixel value of the image B beyond the allowable range of error. May be determined. The difference between the images C and B exceeding the allowable range indicates that the moving image M is not in a still state or that the moving image M has not been determined yet. When a moving object such as a hand is moving, a plurality of continuously acquired images C are different from each other, the moving object image 40 in the image (410) is blurred, and the images B and C are Does not match within tolerance. When the number of synthesized images C is less than a certain number N (for example, N = 8 or 10), the moving body video 40 in the image 410 is mixed with the corresponding position portion of the background image A and half. The image becomes transparent and the synthesized image B and the latest image C do not match.

  If it is determined in step 842 that the image C does not represent a stationary state, the processor 112 (moving object stationary determination unit 1130) determines whether a total of N moving body images M have been combined with the initial image B in step 844. Determine if. If it is determined that N moving body images M have not been combined, the procedure returns to step 706. If it is determined in step 844 that N moving body images M have been combined, the processor 112 (image generation unit 1124) discards the current image B and copies the image A in the memory 114 in step 846. The initial image B is generated again and stored in the memory 114. Accordingly, the image B including the blurred moving body image 40 of one or more moving body images M that are not stationary is not used. The processor 112 (the image generation unit 1124) may further update the image B with the latest moving body image M by adding one latest moving body image M to the initial image B with a certain weight. Thereafter, the procedure returns to step 706. Alternatively, if it is determined in step 842 that image C does not represent a stationary state, the procedure may return to step 706 without performing steps 844, 846.

  If it is determined in step 842 that the image C represents a stationary state, the procedure proceeds to step 850 in FIG. 11C. In step 842 of FIG. 11B, the fact that there is no difference between the image C and the image B exceeding the allowable range indicates that the moving object is substantially stationary over a period of a certain number N of images C. Represents.

12A and 12B show examples of updating the synthesized image B. FIG.
As described above, the initial image B is the same as the image A. The processor 112 (the image generation unit 1124) performs a certain weighting w on pixel values corresponding to the positions of the moving body images 40 to 44 of each moving body image M generated from one or more latest images C that are continuously acquired. Multiply _F to generate a weighted moving image Mw pixel. The processor 112 (the image generation unit 1124) is multiplied by certain weights w _B only pixel values of positions corresponding to the moving object image 40-44 of the moving object image M in the image B, and the pixels of the weighted image Bw Generate. The processor 112 (image generation unit 1124) further adds the pixel values of the moving body images 40 to 44 in the moving body image Mw to the pixel values of the image Bw at the position corresponding to the moving body images 40 to 44. Here, for example, the total number of images C to be combined × w _F + w _B = 1. For example, when a certain number is N = 16, the combination is (w _F = 1/5, w _B = 4/5) for one image C. For example, when the image B is sequentially updated with 16 moving image M, the ratio of the same moving image M to the initial image B becomes 0.97: 0.03, and the moving image M is approximately averaged (weighted average). ). If the image B is updated with a smaller number (for example, one) of moving body images M in each update of the image B, the processing load of the processor 112 can be distributed in time and more uniform. The synthesis produces an image B that includes a thin translucent moving video 40-44 close to image A or initial image B, as shown in image 408 of FIG. . In step 840, the image including the darker translucent moving image 40-44 as shown in the image 410 of FIG. 12B by updating the image B with the moving image M of the more recent image C. B is generated.

  By using the image A as the initial image B, the background image of the image B can be made the same as the image A even if the brightness changes in the equipment or the apparatus 20 and the brightness of the image C changes. Thereby, when comparing the image C and the image B, the images of the background portions thereof are the same, and the comparison of the moving body images 40 to 44 is facilitated.

  In addition, when a moving body is moving, a plurality of the latest N consecutive images C and their moving body images 40 to 44 are different from each other, and the moving body images 40 to 44 of the image 410 are blurred, in particular, the outline is blurred. Thus, the pixel values at the corresponding positions in the images B and C do not completely match. When a plurality of N moving body images 40 to 44 move in position, if the area of the difference in motion near the contour is within an allowable error range, it is determined that the moving body images 40 to 44 are almost stationary. It's okay. When the moving object is almost stationary, the corresponding pixel values of the N consecutive latest images C are the same within the tolerances of the area and level errors, and the corresponding large values of the images B and C are large. The pixel values of the parts substantially match.

  Furthermore, even when the moving body image 44 is also in the prohibited areas 26 to 29 below the boundary lines 304 and 308 and does not penetrate the boundary lines 304 and 308 and is divided by the boundary lines 304 and 308, FIG. As in the case of 9B, it can be regarded as a normal operation.

  In FIG. 11C, steps 850 to 856 after step 842 (YES) are the same as those in FIGS. 10A and 10B. If it is determined in step 852 that the latest moving body image M does not exist in the operation prohibited areas 26 to 29, the procedure proceeds to step 860. When it is determined in step 856 that the moving body images 40 and 44 do not penetrate the video 30 or the boundary lines 304 and 308 of the marking 30 or are divided by the boundary lines 304 and 308 for any moving body image M. The procedure proceeds to step 860.

  In step 860, the processor 112 (signal generator 1134) sets a flag in the memory 114 for a certain period of time (for example, 2 seconds) or turns it on to generate a signal representing normal operation. The processor 112 (signal generation unit 1134) may supply a signal representing normal operation to the speaker 124 to cause the speaker 124 to generate a sound such as a chime sound for a certain length of time. Alternatively or additionally, the processor 112 (signal generator 1134) may display a message representing normal operation on the display device 122. After that, the processor 112 (signal generation unit 1134) releases the flag or turns it off. Thereby, the worker can confirm or recognize an appropriate operation in the operation area 24. A signal or message indicating the normal operation may be transmitted to the monitoring center apparatus 300 by the processor 112 (control unit 1120, processing unit 1136). Thereby, the operator's erroneous operation can be monitored by the monitoring center device 300. In this way, as viewed three-dimensionally, the moving object is in the operation area 24 or only deeply penetrates into the operation area 24 and remains in the operation area 24 and enters the non-operation areas 26 to 29. Even if there is not, it can be confirmed that the operation is normal. Thereafter, the procedure returns to step 706.

  If it is determined in step 856 that the moving body image 42 passes through the boundary lines 304 and 308 for any moving body image M, the procedure returns to step 706.

  Instead of or in addition to generating a signal representing normal operation in step 860 of FIG. 11B, in step 858 of FIG. 11C, processor 112 (signal generator 1134) may generate a signal representing a warning. That is, step 858 is executed when it is determined in step 856 that the moving body image 42 of any moving body image M passes through the boundary lines 304 and 308. In step 852, it is determined that the moving body image 40 does not exist in the prohibited areas 26 to 29, or in step 856, it is determined that the moving body image 44 does not penetrate the image 30 or the boundary lines 304 and 308 of the marking 30 for any moving body image M. If so, the procedure may return to step 706 of FIG. 11A. In this case, step 860 is not executed.

  As an alternative, in the flowcharts of FIGS. 11A to 11C, if the moving body images 42 and 44 are in the prohibited areas 26 to 29 below the boundary lines 304 and 308 in FIGS. 12A and 12B, do not regard them as normal operations. Also good. In this case, the worker places a hand, for example, on the surface area of the marking 30 in the operation area 24 and confirms that the position is in the operation area 24 without inserting a hand in the back. In this case, in Steps 820 to 842, instead of the moving object image M, the moving object image M ′ including the moving object image 40 only in the region above the boundary lines 304 and 308 is used, and Steps 850 to 858 in FIG. 11C are executed. Instead, step 960 may be executed after step 842 (YES). Thereby, the processing of the processor 112 is simplified and the load is reduced.

  In steps 820 to 856 in the flowcharts of FIGS. 11A to 11C, the moving image is not normally invaded into the operation prohibited areas 26 to 29 in three dimensions with respect to the image C generated by all the cameras 202 to 208, and the normal operation is performed by the logical product. When the above condition is satisfied, a signal indicating a normal operation is generated in step 860. In Steps 820 to 856, when the moving image enters the operation prohibited areas 26 to 29 three-dimensionally with respect to the image C generated by any of the cameras 202 to 208, and an error operation condition is established by logical sum, In step 858, a signal representing a warning is generated.

  As an alternative, in FIGS. 11A to 11C, an image B ′ generated by synthesizing a moving body image M of a plurality of N images C may be used instead of the image B including a part of the image A as a background. . In this case, in step 840, the moving body image M may be compared with the image B ′ using the moving body image M of the image C instead of the latest image C for comparison. In this case, the pixel values of the initial image B ′ are all 0 (zero). The updated image B ′ does not include the pixels of the image A, that is, the pixel values of the background image are all 0 (zero). In the image B ′, the moving body image M may be updated in the same form as the update of the image B.

In this case, the processor 112 (the image generation unit 1124) multiplies the pixel values at the positions of the moving body images 40 to 44 of the moving body images M generated from the continuously acquired images C by a certain weight w _F. Then, a pixel of the weighted moving image Mw is generated. The processor 112 (image generation unit 1124) further updates the image B ′ by adding the pixel values of the moving body images 40 to 44 in the moving body image Mw to the pixel values of the image B ′ at the position corresponding to the pixel values. For example, when a certain number is N = 8, w _F = 1/8 for one moving body image M. For example, when the image B ′ is sequentially updated with eight moving body images M, the sum of the pixel positions of the videos 40 to 44 of the same moving body image M has a pixel value that is 8 times and is multiplied by the weight w _F = 1/8. Will average it out.

  When the moving body is moving, the moving body images 40 to 44 of a plurality of N moving body images M that are consecutive are different from each other, and the moving body images 40 to 44 of the image B ′ or the image 410 are blurred, in particular, the outline is blurred. Thus, the pixel values at corresponding positions in the image B ′ and the latest moving object image M do not completely match. Compared with the above-mentioned image B, the level or gradation of the pixel value near the blurred outline of the image B ′ becomes small, and the hue of the pixel may change in some cases. However, when the moving body images 40 to 44 move in position, it may be determined that the moving body images 40 to 44 are substantially stationary if the area of the difference in motion near the contour is within an allowable error range. . When the moving object is almost stationary, the corresponding pixel values of the continuous N latest moving object images M are the same within the tolerances of the area and level errors. Most corresponding pixel values are substantially matched.

  After the work or construction in the facility or apparatus 20 is complete, the worker clicks on the soft key 428 indicating completion, retrieves the cameras 202-210, and removes the marking 30.

  The steps of each flowchart illustrated in FIGS. 5, 8A and 8B, 10A and 10B, 11A-11C, 13A-13C are one or more integrated circuits or devices (units) that implement the respective functions, and It can also be implemented in hardware with other circuits or devices.

  All examples and conditional expressions given here are intended to help the reader understand the inventions and concepts that have contributed to the promotion of technology, such examples and It is understood that the present invention is not limited to the conditions, and that the organization of such examples in the specification is not related to the superiority or inferiority of the present invention. Although embodiments of the present invention have been described in detail, it will be understood that various changes, substitutions and variations can be made thereto without departing from the spirit and scope of the invention.

Regarding the embodiment including the above examples, the following additional notes are further disclosed.
(Supplementary Note 1) A storage unit that captures and stores an image signal via the input unit;
A determining unit that determines a boundary line in the image of the first image signal stored in the storage unit and a non-operation area partitioned by the boundary line;
A detection unit for detecting a moving image in the second image signal stored in the storage unit;
A determination unit that determines a state of the moving body image with respect to the boundary line based on an overlapping relationship between the detected moving body image and the boundary line in the image of the second image signal;
A signal generator for generating a signal based on the determined state;
Including an information processing apparatus.
(Supplementary Note 2) When the detected moving body image does not penetrate the boundary line, the determination unit determines that the state of the moving body image is in the first state, and the detected moving body image is The information processing apparatus according to appendix 1, wherein the moving body image is determined to be in a second state when passing through the boundary line.
(Additional remark 3) The said determination part is located in the area | region which is not the said non-operation area | region beyond the said boundary line from the 1st position on the said detected moving body image located in the said non-operation area | region in the said 2nd image signal. The color of the pixel from the first position to the boundary line and the color of the pixel from the first position to the second position are within an allowable error range. If the color values are not the same, the moving image is determined to be in the first state, and the pixel color from the first position to the boundary line and the second position are determined. The information according to appendix 2, wherein the state of the moving image is determined to be in the second state when the pixel color has the same color value within an allowable error range. Processing equipment.
(Additional remark 4) The said determination part searches in a certain direction from a certain starting point in a temporary non-operation area | region in the image of the said 1st image signal, detects the boundary of the area | region of a specific color, When it is detected that the boundary of the color region exists at least within a certain distance range, a line formed by the boundary is extended, and the extended line is used as at least a part of the boundary line. The information processing apparatus according to any one of appendices 1 to 3, wherein the information processing apparatus is determined.
(Additional remark 5) The said determination part is based on the overlapping relationship of the said detected moving body image | video and the said boundary line in the image of a said 2nd image signal, when the detected moving body image | video is in a stationary state. The information processing apparatus according to any one of appendices 1 to 4, wherein the information processing apparatus determines a state of the moving body image.
(Supplementary Note 6) An image signal is captured via the input unit and stored in the storage unit.
Determining a boundary line in the image of the first image signal stored in the storage unit and a non-operation area partitioned by the boundary line;
Detecting a moving image in the second image signal stored in the storage unit;
Determining a state of the moving body image with respect to the boundary line based on an overlapping relationship between the detected moving body image and the boundary line in the image of the second image signal;
Generating a signal based on the determined state;
A program that causes an information processing apparatus to execute processing.

DESCRIPTION OF SYMBOLS 100,102 Information processing apparatus 112 Processor 114 Memory 116 Input / output interface 122 Display apparatus 124 Speaker 126 Input part 128 Drive for recording medium reading 130 Hard disk drive 142 USB interface 144 Network interface 1120 Control part 1122 Setting part 1124 Image generation part DESCRIPTION OF SYMBOLS 1126 Area | region determination part 1128 Moving body detection part 1130 Moving body stationary determination part 1132 State determination part 1134 Signal generation part

Claims (5)

A storage unit for capturing and storing image signals via the input unit;
A determining unit that determines a boundary line in the image of the first image signal stored in the storage unit and a non-operation area partitioned by the boundary line;
A detection unit for detecting a moving image in the second image signal stored in the storage unit;
And wherein the detected moving object image in the image of the second image signal based on whether to penetrate the boundary line, judging section that judges the state of the moving object image with respect to the boundary line,
A signal generator for generating a signal based on the determined state;
Including an information processing apparatus.   The determination unit determines that the state of the moving body image is in the first state when the detected moving body image does not penetrate the boundary line, and the detected moving body image indicates the boundary line. 2. The information processing apparatus according to claim 1, wherein if it is penetrating, it is determined that the state of the moving object image is in the second state.   The determining unit detects a boundary of a specific color region by searching in a certain direction from a certain start point in the temporary non-operation region in the image of the first image signal, and detects the boundary of the specific color region. A line formed by the boundary is extended when it is detected that the boundary exists at least within a certain distance, and the extended line is determined as at least part of the boundary line; The information processing apparatus according to claim 1, wherein the information processing apparatus is provided. The determination unit, when the detected moving body image was in a stationary state, based on the overlapping relationship of the detected moving object image with the boundary line in the image of the second image signal, the moving object image The information processing apparatus according to claim 1, wherein the information processing apparatus determines a state of the information processing apparatus. Capture the image signal via the input unit, store it in the storage unit,
Determining a boundary line in the image of the first image signal stored in the storage unit and a non-operation area partitioned by the boundary line;
Detecting a moving image in the second image signal stored in the storage unit;
Based on whether the detected moving object image in the image of the second image signal penetrates the boundary, to determine the status of the moving object image with respect to the boundary line,
Generating a signal based on the determined state;
A program that causes an information processing apparatus to execute processing.

Images