JP7347960B2 - Shooting direction setting device, shooting direction setting program, shooting direction setting method, and intrusion detection method of intrusion detection system - Google Patents

Description

The present invention relates to a photographing direction setting device, a photographing direction setting program, a photographing direction setting method, and an intrusion detection method for an intrusion detection system. The present invention relates to a photographing direction setting device, a photographing direction setting program, a photographing direction setting method, and an intrusion detection method for an intrusion detection system that can detect human intrusion into a dangerous area.

This type of technology can be used, for example, to ensure the safety of workers working around machines and the like.

An example of a working environment monitoring device that forms the background of this invention is disclosed in Patent Document 1. The work environment monitoring device of Patent Document 1 includes a camera that takes a picture of a robot's work space, a detection means that detects the presence or absence of a moving object in the work space and its position based on images taken by the camera, and a detection means that detects the presence or absence of a moving object in the work space and its position. and a warning means for issuing a warning when the vehicle is located close to the vehicle.

Further, the work environment monitoring device of Patent Document 1 includes two cameras, one camera that takes a picture of the robot work space from above, and the other camera that takes a picture of the robot work space from the side. It is possible to treat the work space as a three-dimensional image, detect the position of the moving object in the three-dimensional space, and determine the possibility of contact between the robot and the moving object.

By having the above configuration, the work environment monitoring device of Patent Document 1 can detect height differences using images from the side, even when it is determined that there is a risk of contact based only on images from above. If it can be determined that there will be no contact, unnecessary warnings and robot stoppages can be reduced as much as possible.

Japanese Patent Application Publication No. 05-261692 [G01B 11/00]

However, in the work environment monitoring device of Patent Document 1, in order to detect the position of a moving object in three-dimensional space, it is necessary to provide two or more cameras, which complicates the configuration and increases manufacturing costs. There is a problem. Furthermore, depending on the work environment, it may be difficult to install two or more cameras.

Therefore, the main object of the present invention is to provide a novel photographing direction setting device, photographing direction setting program, photographing direction setting method , and intrusion detection method for an intrusion detection system .

Another object of the present invention is a shadow direction setting device, a photographing direction setting program, a photographing direction setting method , and an intrusion detection method for an intrusion detection system, which can detect a person's intrusion into a dangerous area by simple distance image processing. The goal is to provide the following .

In order to solve the above problems, the present invention employs the following configuration. Note that the reference numerals, supplementary explanations, etc. in parentheses indicate correspondence with the embodiments described later to aid understanding of the present invention, and do not limit the present invention in any way.

The first invention provides a determination area that includes a dangerous area that is dangerous if a person enters, a safe area that is safe even if a person enters, a shielded area that is blocked by an obstacle, and an open area where no obstacle exists. The 3D map is equipped with a map generation unit that generates a 3D map of , a plurality of voxels, each voxel is written with either a dangerous area, a safe area, a shielded area, or an open area, and the setting unit is configured to: This is a photographing direction setting device that counts the number of devices that can reach a dangerous area without passing through a shielded area, and sets the direction with the largest number as the photographing direction.

In the first invention, the map generation unit (14, S1, S3, S5) generates a dangerous area that is dangerous if a person enters, a safe area that is safe even if a person enters, and a shielded area that is blocked by an obstacle. A three-dimensional map is generated for the determination area including the area and the open area where no obstacles exist. The setting unit (14, S7 to S17) sets the photographing direction for intrusion detection to the direction with the least blind spots when photographing the dangerous area, based on the three-dimensional map. Specifically, the map generation unit generates a three-dimensional map that includes a plurality of voxels and in which each voxel is written with either a dangerous area, a safe area, a shielded area, or an open area, and the setting unit generates a three-dimensional map that includes a plurality of voxels. In each of a plurality of directions for each voxel included in such a three-dimensional map, the number of directions that can reach the dangerous area without passing through the shielded area is counted, and the direction with the largest number is set as the imaging direction.

According to the first invention, since the photographing direction for intrusion detection is set in the direction with the least blind spots, it is possible to detect the intrusion of a person into a dangerous area by simple distance image processing.

A second invention is a photographing direction setting device according to the first invention , wherein the map generation unit generates a dangerous area map including a dangerous area and a safe area, and a shielded area map including a shielded area and an open area. A three-dimensional map is generated by combining the dangerous area map and the shielded area map .

According to the second invention, it is possible to appropriately generate a three-dimensional map and to appropriately set the photographing direction for intrusion detection.

The third aspect of the invention is to configure the computer of the photographing direction setting device into a dangerous area where it is dangerous if a person enters, a safe area where it is safe even if a person enters, a shielding area that is blocked by an obstacle, and an area where the obstacle exists. A map generation unit that generates a three-dimensional map for a determination area including an open area that is not open, and a setting that sets a photographing direction for intrusion detection to a direction with the least blind spots when photographing a dangerous area based on the three-dimensional map. The 3D map includes a plurality of voxels, and each voxel is written with either a danger area, a safe area, a shielded area, or an open area. This is an imaging direction setting program that counts the number of directions that can reach a dangerous area without passing through a shielded area in each of a plurality of directions for voxels, and sets the direction with the largest number as the imaging direction.

The fourth invention provides (a) a dangerous area that is dangerous if a person enters, a safe area that is safe even if a person enters, a shielded area that is blocked by an obstacle, and an open area that is free of obstacles. (b) a map generation step of generating a three-dimensional map for the included judgment area; The step (a) includes a setting step of setting a shooting direction, and step (a) generates a three-dimensional map including a plurality of voxels in which one of a dangerous area, a safe area, a shielded area, and an open area is written in each voxel; In (b), in each of the multiple directions for each voxel included in the 3D map, the number of people who can reach the dangerous area without passing through the shielding area is counted, and the direction with the largest number is set as the shooting direction. This is the shooting direction setting method.

According to the third and fourth inventions, the same effects as the first invention can be expected.

A fifth invention is an intrusion detection method for an intrusion detection system that includes a plurality of distance image sensors that photograph a photographing area including a dangerous area that is dangerous if a person enters from different directions and outputs distance images. a) Map generation that generates a three-dimensional map of a determination area that includes a dangerous area, a safe area where it is safe for people to enter, a shielded area that is blocked by obstacles, and an open area where no obstacles exist. and (b) a distance image sensor for intrusion detection that is capable of photographing a dangerous area from a direction with the least blind spots among a plurality of distance image sensors based on the three-dimensional map generated in step (a). (c) A setting step of setting a plurality of distance images output from the distance image sensor for intrusion detection in a cutting direction perpendicular to the photographing direction of the distance image sensor for intrusion detection set in step (b). and (d) an intrusion detection step of detecting whether a person has entered the dangerous area based on the plurality of cross-sectional images generated in step (c). This is a detection method.

According to this invention, human intrusion into a dangerous area can be detected by simple distance image processing.

The above objects, other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of a safety control system according to an embodiment of the present invention. FIG. 2 is an illustrative diagram showing an example of a memory map of the memory shown in FIG. FIG. 3 is a flowchart showing an example of the shooting direction setting operation executed by the computer shown in FIG. FIG. 4 is a side view showing an example of a determination area including a dangerous area. FIG. 5 is a top view of the determination area in the embodiment of FIG. FIG. 6 is an illustrative diagram showing an example of a shielding area map based on the determination area of the embodiment in FIG. FIG. 7 is an illustrative diagram showing an example of a dangerous area map based on the determination area of the embodiment in FIG. FIG. 8 is an illustrative diagram showing an example of a composite map based on the determination area of the embodiment in FIG. FIG. 9 is an illustrative diagram showing an example of a method for calculating a score based on the determination area of the embodiment in FIG. FIG. 10 is an illustrative diagram showing scores for each photographing direction in the embodiment shown in FIG. FIG. 11 is a side view showing another example of a determination area including a dangerous area. FIG. 12 is a top view of the determination area in the embodiment of FIG. 11. FIG. 13 is an illustrative diagram showing scores based on the determination areas of the example in FIG. 11. FIG. 14 is a flow diagram showing an example of an intrusion detection operation executed by the computer shown in FIG. FIG. 15 is a flow diagram showing details of the intrusion detection step. FIG. 16 is an illustrative diagram showing the configuration of the safety control system of the second embodiment. FIG. 17 is a flowchart showing an example of the shooting direction setting operation in the second embodiment.

<First example>
Referring to FIG. 1, a safety control system 10 of this embodiment includes a distance image sensor 12 that photographs a predetermined photographing area. As an example, the distance image sensor 12 has a viewing angle or angle of view of 40° in the front, rear, left, and right directions.

The distance image sensor 12 is a sensor for acquiring a distance image of a shooting area. However, the imaging area of the distance image sensor 12 includes at least the determination area. The judgment area includes a dangerous area. A dangerous area is an area that is dangerous for humans to enter, and is, for example, an area set around a machine such as a machine tool or a robot. That is, the distance image sensor 12 is a sensor for acquiring a distance image of the dangerous area and its surrounding area.

This distance image sensor 12 irradiates light such as infrared light or laser and captures the light reflected from the object (reflected light) with an optical sensor such as a CCD (Charge Coupled Device) sensor. The distance image sensor 12 measures the actual distance to the object by measuring the time it takes for light to return for each pixel. As the distance image sensor 12 of the embodiment, a product called Xtion (registered trademark) manufactured by ASUS (registered trademark) is adopted. In other embodiments, the distance image sensor 12 may be a Kinect (registered trademark) sensor manufactured by Microsoft (registered trademark), a three-dimensional distance image sensor D-IMager (registered trademark) manufactured by Panasonic (registered trademark), or the like. It is also possible to do so. This type of sensor is sometimes referred to as a three-dimensional distance measurement sensor, a 3D scanner, or the like.

Note that as the distance image sensor, an omnidirectional laser distance meter such as LiDAR (for example, imaging unit LiDAR (HDL-32e) (trade name) manufactured by Velodine), a stereo camera, etc. can also be used.

The distance image signal from the distance image sensor 12 is input to the computer 14, and the computer 14 stores the distance image signal as digital data in a memory 16 such as a RAM. Although details will be described later, the computer 14 detects a person's intrusion into a dangerous area from the results of processing the distance image data stored in the memory 16. When human intrusion into the dangerous area is detected, the computer 14 displays a warning screen on the display 18 to the effect that human intrusion into the dangerous area has been detected, or outputs a warning sound from the speaker 24. issue warnings (alerts) such as sending warning messages to other devices such as administrator terminals.

Memory 16 includes a program storage area 20 and a data storage area 22, as shown in FIG.

In the program storage area 20, a map generation program 20a, a photographing direction setting program 20b, a data acquisition program 20c, a preprocessing program 20d, a lump recognition program 20e, an intrusion detection program 20f, and a warning program 20g are preset.

The map generation program 20a generates composite map data for a composite map that is a three-dimensional map corresponding to the determination area 30 (see FIG. 4) corresponding to the photographing area, and stores it in the composite map data area 22c of the data storage area 22. It is a program for memorizing. However, the determination area includes a dangerous area, a safe area where it is safe for people to enter, a shielded area that is blocked by an obstacle, and an open area that is free of obstacles and is not shielded (non-shielded area). include.

The shield area is an area that becomes a blind spot for the distance image sensor 12 due to an obstacle. The obstacle is typically a structure such as a machine included in the determination area. However, in the present invention, an area where a person operating a machine or the like is present (an area near the front of the operating section of the machine or the like) is also treated as an obstacle. This is because when detecting an intrusion, a blind spot of the distance image sensor 12 may occur (occupied by the worker) due to the body of the person operating the machine or the like.

The map generation program 20a also generates dangerous area map data for a dangerous area map that is a three-dimensional map corresponding to the determination area and includes a dangerous area and a safe area, and generates dangerous area map data in the data storage area 22. It is also a program (dangerous area map generation program) for storing in the area 22a.

Furthermore, the map generation program 20a generates shielded area map data for a shielded area map that is a three-dimensional map corresponding to the determination area and includes a shielded area and an open area, and generates shielded area map data in the data storage area 22. It is also a program (blocked area map generation program) for storing in the area 22b. Although the details will be described later, the composite map is generated by combining the dangerous area map and the shielded area map.

The photographing direction setting program 20b sets a photographing direction for intrusion detection based on the composite map, generates photographing direction data regarding the photographing direction for intrusion detection, and stores the photographing direction data area 22d in the data storage area 22. This is a program for storing data in However, the photographing direction for intrusion detection is set to the direction with the fewest blind spots when photographing the dangerous area, depending on the positional relationship of the dangerous area, safe area, shielded area, and open area included in the composite map.

The data acquisition program 20c captures the distance image signal from the distance image sensor 12 that has photographed the determination area from the shooting direction for intrusion detection, as distance image data (three-dimensional distance image data) regarding the entire distance image of the determination area. This is a program for storing in the acquired distance image data area 22e of the data storage area 22.

The preprocessing program 20d is a program for performing preprocessing such as noise removal on the distance image data stored in the acquired distance image data area 22e.

The lump recognition program 20e extracts distance image data (clump distance image data) regarding a lump distance image of a predetermined shape corresponding to a part of a person's body from the preprocessed distance image data, and extracts the data. This is a program for storing in the block distance image data area 22g of the storage area 22.

The intrusion detection program 20f generates a plurality of cross-sectional images from the mass distance image in a cutting direction perpendicular to the photographing direction for intrusion detection, and detects whether a person has entered the dangerous area based on the plurality of cross-sectional images and the dangerous area map. This is a program for detecting whether an intrusion has occurred and storing the detection result in the intrusion detection data area 22h of the data storage area 22.

The warning program 20g is a program for issuing a warning (alert) when a person enters a dangerous area.

The data storage area 22 includes the above-mentioned dangerous area map data area 22a, shielded area map data area 22b, composite map data area 22c, photographing direction data area 22d, acquired distance image data area 22e, lump distance image data area 22g, and intrusion area. In addition to the detection data area 22h, it includes a background distance image data area 22f. The background distance image data area 22f is an area for storing a distance image (background distance image) from the distance image sensor 12 that shows only the background with no people present in the photographing area, and this background distance image data is used as described below. It is used for preprocessing by the preprocessing program 20d.

FIG. 3 is a flowchart showing the processing contents of the shooting direction setting operation of the computer 14 in the safety control system 10 of this embodiment.

When the computer 14 starts the photographing direction setting operation, in step S1, the computer 14 generates shielding area map data for the shielding area map corresponding to the determination area 30 according to the map generation program 20a, and stores it in the shielding area map data area 22b. .

An example of the determination area 30 is shown in FIGS. 4 and 5. As shown in FIGS. 4 and 5, the determination area 30 is set to include the machine M operated by the person PSN, and a dangerous area 32 is set around the machine M. The dangerous area 32 is set according to the shape, function, operation, etc. of the machine M to include all areas that are dangerous if a person enters. For example, the dangerous area 32 is set along the external shape of the machine M.

In the embodiment shown in FIG. 4, the front end of the machine M is provided with space portions 34 and 36 in which no structures exist. The space portions 34 and 36 are formed to extend in the left-right direction around the height of the arms of the person PSN and around the feet of the person PSN. Further, the spatial portions 34 and 36 do not become blind spots depending on the photographing direction, and therefore are treated as the above-mentioned open areas. However, a space 34 around the height of the arm of the person PSN is included in the danger area 32, and a space 36 around the feet of the person PSN is not included in the danger area 32.

Further, in this embodiment, the shielding area map is expressed as a voxel map including a plurality of voxels. An example of an occluded area map is shown in FIG. As shown in FIG. 6, the occluded area map is expressed as a voxel map including 64 voxels of 4×4×4. That is, the shielding area map is a voxel map in which the determination area 30 is divided into 64 spaces and each space is expressed as a voxel.

Further, a danger area map and a composite map, which will be described later, are also expressed as voxel maps including 64 voxels, and in each of the shielding area map, danger area map, and composite map, voxels are arranged at positions corresponding to each other. Furthermore, voxels arranged at the same position in each of the shielded area map, the dangerous area map, and the composite map are associated with each other.

Information about either a shielded area or an open area is written in each voxel included in the shielded area map. However, the information about the shielding region includes information about the first type of shielding region (shielding region 1) and information about the second type of shielding region (shielding region 2). The first type of shielding area is an area where a person operating a machine or the like is present, and the second type of shielding area is an area where a structure such as a machine is present.

Next, in step S3, according to the map generation program 20a, dangerous area map data for the dangerous area map corresponding to the determination area 30 is generated and stored in the dangerous area map data area 22a. An example of a danger area map is shown in FIG. As shown in FIG. 7, information about either a safe area or a dangerous area is written in each voxel included in the dangerous area map.

Note that the shielding area map and the dangerous area map may be generated in advance based on distance images taken of the actual environment of the determination area 30 from a plurality of directions, for example, three axes, or may be generated based on a three-dimensional image of the environment of the determination area 30. It may be generated based on data virtually reproduced in .

Next, in step S5, according to the map generation program 20a, composite map data regarding a composite map obtained by combining the dangerous area map and the shielded area map is generated and stored in the composite map data area 22c. An example of a composite map is shown in FIG. The composite map shown in FIG. 8 is a map generated by combining the shielded area map shown in FIG. 6 and the dangerous area map shown in FIG. 7.

As shown in FIG. 8, in the composite map, voxels that are occluded areas on the occluded area map are treated as occluded areas, regardless of whether they are dangerous areas or safe areas on the dangerous area map. On the other hand, a voxel that is an open area on the shielded area map is treated as a dangerous area if it is a dangerous area on the dangerous area map, and is treated as an open area or a safe area if it is a safe area on the dangerous area map.

Next, in step S7, the photographing direction is set to the first direction. However, the first direction refers to any one of a plurality of directions that can be photographed directions for intrusion detection. The plurality of directions that can be photographed directions for intrusion detection include, for example, the front-rear direction, the left-right direction, the up-down direction, and other arbitrary directions.

Next, in step S9, in the composite map, from the first contacting voxel (reference voxel) when viewed from the shooting direction, the number of points that can reach the dangerous area without passing through the occlusion area (score from the reference voxel) is calculated. .

For example, as shown in FIG. 9(A), when the shooting direction is downward (in the direction of the arrow "b"), the 16 voxels located at the top end in the vertical direction are This is the reference voxel that is first in contact with . Furthermore, when the photographing direction is the front direction (the direction of the arrow "c"), the 16 voxels located at the rear end in the front-back direction become the reference voxels. Furthermore, when the photographing direction is rightward (direction of the arrow "d"), the 16 voxels located at the left end in the left-right direction become the reference voxels.

Then, the number of dangerous areas passed through from the reference voxel to the opposite side of the voxel map in the photographing direction is counted. However, if the object passes through a shielded area from the reference voxel to the other side of the voxel map, the number of dangerous areas thereafter is not counted. That is, only the number of dangerous areas passed before passing through the shielded area is counted. This is because the far side of the shielding area becomes a blind spot for the distance image sensor 12. In this way, the number of visible (photographable) dangerous areas is counted when viewed from the distance image sensor 12 (photographing direction).

For example, when the shooting direction is downward and the reference voxel touching the arrow "b" exits to the opposite side of the voxel map, as shown in FIG. 9(B), open areas, dangerous areas, and shielded areas , and the occluded areas in this order. In this case, since the number of dangerous regions passed through before passing through the shielding region is one, the score from the reference voxel touching the arrow "b" is "1".

In addition, when the shooting direction is the front direction and the voxel map exits from the reference voxel touching the arrow "c" to the opposite side of the voxel map, as shown in FIG. , and the occluded areas in this order. In this case, since the shielding area is passed through first, the number of dangerous areas after this is not counted, and the score from the reference voxel touching the arrow "c" is "0".

Furthermore, when the shooting direction is to the left and the voxel map exits from the reference voxel touching the arrow "d" to the opposite side of the voxel map, as shown in FIG. 9(D), a dangerous area, a dangerous area, a dangerous area , pass through the danger area in that order. In this case, since the shielded area is not passed through, the number of dangerous areas (4) passed through is counted as is. Therefore, the score from the reference voxel touching the arrow "d" is "4".

With the above procedure, scores from all the reference voxels that first come into contact when viewed from the shooting direction are calculated, and the scores from all the reference voxels are summed up to obtain the total score for that shooting direction. FIG. 10 is an illustrative diagram showing the total score for the photographing direction in the example of FIG. As shown in FIGS. 10(A) and 10(B), in the example shown in FIG. ”, as shown in FIG. 10(C), the total score from the top (the shooting direction is downward) is “4”, and as shown in FIG. 10(D), the total score from the right direction (the shooting direction is downward). (direction is leftward), the total score is "8".

Next, in step S11, the total score for the photographing direction is registered in the score list in association with the photographing direction, and the process proceeds to step S13. Note that the score list is a list for recording the total score of each of a plurality of directions that can be photographed directions for intrusion detection. This score list is stored in the photographing direction data area 22d.

Next, in step S13, it is determined whether there are any other directions that can be photographed directions for intrusion detection. Here, it is determined whether the total score of all possible photographing directions for intrusion detection has been recorded in the score list. If "YES" in step S13, that is, if it is determined that there is another photographing direction, the photographing direction is changed to another direction that can be the photographing direction for intrusion detection in step S15, and step S9 Return to

On the other hand, if "NO" in step S13, that is, if it is determined that there is no other direction that can be the shooting direction for intrusion detection, then in step S17, the direction with the largest total score for the shooting directions is determined. , the photographing direction is set as the photographing direction for intrusion detection, and the photographing direction setting operation is completed. For example, as shown in FIG. 10, in the embodiment of FIG. 4, the total score when viewed from the right direction, that is, the total score when the photographing direction is to the left, is the largest, so the photographing direction for intrusion detection is set to the left. Set.

In the example shown in FIGS. 11 and 12, as in the example shown in FIG. Set as the shooting direction for detection.

In the example shown in FIGS. 11 and 12, the dangerous area 32 is set around the machine M. However, the area around the machine M refers to an area within a predetermined distance from the machine M in the upward direction and in the front, rear, left and right directions.

FIG. 13 is an illustrative diagram showing the total score for the photographing direction in the example of FIG. As shown in FIG. 13(A), in the example shown in FIG. 11, the total score from the front direction (the shooting direction is the rear direction) is "12", and as shown in FIG. 13(B), the total score from the rear direction is "12". (The shooting direction is forward) The total score is "40," and as shown in FIG. 13(C), the total score from the top (the shooting direction is downward) is "42." As shown in 13(D), the total score from the right direction (photographing direction is left) is "30".

Therefore, in the embodiment of FIG. 11, since the total score when viewed from above, that is, the total score when the photographing direction is downward, is the largest, the photographing direction for intrusion detection is set downward.

As described above, the computer 14 has a function as a photographing direction setting device that cooperates with the memory 16 and the like to set a photographing direction for intrusion detection.

Then, after the photographing direction for intrusion detection is set, the computer 14 starts the main processing of the intrusion detection operation. When the computer 14 starts the main processing of the intrusion detection operation, in step S21, the computer 14 acquires the distance image signal from the distance image sensor 12 as distance image data and stores it in the acquired distance image data area 22e according to the data acquisition program 20c. do. However, the photographing direction of the distance image sensor 12 in the main processing of the intrusion detection operation is the photographing direction for intrusion detection.

Next, in steps S23, S25, and S27, the computer 14 performs preprocessing such as noise removal (step S23), distortion correction (step S25), and background information removal (step S27) on the distance image according to the preprocessing program 20d. do.

Since various noises are mixed in the measured values of the distance image sensor 12, this type of noise is removed using a filter such as smoothing or a median filter in step S23, if necessary. Specifically, outliers in the measured values are removed, smoothed (coarse quantization), objects that are too small or too thin are removed, and the results are acquired in the range image data area 22e or another area (not shown). z).

In the next step S25, distortion correction is performed as part of preprocessing. In the case of a distance image sensor such as an infrared laser that bends the optical axis with a lens or the like to measure a wide range, distortion aberration etc. occurs, so if necessary, this correction is performed in step S25 and the result is used as an acquired distance image. The information is stored in the data area 22e or another area (not shown).

In step S27, background information removal processing is executed. In order to detect only people and moving objects, the system learns the reflections of static objects and removes them from the measured values. Specifically, background distance image data stored in advance is read out from the background distance image data area 22f of the data storage area 22, and this background distance image data is subtracted from the distance image data being processed. At the same time, data outside the preset determination area is erased, and the result (distance image data for which preprocessing has been completed) is stored in the acquired distance image data area 22e.

In step S29, lump separation processing is executed. Here, from the distance image data that has been preprocessed, lumps similar to the shape of a person are selected and extracted as an image of a predetermined shape (clump distance image) including the lump, and each lump distance image is Separate into distance image data (clump distance image data). As a method for selecting clustered distance images, for example, considering the two-dimensional size of images showing objects included in distance images that have completed preprocessing, Each image has a predetermined length or less, the ratio of the area of a rectangle circumscribing the image to the area of the image is greater than a certain value, and the aspect ratio is less than or equal to a predetermined ratio. An image similar to the shape of is extracted as a cluster distance image. Therefore, images that are not similar to the shape of a person, such as images that are not rounded such as rectangular shapes, and images that are too large or too small, are not extracted as cluster distance images. That is, images that do not resemble the shape of a person are excluded from the targets of the intrusion detection operation.

Note that another method for selecting lump distance images is to generate a cross-sectional image of a predetermined height (for example, 1200 mm) instead of the distance image, and to select images included in the cross-sectional image that have human contours. An image similar to the shape of may be extracted as a cluster distance image.

Next, in step S31, it is determined whether a person exists in the connection area. Here, it is determined whether a person (clump distance image) exists within an area (connection area) up to a predetermined distance from the dangerous area when viewed from the shooting direction of the distance image sensor 12. For example, it is determined whether the center of the lump distance image is within a predetermined distance from the dangerous area.

If "NO" in step S31, that is, if it is determined that no person exists in the connection area, the process returns to step S21. On the other hand, if "YES" in step S31, that is, if it is determined that there is a person in the connection area, the process advances to step S33.

In the following step S33, intrusion detection processing is executed according to the subroutine shown in FIG. Briefly, intrusion detection processing generates multiple cross-sectional images from the cluster distance image data regarding the cluster distance image, and detects whether a person has entered a dangerous area based on the multiple cross-sectional images. This is the process to do so.

When the computer 14 starts the intrusion detection process, in a first step S41, the computer 14 generates a plurality of cross-sectional images cut in a cutting direction perpendicular to the photographing direction for intrusion detection from the lump distance image data regarding the lump distance image. do.

Next, in step S43, it is determined whether a person has entered the dangerous area. Here, images showing objects included in multiple cross-sectional images and dangerous area maps are superimposed (verified) to create an image showing objects included in multiple cross-sectional images and a dangerous area map included in the dangerous area map. Determine whether the areas overlap.

If "NO" in step S43, that is, if it is determined that no person has entered the dangerous area, the process proceeds to step S47, which will be described later. On the other hand, if "YES" in step S43, that is, if it is determined that a person has entered the dangerous area, a warning process is executed in step S45, and the process proceeds to step S47. In addition, in the warning process of step S45, a warning screen to the effect that a person has been detected to have entered the dangerous area may be displayed on the display 18, a warning sound may be output from the speaker 24, or other devices such as an administrator terminal may be displayed. A warning will be issued, such as a warning message being sent.

Subsequently, in step S47, it is determined whether or not to end the intrusion detection process. Here, it is determined whether there is no longer a person in the connection area. If "NO" in step S47, that is, if it is determined that the intrusion detection process is not to be completed, the process returns to step S41. On the other hand, if "YES" in step S47, that is, if it is determined that the intrusion detection process is to be ended, the process returns to the main process of the intrusion detection operation.

As described above, the computer 14 also has the function of an intrusion detection device that works with the distance image sensor 12, the memory 16, and the like to detect whether or not a person has entered a dangerous area.

As described above, according to the present embodiment, a plurality of cross-sectional images are generated by cutting in a cutting direction perpendicular to the photographing direction for intrusion detection, and it is determined whether a person has entered a dangerous area based on the plurality of cross-sectional images. Detect. However, the shooting direction for intrusion detection is determined in accordance with the positional relationship of the dangerous area, safe area, shielded area, and open area included in the composite map, which is a three-dimensional map. is set to In this way, it is possible to detect the intrusion of a person into a dangerous area by simple distance image processing.

Further, according to this embodiment, the three-dimensional map is expressed as a voxel map including a plurality of voxels, and each voxel is written with a dangerous area, a safe area, a shielded area, and an open area. In this way, the photographing direction for intrusion detection can be appropriately set.

Furthermore, according to this embodiment, in each of a plurality of directions for each voxel included in the three-dimensional map, a score corresponding to the number of people who can reach the dangerous area without passing through the shielding area is counted, and the score is Set the larger direction as the shooting direction. In this way, the photographing direction for intrusion detection can be appropriately set.

Furthermore, according to this embodiment, the three-dimensional map includes a dangerous area map including a dangerous area and a safe area, and a shielded area map including a shielded area and an open area, and the map generation unit is configured to , and the occluded area map to generate a three-dimensional map. In this way, the photographing direction for intrusion detection can be appropriately set.

<Second example>
The safety control system 10 of the second embodiment includes a plurality of distance image sensors 12, and among the plurality of distance image sensors 12, a distance image sensor for intrusion detection that can photograph a dangerous area from a direction with the least blind spots. The second embodiment is the same as the first embodiment except that whether or not a person has entered the dangerous area is detected based on the distance image output from the first embodiment. Hereinafter, the safety control system 10 of the second embodiment will be described, but content that overlaps with the content explained in the first embodiment will be omitted.

As shown in FIG. 16, the safety control system 10 of the second embodiment includes a plurality of distance image sensors 12. Each of the plurality of distance image sensors 12 is arranged so as to photograph the determination area 30 from different directions. In the example shown in FIG. 16, the safety control system 10 includes four distance image sensors 12a to 12d.

FIG. 17 is a flowchart showing the processing contents of the shooting direction setting operation of the computer 14 in the safety control system 10 of the second embodiment.

The photographing direction setting operation in the second embodiment will be explained below using a flowchart. Processes that are the same as the photographing direction setting operation explained in the first embodiment will be given the same reference numerals, and duplicate contents will be omitted. , the explanation will be omitted or explained simply.

When the computer 14 starts the photographing direction setting operation, in step S5, the computer 14 generates composite map data for the composite map and stores it in the composite map data area 22c, and in the subsequent step S61, in the composite map, multiple distance image sensors When viewed from any one of the 12 distance image sensors 12, a score is calculated from the reference voxel that comes into contact first. Then, the scores from all reference voxels are totaled to obtain the total score for that distance image sensor 12. Note that the score calculation method is the same as in the first embodiment, so detailed explanation will be omitted.

Next, in step S63, the total score for the distance image sensor 12 is registered in the score list in association with the distance image sensor 12, and the process proceeds to step S65.

Next, in step S65, it is determined whether there is another distance image sensor 12 whose score has not been calculated. Here, it is determined whether the total score for all distance image sensors 12 included in the safety control system 10 has been recorded in the score list. If "YES" in step S65, that is, if it is determined that there is another distance image sensor 12, the distance image sensor 12 is changed in step S67, and the process returns to step S61.

On the other hand, if "NO" in step S65, that is, if it is determined that there is no other distance image sensor 12 whose score has not been calculated, in step S69, the distance image sensor 12 with the largest total score is selected. It is set as the distance image sensor 12 for intrusion detection, and the photographing direction setting operation is completed.

In the second embodiment, the photographing direction of the distance image sensor 12 for intrusion detection (the center of the angle of view of the distance image sensor 12) is the photographing direction for intrusion detection. Further, in the main processing of the intrusion detection operation, processing is performed according to a distance image signal from the distance image sensor 12 for intrusion detection. However, the main processing of the intrusion detection operation and the contents of the intrusion detection processing in step S33 are the same as in the first embodiment, and therefore the description thereof will be omitted.

As described above, according to the second embodiment, among the plurality of distance image sensors 12, the output is output from the distance image sensor for intrusion detection, which is capable of photographing the dangerous area from the direction with the least blind spots. Based on the distance images captured, it is detected whether a person has entered a dangerous area. In this way, similar to the first embodiment, it is possible to detect a person's intrusion into a dangerous area by simple distance image processing.

In addition, in the second embodiment, since a plurality of distance image sensors 12 are provided, if a shielded area map, a dangerous area map, and a composite map are not created, the distance images taken from each distance image sensor 12 are used. It is possible to create a shielded area map, a dangerous area map, and a composite map.

Note that the number of distance image sensors 12 and the arrangement of each distance image sensor 12 mentioned in the second embodiment are merely examples, and can be changed as necessary.

In the above embodiment, all steps in FIGS. 3, 14 (FIG. 15), and 17 are explained as being executed by one computer 14, but multiple computers may be used, or specific processing may be executed by one computer 14. The processing may be performed using a dedicated processing circuit such as a DSP instead of a computer.

Note that the specific numerical values such as the angle, distance, ratio, and change range listed above are merely examples, and can be changed as necessary.

Further, the number of voxels included in the voxel map mentioned above, the arrangement of each voxel, etc. are merely examples, and can be changed as necessary.

10...Safety control system 12...Distance image sensor 14...Computer 16...Memory 18...Display

Claims (5)

A three-dimensional map of the determination area, which includes dangerous areas where it is dangerous for people to enter, safe areas where it is safe for people to enter, shielded areas that are blocked by obstacles, and open areas where there are no obstacles. a map generation unit that generates a map, and a setting unit that sets a photographing direction for intrusion detection to a direction with the least blind spots when photographing the dangerous area based on the three-dimensional map,
The three-dimensional map includes a plurality of voxels, and one of the dangerous area, the safe area, the shielded area, and the open area is written in each voxel,
The setting unit counts the number of directions in which the dangerous area can be reached without passing through the shielding area in each of a plurality of directions for each voxel included in the three-dimensional map, and determines the direction in which the number is larger. A photographing direction setting device configured to set the photographing direction. The map generation unit generates a dangerous area map including the dangerous area and the safe area, and a shielded area map including the shielded area and the open area, and synthesizes the dangerous area map and the shielded area map. The photographing direction setting device according to claim 1 , wherein the photographing direction setting device generates the three-dimensional map. The computer of the shooting direction setting device,
A three-dimensional map of the determination area, which includes dangerous areas where it is dangerous for people to enter, safe areas where it is safe for people to enter, shielded areas that are blocked by obstacles, and open areas where there are no obstacles. a map generation unit that generates a map, and a setting unit that sets a photographing direction for intrusion detection to a direction with the least blind spots when photographing the dangerous area based on the three-dimensional map;
The three-dimensional map includes a plurality of voxels, and one of the dangerous area, the safe area, the shielded area, and the open area is written in each voxel,
The setting unit counts the number of directions in which the dangerous area can be reached without passing through the shielding area in each of a plurality of directions for each voxel included in the three-dimensional map, and determines the direction in which the number is larger. A photographing direction setting program for setting the photographing direction. (a) 3 regarding determination areas including dangerous areas where it is dangerous for people to enter, safe areas where it is safe for people to enter, shielded areas that are blocked by obstacles, and open areas where there are no obstacles. a map generation step of generating a dimensional map;
(b) a setting step of setting a photographing direction for intrusion detection to a direction with the least blind spots when photographing the dangerous area, based on the three-dimensional map generated in step (a);
In the step (a), a three-dimensional map is generated that includes a plurality of voxels, and one of the dangerous area, the safe area, the shielded area, and the open area is written in each voxel;
In the step (b), in each of a plurality of directions for each voxel included in the three-dimensional map, the number of people who can reach the dangerous area without passing through the shielding area is counted, and the direction in which the number is larger is counted. is set as the photographing direction. An intrusion detection method for an intrusion detection system comprising a plurality of distance image sensors that photograph a photographing area including a dangerous area that is dangerous if a person enters from different directions and output a distance image, the method comprising:
(a) Generate a three-dimensional map of a determination area that includes the dangerous area, a safe area where it is safe for people to enter, a shielded area that is blocked by obstacles, and an open area where no obstacles exist. a map generation step;
(b) Based on the three-dimensional map generated in step (a), a distance image for intrusion detection that can photograph the dangerous area from a direction with the least blind spots among the plurality of distance image sensors. a configuration step for configuring the sensor;
(c) Cut a plurality of cross-sectional images from the range images output from the range image sensor for intrusion detection in a cutting direction perpendicular to the photographing direction of the range image sensor for intrusion detection set in step (b). a step of generating a cross-sectional image;
(d) An intrusion detection method including an intrusion detection step of detecting whether a person has entered the dangerous area based on the plurality of cross-sectional images generated in the step (c).

Images