JP4668684B2 - Monitoring device and software program - Google Patents

Description

  The present invention relates to a monitoring device and a software program, and more particularly to a monitoring device and a software program capable of gradually adapting to monitoring conditions without repeating the same error when making an erroneous determination while operating the device. It is about.

2. Description of the Related Art Conventionally, a motion detection sensor has been proposed as a motion detection device that detects a motion of an object such as a person in a space such as a toilet, a bathroom, or a bed. As a typical example, a spot (bright spot) divided into a two-dimensional grid pattern is projected into a room such as a toilet, and the change in the coordinates of each spot of an image obtained by imaging the projected spot results in the indoor There is an apparatus for monitoring the position / posture state of a person at (see, for example, Patent Document 1).
JP-A-2005-005912 (Page 3-11, Figure 1-12)

  However, according to the conventional apparatus as described above, for example, when a posture that is not assumed in advance is taken with respect to a posture that should be determined to be in a dangerous state, Dangerous postures may vary depending on the space being monitored. Regarding such various monitoring conditions, it was impossible to cover all postures in advance.

  Therefore, an object of the present invention is to provide a monitoring device and a software program that can gradually adapt to the monitoring conditions without repeating the same error when making an erroneous determination while operating the device. .

In order to achieve the above object, a monitoring device according to a first aspect of the present invention is a monitoring device for monitoring an object 2 existing in a monitoring area 3 as shown in FIG. 5 or FIG. Measuring means 14 for obtaining information relating to the shape or movement; and based on the output obtained from the output layer R by inputting information relating to the shape or movement of the object 2 obtained by the measuring means 14 to the input layer S of the neural network. Determining means 23 for determining the state of the object 2; alarm means 41 for issuing an alarm when the determining means 23 determines that the object 2 is in a dangerous state; and an alarm releasing means 42 for releasing the alarm; comprising a; determination means 23, when the alarm by alarm release means 42 is released, configured to perform additional learning for changing the weighting factor W _i to associate the input layer S and the output layer R.

  With this configuration, the measurement unit acquires information on the shape or movement of the object, and the determination unit inputs the acquired information to the input layer of the neural network based on the output obtained from the output layer. Determine the state of the object. Further, the alarm means issues an alarm when the determination means determines that the object is in a dangerous state. Furthermore, the determination means performs additional learning to change the weighting coefficient that associates the input layer with the output layer when the alarm is canceled by the alarm cancellation means, so when an erroneous determination is made while operating the device Thus, it is possible to provide a monitoring apparatus that can gradually adapt to the monitoring conditions without repeating the same error.

Also, the monitoring device according to claim 1, for example, as shown in FIG. 5 or FIG. 17, the state of the object 2 to be determined by the determining means 23, a safe condition object 2 and unsafe conditions The determination means 23 is first information regarding the shape or movement of the object 2 obtained by the measurement means 14 set in advance, and the object 2 is determined to be in a safe state based on the information. First information that is not performed, and second information relating to the shape or movement of the object 2 obtained by the measuring means 14 for a certain period immediately before the released alarm is issued, and based on the information, the object 2 by using the second information is determined to hazardous conditions, it is configured to perform additional learning for changing the weighting factor W _i.

  If comprised in this way, a judgment means is the 1st information about the shape or movement of a preset object, and based on this information, the 1st information which does not judge a target as a safe state, Using the second information related to the shape or movement of the object for a certain period immediately before the released alarm is issued, and based on the second information that the object is determined to be in a dangerous state based on the information, Since additional learning for changing the weighting factor is performed, for example, information that has been erroneously determined to be in a dangerous state can be corrected to be determined to be in a safe state. Furthermore, it is possible to prevent the information that should not be determined to be absolutely safe from being determined to be safe. Therefore, a more accurate state determination can be performed.

Further, as described in claim 2 , in the monitoring device described in claim 1, when the determination unit 23 determines the state of the object, for example, as shown in FIG. Information regarding the shape or movement of the object 2 obtained by the measuring means 14 for a certain period is temporarily stored, and when the warning means 41 issues a warning, the storage means 31 is maintained so as to keep the stored information. You may comprise.

  With this configuration, when the determination unit determines the state of the object, the storage unit temporarily stores information on the shape or movement of the object obtained by the measurement unit for a certain period immediately before the determination. To do. Furthermore, since the stored information is continuously held when the warning means issues a warning, the information that has been held can be used as the second information.

Further, as described in claim 3 , in the monitoring device according to claim 1 or 2 , the projection device that projects pattern light for forming a pattern on the monitoring region 3, as shown in FIG. 5 or FIG. 17, for example. 11; and an imaging device 12 that images the monitoring area on which the target object is projected, on which the pattern light is projected; and the measuring means 14 is configured to measure the movement of the pattern on the captured image, and You may comprise so that the information regarding the shape or movement of the thing 2 may be the information regarding the movement of the measured pattern.

  With this configuration, the projection device projects pattern light that forms a pattern on the monitoring region, and the imaging device images the monitoring region on which the object exists, on which the pattern light is projected. Since the measurement means measures the movement of the pattern on the imaged image, the three-dimensional position of the pattern light projected onto the object can be calculated based on the movement of the measured pattern, Based on the three-dimensional position, information on the movement of the pattern measured as information on the shape or movement of the object can be calculated. Therefore, it is possible to obtain more accurate information on the shape or movement of the object.

In order to achieve the above object, a software program according to a fourth aspect of the present invention is a software program that is installed in a computer and operates as a monitoring device, for example, as shown in FIG. 5 or FIG. In the process of monitoring the object 2 existing in the monitoring area 3; the process of obtaining information on the shape or movement of the object 2 (S114); and the information on the shape or movement of the obtained object 2 in the neural network Processing for determining the state of the object 2 based on the output obtained from the output layer R (for example, see FIG. 17) by inputting to the input layer S (for example, see FIG. 17) (S116); When it is determined in (S116) that the object 2 is in a dangerous state, a process for issuing an alarm (S124); a process for canceling the alarm (S1) 6); when the alarm is canceled by the processing (S126) for releasing an alarm, and a process for performing additional learning for changing the weighting factor W _i to associate the input layer S and the output layer R (S132) The state of the object 2 determined by controlling the computer as described above includes a state in which the object 2 is dangerous and a safe state, and the process of performing additional learning (S132) is a process of obtaining preset information. The first information regarding the shape or movement of the object 2 obtained in (S114), the first information that the object 2 is not determined to be in a safe state based on the information, and the released alarm The second information related to the shape or movement of the object 2 obtained by the process (S114) for obtaining information for a certain period immediately before being emitted, and the object 2 is determined to be in a dangerous state based on the information. With the information of 2 Configured to change only coefficient W _i.

  If comprised in this way, the information regarding the shape or movement of a target object will be acquired, and the state of a target object will be judged based on the output obtained from an output layer by inputting the acquired information into the input layer of a neural network . Furthermore, an alarm is issued when it is determined that the object is in a dangerous state by the determination process. In addition, when the alarm is released, additional learning is performed to change the weighting coefficient that associates the input layer with the output layer, so that when making an incorrect decision while operating the device, the same error is gradually repeated without repeating. It is possible to provide a software program that can adapt to the monitoring conditions.

  As described above, according to the present invention, in the monitoring device that monitors the object existing in the monitoring region, the measurement unit that obtains information on the shape or movement of the object, and the shape or movement of the object obtained by the measurement unit When the determination means determines that the object is in a dangerous state based on the output obtained from the output layer by inputting information on the input layer of the neural network And an alarm means for issuing an alarm and an alarm canceling means for canceling the alarm, and when the alarm is canceled by the alarm canceling means, the determination means performs additional learning for changing a weighting coefficient for associating the input layer with the output layer. Therefore, when an erroneous determination is made while operating the apparatus, a monitoring apparatus capable of gradually adapting to the monitoring conditions without repeating the same error is provided. Door can be.

  Embodiments of the present invention will be described below with reference to the drawings. In addition, in each figure, the same code | symbol is attached | subjected to the mutually same or equivalent member, and the overlapping description is abbreviate | omitted. Further, the main mathematical formulas used in the present embodiment include the mathematical formulas (1) to (24). These mathematical expressions will be described sequentially. FIG. 19 is a diagram summarizing the formulas (1) to (24). In the following description, FIG. 19 will be referred to as appropriate.

  FIG. 1 is a schematic perspective view of an indoor monitoring apparatus 1 as a monitoring apparatus according to an embodiment of the present invention. The indoor monitoring apparatus 1 is configured to monitor an object existing in a monitoring area defined by a first plane and a second plane different from the first plane. In the present embodiment, the monitoring area is the toilet 3, and the indoor monitoring device 1 is typically configured to monitor the object by measuring the position of the object. In addition, the first plane and the second plane typically intersect, and more typically are orthogonal. Further, here, the first plane is a plane parallel to the floor surface 4, and the second plane is a plane parallel to the front wall surface 5a as a wall surface, and a plane parallel to the side wall surface 5c and the side wall surface 5d. The toilet 3 is a space defined by a surface parallel to the floor surface 4, a surface parallel to the front wall surface 5a, and a surface parallel to the side wall surface 5c and the side wall surface 5d. A plane parallel to the floor surface 4 may coincide with the floor surface 4. A surface parallel to the front wall surface 5a may also coincide with the front wall surface 5a. Similarly, surfaces parallel to the side wall surface 5c and the side wall surface 5d may also coincide with the side wall surface 5c and the side wall surface 5d. That is, the surface parallel to the floor surface 4 may be the floor surface 4, and the surface parallel to the front wall surface 5a may be the front wall surface 5a. In the present embodiment, the first plane is the floor surface 4 and the second plane is the front wall surface 5a. Furthermore, the surface parallel to the side wall surface 5c and the side wall surface 5d may be the side wall surface 5c and the side wall surface 5d. In the present embodiment, the first plane is the floor surface 4, and the second plane is the front wall surface 5a, the side wall surface 5c, and the side wall surface 5d. Here, “parallel” is a concept including being approximately parallel. Further, the parallel plane includes not only an actual plane but also a virtual plane. The space is a closed space typically having a ceiling 6 facing the floor 4 and a back wall 5b facing the front wall 5a, a side wall 5c, and a side wall 5d. The closed space forms a toilet 3. In addition, the object typically changes its posture in the toilet 3. In the present embodiment, the case where the object is the person 2 will be described.

  Here, the toilet 3 used in the present embodiment will be described. The toilet 3 has a floor surface 4 and four planes formed so as to be orthogonal to the floor surface 4 (typically arranged vertically). The plane is a front wall surface 5a (front side in the drawing), a back wall surface 5b facing the front wall surface 5a (back in the drawing), and two side wall surfaces 5c as second planes orthogonal to the front wall surface 5a and facing each other. (Right in the figure) and 5d (left in the figure). Furthermore, a ceiling 6 disposed approximately in parallel with the floor surface 4 is formed at the top of each wall surface. Further, a toilet 7 is placed on the floor surface 4. The toilet bowl 7 is placed near the back wall surface 5b of the floor surface 4. Further, a door (not shown) is formed on the front wall surface 5a, and the person 2 can enter the toilet 3 from the front wall surface 5a or leave the toilet 3 by using this door. In the present embodiment, with respect to the floor surface 4 as the first plane, the three planes of the front wall surface 5a, the side wall surface 5c, and the side wall surface 5d are the planes in the relationship of the second plane of the present invention. However, there may be an embodiment in which there is only one side. In the present embodiment, the back wall surface 5b is not the second plane on which the pattern light is projected by the projection device 11 because of the relationship such as the installation angle of the projection device 11 to be described later. The back wall surface 5b may also be a second plane on which pattern light is projected.

  The indoor monitoring device 1 projects pattern light that forms a pattern on the floor surface 4 as the first plane of the toilet 3 that is the monitoring area and the front wall surface 5a, the side wall surface 5c, and the side wall surface 5d as the second plane. And an imaging device 12 that images the toilet 3 on which the person 2 as an object is projected, on which the pattern light is projected. In addition, the indoor monitoring device 1 includes a calculation device 20. The arithmetic device 20 controls the entire indoor monitoring device 1. The projection device 11 and the imaging device 12 are disposed at positions facing the floor surface 4 and the front wall surface 5a. Furthermore, the projection device 11 and the imaging device 12 are arranged in the vicinity of the ceiling 6 and the back wall surface 5b. In the present embodiment, the projection device 11 and the imaging device 12 are arranged approximately on the back wall surface 5b. When the person 2 exists in the toilet 3, the pattern is also projected on the person 2. Furthermore, in the present embodiment, the projected pattern is formed by a plurality of bright spots 11b arranged in a substantially square lattice shape, which will be described later with reference to FIG.

  Furthermore, the indoor monitoring device 1 includes a measuring device 14 that obtains information on the shape or movement of the person 2 as an object. As will be described later, the measurement device 14 is typically configured to measure movement of a pattern on an image captured by the imaging device 12, where information on the shape or movement of the person 2 is the measurement This is information regarding the movement of the pattern. The projection device 11 and the imaging device 12 are electrically connected to the measurement device 14 and controlled by the measurement device 14. In the present embodiment, the measurement device 14 is configured integrally with the arithmetic device 20.

  First, installation of the projection device 11 and the imaging device 12 will be described. As described above, the projection device 11 and the imaging device 12 are disposed above the back wall surface 5b. The imaging device 12 is disposed at a distance d from the projection device 11. Note that the distance (the distance d) between the projection device 11 and the imaging device 12 is referred to as a baseline length. The baseline length is the distance between the projection device 11 and the imaging device 12 in the baseline direction of the triangulation method. In the present embodiment, the base line direction is a direction approximately parallel to the floor surface 4 and the front wall surface 5a.

  The projection device 11 and the imaging device 12 are installed with their optical axes inclined with respect to the vertical direction of the floor surface 4. That is, in the figure, the installation angle with respect to the floor surface 4 of the projection device 11 and the imaging device 12 is illustrated as being inclined at an angle θ with respect to the vertical direction of the floor surface 4. Further, the respective optical axes of the projection device 11 and the imaging device 12 are installed in directions substantially parallel to each other. In the present embodiment, θ≈30 °, but is appropriately determined according to the aspect ratio of the room (toilet 3), but may be 15 to 60 °. In the present embodiment, the projection device 11 is disposed so that the optical axis intersects the floor surface 4.

  In the present embodiment, the projection device 11 and the imaging device 12 are housed and installed in the housing 18. In this way, for example, the person 2 who uses the toilet 3 does not need to be aware of the installation of the imaging device (CCD camera). In addition, the projection device 11 and the imaging device 12 can be protected by the housing 18.

  The indoor monitoring device 1 is configured to measure the movement of the pattern on the image captured by the imaging device 12 by the measuring device 14. For example, as shown in FIG. 2, when the projection device 11 and the imaging device 12 are arranged at approximately the center of the ceiling 6 and their optical axes are oriented in the vertical direction of the floor surface 4, the object (here, The greater the movement of the person 2) in the height or height direction, the greater the amount of movement of the bright spots forming the pattern. For this reason, according to the concept described later with reference to FIG. 6, if the amount of movement of the bright spot is large, a phenomenon may occur in which the bright spot to be compared jumps over the reference position of the adjacent bright spot. In this case, it is determined that the light has moved from the adjacent bright spot, and the amount of movement of the bright spot to be measured is small. That is, the movement amount of the bright spot cannot be measured accurately. Furthermore, the distance to the person 2 may not be measured accurately. For example, if this phenomenon occurs in the head of the person 2 as in the case of FIG. 2, the height of the head cannot be measured accurately. In other words, the head becomes a blind spot. As a result, the presence of the person 2 cannot be measured accurately. Further, since the measurement range and the resolution at the time of measuring the movement amount of the pattern (bright spot) are contradictory, for example, if the measurement range is widened so that this jump does not occur, sufficient resolution cannot be obtained. The blind spot referred to here is, for example, a region where the movement of the pattern cannot be measured accurately. As a result, it refers to a region where the above bright spot jump occurs within the angle of view of the imaging device 12.

  For this reason, the indoor monitoring device 1 can significantly reduce the influence of the bright spot jumping by installing the projection device 11 and the imaging device 12 as described above with reference to FIG. For example, the case shown in FIG. 3 will be described. In this embodiment, a reference pattern is projected onto the floor surface 4 as the first plane and the front wall surface 5a, the side wall surface 5c (see FIG. 1), and the side wall surface 5d (see FIG. 1) as the second plane. Has been. The projection device 11 and the imaging device 12 are installed above the toilet bowl 7. For example, the distance from the projection device 11 and the imaging device 12 to the front wall surface 5a is smaller than the distance to the floor surface 4. Therefore, in the direction from the projection device 11 and the imaging device 12 toward the front wall surface 5a, the region where the bright spot jumps is small. That is, the blind spot becomes a small portion above the toilet bowl 7, and it is less likely that the person 2 squatting on the toilet bowl 7 or sitting on the toilet bowl 7 enters the blind spot. In other words, when the person 2 is standing, the bright spot is projected onto the body of the person 2 outside the range where the bright spot jumps over, so that accurate measurement can be performed (state (a)). . Further, even when the person 2 is sitting on the toilet bowl 7, the person 2 exists outside the range where the bright spot jump occurs, so that accurate measurement can be performed (state (b)). As described above, the indoor monitoring device 1 can determine the state of the person 2 described later stably and accurately by installing the projection device 11 and the imaging device 12 as described above with reference to FIG. .

  In addition, when the projection device 11 and the imaging device 12 are installed in this way, for example, when the person 2 stands at the center of the toilet 3 as compared with the case where the projection device 11 and the imaging device 12 are installed at the center of the ceiling 6 (see FIG. 2). The pattern (number of bright spots) projected on the person 2 increases dramatically, and the amount of information that can be acquired increases.

  With reference to the schematic perspective view of FIG. 4, the projection apparatus 11 suitable for the indoor monitoring apparatus 1 is demonstrated. Here, for the sake of explanation, a case will be described in which the plane that faces the imaging device 12 included in the monitoring area is the plane 102, and a laser beam L1 described later is projected perpendicularly to the plane 102. Note that to face directly means to face straight, for example. The projection apparatus 11 includes a light beam generation unit 105 serving as a light beam generation unit that generates a coherent light beam, and a fiber grating 120 (hereinafter simply referred to as a grating 120). The coherent light beam projected by the light beam generation unit 105 is typically a near infrared laser. The light beam generation unit 105 is configured to generate a parallel light beam. The light flux generation unit 105 is typically a semiconductor laser device including a collimator lens (not shown), and the generated parallel light flux is a laser light flux L1. The laser light beam L1 is a light beam having a substantially circular cross section. In addition, this substantially circular shape is a concept including a substantially elliptical shape. Here, the parallel light flux only needs to be substantially parallel, and includes a nearly parallel light flux.

  Further, here, the grating 120 is disposed in parallel to the plane 102 (perpendicular to the Z axis). A laser beam L1 is incident on the grating 120 in the Z-axis direction. Then, the laser beam L1 is collected in a plane having the lens effect by each optical fiber 121, and then spreads as a divergent wave, interferes, and interferes with a plurality of bright spot arrays on the plane 102 as a projection plane. A pattern 11a is projected. Note that the arrangement of the grating 120 in parallel with the plane 102 means, for example, that the plane including the axis of each optical fiber 121 of the FG element 122 constituting the grating 120 and the plane 102 are parallel.

  The grating 120 includes two FG elements 122. In the present embodiment, the planes of the FG elements 122 are parallel to each other. Hereinafter, the plane of each FG element 122 is referred to as an element plane. In the present embodiment, the axes of the optical fibers 121 of the two FG elements 122 are substantially orthogonal to each other.

  The FG element 122 is configured by arranging, for example, several tens to several hundreds of optical fibers 121 having a diameter of several tens of microns and a length of about 10 mm in parallel in a sheet shape. Further, the two FG elements 122 may be arranged in contact with each other, or may be arranged at a distance from each other in the normal direction of the element plane. In this case, the distance between the two FG elements 122 is set so as not to interfere with the projection of the pattern 11a. The laser beam L1 is typically incident perpendicular to the element plane of the grating 120.

  Thus, since the grating 120 configured to include the two FG elements 122 serves as an optical system, the optical housing can be downsized without requiring a complicated optical system. Furthermore, by using the grating 120, the projection device 11 can project the bright spots 11b as a plurality of pattern lights as a pattern 11a onto the monitoring area with a simple configuration. The pattern 11a is typically a plurality of bright spots 11b arranged in a square lattice pattern. Further, when the optical axis of the light beam incident on the FG element 122 is perpendicular to the plane 102, the pattern 11a becomes a plurality of bright spots 11b arranged in a square lattice pattern near the optical axis, but as the distance from the optical axis increases. A slight deviation occurs in the lattice spacing. The bright spot has a substantially circular shape including an ellipse.

  Returning to FIG. The imaging device 12 is typically a CCD camera. The imaging device 12 includes an imaging optical system 13a (see FIG. 6) and an imaging element 15 (see FIG. 6). The image sensor 15 is typically a CCD image sensor. In addition to the CCD, an element having a CMOS structure has recently been actively announced as the image pickup element 15, and these can naturally be used. In particular, some of the elements themselves have inter-frame difference calculation and binarization functions, and it is preferable to use these elements.

  The imaging device 12 may include a filter 13b (see FIG. 6) that attenuates light having a wavelength other than the peripheral portion of the wavelength of the laser light beam L1 generated by the light beam generation unit 105 (see FIG. 4). The filter 13b is typically an optical filter such as an interference filter, and may be disposed on the optical axis of the imaging optical system 13a. In this way, the imaging device 12 can reduce the influence of disturbance light because the light intensity of the pattern 11a projected from the projection device 11 out of the light received by the imaging device 15 is relatively increased. The laser beam L1 generated by the beam generator 105 is typically an infrared laser beam. Further, the laser beam L1 may be irradiated continuously or intermittently. When irradiating intermittently, imaging by the imaging device 12 is performed in synchronization with the timing of irradiation.

  FIG. 5 is a block diagram showing a configuration example of the indoor monitoring apparatus 1 according to the embodiment of the present invention. With reference to this figure, the structural example of the indoor monitoring apparatus 1 is demonstrated. As described above, the arithmetic device 20 is configured integrally with the measuring device 14. Further, here, the measuring device 14 is configured integrally with a control unit 21 described later. The projection device 11 and the imaging device 12 are electrically connected to the measurement device 14 and controlled. In the present embodiment, the arithmetic device 20 is remotely arranged with respect to the projection device 11 and the imaging device 12. Specifically, for example, it is installed in a room different from the toilet 3. The arithmetic device 20 is typically a computer such as a personal computer. In the present embodiment, the calculation device 20 is described as being installed separately from the projection device 11 and the imaging device 12, but may be installed integrally with either or both of the projection device 11 and the imaging device 12. For example, the projector 11 and the imaging device 12 may be housed and installed in the housing 18 in which the projector 11 and the imaging device 12 are housed.

  As described above, the arithmetic device 20 includes the control unit 21 that controls the indoor monitoring device 1. Further, a storage unit 31 as a storage unit is connected to the control unit 21. The storage unit 31 may store the images acquired from the imaging device 12 in time series. The storage unit 31 can store data such as calculated information. Note that the storage unit 31 is not an image itself, but a bright spot (centroid) position in the image (a reference position when no person is present and a bright spot position as data at each time point) or a space calculated from the bright spot position. You may comprise so that the three-dimensional position of inside may be memorize | stored. In this case, the memory can be saved rather than storing the image itself.

  The control unit 21 obtains information related to the movement of the measured pattern as information related to the shape or movement of the object, and information related to the shape or movement of the object obtained by the measurement apparatus 14. Determination of determining the state of the person 2 based on the output obtained from the output layer R (see FIG. 17) by inputting information on the measured movement of the pattern to the input layer S (see FIG. 17) of the neural network And a state determination unit 23 as means. Here, in other words, the state determination unit 23 is means for determining the state of the person 2 based on the movement of the pattern measured by the measurement device 14. Further, in the present embodiment, the state determination unit 23 blocks information relating to movement of the measured pattern as information relating to the shape or movement of the object obtained by the measuring device 14, and inputs the input layer of the neural network. S (see FIG. 17) is input.

Furthermore, the control unit 21 includes an alarm signal unit 28 that transmits an alarm signal to an alarm device 41 as an alarm unit that issues an alarm when the state determination unit 23 determines that the person 2 is in a dangerous state. Is done. In addition, the control unit 21 includes an alarm release button 42 as an alarm release unit for releasing an alarm issued by the alarm device 41 via the alarm signal unit 28, and a state determination unit 23 in a state where the person 2 is in danger. When it is determined that there is not, an emergency call button 43 for causing the alarm device 41 to issue an alarm with the intention of the person 2 for some reason is connected. When the alarm is canceled by the alarm cancel button 42, the state determination unit 23 weights each connection as a weighting coefficient that associates the input layer S (see FIG. 17) and the output layer R (see FIG. 17) of the neural network. configured to perform the additional learning to change the W _i.

  Furthermore, in the present embodiment, when the state determination unit 23 determines the state of the person 2, the storage unit 31 described above has a measured pattern obtained by the measurement device 14 for a certain period immediately before the determination. Information regarding movement, or information regarding the position or movement of an object is temporarily stored, and when the alarm device 41 issues an alarm, the information stored temporarily is kept.

    Further, the control unit 21 projects light on a virtual plane 8 (see FIG. 9) that is assumed to be sufficiently wide compared to the projection range of the pattern light 11b, and is based on the optical arrangement of the projection device 11 and the imaging device 12. The position at which the measurement device 14 measures the movement of the pattern on the image captured by the imaging device 12 by comparing the position of the virtual pattern light calculated in this way with the position of the pattern light actually captured. And a reference position calculation unit 24 as reference position calculation means for calculating a reference position on the wall surface 5a (see FIG. 1), the side wall surface 5c (see FIG. 1), and the side wall surface 5d (see FIG. 1). Has been.

  The control unit 21 captures the pattern light 11b on the optical axis of the projection device 11 on the floor surface 4 (see FIG. 1), which is calculated based on the optical arrangement of the projection device 11 and the imaging device 12. The pattern light 11b that is actually projected on the floor surface 4 (see FIG. 1) and picked up by the image pickup device 12, and is the position of the projection device 11 on the floor surface 4 (see FIG. 1). An optical axis angle correcting unit 25 as optical axis angle correcting means for correcting the inclination of the optical axis of the projection device 11 based on the position of the pattern light 11b closest to the position where the pattern light 11b on the optical axis is to be imaged; A virtual pattern formed by performing a Hough transform on an actual pattern on the floor surface 4 to calculate a first angle peak and forming a certain range of virtual pattern light on the virtual plane 8 (see FIG. 9). Perform a Hough transform on the projection The second angle peak is calculated every time the projector 11 is rotated by a certain angle around the optical axis of the projection device 11, and the rotation angle around the optical axis when the first angle peak and the second angle peak substantially coincide with each other. Is configured to include a rotation angle calculation unit 26 as a rotation angle calculation unit that sets the actual rotation angle of the projection device 11. Furthermore, the control unit 21 responds to the frequency histogram regarding the position in the direction perpendicular to the front wall surface 5a (see FIG. 1), the side wall surface 5c (see FIG. 1), and the side wall surface 5d (see FIG. 1) at each reference position. It includes a wall surface specifying unit 27 as second plane specifying means for specifying the positions of the front wall surface 5a (see FIG. 1), the side wall surface 5c (see FIG. 1), and the side wall surface 5d (see FIG. 1).

Connected to the control unit 21 are a display 40 as information output means for outputting the determination result of the state determination unit 23 and an input device 35 for inputting information for operating the room monitoring device 1. The display 40 is typically an LCD. The input device 35 is, for example, a touch panel, a keyboard, or a mouse. In this figure, the input device 35 is illustrated as being externally attached to the arithmetic device 20, but may be built in. Hereinafter, each of the above-described configurations will be described in detail.

  First, the measuring device 14 will be described. The measuring device 14 measures the movement of the pattern on the image picked up by the image pickup device 12. The measuring device 14 is configured to acquire an image captured by the imaging device 12. Note that the measurement device 14 acquires an image captured by the imaging device 12 as data, for example, more typically as digital data. Further, the measuring device 14 is configured to measure the movement of each bright spot on the image picked up by the image pickup device 12. Here, the projected bright spot and the image of the bright spot on the captured image are simply referred to as bright spot for convenience. Here, measuring the movement of the bright spot means measuring the amount of movement of the bright spot (hereinafter referred to as the movement amount). Further, the measured moving amount of the bright spot is a concept including the moving direction of the bright spot. That is, it is assumed that the moving amount of the bright spot to be measured includes information on the moving direction. Further, here, the measuring device 14 is configured to calculate the three-dimensional coordinates of each bright spot forming the pattern projected on the person 2 based on the movement of the measured pattern.

  Furthermore, when the person 2 does not exist in the toilet 3, the measuring device 14 has a floor surface 4 (see FIG. 1), a front wall surface 5a (see FIG. 1), a side wall surface 5c (see FIG. 1), and a side wall surface 5d (FIG. 1). The movement of each bright spot forming the pattern is measured with reference to the bright spot as the pattern light projected on the reference).

  Furthermore, the measuring device 14 determines each position and installation angle of the projection device 11 and the imaging device 12 and each position of the floor surface 4, the front wall surface 5 a, the side wall surface 5 c, and the side wall surface 5 d based on the movement of the measured pattern. The three-dimensional coordinates as the three-dimensional position of the pattern light projected on the person 2 according to the above are calculated.

  Here, the measurement of the movement of the bright spot (pattern) by the measuring device 14 will be described in detail. The measuring device 14 is configured to measure the movement of the bright spot based on two different time points acquired from the imaging device 12. Two different time points are an arbitrary time point (current) and a time point when the person 2 does not exist in the toilet 3. Hereinafter, an image acquired at an arbitrary time (current) will be described as an acquired image, and an image acquired when no person 2 is present in the toilet 3 will be described as a reference image. That is, the movement of the bright spot (pattern) is measured with reference to the reference image. The reference image is stored in the storage unit 31.

  The acquisition interval of acquired images may be appropriately determined depending on, for example, the processing speed of the apparatus and the content of movement to be detected as described above, but is, for example, 0.1 to 3 seconds, preferably 0.1 to 0.5 seconds. It is good to have a degree. Here, it is set to 0.1 to 0.25 seconds. In addition, it is effective to acquire images at shorter time intervals and perform averaging or filtering to reduce the influence of random noise, for example.

  Here, the acquired image and the reference image are images picked up by the image pickup device 12, for example, and are concepts including the position information of the bright spot on each image. That is, the acquired image and the reference image are images of the pattern 11a (see FIG. 6) formed by the projection of the projection device 11 at each time point. In the present embodiment, the reference image is stored in the storage unit 31 in the form of positional information such as coordinates regarding the position of each bright spot, for example, not as a so-called image. In this way, when measuring the movement amount of the bright spot, which will be described later, for example, it is only necessary to compare the coordinates and direction of the bright spot, so the processing becomes simple. Further, here, the position of the bright spot is the barycentric position of the bright spot. By doing in this way, the movement of a slight bright spot can also be measured.

  Further, the movement amount of the bright spot is obtained by comparing the position information of each bright spot on the reference image stored in the storage unit 31 with the position information of each bright spot on the acquired image. Measure. Each amount of movement can be obtained, for example, by counting the number of pixels to which the position of the bright spot has moved (how many pixels have moved). In this way, as will be described later, it is not necessary to generate a difference image, so that the processing can be simplified.

  In the above description, the position information of the bright spots is compared. However, a difference image between the reference image and the acquired image may be created. In this case, the movement amount of the bright spot is measured based on the position of the corresponding bright spot from the difference image. In this way, since only the moved bright spot remains on the difference image, the processing amount can be reduced.

  Furthermore, the moving amount of the bright spot measured by the measuring device 14 may be a moving average value or a period average value of the moving amount of the bright spot that has been measured a certain number of times in the past or measured within the past fixed period. By doing so, it is possible to reduce random noise and sudden noise caused by sunlight flickering through a window, and the reliability of the measured moving amount of bright spots is improved.

  The projection device 11 and the imaging device 12 may be installed with a certain distance. By doing so, the distance d (base line length d, see FIG. 13) becomes longer, so that the movement of the bright spot can be detected sensitively (detection sensitivity is improved). The base line length is preferably long, but may be short. However, in this case, it is difficult to detect a small movement, but if a barycentric position of a bright spot is detected, a small movement can be detected. When the base line length is shortened, the blind spot area described with reference to FIGS. 2 and 3 becomes small.

  The measuring device 14 is configured to measure the movement of the bright spot as described above for each bright spot 11b (see FIG. 6) forming the pattern 11a (see FIG. 6). That is, the positions of the plurality of bright spots 11b become a plurality of measurement points.

  FIG. 6 is a conceptual perspective view for explaining the concept of bright spot movement in the embodiment of the present invention. Here, the concept of bright spot movement will be described with reference to FIG. Here, it is easy to understand, and the target area will be described as the plane 102 and the target object as the object 103. Further, here, for the sake of explanation, the case where the above-described reference image and acquired image are used will be described. Note that the reference image is an image of the pattern 11 a when the object 103 is not present on the plane 102, and the acquired image is described as the pattern 11 a when the object 103 is present on the plane 102.

  In the figure, an object 103 is placed on the plane 102. Further, the orthogonal coordinate system XYZ is taken so that the XY axis is placed in the plane 102, and the object 103 is placed in the first quadrant of the XY coordinate system. On the other hand, a projection device 11 and an imaging device 12 are arranged above the plane 102 on the Z axis in the drawing. The imaging device 12 images the plane 102 on which the pattern 11 a is projected by the projection device 11. In other words, the object 103 placed on the plane 102 is imaged.

  Here, the imaging lens 13a as the imaging optical system of the imaging device 12 is disposed so that its optical axis coincides with the Z-axis. Here, the origin of the coordinate system XYZ is the position of the imaging lens 13a. The imaging lens 13 a forms an image of the pattern 11 a on the plane 102 or the object 103 on the imaging surface 15 ′ (image plane) of the imaging device 15 of the imaging device 12. The imaging plane 15 'is disposed at a position of 1 (el) from the imaging lens 13a (substantially equal to the focal point of the imaging lens 13a). The image plane 15 'is typically a plane orthogonal to the Z axis. Further, an orthogonal coordinate system uv is taken in the image plane 15 'so that the Z axis passes through the origin (0, 0, l) of the uv coordinate system. The projection device 11 is disposed at a distance equidistant from the imaging lens 13a from the plane 102 and a distance d (base line length d) from the imaging lens 13a in the negative direction of the Y axis. That is, the coordinates at which the projection device 11 is arranged are (0, −d, 0). A pattern 11 a formed by a plurality of bright spots 11 b is projected onto the object 103 and the plane 102 by the projection device 11. The v-axis direction is also the baseline direction of the triangulation method.

  The pattern 11 a projected onto the plane 102 by the projection device 11 is blocked by the object 103 and does not reach the plane 102 in a portion where the object 103 exists. Here, if the object 103 exists, the bright spot 11 b to be projected onto the point 102 a on the plane 102 is projected onto the point 103 a on the object 103. Since the bright spot 11b has moved from the point 102a to the point 103a and the imaging lens 13a and the projection device 11 are separated from each other by a distance d (baseline length d), the point 102a is formed on the imaging plane 15 ′. An image to be imaged at '(u, v) is imaged at a point 103a' (u, v + δ). That is, when the object 103 does not exist and when the object 103 exists, the image of the bright spot 11b moves by δ in the v-axis direction.

  For example, as shown in FIG. 7, the bright spot imaged on the imaging surface 15 ′ of the image sensor 15 moves in the v-axis direction by δ by the object 103 having a height.

  Thus, by measuring the movement amount δ of the bright spot, the position of the point 103a on the object 103 can be specified three-dimensionally. That is, for example, the three-dimensional coordinates of the point 103a are known. In this way, by measuring the difference between a point that should be imaged on the imaging plane 15 ′ if the object 103 is not present and the actual imaging position on the imaging plane 15 ′, The three-dimensional coordinates of the object 103 can be measured. Alternatively, the distance from the imaging lens 13a to each point on the object 103 can be measured. Further, if the pitch of the pattern 11a, that is, the pitch of the bright spot 11b is made so fine that the correspondence relationship of the bright spot 11b is not unknown, the three-dimensional coordinates of the object 103 can be measured in detail.

  In this embodiment, when an image of 640 × 480 pixels is acquired, when a luminescent spot exists within the range of 50 pixels vertically and 1 pixel horizontally from the position of the bright spot (center of gravity position). The movement amount δ is measured. If the movement amount δ is 1 pixel or more, the amount of data to be processed may be reduced by determining that the bright spot has moved.

Further, returning to FIG. 6, the calculation of the three-dimensional coordinates will be described. The three-dimensional coordinates (X, Y, Z) can be calculated by equations (1), (2), and (3) shown in FIG. 19 where h is the distance from the imaging lens 13a to the plane 102.
By calculating the movement amount δ as described above, the three-dimensional coordinates of the bright spot projected on the person 2 can be calculated, and the three-dimensional shape information of the object can be obtained.

  Here, calculation of the three-dimensional coordinates of each bright spot forming the pattern of the reference image in the toilet 3 will be described. As described above, here, the reference image is position information including coordinates relating to the position of each bright spot when the person 2 does not exist in the toilet 3 (hereinafter referred to as “vacant”).

  FIG. 8 shows an image captured by the imaging device 12 when the person 2 is not present on the boundary between the floor surface 4, the front wall surface 5a, the side wall surface 5c, and the side wall surface 5d of the toilet 3 and the toilet wall 3 according to the embodiment of the present invention. It is a schematic plan view which shows an example. Here, the projection device 11 and the imaging device 12 are actually installed with the optical axis inclined at an angle θ with respect to the vertical direction of the floor surface 4 as the installation angle (see FIG. 1). Therefore, the pattern 11a projected from the projection device 11 (see FIG. 1) is not projected onto a simple plane (for example, the plane 102 shown in FIG. 6) as described in FIG. That is, as shown in this figure, the projection is projected onto a plane in which the floor surface 4, the front wall surface 5a, the side wall surface 5c, and the side wall surface 5d are combined. Since the movement of the bright spot (pattern) is measured with reference to the reference image as described above, it is necessary to calculate the position of the reference image on each plane.

  Therefore, the reference position calculation unit 24 projects light on a virtual plane 8 (see FIG. 9) that is assumed to be sufficiently wide compared to the projection range of the actual pattern light 11b, and the projection device 11 and the imaging device 12 The position of the virtual pattern light 11b calculated based on the optical arrangement is compared with the position of the pattern light 11b that is actually picked up, and the pattern on the image picked up by the measuring device 12 by the measuring device 14 is moved. Is configured to calculate a reference position on the front wall surface 5a (see FIG. 1), the side wall surface 5c (see FIG. 1), and the side wall surface 5d (see FIG. 1), that is, a reference image. .

  FIG. 9 is a diagram illustrating an example of a pattern 11a projected by the projection device 11 according to the embodiment of the present invention. FIG. 9A is a schematic perspective view when projected onto the toilet 3, FIG. FIG. 4 is a schematic perspective view when projected onto a virtual plane 8. In (a), the ceiling 6, the front wall surface 5a, and the side wall surface 5d are illustrated by imaginary lines in order to make it easy to see the inside of the toilet 3 room.

  Here, the virtual plane 8 that is assumed to be sufficiently wide compared to the projection range of the pattern light 11b is, for example, a pattern 11a projected from the projection device 11 like the plane 102 described in FIG. Is an ideal plane that is projected without being blocked by another plane that intersects the virtual plane 8. Here, the virtual plane may be a plane that is wide enough to calculate the position of the reference image by the reference position calculation unit 24 described below.

  As shown in FIG. 9B, the virtual plane 8 is a plane substantially parallel to the floor surface 4 in the present embodiment, and the distance from the projection device 11 is the distance between the floor surface 4 and the projection device 11. It is a plane at a position substantially equal to. Further, the projection device 11 and the imaging device 12 are installed with their optical axes inclined by an angle θ with respect to the vertical direction of the floor surface 4 as in the case of installing in the toilet 3 room. As virtual. In other words, in the present embodiment, the virtual plane 8 is the floor surface 4 in a state where the front wall surface 5a, the side wall surface 5c, the side wall surface 5d, the toilet bowl 7 and the like are removed from the toilet 3, and is expanded to a sufficiently wide area. What is necessary is just to assume the floor 4 made. The position on the imaging surface 15 ′ (see FIG. 6) of the bright spot 11b as the virtual pattern light that forms the virtual pattern 11a projected on the virtual plane 8 is the position between the projection device 11 and the imaging device 12. It can be calculated based on the optical arrangement.

Here, the optical arrangement of the projection device 11 and the imaging device 12 is at least the installation height relative to the floor surface 4 with respect to the projection device 11 or the imaging device 12, here, from the imaging lens 13a (see FIG. 6) to the floor surface 4. Is determined based on the installation height h and the optical axis angle θ as the installation angle described above. Further, the optical arrangement typically includes a numerical value predetermined in design in addition to the installation height h and the optical axis angle θ, that is, the base line length d and the imaging plane 15 ′ of the imaging device 12 (see FIG. 6) and the imaging lens 13a (see FIG. 6), the distance l (≈imaging lens focal length, see FIG. 6), the laser wavelength λ of the projection device 11, the fiber diameters r ₁ and r ₂ , and the laser beam is FG It is determined by various values such as output angles φ _X and φ _Y when passing through the element 122 (see FIG. 4). Therefore, calculating based on the optical arrangement of the projection device 11 and the imaging device 12 typically means calculating based on these various numerical values that determine the optical arrangement. When calculating based on the optical arrangement of the projection device 11 and the imaging device 12, not only the calculation using all the various numerical values that determine the optical arrangement, but also at least the installation height of the various numerical values. This includes the case where calculation is performed using several numerical values including h and the optical axis angle θ. That is, it can be said that the calculation is based on the optical arrangement of the projection device 11 and the imaging device 12 by calculating at least the installation height h and the optical axis angle θ among the numerical values that determine the optical arrangement.

Here, the projection angle of the bright spot 11b output from the FG element 122 (see FIG. 4) depends on the fiber diameters r ₁ and r ₂ of the FG element 122 (see FIG. 4) and the laser wavelength λ. When the laser light passes through the FG element 122, the bright spot 11b spreads as diffracted light, so that the reference position calculation unit 24 outputs the output angles φ _X and φ _Y when the laser light passes through the FG element 122 (see FIG. 4). Can be approximated and calculated by Equations (4) and (5) shown in FIG. Here, n and m are positive numbers and are indices of the bright spot 11b.

Further, the reference position calculation unit 24 uses the formulas (6) and (7) shown in FIG. 19 based on the installation height h and the optical axis angle θ to create virtual luminescent spots corresponding to the indices m and n of the luminescent spot 11b. The position coordinates u _{ideal, m, n} , v _{ideal, m, n} on the imaging plane 15 ′ (see FIG. 6) according to the three-dimensional coordinates of 11b can be calculated. The reference position calculation unit 24 actually uses the position coordinates on the image plane 15 ′ of the virtual bright spot 11b group calculated as described above as a reference, as shown in FIG. , Position coordinates u _{real, m,} b on the imaging surface 15 ′ (see FIG. 6) of the bright spot 11b actually picked up by the front wall surface 5a, the side wall surface 5c, and the bright spot 11b projected on the side wall surface 5. The virtual movement amount δ of each bright spot 11b at the time of vacancy is calculated from _n, v _{real, m, n} . The reference position calculation unit 24 calculates each bright spot 11b according to the shape of the toilet 3 when the room is empty from each calculated movement amount δ and the mathematical expressions (1), (2), (3) (see FIG. 19). The reference three-dimensional coordinates for measuring the amount of movement on the floor surface 4, the front wall surface 5a, the side wall surface 5c, and the side wall surface 5d are calculated.

By the way, it is ideal that the optical axis of the imaging device 12 and the optical axis of the projection device 11 are parallel, but in reality, some error occurs. The actual optical axis and bright spot group are actually affected by this inclination. Therefore, the optical axis angle correction unit 25 (see FIG. 5) is on the optical axis of the projection device 11 on the floor surface 4 (see FIG. 1) calculated based on the optical arrangement of the projection device 11 and the imaging device 12. The bright spot 11b (see FIG. 6) as the pattern light is projected onto the floor surface 4 (see FIG. 1) and the positions u _{ideal, 0,0} and v _{ideal, 0,0} to be imaged. Based on the position of the bright spot 11b as the pattern light imaged by the apparatus 12 and closest to the position u _{ideal, 0,0} , v _{ideal, 0,0} , the optical axis of the projection apparatus 11 It is configured to correct the tilt.

Positional coordinates u _{ideal, optical axis} , v _{ideal, and optical axis} on the imaging plane 15 ′ (see FIG. 6) of the laser optical axis of the projection apparatus 11 in an ideal state with no inclination are the installation height h and the optical axis. Based on the angle θ and the base line length d, equations (8) and (9) shown in FIG. 19 are obtained. Therefore, the bright spot 11b (see FIG. 6) on the optical axis of the projection device 11 on the floor surface 4 (see FIG. 1) calculated based on the optical arrangement of the projection device 11 and the imaging device 12 is imaged. The coordinates u _{ideal, 0,0} and v _{ideal, 0,0} on the imaging plane 15 ′ (see FIG. 6) of the power position are similarly expressed by the equations (8) and (9) shown in FIG. be able to. Among the actually acquired bright spot groups, the position coordinates on the imaging plane 15 ′ (see FIG. 6) of the bright spot 11b close to the position coordinates u _{ideal, 0,0} , v _{ideal, 0,0} are represented by u _real , Assume v _real . The inclinations Gap _u and Gap _v of the optical axis of the imaging device 12 and the laser optical axis at that time are expressed by equations (10) and (11) shown in FIG.

When the inclinations Gap _u and Gap _v of the laser optical axis are corrected to the output angles φ _X and φ _Y calculated by the above formulas (4) and (5) (see FIG. 19), the output angle φ ′ after the optical axis correction is performed. _{X 1} and φ ′ _Y can be calculated by Equations (12) and (13) shown in FIG. The optical axis angle correction unit 25 (see FIG. 5), as described above, the bright spot 11b (on the optical axis of the projection device 11 on the floor surface 4 (see FIG. 1)) calculated based on the optical arrangement. (See FIG. 6), the position coordinates u _{ideal, 0,0} , v _{ideal, 0,0} on the imaging plane 15 ′ (see FIG. 6) according to the position to be imaged, and actually on the floor surface 4 The position coordinates u _real and v _real on the imaging plane 15 ′ (see FIG. 6) according to the position of the bright spot 11b (see FIG. 6) that is projected and closest to the position of the optical axis on the floor surface 4 Based on the installation angle of the projection device 11 and the imaging device 12, that is, the deviation of the optical axis angle of the actual projection device 11 with respect to the optical axis angle θ as a predetermined optical arrangement is calculated. The inclination of the optical axis can be corrected. In other words, it is possible to correct the inclination of the optical axis of the projection device 11 by calculating a correction amount of deviation from the parallel between the optical axis of the projection device 11 and the optical axis of the imaging device 12.

Reference position calculating section 24, the optical axis angle correcting unit 25 outputs the angle φ _'X, φ' which is calculated by using the _Y, above-mentioned equation (6), (7) the output angle (see FIG. 19) phi _X = Φ ′ _X , φ _Y = φ ′ _Y is substituted. Thereby, the reference position calculation unit 24 is on the image plane 15 ′ (see FIG. 6) of the virtual bright spot 11b after the optical axis correction according to the inclination of the optical axis corrected by the optical axis angle correction unit 25. Calculate the position coordinates u ′ _{ideal, m, n} , v ′ _{ideal, m, n} at, and as described above, the position coordinates u ′ _{ideal, m, n} , v ′ _{ideal, m, n} and the actual position Three-dimensional reference for measuring the amount of movement of each bright spot 11b on the floor surface 4, front wall surface 5a, side wall surface 5c, and side wall surface 5d from the coordinates u _{real, m, n,} v _{real, m, n} Coordinates can be calculated.

  Reference is again made to FIG. The bright spot movement amount δ that can be measured by the measuring device 14 (see FIG. 5) is limited to a range in which the above jump does not occur, that is, a distance to an adjacent bright spot. Therefore, in this embodiment, in order to widen this detection range, the FG element 122 (see FIG. 4) of the projection apparatus 11 is used by rotating around the optical axis with respect to the baseline direction (v-axis direction). By rotating and using the FG element 122, the square lattice pattern 11a has an inclination with respect to the base line direction as shown in FIG. As in the present embodiment, when the rotation amount around the optical axis of the projection device 11 with respect to the baseline direction is the rotation angle α of the FG element 122, the rotation angle α is calculated and each brightness corresponding to the rotation angle α is calculated. If the reference three-dimensional coordinates for measuring the amount of movement of the point 11b on the floor surface 4, the front wall surface 5a, the side wall surface 5c, and the side wall surface 5d are calculated, the reference three-dimensional coordinates are more accurate. Can do. Therefore, the state of the person 2 can be more accurately determined. The rotation amount around the optical axis of the projection device 11 with respect to the base line direction, that is, the rotation angle α of the FG element 122 can also be referred to as the rotation angle of the square lattice pattern 11a with respect to the base line direction. In addition, here, when the projection device 11 is used while being rotated around the optical axis, in short, it may be used by rotating the square lattice pattern 11a so as to have an inclination with respect to the baseline direction. It is not necessary to rotate the entire projection apparatus 11 for use.

  Therefore, the rotation angle calculation unit 26 performs a Hough transform (Hough transform) on the actual pattern 11a on the floor surface 4 to calculate a first angle peak, and is constant on the virtual plane 8 (see FIG. 9). Every time the FG element 122 of the projection device 11 is rotated by a certain angle around the optical axis of the projection device 11 by performing the Hough transform on the virtual pattern 11a formed by the virtual pattern light 11b in the range, here, When the second angle peak is calculated for each rotation angle α when assumed from 0 ° to 180 ° in increments of 0.1 °, the first angle peak and the second angle peak substantially coincide with each other. The rotation angle α around the optical axis is set as the actual rotation angle α of the projection device 11.

  Here, each position coordinate on the imaging plane 15 ′ (see FIG. 6) according to the three-dimensional coordinates of the virtual pattern 11a in a certain range of region on the virtual plane 8 (see FIG. 9) is the virtual plane 8 It should match only the position coordinates on the imaging plane 15 ′ (see FIG. 6) corresponding to the three-dimensional coordinates of the pattern 11a on the floor surface 4 located on the same plane as (see FIG. 9). is there. Therefore, here, a floor surface region including the floor surface 4 indicated by a broken line in the acquired image illustrated in FIG. 8 is assumed as a certain range of the region on the virtual plane 8 (see FIG. 9). The rotation angle calculation unit 26 performs a Hough transform (Hough transform) on the actual pattern 11a on the floor surface 4 to calculate a first angle peak. Further, the rotation angle calculation unit 26 calculates the second angle peak by performing Hough transform on the virtual pattern 11a of the floor area on the virtual plane 8 (see FIG. 9).

  FIG. 10 is a diagram for explaining the concept of the Hough transform according to the embodiment of the present invention. Here, the Hough transform will be described with reference to FIG. In digital image processing such as analysis of line figures, finding a straight line segment is an important task. Among the methods for finding such a line segment, there is a Hough transform as an example of a line segment detection method based on mathematical calculation logic. The Hough transform is based on P.I. V. C. This is a method devised by Hough, and is used to detect straight lines and circles. This method is based on the majority rule, and is effective when the points (here, the bright spots 11b) are remarkably distributed in a linear shape (here, a square lattice shape) as in the present embodiment. There is also an advantage that a plurality of straight lines can be detected simultaneously.

Hereinafter, a general outline of the Hough transform will be described with reference to FIG. In general, a straight line expression can be expressed by a mathematical expression (a) shown in FIG. Here, in Formula (a), a ₀ and b ₀ are constants. If arbitrary M points {(x ₁ , y ₁ ), (x ₂ , y ₂ ),..., (X _M , y _M )} are selected on this straight line, the mathematical formula ( b) holds. Here, i = 1, 2,..., M in the formula (b).

Here, considering a ₀ and b ₀ as variables and considering the (a, b) plane, when a certain point (x _i , y _i ) in the xy coordinate system is given, all straight lines passing there The formula representing the formula is the formula (c) shown in FIG.

Straight line shown by Equation (c), the point _(x i, _{y i)} is different for each, by the points as described above _(x i, _{y i)} is, as a point on the straight line shown by Equation (a) All should pass through the points (a ₀ , b ₀ ). Considering the same thing as the above in the polar coordinate system, when a certain point (x _i , y _i ) in the xy coordinate system is given, all straight line groups passing through the point are expressed by the formula (d) shown in FIG. Can be expressed as

Here, as shown in the graph of FIG. 10, ρ is the length of the perpendicular line from the origin to θ, and _θα is the angle between the perpendicular line and the x axis. In this case, [rho, value [rho ₀ of theta _alpha, be fixed to the _θ α0, (x, y)
One straight line on the plane is specified.

Therefore, it corresponds to arbitrary M points on ρ ₀ = x cos θ _α0 + ysin θ _α0 , {(x ₁ , y ₁ ), (x ₂ , y ₂ ),..., (X _M , y _M )}. The curves on the (θ _α , ρ) plane all intersect at (θ _α0 , ρ ₀ ). Since θ _α only needs to be subdivided into a finite range from −π to π, and ρ needs only to have a certain size, straight line detection by the Hough transform is performed on the (θ _α , ρ) plane. It is common. That is, when a set of candidate points (x _i , y _i ) is given, if a point (θ _α0 , ρ ₀ ) where all the curves intersect most frequently is found, a straight line represented by the formula (a) is extracted. Can do.

For example, when three points of point A: (−2, 0), point B: (0, 2), point C: (4, 6) are given, point A: ρ ₀ = −2 cos θ _α0 , point B: ρ ₀ = 2 sin θ _α0 , point C: ρ ₀ = 4 cos θ _α0 +6 sin θ _α0 . Here, when θ _α0 and ρ ₀ are solved for the three equations, θ _α0 = 135 ° and ρ ₀ = √2. Therefore, the straight line extracted from these three points is y = x + 2.

  In addition, for detection of straight lines by Hough transform, for example, one line can be extracted even if it is a solid line, a broken line, or partly blurred, the thickness of the two lines may vary, and it is independent for every three pixels. There are advantages such as high speed by parallel processing for mapping, and batch processing of any number of 4 lines in the image.

The rotation angle calculation unit 26 performs Hough transform on the actual pattern 11a of the floor area on the floor 4, and controls the straight line extracted from the actual bright spot 11b group as the first angle peak. The angles θ _{real and α0} are calculated. Further, the rotation angle calculation unit 26 performs Hough transform on the virtual pattern 11a of the floor area on the virtual plane 8 (see FIG. 9), and uses the virtual plane 8 (see FIG. 9) as the second angle peak. A dominant angle θ _{ideal, α0} of a straight line extracted from the virtual bright spot 11b group in the upper floor area is calculated.

Here, when the rotation angle calculation unit 26 calculates the angle θ _{ideal, α0} , first, for each rotation angle α when the rotation angle α is assumed in increments of 0.1 ° from 0 ° to 180 °. The position coordinates of the virtual pattern 11a are calculated. Specifically, the rotation angle calculation unit 26 adds the rotation angle α to the output angles φ ′ _X and φ ′ _Y after optical axis correction calculated by the above-described mathematical expressions (12) and (13) (see FIG. 19). output angle after the rotation angle correction obtained by introducing φ _{'' X,} φ _{'a' Y,} equation shown in FIG. 19 (14), is calculated for each rotation angle based on (15). The rotation angle calculation unit 26 uses the output angles φ ″ _X , φ ″ _Y and outputs the output angles φ _X = φ ″ _X , φ _Y to the above-described formulas (6) and (7) (see FIG. 19). = Φ '' _Y is substituted. Thereby, each position coordinate u ′ on the imaging plane 15 ′ (see FIG. 6) of the virtual bright spot 11b according to each rotation angle α assumed in increments of 0.1 ° from 0 ° to 180 °. ' _{ideal, m, n} , v''Calculate _{ideal, m, n} . The rotation angle calculation unit 26 performs the Hough transform for each rotation angle α when the hypothetical pattern 11a formed by the virtual bright spot 11b in the floor area is assumed in increments of 0.1 ° from 0 ° to 180 °. To calculate the angle θ _{ideal, α0} .

The rotation angle calculation unit 26 compares the angles θ _{real and α0} with the angles θ _{ideal and α0} for each rotation angle _, and the rotation angle when the angles θ _{real and α0} substantially match the angles θ _{ideal and α0.} α is set as the actual rotation angle α of the FG element 122. Here, the angle theta _{real, .alpha.0} the angle theta _ideal, the α rotation angle at where the _.alpha.0 was substantially matches the angle theta _{real, .alpha.0} the angle theta _ideal, rotation angle at where the _.alpha.0 exactly matches α In addition to the angle θ _{real, α0} and the angle θ _{ideal, α0} may be the rotation angle α. In this case, the rotation angle in the closest case is set as the actual rotation angle α of the FG element 122. Here, since the rotation angle α is assumed to be in increments of 0.1 ° from 0 ° to 180 °, the rotation angle when the angle θ _{real, α0} and the angle θ _{ideal, α0} are closest is the actual rotation angle α. There is almost no error.

The reference position calculation unit 24 uses the output angles φ ″ _X and φ ″ _Y after the rotation angle correction according to the rotation angle α of the actual FG element 122 set by the rotation angle calculation unit 26 as described above. The output angles φ _X = φ ″ _X , φ _Y = φ ″ _Y are substituted into Equations (6) and (7) (see FIG. 19). As a result, the reference position calculation unit 24 determines the image plane 15 ′ of the virtual bright spot 11b after correction of the rotation angle according to the actual rotation angle α of the FG element 122 set by the rotation angle calculation unit 26 (FIG. 6) and calculate the position coordinates u ″ _{ideal, m, n} , v ″ _{ideal, m, n} above, and as described above, the position coordinates u ″ _{ideal, m, n} , v ″ _{ideal , M, n} and the actual position coordinates u _{real, m, n,} v _{real, m, n} , the amount of movement of each bright spot 11b on the floor 4, front wall 5a, side wall 5c, and side wall 5d. A reference three-dimensional coordinate for measurement can be calculated. In other words, the reference position calculation unit 24 sets the position of the virtual bright spot 11b at the actual rotation angle α of the FG element 122 set by the rotation angle calculation unit 26 and the position of the pattern bright spot 11b actually captured. Based on this, it can be said that the reference three-dimensional coordinates for measuring the amount of movement of each bright spot 11b on the floor surface 4, front wall surface 5a, side wall surface 5c, and side wall surface 5d are calculated.

The reference position calculation unit 24 calculates output angles φ ″ _X and φ ″ _Y for all rotation angles, and calculates position coordinates u ″ _{ideal, m, n} , and v ″ _{ideal, m, n} . The angles θ _{ideal and α0} can be calculated together, and then the angles θ _{ideal and α0} can be compared with θ _{real and α0} to set the actual rotation angle α or output for each rotation angle. Calculate the angle φ ″ _X , φ ″ _Y , position coordinates u ″ _{ideal, m, n} , v ″ _{ideal, m, n} , angle θ _{ideal, α0} and compare each time to calculate the rotation angle α It may be set. When the rotation angle α is set by comparing each rotation angle, the calculation amount can be reduced by omitting the subsequent processing when an appropriate rotation angle α is found. Further, the reference position calculation unit 24 performs rotation when calculating the position coordinates u ″ _{ideal, m, n} , v ″ _{ideal, m, n} of each virtual bright spot 11b according to the actual rotation angle α. By reading out each position coordinate u ″ _{ideal, m, n} , v ″ _{ideal, m, n} calculated when the angle calculation unit 26 sets the actual rotation angle α, the position coordinate u ″ _{ideal, m , N} , v ″ _ideal may be calculated. In this case, by storing each position coordinate u ″ _{ideal, m, n} , v ″ _{ideal, m, n} calculated by the rotation angle calculation unit 26 in the storage unit 31 (see FIG. 5), the calculation amount Can be reduced.

As described above, the position coordinates u ″ _ideal on the imaging plane 15 ′ (see FIG. 6) of the virtual bright spot 11b group after the optical axis correction and the rotation angle correction calculated by the reference position calculation unit 24 _, The virtual movement amount δ of each bright spot 11b is calculated from _{m, n} , v ″ _{ideal, m, n} and the actual position coordinates u _{real, m, n,} v _{real, m, n} , and Expression (1) , (2), (3) (see FIG. 19), the three-dimensional position of the bright spot 11b group on each wall surface at the time of vacant space, that is, the floor surface 4, the front wall surface 5a, and the side wall surface 5c. The three-dimensional coordinates serving as a reference for measuring the movement amount of each bright spot 11b on the side wall surface 5d can be calculated. More specifically, the toilet 3 is defined based on the three-dimensional coordinates serving as a reference for measuring the amount of movement of each bright spot 11b on the floor surface 4, the front wall surface 5a, the side wall surface 5c, and the side wall surface 5d. The position of the wall surface can be specified.

  However, ideally, the bright spot on the wall should show the shape of the toilet neatly, but in reality, the shape of the toilet may not appear neatly due to the influence of noise and parameters that cannot be corrected. .

  FIG. 11 is a diagram of the floor surface 4, the front wall surface 5a, the side wall surface 5c, and the side wall surface 5d calculated based on the virtual bright spot 11b group after the optical axis correction and the rotation angle correction according to the embodiment of the present invention. It is the projection figure which plotted the three-dimensional coordinate used as the reference position of movement amount measurement. (A) is the figure which looked at the back wall surface 5b direction from the front wall surface 5a side, (b) is the figure which looked at the side wall surface 5d direction from the side wall surface 5c side.

  The wall surface specifying unit 27 specifies the positions of the front wall surface 5a, the side wall surface 5c, and the side wall surface 5d according to the frequency histogram regarding the positions in the direction perpendicular to the front wall surface 5a, the side wall surface 5c, and the side wall surface 5d at each reference position. Configured as follows. That is, by taking a frequency histogram of the position of the bright spot 11b on each plane shown in the lower part of the figure, the wall surface position can be obtained at the place where the frequency is the highest.

  Here, the positions of the rear wall surface 5b to which the projection device 11 and the imaging device 12 are attached and the floor surface 4 serving as a reference for the installation height h have already been specified. Since it can be estimated that there is a wall surface at the position where the peak of the frequency histogram is taken, the positions of the side wall surface 5c and the side wall surface 5d can be specified from the frequency histogram in the lower part of FIG. The width W of the toilet 3 (see also FIG. 9A) can be specified from the positional relationship. Similarly, the value of the front wall surface 5a can be identified from the frequency histogram in the lower part of FIG. 11B, and the depth D of the toilet 3 (FIG. 9 (FIG. 9) from the relative positional relationship between the front wall surface 5a and the back wall surface 5b. see also a)).

  In this figure, for example, in the projection view of FIG. 11A, there is a bright spot 11b plotted outside the specified side wall surface 5d, but the bright spot 11b is as described above. This can be regarded as an error due to the influence of noise or parameters that cannot be corrected. Therefore, in the present embodiment, in order to reduce the error and measure the movement amount δ more accurately, the reference position calculation unit 24 is positioned at the specified positions of the front wall surface 5a, the side wall surface 5c, and the side wall surface 5d. A three-dimensional coordinate serving as a reference position for movement amount measurement corrected based on the three-dimensional coordinate is calculated.

  FIG. 12 is a schematic diagram of a cross section passing through the X-axis (see FIG. 6) of the toilet 3 according to the embodiment of the present invention. In addition, this figure is shown so that it may look at the side wall surface 5c direction, and what can be abbreviate | omitted according to the following description, such as structures other than the imaging lens 13a of the toilet bowl 7 and the imaging device 12, and imaging plane 15 ', is suitably used. The illustration is omitted.

  Here, in order to calculate the three-dimensional coordinates of each bright spot 11b serving as a reference on each surface based on the specified positions of the front wall surface 5a, the side wall surface 5c, and the side wall surface 5d, It is necessary to obtain a linear equation connecting the centers of the imaging lens 13a. The bright spot forms an image on the imaging surface 15 'through the imaging lens 13a. In other words, the bright spot projected on the floor surface 4, the front wall surface 5a, the side wall surface 5c, and the side wall surface 5d is a straight line connecting the bright spot imaged on the imaging plane 15 'and the central point of the imaging lens 13a. It is on the extension line. Accordingly, the bright spot position (Δu, Δv) on the imaging plane 15 ′ and the focal length l of the imaging lens from the imaging lens 13a to the imaging plane 15 ′ are used to connect the bright spot to the imaging lens 13a. A linear equation is obtained as Equation (16) shown in FIG.

  From Equation (16), Z = (l / Δu) · X, Y = (Δv / Δu) · X is derived, and when this is substituted into the general form aX + bY + cZ = d of the plane equation, The X coordinate of the three-dimensional coordinates of the bright spot is obtained by Expression (17) shown in FIG. Similarly, when Z = (l / Δv) · Y and X = (Δu / Δv) · Y are derived from Equation (16) and substituted into aX + bY + cZ = d, the three-dimensional coordinates of each bright spot on the plane Y coordinate is obtained by the mathematical formula (18) shown in FIG. Furthermore, when X = (Δu / l) · Z and Y = (Δv / l) · Z are derived from Equation (16) and substituted into aX + bY + cZ = d, the three-dimensional coordinates of each bright spot on the plane are calculated. Among them, the Z coordinate is obtained by Expression (19) shown in FIG.

It is assumed that the toilet 3 has five planes including a floor surface 4, a front wall surface 5a, side wall surfaces 5c and 5d, and an upper surface of the toilet bowl 7. Here, the depth D of the toilet 3, the width W, the installation height h of the projection device 11 and the imaging device 12, the optical axis angle θ, and the height H _seat of the toilet 7 are input (see also FIG. 9A). ). In addition, although the toilet height H may be used instead of the installation height h of the projection device 11 and the imaging device 12, since the projection device 11 projects the bright spot 11b in the direction of the floor surface 4, The wall surface above the projection device 11 is not substantially related to the measurement of the movement of the bright spot 11b. Therefore, the installation height h is used here.

19 are the five planes of the floor surface 4, the front wall surface 5a, the side wall surfaces 5c and 5d, and the upper surface of the toilet bowl 7. In the equations (20), (21), (22), (23), and (24) shown in FIG. The surface equation is shown. Here, it is considered that each corresponds to the above-described surface equation aX + bY + cZ = d, and the equations (20), (21) are added to a, b, c, d in the equations (17), (18), (19). ), (22), (23), and the corresponding values of (24) are substituted, the three-dimensional coordinates of the bright surface 11b on the floor surface 4, the front wall surface 5a, the side wall surfaces 5c, 5d, and the upper surface of the toilet 7 are obtained. , From the position (Δu, Δv) of the bright spot on the image plane 15 ′. Here, Δu and Δv correspond to the position coordinates u _{real, m, n,} v _{real, m, n} of the actual bright spot on the image plane 15 ′ described above.

  For example, in the case of the bright spot 11b on the floor surface 4, by substituting a = 0, b = −sin θ, c = cos θ, and d = h into equations (17), (18), and (19), The three-dimensional coordinates of the bright spot 11b on the upper surface of the floor surface 4 are calculated. As described above, since the approximate position of each bright spot 11b has already been obtained by comparison with each virtual bright spot 11b, it is understood that on which surface each bright spot 11b is located. Then, mathematical expressions (20), (21), (22), (23), (24) and mathematical expressions (17), (18), (19) are used as surface equations corresponding to each bright spot 11b. Thus, the three-dimensional coordinates of the reference bright spot for measuring the amount of movement of each bright spot 11b on the floor 4, front wall 5a, side wall 5c, and side wall 5d can be calculated.

  As described above, the reference position calculation unit 24 is a bright spot that serves as a reference position for movement amount correction based on the positions of the front wall surface 5a, the side wall surface 5c, and the side wall surface 5d specified by the wall surface specification unit 27. By calculating the three-dimensional coordinates, it is possible to reduce errors due to the influence of noise and parameters that cannot be corrected.

  Note that the calculation of the three-dimensional coordinates by the measuring device 14 when the object (here, the person 2) is present is performed as follows. First, assuming that the reference bright spot is included in the plane directly facing the imaging device 12, the amount of movement δ from the reference position of the bright spot is shown in FIG. 6 according to Formula (1) to Formula (3). The position (three-dimensional coordinate) of the bright spot in the coordinate system is obtained. Next, this position is coordinated to the position of the monitoring region (here, toilet 3) from the position of the main surface of the imaging lens 13a (see FIG. 6) of the imaging device 12 and the direction θ of the optical axis (see FIG. 1). Convert.

As described above, the state determination unit 23 (see FIG. 5) is configured to determine the state of the person 2 based on the movement of the pattern measured by the measurement device 14. Specifically, the state determination unit 23 blocks information on the shape or movement of the person 2 obtained by the measuring device 14, in this case, shape information as information on movement of the pattern 11a, and blocks the input layer of the neural network. The state of the person 2 is determined based on the output obtained from the output layer. The state of the person 2 determined by the state determination unit 23 typically includes two types, a dangerous state and a safe state. The dangerous state is a state where it can be inferred that an abnormality has occurred in the body of the person 2 such as a state where the person 2 has fallen.
Hereinafter, the determination of the posture of the person 2 by the state determination unit 23 will be described.

  FIG. 13 is a schematic diagram illustrating an example of shape information used in the embodiment of the present invention and information obtained by blocking the shape information. In the figure, the shape information is shown in the upper part, and the block information is shown in the lower part. In addition, this figure (a) is a schematic diagram which shows the attitude | position which the person 2 stands, (b) is a schematic diagram which shows the attitude | position in which the person 2 is sitting on the upper surface of the toilet bowl 7, (c) is the person 2 It is a schematic diagram which shows the attitude | position which has fallen. Moreover, in this figure, in order to make it easy to understand the shape information and the information obtained by blocking the shape information, these pieces of information are superimposed on an image displaying the position and shape of the floor surface 4 and the toilet bowl 7 and are viewed from the side wall surface 5d side. Shown as a seen image.

  Here, the measurement device 14 described above is configured to generate shape information of the person 2 based on the calculated three-dimensional coordinates of each bright spot projected onto the person 2. The shape information is information indicating the shape of the person 2. Here, the three-dimensional coordinates of each bright spot projected on the person 2 calculated by the measuring device 14 are used for all the bright spots that have been moved in the toilet 3 room. Information plotted in the three-dimensional space.

  The state determination unit 23 is further configured to block the shape information generated by the measurement device 14. Furthermore, the state determination unit 23 converts the shape information into information for each block in a region wider than the projected bright spot interval. As a specific example, an area obtained by dividing the space in the toilet 3 at a predetermined interval is used as a block, and the shape information in the block is converted into data with a small amount of information. The predetermined interval is an interval corresponding to about 15 cm in the toilet 3, for example. Here, the space of the toilet 3 was divided into 15 cm cubic meshes, and the blocks were divided into 6 blocks in the width direction × 14 blocks in the height direction × 12 blocks in the depth direction, for a total of 1008 blocks. The information for each block may be indicated by binary data, for example. In this case, for example, it may be converted to 1 if there is shape information (bright spot) in the block and 0 if there is no shape information.

Further, the state determination unit 23 is configured to obtain the determination result of the state of the person 2 by inputting the block information to the neural network. In the present embodiment, the determination result of the state of the person 2 obtained by inputting to the neural network is two states of “dangerous state” and “safe state”. The neural network is typically a feedforward neural network.
The block diagram of FIG. 14 shows the concept of determination of the state of the person 2 by the state determination unit 23.

  Here, the neural network will be described. The neural network model is an information processing model devised based on the information processing mechanism in the human brain. In the present embodiment, a case will be described where a back-propagation algorithm (error back propagation algorithm) is used in a feed-forward network that is one of neural network models.

  The backpropagation algorithm will be described. As shown in FIG. 15, the neural network model has a multi-input called unit and one-output element called synaptic connection, a signal is transmitted only in one direction, and connected by a connection having a certain weight. Further, the unit receives the sum of values obtained by multiplying those weights by the input value, and outputs the result after transformation by the response function f (x). This output becomes the state of each unit. In this case, a sigmoid function is used for the response function f (x) in order to have a quasi-linear saturation type characteristic.

Also, as shown in FIG. 16, a network in which the units of FIG. 15 are connected in a hierarchical structure including an input layer, an intermediate layer, and an output layer is called a multilayer perceptron. Then, the weight W of each connection as a weighting coefficient that correlates the input layer, the intermediate layer, and the output layer is compared so that the data output from this network is compared with the correct teacher signal and the error is minimized. A method of making the network learn by changing and updating _i is called a back-propagation algorithm. Bakkupuropage - Deployment algorithm output for each input, using the error of the ideal output (teaching signal), but to continue to update the weights W _i between the units (learning). The backpropagation algorithm is said to be a method capable of learning complicated input / output relationships.

  As shown in FIG. 17, the state determination unit 23 determines the state of the person 2 using the above neural network. In this example, the state of the person 2 is determined based on learned neural network data. The input to the neural network is the block information described above with reference to FIG. Further, the output of each block corresponds to the output of each input layer S. That is, here, the number of input layers S is 1008, which is equal to the number of blocks. Further, the number of intermediate layers A is, for example, about 2 to 5, and three here. There are two output layers R, a safety node and a dangerous node.

  When the output of the safety node indicates a higher value than the output of the dangerous node, the state determination unit 23 determines that the person 2 in the toilet 3 is in a safe state, and conversely, the output of the dangerous node is the safety node. When a value higher than the output of is shown, it is determined that the state is dangerous. For example, in this figure, as an example of the safe state, when information when the person 2 shown in FIG. 13A is “standing” is input, the safety node in the output layer R It has a higher value than the dangerous node, and it can be seen that the input data is in a “safe state”.

Here, first, as a learning stage of the neural network, using a simulated toilet, the subject performs the posture of actually using the toilet and the posture at the time of the fall, and the safe state 2500 is then selected. The shape information of the scene and the shape information of the dangerous state 2500 scene are extracted as a learning set. As the teacher signal, the safe state shape information is given to the safety node as 1 and the dangerous node as 0, and the dangerous state shape information as 0 as the safety node and 1 to the dangerous node. -Set a weight W _i for each connection with the algorithm. One scene is image data corresponding to one acquired image. Therefore, for example, when acquiring shape information of 2500 scenes, it means acquiring image data corresponding to 2500 acquired images. It is.

  Here, FIG. 5 will be referred to again. As described above, the state determination unit 23 determines the safety of the person 2 in the toilet 3 based on the learned data. The state determination unit 23 determines that the person 2 is in a dangerous state, and when the dangerous state continues, for example, from 3 seconds to 60 seconds, preferably from 5 seconds to 15 seconds, in this case, 5 seconds or more, A danger signal is transmitted to the alarm signal unit 28 described above. When the warning signal unit 28 receives the danger signal, the warning signal unit 28 is connected to the warning signal unit 28 and transmits the warning signal to the warning device 41 installed in the toilet 3. For example, the alarm device 41 is configured by a speaker that generates sound, alarm sound, and the like. When an alarm signal is received, the alarm device 41 generates sound, alarm sound, and the like, and issues an alarm to the person 2 in the toilet 3. The alarm device 41 may be anything that can issue an alarm without being a speaker. For example, a monitor that issues an alarm using character information or the like may be used.

  Furthermore, an alarm release button 42 and an emergency call button 43 installed in the toilet 3 are connected to the alarm signal unit 28. For example, when the alarm is issued by the alarm device 41, the alarm release button 42 is actually in a safe state when the person 2 is in a safe state, that is, the dangerous state is determined by the state determination unit 23. In this case, when the person 2 presses the alarm release button 42, the alarm release signal is transmitted to the alarm signal unit 28. The alarm signal unit 28 cancels the alarm when receiving the alarm cancel signal.

  The alarm signal unit 28 transmits a notification signal to a notification device (not shown), for example, when the alarm is not canceled, for example, 3 seconds to 30 seconds, preferably 5 seconds to 10 seconds. The person 2 is in a dangerous state with a doctor or nurse waiting in a separate room. The emergency call button 43 is configured to transmit an emergency call signal to the alarm signal unit 28 when the person 2 presses the emergency call button 43 regardless of the state determination by the state determination unit 23. When receiving the emergency notification signal, the alarm signal unit 28 similarly transmits the notification signal to a notification device (not shown).

The state determination unit 23 further changes and updates the connection weight Wi for associating the input layer S (see FIG. 17) and the output layer R (see FIG. 17) when the alarm is released by the alarm release button 42. Configured to perform additional learning. As described above, numerical values determined by learning in advance using many common learning patterns are stored in advance as the weights Wi of these connections. However, in actual use, depending on the characteristics and usage environment of the user, it is possible that a state that did not apply in advance learning may occur, and additional learning according to it is necessary for actual use. Specifically, the state determination unit 23 is first information on the shape or movement of the person 2 obtained by the measurement device 14 set in advance, and based on the information, the person 2 is in a safe state. First shape information as first information that is not determined, and second information related to the shape or movement of the person 2 obtained by the measurement device 14 for a certain period immediately before the alarm released by the alarm release button 42 is issued. The second shape information as the second information that the person 2 determines to be in a dangerous state based on the information is used to perform additional learning for changing the weight W _i of the combination. The

  Here, the first shape information is shape information that is determined to be a dangerous state under any monitoring condition, for example, a state in which the user is completely lying on the floor. In other words, the second shape information is determined that the person 2 is in a dangerous state, but in reality, it is determined that the person 2 is in a safe state, that is, erroneously determined to be in a dangerous state. It is the shape information. When the warning is canceled by the person 2 who is the user of the toilet 3, the state determination unit 23 performs additional learning on the assumption that the posture is in a safe state.

  When the state determination unit 23 determines that the person 2 is in a dangerous state based on the output of the neural network, the state determination unit 23 first warns the toilet 3 as described above. Since the output of the neural network is not always correct, a false alarm may occur if the determination is wrong. Even if a certain posture of the person 2 is actually in a safe state, if it is judged as a dangerous state by the output of the neural network, it is very possible that the same person 2 takes the same posture again in the same monitoring area Very expensive. In this case, erroneous determination and false alarm occur again. Frequent false alarms are not desirable as a system for detecting danger, and the user's willingness to use is lost.

Therefore, in this embodiment, the state determination unit 23, using the first shape information and the second shape information, and additional learning for changing the weight W _i of the coupling. Here, the two types of shape information, the first shape information and the second shape information, are used for additional learning using only the second shape information that has been erroneously determined to be a dangerous state. This is because there is a possibility that the shape information that should originally be determined to be a dangerous state may be affected. In other words, in order to prevent the shape information that should never be determined as being in a safe state from being determined as being in a safe state, it is erroneously determined as a dangerous state. Additional learning is performed using additional learning data obtained by adding the first shape information that is determined to be in a dangerous state to any shape under the monitoring information.

  When the alarm is canceled by the alarm cancel button 42, the second shape information that has been erroneously determined to be a dangerous state is used in the initial neural network learning stage 2500. In addition to the shape information of the scene and the shape information of the dangerous state 2500 scene, there is no doubt that relearning with all data is the ideal learning. However, in this method, generally, there is a great deal of shape information used in the initial learning, and if it is re-learned together with the newly added second shape information, it takes a very long learning time. In addition, each time the alarm is canceled by the alarm cancellation button 42, it is necessary to re-learn over an enormous amount of time, and the second shape information to be added continues to increase each time. become. As in the present embodiment, by performing additional learning using additional learning data obtained by adding the first shape information to the second shape information, for example, the amount is smaller than the relearning as described above. The weight Wi of each connection can be updated at high speed with the retained data. In addition, there is no case where the learning data continues to increase.

  Specifically, first, shape information about a posture that can be estimated to be a dangerous state in any use state from the shape information of the dangerous state 2500 scene used in the learning stage of the initial neural network. For example, 30 scenes to 500 scenes, preferably 50 scenes to 200 scenes, here, 100 scenes are selected and stored in the storage unit 31. In the toilet (monitoring area) actually used for the toilet (monitoring area) from which general learning data has been acquired, objects may be placed or the shape may be different. ) Is not always a dangerous situation. Accordingly, the shape information of the posture that can be determined to be a dangerous state regardless of the use situation of the toilet (monitoring area), the user's characteristics, and the like is selected by human eyes. The selected shape information is set as first shape information.

  In the present embodiment, the second shape information is a pattern of a certain period immediately before the determination that is temporarily stored in the storage unit 31 when the state determination unit 23 determines the state of the person 2. As the shape information related to movement, when the alarm device 41 issues an alarm, information that is kept in the storage unit 31 as it is is used. Here, the shape information of a certain period immediately before the determination is, for example, the shape information of 10 scenes to 100 scenes, preferably 20 scenes to 60 scenes, in this case, 40 scenes in 3 to 30 seconds before the determination. .

  That is, the shape information temporarily stored in the storage unit 31 is updated at any time as the state determination unit 23 determines the state of the person 2. When the warning device 41 issues a warning, the storage unit 31 continues to hold the shape information of the immediately preceding 40 scenes without giving up. Here, as described above, the alarm device 41 issues an alarm when a dangerous state continues for a certain period of time, in this case, 5 seconds or more. Therefore, the alarm device 41 is dangerous for a certain period immediately before the alarm device 41 issues an alarm. It will be the judgment of the state. Thereafter, when the alarm is canceled by the alarm cancel button 42, the shape information kept in the storage unit 31 here is erroneously dangerous although it is actually in a safe state. This is the second shape information that has been determined.

  When the storage unit 31 continues to hold the shape information for a certain period without giving up, the storage unit 31 may continue to hold the information by interrupting the temporary storage update, or the storage unit 31 may store the second shape information. An area for storing the shape information may be formed, and the shape information may be stored in the area. In short, it can be called during additional learning without being abandoned.

As the teacher signal for additional learning, the state determination unit 23 sets the safety node to 0 for the first shape information of 100 scenes determined to be in a dangerous state under any monitoring condition. With regard to the second shape information of 40 scenes that have been erroneously determined to be in a dangerous state, 1 is assigned to the dangerous node, 1 is assigned to the safety node, 0 is assigned to the dangerous node, and the back propagation algorithm is used so far. Update with each connection weight W _i as a starting point.

  FIG. 18 is a flowchart showing an outline of processing steps by the room monitoring apparatus 1 according to the embodiment of the present invention. A method for measuring the position of a person 2 existing in a toilet 3 defined by a floor wall 4, a side wall surface 5 c, and a side wall surface 5 d different from the floor surface 4 by the indoor monitoring apparatus 1 with reference to this figure Here, a method for measuring the position and monitoring the state of the person 2 will be described. For the configuration of the room monitoring apparatus 1, refer to FIG. In addition, the description which overlaps with description of the structure of the indoor monitoring apparatus 1 shall be abbreviate | omitted as much as possible.

  First, as a projection process, projection of pattern light (bright spot) 11b forming the pattern 11a by the projection device 11 is started on the floor surface 4, front wall surface 5a, side wall surface 5c, and side wall surface 5d of the toilet 3 ( S100) As an imaging process, imaging by the imaging device 12 of the toilet 3 on which the pattern light (bright spot) 11b is projected is started (S102). The projection of the pattern light (bright spot) 11b by the projection device 11 is continued until the monitoring is completed. Similarly, the imaging by the imaging device 12 is continued at an acquisition interval of about 0.1 to 0.25 seconds until the monitoring is completed. Further, here, the toilet 3 is in a vacant state, that is, a state in which the person 2 does not exist, and each step is executed in the vacant state until the second reference position calculation step (S112) described later. The In other words, each process up to the second reference position calculation step (S112) can be said to be a process for automatic setting of the room monitoring apparatus 1.

  Next, as the optical axis angle correction step, the optical axis angle correction unit 25 calculates the optical axis of the projection apparatus 11 on the floor 4 calculated based on the optical arrangement of the projection apparatus 11 and the imaging apparatus 12. The position where the pattern light 11b is to be imaged and the pattern light 11b actually projected on the floor surface 4 and imaged by the imaging device 12, and the pattern on the optical axis of the projection device 11 on the floor surface 4 The inclination of the optical axis of the projection device 11 is corrected based on the position of the pattern light 11b closest to the position where the light 11b is to be imaged (S104).

Further, as the rotation angle calculation step, the rotation angle calculation unit 26 performs the Hough transform on the actual pattern 11a of the floor surface area on the floor surface 4 to obtain the actual angle on the floor surface 4 as the first angle peak. The dominant angles θ _{real and α0} of the straight lines extracted from the bright spot 11b group are calculated. Further, the Hough transform is performed on the virtual pattern 11a in the floor area on the virtual plane 8, and the FG element 122 of the projection apparatus 11 is incremented by 0.1 ° from 0 ° to 180 ° around the optical axis of the projection apparatus 11. As a second angle peak, the dominant angle θ _{ideal, α0} of the straight line extracted from the virtual bright spot 11b group in the floor area on the virtual plane 8 is calculated for each rotation angle α when To do. The rotation angle α around the optical axis when the angle θ _{real, α0} and the angle θ _{ideal , α0} for each rotation angle α substantially coincide with each other is set as the actual rotation angle α of the projection device 11 (S106).

  Next, as a first reference position calculation step, the reference position calculation unit 24 projects and projects on a virtual plane 8 (see FIG. 9) that is assumed to be sufficiently wider than the projection range of the pattern light 11b. The position of the virtual pattern light (bright spot) 11b after the optical axis correction and the rotation angle correction calculated based on the optical arrangement of the device 11 and the imaging device 12 is calculated. Further, the position of the virtual pattern light (bright spot) 11b after optical axis correction and rotation angle correction is compared with the position of the pattern light 11b that is actually imaged, and the measuring device 14 is imaged by the imaging device 12. A reference position on the front wall surface 5a (see FIG. 1), the side wall surface 5c (see FIG. 1), and the side wall surface 5d (see FIG. 1) when measuring the movement of the pattern on the image is calculated. That is, a reference image is calculated, and reference three-dimensional coordinates for measuring the amount of movement of each bright spot 11b on the floor 4, front wall 5a, side wall 5c, and side wall 5d are calculated (S108).

  Next, as the wall surface specifying step, the front wall surface 5a is determined by the wall surface specifying unit 27 according to the frequency histogram (see FIG. 11) regarding the positions in the direction perpendicular to the front wall surface 5a, the side wall surface 5c, and the side wall surface 5d. The positions of the side wall surface 5c and the side wall surface 5d are specified. That is, by taking a frequency histogram of the position of the bright spot 11b on each plane, the wall surface position is obtained at the place with the highest frequency (S110).

  Subsequently, as a second reference position calculation step, the reference position calculation unit 24 serves as a reference position for movement amount measurement corrected based on the specified positions of the front wall surface 5a, the side wall surface 5c, and the side wall surface 5d. The original coordinates are calculated (S112). Thus, the process for automatically setting the room monitoring device 1 is completed. Subsequent processing is executed in a state where the person 2 is present in the toilet 3.

  Next, as a measurement process, the pattern light (bright spot) calculated on the floor surface 4, front wall surface 5a, side wall surface 5c, and side wall surface 5d of each bright spot 11b when the toilet 3 is empty is calculated as described above. ) The moving amount δ of each pattern light (bright spot) 11b on the image actually picked up by the image pickup device 12 is measured by the measuring device 14 using the three-dimensional coordinates of 11b as a reference. Furthermore, the three-dimensional coordinates of each pattern light (bright spot) 11b projected on the person 2 is calculated based on the measured movement amount, and shape information (see FIG. 13) as information on the shape or movement of the person 2 is calculated. Generate (S114).

  Further, as the state determination step, the state determination unit 23 blocks the shape information generated by the measurement device 14 (see FIG. 13) and inputs it into the input layer S (see FIG. 17) of the neural network, thereby outputting the output layer R ( The state of the person 2 is determined based on the output obtained from (see FIG. 17) (S116). Here, the storage unit 31 temporarily stores the shape information of the 40 scenes immediately before the determination.

  When the output of the safety node in the output layer R shows a higher value than the output of the dangerous node, the state determination unit 23 determines that the person 2 in the toilet 3 is in a safe state (S118; NO) When the process proceeds to S120 and monitoring is further continued (S120; YES), the process returns to the measurement step (S114) and the subsequent processes are repeated. If the monitoring is not continued (S120; NO), the monitoring is terminated.

When the output of the dangerous node in the output layer R shows a higher value than the output of the safety node, it is determined that the state is dangerous (S118; YES), and whether or not the dangerous state has continued for 5 seconds or more. Determination is made (S122). If it has not continued for more than 5 seconds (S122; NO), the process returns to the measurement step (S114) and the subsequent processes are repeated. When it continues for 5 seconds or more (S122; YES), the state determination unit 23 transmits a danger signal to the alarm signal unit 28, and proceeds to the next alarm generation step (S124).

  In the alarm issuing step, an alarm signal is transmitted to the alarm device 41 by the alarm signal unit 28 that has received the danger signal, and an alarm is issued by the alarm device 41 to the person 2 in the toilet 3 (S124). At this time, in the state determination step (S116), the shape information of the 40 scenes temporarily stored in the storage unit 31 is not abandoned, but is used in the second learning step (S132) to be described later. Continue to hold as shape information.

  Subsequently, the alarm signal unit 28 determines whether or not the alarm has been canceled by the person 2 pressing the alarm cancel button 42 (S126). If it is determined that the alarm has been canceled (S126; YES) Then, the process proceeds to the additional learning step (S132).

  When it is determined that the alarm is not released (S126; NO), the alarm signal unit 28 determines whether or not the period during which the alarm is not released continues for 7 seconds or more (S128), and continues for 7 seconds or more. If not (S128; NO), the process returns to S126 and the subsequent processes are repeated.

  When it continues for 7 seconds or more (S128; YES), as a notification process, the alarm signal unit 28 transmits a notification signal to a notification device (not shown), for example, a doctor or a nurse waiting in a separate room, etc. Is notified that person 2 is in a dangerous state (S130), and the monitoring is terminated.

In the additional learning step of S132, the first shape information of 100 scenes determined by the state determination unit 23 to be in a dangerous state under any monitoring condition stored in the storage unit 31 in advance Then, the neural network is used by using the second shape information of the 40 scenes that have been erroneously determined to be in a dangerous state that has been stored in the storage unit 31 without being abandoned in the alarm issuing step (S124). Perform additional learning. As a teacher signal for additional learning, the state determination unit 23 sets 0 for the safety node, 1 for the dangerous node, 1 for the safety node, and 0 for the safety node for the second shape information. And the weight W _{i of} each connection is updated by the backpropagation algorithm.

  When the additional learning process (S132) is executed, the process subsequently proceeds to S120, and when monitoring is continued (S120; YES), the process returns to the measurement process (S114) and the subsequent processes are repeatedly executed. . If the monitoring is not continued (S120; NO), the monitoring is terminated.

  Here, each process in each part of the arithmetic unit 20 and the above-described method are installed in a computer, operate the computer as an indoor monitoring apparatus, and control the computer so as to execute each process. And can also be realized as a recording medium for recording the software program. The software program may be recorded and used in a program unit (not shown) built in the computer, recorded in an external storage device or CD-ROM, and read and used by the program unit (not shown). Alternatively, it may be downloaded from the Internet to a program unit (not shown) and used.

According to the indoor monitoring device 1 or the software program according to the embodiment of the present invention described above, the state determination unit 23 can detect the input layer S (FIG. 17) when the alarm is released by the alarm release button 42. (See FIG. 17) and the output layer R (see FIG. 17). Therefore, for example, when a posture that is not supposed to be assumed in advance is taken with respect to a posture that should be determined to be a dangerous state, or a dangerous state for each monitored space When making misjudgment while operating the device for various monitoring conditions such as when the attitude of the person is different, the error is not gradually repeated without adapting to the same error. It is possible to provide a monitoring device and a software program that can make an accurate determination with less.

  In other words, since it is not necessary to make detailed settings corresponding to the monitoring space to be installed when installing the device, it is not always necessary to install a device having specialized knowledge, that is, easy The indoor monitoring device 1 can be installed in the room. Further, in additional learning, additional learning is performed using additional learning data obtained by adding the first shape information to the second shape information. For example, as described above, less retained data compared to relearning. Thus, the weight Wi of each connection can be updated at high speed. In addition, there is no case where the learning data continues to increase.

The state determination unit 23 is first information regarding the shape or movement of the person 2 obtained by the measuring device 14 set in advance, and the person 2 does not determine that the person 2 is in a safe state based on the information. Using the shape information of 1 and the second shape information that has been erroneously determined to be a dangerous state, additional learning is performed to change the weight W _i of the connection. As a result, in the determination of the state after additional learning, not only the shape information that has been erroneously determined to be a dangerous state before additional learning is determined to be a safe state, but also in an absolutely safe state. It is possible to prevent the shape information that should not be determined to be present from being determined as being in a safe state.

  In addition, according to the indoor monitoring device 1 or the software program according to the embodiment of the present invention described above, the reference position calculation unit 24 is assumed to be sufficiently wide compared to the projection range of the pattern light 11b. Measurement is performed by comparing the position of the virtual pattern light projected on the virtual plane 8 and calculated based on the optical arrangement of the projection device 11 and the imaging device 12 with the position of the pattern light 11b that is actually imaged. The reference position of each bright spot 11b on the floor surface 4, the front wall surface 5a, the side wall surface 5c, and the side wall surface 5d when the device 14 measures the movement of the pattern on the image captured by the imaging device 12 is calculated. Configured as follows. Therefore, for example, in accordance with various conditions such as the shape and size of the space to be monitored, what is installed in the space, the position of the toilet seat, size and shape, etc. It is possible to provide the indoor monitoring apparatus 1 or the software program that does not need to be performed manually and can simplify the installation work of the apparatus.

  Furthermore, the optical axis angle correction unit 25 should image pattern light 11b on the optical axis of the projection device 11 on the floor surface 4 calculated based on the optical arrangement of the projection device 11 and the imaging device 12. Position and pattern light 11b that is actually projected on the floor surface 4 and imaged by the imaging device 12, and the pattern light 11b on the optical axis of the projection device 11 on the floor surface 4 is to be imaged Is configured to correct the inclination of the optical axis of the projection device 11 based on the position of the pattern light 11b closest to the. Therefore, an error in tilt between the optical axis of the imaging device 12 and the optical axis of the projection device 11 can be corrected, and the reference position calculation unit 24 can correct the optical axis corresponding to the corrected tilt of the optical axis of the projection device 11. Based on the virtual pattern light (bright spot) 11b after the angle correction, the reference of each bright spot 11b on the floor surface 4, the front wall surface 5a, the side wall surface 5c, and the side wall surface 5d when measuring the movement of the pattern. By calculating the position, the influence of the tilt can be eliminated. Therefore, movement can be measured more accurately, and as a result, the state determination unit 23 can also determine the state of the person 2 more accurately.

  Further, the rotation angle calculation unit 26 calculates the first angle peak by executing the Hough transform on the actual pattern 11a on the floor surface 4, and is formed by the virtual pattern light 11b in a certain range on the virtual plane 8. A Hough transform is performed on the virtual pattern 11a, and a second angle peak is calculated each time the FG element 122 of the projection apparatus 11 is rotated by a certain angle around the optical axis of the projection apparatus 11. The rotation angle calculation unit 26 can set the rotation angle α around the optical axis when the first angle peak and the second angle peak substantially coincide with each other as the actual rotation angle α of the projection device 11. . Therefore, the reference position calculation unit 24 measures the movement of the pattern on the floor surface 4 and the front wall surface based on the virtual pattern light (bright spot) 11b after the rotation angle correction corresponding to the set rotation angle α. By calculating the reference position of each bright spot 11b on 5a, the side wall surface 5c, and the side wall surface 5d, the influence of the rotation angle α can be eliminated. Therefore, in the measurement of the bright spot movement amount δ by the measurement device 14 (see FIG. 5), the FG element 122 (FIG. 4) of the projection device 11 is used in order to increase the range where the above-mentioned jump does not occur, that is, the detection range. The movement can be accurately measured even if the reference is rotated with respect to the base line direction (v-axis direction). As a result, the state determination unit 23 can also determine the state of the person 2 more accurately.

  Further, the wall surface specifying unit 27 calculates a frequency histogram (position histograms) regarding the positions in the direction perpendicular to the front wall surface 5a, the side wall surface 5c, and the side wall surface 5d of each reference position of the pattern light (bright spot) 11b calculated as described above. The positions of the front wall surface 5a, the side wall surface 5c, and the side wall surface 5d are specified according to FIG. Based on the positions of the front wall surface 5a, the side wall surface 5c, and the side wall surface 5d specified by the wall surface specifying unit 27, the reference position calculation unit 24 measures the floor surface 4 and the front wall surface 5a when measuring the movement of the corrected pattern. The reference position of each bright spot 11b on the side wall surface 5c and the side wall surface 5d is calculated. As a result, it is possible to reduce errors due to the influence of noise and uncorrectable parameters in the calculation of the reference position.

  As described above, the indoor monitoring device 1 according to the present embodiment projects the pattern light that forms the pattern on the floor surface 4 and the front wall surface 5a, the side wall surface 5c, and the side wall surface 5d of the toilet 3. And the imaging device 12 that images the toilet 3 on which the person 2 as an object is projected, on which the pattern light is projected, and the floor surface 4, the front wall surface 5a, and the side wall surface 5c when the person 2 is not present on the toilet 3. The movement of the pattern on the captured image is measured with reference to the pattern light projected on the side wall surface 5d, and the three-dimensional position of the pattern light projected on the person 2 based on the measured movement of the pattern A measuring device 14 for calculating

  With this configuration, for example, when the person 2 is present in the toilet 3, the pattern is projected onto the person 2, so that the pattern moves on the image captured by the imaging device 12. The shape of the person 2 can be measured by measuring the movement amount. Thereby, the person 2 existing in the toilet 3 can be monitored more accurately.

  Furthermore, since the projection device 11 and the imaging device 12 are disposed at positions facing the floor surface 4 and the front wall surface 5a, for example, a necessary measurement range and resolution can be easily ensured. Furthermore, the influence of the bright spot jumping that occurs when measuring the movement amount of the bright spot forming the pattern can be greatly reduced. In addition, many bright spots hit the person 2 in the toilet 3, and a lot of position information (three-dimensional coordinates) can be obtained.

  The indoor monitoring device 1 also includes a state determination unit 23 that determines the state of the person 2 based on the movement of the pattern measured by the measurement device 14, and the state determination unit 23 includes the pattern measured by the measurement device 14. It is configured to obtain a determination result by making shape information as information relating to movement of the block into blocks and inputting them into a neural network.

  With this configuration, for example, the state determination unit 23 further blocks the shape information of the person 2 based on the three-dimensional coordinates of each bright spot projected on the person 2 calculated by the measurement device 14. The amount of information input to the neural network can be reduced. That is, processing becomes easy. Moreover, since the determination result of the state of the person 2 is obtained by inputting the result into the neural network, the response of the system for determining the state of the person 2 can be optimized, and the person 2 existing in the toilet 3 can be monitored more accurately. .

  Further, the indoor monitoring device can follow a series of movements such as whether the person 2 enters the toilet 3, what state the person 2 is in, and whether the person 2 has exited with a simple device. As described above, the indoor monitoring apparatus 1 can very easily monitor, for example, whether or not the person 2 has fallen by judging the state of the person 2. Furthermore, high-speed processing is possible with a simple device.

  In addition, the indoor monitoring apparatus 1 or software program which is embodiment of this invention is not limited to embodiment mentioned above, A various change is possible in the range described in the claim. In the above description, the indoor monitoring device 1 has been described as the monitoring device, but is not limited thereto. Note that monitoring is a concept that includes measurement. For example, the indoor monitoring device 1 of the present embodiment can also measure the position distribution of an object. Further, the information that the state determination unit 23 blocks and inputs to the neural network has been described as information related to the movement of the pattern measured by the measurement device 14, but is not limited thereto, and is related to the shape or movement of the object. Any information is acceptable.

  In the above description, the reference position calculation unit 24 measures the movement of the pattern based on the virtual pattern light (bright spot) 11b after the optical axis angle correction and the rotation angle correction. The reference position of each bright spot 11b on the wall surface 5a, the side wall face 5c, and the side wall face 5d is calculated, and further, the bright spot 11b on the floor surface 4, the front wall face 5a, the side wall face 5c, and the side wall face 5d is calculated. It has been described that the position is specified, and the corrected reference position is calculated based on the specified position. However, the three corrections of the optical axis angle correction, the rotation angle correction, and the correction after specifying the wall surface position are not necessarily performed. For example, two of the three corrections may be combined. However, it goes without saying that the reference position can be calculated more accurately by performing the above three corrections.

  In the above description, the first plane has been described as the floor surface 4. However, for example, when the optical axes of the projection device 11 and the imaging device 12 are arranged so as to intersect the front wall surface 5a. Alternatively, the front wall surface 5a may be a first plane, the floor surface 4, the side wall surface 5c, and the side wall surface 5d may be a second plane. In this case, it is only necessary to know the horizontal distance from the imaging lens 13a to the front wall surface 5a instead of the installation height h from the imaging lens 13a to the floor surface 4 as an optical arrangement. In this case, the virtual plane 8 may be a virtual plane that is sufficiently wide as compared to the projection range of the pattern light 11b including the front wall surface 5a.

  Further, the optical axis angle correction unit 25 (see FIG. 5) is configured so that the installation angle of the projection apparatus 11 and the imaging apparatus 12, that is, the optical axis of the actual projection apparatus 11 with respect to the optical axis angle θ as a predetermined optical arrangement Although it has been described that the angle deviation is calculated and the inclination of the optical axis of the projection apparatus 11 is corrected, the actual deviation of the optical axis angle of the imaging apparatus 12 with respect to the optical axis angle θ as a predetermined optical arrangement is described. May be calculated to correct the tilt of the optical axis of the imaging device 12, or the tilt of the optical axis of both the projection device 11 and the imaging device 12 may be corrected. However, the installation height h or the like is large as long as the optical axis angle of the imaging device 12 is slightly shifted without correcting the optical axis angle shift of the imaging device 12 as in the present embodiment. Since there is no error, there is no particular problem.

  In the above description, the state determination unit 23 blocks the information about the movement of the measured pattern as the information about the shape or movement of the object obtained by the measuring device 14 and blocks the input layer S of the neural network. Although described as an input to (see FIG. 17), it does not necessarily have to be blocked. For example, the barycentric position of the bright spot group detected outside the wall, the direction of the principal axis of inertia, and the like can be used as the input parameters of the neural network. Moreover, when inputting using the information regarding the movement of the object obtained by the measuring device 14, it is not always necessary to make a block.

  In the above description, the input device 35 and the display 40 have been described. However, instead of these, the switches that perform reset and start / stop of the device, and the status thereof, are operated as a simpler configuration. It is good also as a structure provided with the some indicator to display.

  As for the neural network, it is necessary to examine the number of nodes in each layer according to the characteristics of the identification target. The number of input layers S is determined by how much spatial resolution is required for determining the state of the person 2, and in the above description, the number of input layers S is 1008, which is equal to the number of blocks. The number of output layers R is determined by the number of categories to be identified. In the above description, the safety node and the dangerous node are assumed to be two. It is desirable that the number of intermediate layers A be as small as possible within the range that can be determined. If this is too large, there are many cases where the determination for an unlearned input pattern cannot be obtained correctly, that is, the generalization ability is low. There is. In the above description, the number is three.

  Although the number of output layers R is two, a safety node and a dangerous node, for example, the posture of the person 2 is classified into categories such as a standing state, a sitting state, and a falling state. Thus, safety and danger may be determined from the posture.

  In some cases, a self-organizing map type neural network may be used instead of the hierarchical type neural network used above. Unlike the hierarchical type, this is basically a structure of only the input layer S and the output layer R, and the node that has obtained the highest output in the output layer R and the surrounding nodes have a higher output than the input pattern. It is a neural network that performs unsupervised learning that strengthens the connection so that it can be obtained. While many learning patterns are presented, a node near the output layer is ignited naturally for a similar input pattern. Therefore, the determination function can be provided by making the node position of the output layer R correspond to the determination of safety and danger. Again, additional learning can be performed by presenting additional learning patterns in the same way.

It is a typical external view showing the outline of the indoor monitoring device concerning an embodiment of the invention. It is a typical side view explaining the case where the projector and imaging device which concern on embodiment of this invention are arrange | positioned in the center part of the ceiling. It is a typical side view explaining the case where the projection apparatus and imaging device which concern on embodiment of this invention are arrange | positioned in the position facing a floor surface and a front wall surface. It is a typical perspective view explaining the projection apparatus which concerns on embodiment of this invention. It is a block diagram which shows the structural example of the indoor monitoring apparatus which concerns on embodiment of this invention. It is a conceptual perspective view explaining the concept of the movement of the bright spot in the embodiment of the present invention. It is a schematic diagram explaining the bright spot imaged on the imaging surface in the case of FIG. It is a schematic plan view which shows an example of the image imaged with the imaging device when the person does not exist in the boundary of the floor surface of a toilet, a front wall surface, a side wall surface, a side wall surface, and a toilet in embodiment of this invention. It is a figure which shows an example of the pattern projected by the projection apparatus which concerns on embodiment of this invention. It is a figure explaining the concept of Hough transformation concerning an embodiment of the invention. Tertiary serving as a reference position for measuring the amount of movement on the floor, front wall, side wall, and side wall calculated based on the virtual bright spot group after optical axis correction and rotation angle correction according to the embodiment of the present invention It is the projection figure which plotted the original coordinate. It is a schematic diagram of the cross section which passes along the X-axis (refer FIG. 6) of the toilet which concerns on embodiment of this invention. It is a schematic diagram which shows an example of the shape information used for embodiment of this invention, and the information which blocked the shape information. It is a block diagram which shows the concept of the determination of a person's state by the state determination part used by embodiment of this invention. It is a block diagram explaining the unit of a neural network used by embodiment of this invention. It is a block diagram explaining the back propagation algorithm used by embodiment of this invention. It is a block diagram which shows the concept of the determination of a person's state using the neural network by a state determination part used by embodiment of this invention. It is a flowchart which shows the outline of the process process by the indoor monitoring apparatus which concerns on embodiment of this invention. It is the figure put together about the main numerical formulas used by this Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Indoor monitoring apparatus 2 Person 3 Toilet 4 Floor surface 5a Front wall surface 5b Back wall surface 5c Side wall surface 5d Side wall surface 6 Ceiling 7 Toilet bowl 8 Virtual plane 11a Pattern 11b Pattern light (bright spot)
DESCRIPTION OF SYMBOLS 11 Projector 12 Imaging device 13a Imaging lens 14 Measuring device 15 'Imaging surface 15 Imaging element 18 Case 20 Arithmetic device 21 Control unit 23 State determination unit 24 Reference position calculation unit 25 Optical axis angle correction unit 26 Rotation angle calculation unit 27 Wall surface identification unit 28 Alarm signal unit 31 Storage unit 35 Input device 40 Display 41 Alarm device 42 Alarm release button 43 Emergency call button 102 Plane 103 Object 105 Light flux generation unit 120 Grating 121 Optical fiber 122 FG element R Output layer S Input layer A Intermediate Layer _Wi connection weights

Claims (4)

In a monitoring device for monitoring an object present in a monitoring area;
Measuring means for obtaining information on the shape or movement of the object;
Determining means for determining the state of the object based on an output obtained from an output layer by inputting information on the shape or movement of the object obtained by the measuring means to an input layer of a neural network;
Alarm means for issuing an alarm when the determination means determines that the object is in a dangerous state;
Alarm release means for releasing the alarm;
The determination unit, when the alarm is canceled by the alarm cancellation section, have rows additional learning for changing the weighting coefficient relating and said output layer and the input layer,
The state of the object determined by the determining means includes a dangerous state and a safe state of the object,
The determination means is first information relating to the shape or movement of the object obtained by the measurement means set in advance, and the first means that the object is not determined to be in a safe state based on the information. And the second information regarding the shape or movement of the object obtained by the measuring means for a certain period immediately before the released alarm is issued, and the object is dangerous based on the information. Performing additional learning to change the weighting coefficient using the second information determined to be in an unsatisfactory state,
Monitoring device. When the determination means determines the state of the object, it temporarily stores information on the shape or movement of the object obtained by the measurement means for a certain period immediately before the determination, and the warning means A storage means for continuing to hold the stored information when issuing an alarm;
The monitoring device according to claim 1 . A projection device that projects pattern light that forms a pattern on the monitoring area;
An imaging device that images the monitoring area where the object is present, onto which the pattern light is projected;
The measuring means is configured to measure movement of a pattern on the imaged image;
Information on the shape or movement of the object is information on movement of the measured pattern,
The monitoring apparatus according to claim 1 or 2 . A software program installed on a computer and operating the computer as a monitoring device;
In the process of monitoring objects present in the monitoring area;
Processing to obtain information on the shape or movement of the object;
A process of determining the state of the object based on an output obtained from an output layer by inputting information on the shape or movement of the object obtained to the input layer of a neural network;
A process for issuing an alarm when it is determined in the determination process that the object is in a dangerous state;
Processing to release the alarm;
When the alarm is canceled by the process of canceling the alarm, the computer is controlled to execute a process of performing additional learning to change a weighting coefficient that associates the input layer and the output layer ,
The state of the object to be determined includes a dangerous state and a safe state of the object,
The process of performing the additional learning is first information on the shape or movement of the object obtained by the process of obtaining the information set in advance, and the object is in a safe state based on the information First information that is not determined to be, and second information related to the shape or movement of the object obtained by the process of obtaining the information for a certain period immediately before the released alarm is issued, Changing the weighting factor using the second information based on which the object is determined to be dangerous based on ;
Software program.