JP4990552B2 - Attention position identification system, attention position identification method, and attention position identification program - Google Patents

Description

  The present invention relates to a target position specifying system, a target position specifying method, and a target position specifying program, and in particular, a target position specifying system, a target position specifying method, and a target position specifying program for specifying a target position of a person with high accuracy. About.

  In the past, in order to prevent crimes such as theft (shoplifting), robbery, arson, etc. in the vicinity of houses, surveillance areas such as convenience stores, department stores, hospitals, banks, factories, office buildings, etc. There is a method of identifying the attention target of the person from the action and the line-of-sight direction (see, for example, Patent Document 1).

The technique disclosed in Patent Document 1 arranges a plurality of line-of-sight detection devices so that there is no blind spot, enables line-of-sight detection regardless of the direction of the person, and the position of the monitored person in the space to be monitored The line-of-sight detection device corresponding to is selected, and the target object of the observer is detected from the line-of-sight direction information of the selected line-of-sight detection device.
JP 2001-8197 A

  However, the position information of the person in the above-described conventional method is used only for determining which of the plurality of line-of-sight detection devices is used to detect the line-of-sight of the monitored person. Therefore, for example, when there are a plurality of people whose gaze is detected by the same gaze detection device, the gaze position may not be the same depending on the position of the person even if the angle (gaze) of each person's face is the same. Therefore, there is a problem that an error occurs. Moreover, since the line-of-sight detection device and the person detection device are separately required as a configuration, there are problems such as the cost of the device and the installation location.

  The present invention has been made in view of the above-described problems, and an object thereof is to provide an attention position specifying system, an attention position specifying method, and an attention position specifying program for specifying a person's attention position with high accuracy. .

  In order to solve the above problems, the present invention employs means for solving the problems having the following characteristics.

According to the first aspect of the present invention, in the attention position specifying system for specifying the attention position of a person in a preset monitoring area, the monitoring area is photographed to acquire image information. imaging means, on the basis of image information obtained by the imaging means, the imaging said position coordinates of the person relative to the position of the imaging means and detects as the position information of the person is preset Person position detecting means for detecting an angle from a reference direction of the means to the direction in which the person is present as angle information of the person, line-of-sight information acquiring means for acquiring line-of-sight information of the person from the image information, and the person position detection An attention position specifying hand for specifying the attention position of the person based on the position information and angle information of the person obtained by the means and the line-of-sight information obtained by the line-of-sight information acquisition means If, sets the one or more target objects included in the pre-imaging area of the imaging means, have a storage means for storing position information of the object of interest, said target position specifying means, the storage Based on the position information of the target object accumulated in the means and the position information of the person, the angle information from the person to the target object with respect to the reference direction of the imaging means is acquired, and the acquired person When the attention position of the person is specified based on the angle information from the object to the attention object and the line-of-sight information obtained by the line-of-sight information acquisition unit, and the attention object has a predetermined size The method extracts a plurality of feature points for the target object, acquires angle information at each feature point from the person with respect to a reference direction of the imaging unit, and among the plurality of acquired angles, the maximum angle and the minimum Corner Select a, and selected the maximum angle and the minimum angle was, on the basis of said line of sight information, and identifies the target position of the person.

The invention described in claim 2 further includes a suspicious person determination means for determining whether or not the person is a suspicious person based on the attention position of the person obtained by the attention position specifying means. Features.

In the invention described in claim 3 , the suspicious person determination unit accumulates the number of times the target object is present in the line-of-sight direction of the person. It is characterized by determining that it is a person.

In the invention described in claim 4 , the suspicious person determining means accumulates the time during which the target object is present in the line-of-sight direction of the person, and when the accumulated time is equal to or longer than a predetermined time, the person is suspicious. It is characterized by determining that it is a person.

According to a fifth aspect of the present invention, in the attention position specifying method for specifying the attention position of a person in a preset monitoring area, the imaging of the monitoring area is performed by one or a plurality of imaging means. a method, based on the image information obtained by the imaging step, the position coordinates of the position the in person image relative to the said imaging means and detecting the position information of the person is preset A person position detecting step for detecting an angle from the reference direction of the image pickup means to the direction in which the person is present as the angle information of the person, and based on the image information obtained by the image pickup step, the position information of the person, A person position detecting step for detecting angle information of the person with respect to a preset reference direction of the imaging means; and obtaining the line-of-sight information of the person from the image information Attention to identify the attention position of the person based on the position information and angle information of the person obtained by the step of acquiring the line-of-sight information, the position information and angle information of the person obtained by the step of detecting the person position, and the line-of-sight information obtained by the step of obtaining the line-of-sight information position possess a certain step, the target position specifying step, on the basis of the position information of one or more target objects included in the imaging area of a preset the imaging means, the position information of the person, The angle information from the person to the target object with respect to the reference direction of the imaging means is acquired, the angle information from the acquired person to the target object and the line-of-sight information obtained by the line-of-sight information acquisition step Based on the target position of the person, and when the target object has a predetermined size, a plurality of feature points are extracted for the target object. An angle at each feature point is acquired from the person with respect to a reference direction of the imaging means, a maximum angle and a minimum angle are selected from the acquired plurality of angles, the selected maximum angle and the minimum angle, and the line of sight The attention position of the person is specified based on the information .

The invention described in claim 6 further includes a suspicious person determination step of determining whether or not the person is a suspicious person based on the attention position of the person obtained by the attention position specifying step. Features.

In the invention described in claim 7 , the suspicious person determination step accumulates the number of times that the target object is present in the line-of-sight direction of the person. It is characterized by determining that it is a person.

Further, in the invention described in claim 8 , in the suspicious person determination step, the time during which the object of interest exists in the line-of-sight direction of the person is accumulated, and if the accumulated time is a predetermined time or more, the person is suspicious. It is characterized by determining that it is a person.

The invention described in claim 9 is an attention position specifying program for causing a computer to function as each means included in the attention position specifying system described in any one of claims 1 to 4 .

According to the first aspect of the present invention, a person's attention position can be specified with high accuracy. Further, it is possible to obtain with high accuracy which direction the person is facing in the real space from the position of the person, the position of the imaging unit, and the angle information of the person photographed by the imaging unit. In addition, it is possible to determine with high accuracy whether or not the person is looking at the target object from the gaze information of the person in the real space and the position information of the target object. Furthermore, even if the target object has a certain size, it is possible to determine with high accuracy whether or not the person is looking at the target object by setting an appropriate range of the line-of-sight direction.

According to the second aspect of the present invention, it is possible to determine with high accuracy whether or not the person is a suspicious person based on the specific information of the target position. Thereby, the monitoring burden of a supervisor etc. can be reduced.

According to the third aspect of the present invention, it is possible to determine whether or not the person is a suspicious person with higher accuracy based on the information on the number of times the person has viewed the target object.

According to the fourth aspect of the present invention, it is possible to determine whether or not the person is a suspicious person with higher accuracy based on time information when the person saw the target object.

In addition, according to the fifth aspect of the present invention, it is possible to specify a person's attention position with high accuracy. Further, it is possible to obtain with high accuracy which direction the person is facing in the real space from the position of the person, the position of the imaging unit, and the angle information of the person photographed by the imaging unit. In addition, it is possible to determine with high accuracy whether or not the person is looking at the target object from the gaze information of the person in the real space and the position information of the target object. Furthermore, even if the target object has a certain size, it is possible to determine with high accuracy whether or not the person is looking at the target object by setting an appropriate range of the line-of-sight direction.

According to the sixth aspect of the present invention, it is possible to determine with high accuracy whether or not the person is a suspicious person based on the specific information of the target position. Thereby, the monitoring burden of a supervisor etc. can be reduced.

According to the seventh aspect of the present invention, it is possible to determine whether or not the person is a suspicious person with higher accuracy based on the number of times the person has seen the target object.

According to the eighth aspect of the present invention, it is possible to determine whether or not the person is a suspicious person with higher accuracy based on time information when the person saw the target object.

According to the ninth aspect of the present invention, it is possible to specify the attention position of a person with high accuracy. Further, the attention position specifying process can be easily realized by installing the execution program in the computer.

  Hereinafter, embodiments in which a target position specifying system, a target position specifying method, and a target position specifying program according to the present invention are suitably implemented will be described with reference to the drawings. In the following description, an attention position specifying method having a function of determining whether or not a person photographed by the camera is a suspicious person using the attention position specifying result will be described.

<System configuration>
FIG. 1 is a diagram illustrating an example of a schematic configuration of a target position specifying system according to the present invention. The attention position specifying system 10 shown in FIG. 1 is configured to include one or more cameras 11-1 to 11-n as imaging means and an attention position specifying device 12.

  For example, one or a plurality of cameras 11 are installed in a monitoring area such as the periphery of a house, and images are taken constantly or at predetermined time intervals, and the obtained image information is output to the attention position specifying device 12. Note that the camera 11 may add identification information for identifying which camera has taken the image information to be output. Further, the camera 11 may add shooting time information such as date and time of shooting, audio information around the camera, and the like to the output image information.

  Here, the camera 11 is not particularly limited, and for example, a stereoscopic camera and a stereo camera having a distance measuring function can be used. Further, the camera 11 may be provided with a distance sensor that measures the distance to a person in the shooting range. A technique using a distance sensor is disclosed in, for example, Japanese Patent Application Laid-Open No. 2004-178229.

  Furthermore, the camera 11 may have a function of detecting the movement of an object. For example, the camera 11 may start shooting when an object moving to the shooting range is detected.

  The attention position specifying device 12 detects the position of a person photographed by the camera 11 based on image information obtained from each of the cameras 11-1 to 11-n. The attention position specifying device 12 stores in advance the installation positions of the cameras 11-1 to 11-n in the monitoring area and the positions of obstructions that hinder vision from the positions. Furthermore, the attention position specifying device 12 specifies the position where the person is paying attention based on the position of the person and the face angle (line of sight) from the image information captured by each of the cameras 11-1 to 11-n. With the attention position specifying system 10 described above, a suspicious person can be detected from the monitored person with high accuracy as an action in which the monitored person confirms (observes) the monitoring person.

<Attention position specifying device 12: Functional configuration>
Next, the functional configuration of the above-described attention position specifying device 12 will be described with reference to the drawings. FIG. 2 is a diagram illustrating an example of a functional configuration of the monitoring apparatus. The attention position identification device 12 shown in FIG. 2 includes an input means 21, an output means 22, a person position detection means 23, a line-of-sight information acquisition means 24, a attention position identification means 25, a suspicious person determination means 26, and a notification. Means 27, storage means 28, transmission / reception means 29, and control means 30 are provided.

  The input means 21 accepts input of various processes such as an instruction for detecting the position of a person photographed in an image from a user or the like and an instruction for specifying a position of interest. The input means 21 includes, for example, a keyboard, a pointing device such as a mouse, a microphone for inputting instructions by voice, and the like.

  The output unit 22 outputs the instruction content input by the input unit 21, the image information from the camera 11, the person position detection result obtained from the captured image, the target position specifying result, and the like on a display or the like. In addition to the display, the output unit 22 includes an audio output unit such as a speaker, and can output audio data obtained from an audio input unit such as a microphone provided in the camera 11. Further, the output means 22 may have a function of a printer or the like. In that case, printable data such as the contents of image information is printed on a print medium such as paper and provided to the user or the like. You can also.

  The person position detecting means 23 detects the position of the person from the image taken by the camera 11. Specifically, each of the cameras 11-1 to 11-n is installed so as to be able to move only in a fixed or fixed area in order to photograph a predetermined place or area. Therefore, the position of the person can be specified by accumulating in the accumulating unit 28 which positions are photographable by the cameras 11-1 to 11-n and collating with the accumulating means 28. .

That is, the person position detecting means 23 uses a correspondence table for associating the position in the image with the position in the real space, a distance measuring function using a stereo camera, or the like, Convert position to actual space position. The position is specified by calculating the position of the photographed camera 11 as the origin, detecting the two-dimensional coordinates (x ₁ , y ₁ ) of the person's position, Angle information (θ ₁ ) indicating whether a person is located at an angle is detected.

  It should be noted that the method for specifying the position of a person in the present invention is not limited to this. For example, in a photographed image, the size (number of pixels) occupied by a part or all of a person's face and body is preliminarily determined. The position can be specified by obtaining and accumulating the distance for each distance and collating with the size (number of pixels) of the face and body obtained from the actually captured image. Further, as described above, when the camera 11 is provided with a distance sensor in advance, the position information of the person is detected by the distance sensor.

The line-of-sight information acquisition unit 24 acquires information on the angle of the person's face (line-of-sight information), that is, information on where the person is facing, from the image information captured by the cameras 11-1 to 11-n. Specifically, the conventional image recognition processing first detects whether or not a human face exists from the captured image information, and extracts the feature amount of the face after detecting the face to extract the center of gravity of the person's head 3D coordinates (X 1 , Y 2 , Z) are defined, and the face angle (line of sight information) such as the rotation angle and position of the head is acquired.

  The line-of-sight information is based on the angle of the face. However, the present invention is not limited to this. For example, when the position of the eye (pupil) can be acquired from the image information, the position of the pupil is used as a reference. The line-of-sight information may be acquired. Thereby, it is possible to more accurately acquire the line-of-sight information of the person.

  The attention position specifying unit 25 specifies the attention position of the person based on the position of the target object accumulated in advance and the position information and angle information of the person detected by the person position detection unit 23. Details of the person's attention position specifying method in the attention position specifying means 25 will be described later.

  Further, the suspicious person determination means 26 determines whether or not the person is a suspicious person based on the attention position information of the person specified by the attention position specification means 25. Specifically, the suspicious person determination unit 26 accumulates the number of times the target object is present in the direction of the line of sight of the photographed person, and determines that the person is a suspicious person when the accumulated number is equal to or greater than a predetermined number. . Accordingly, it is possible to determine whether or not the person is a suspicious person with higher accuracy based on the number of times the photographed person has seen the target object.

  The suspicious person determination unit 26 accumulates the time during which the target object is present in the direction of the line of sight of the photographed person, and determines that the person is a suspicious person when the accumulated time is a predetermined time or more. Accordingly, it is possible to determine whether or not the person is a suspicious person with higher accuracy based on the time information when the photographed person saw the target object.

  In addition, when measuring the above-mentioned accumulation frequency and accumulation time, the accumulation using one camera may be sufficient, and the accumulation which totaled the several camera set beforehand may be sufficient. That is, when there are a plurality of cameras in which the same object of interest is included in the shooting range, the suspicious person determination means 26 performs the suspicious person determination based on the total accumulated number of times or the total accumulated time in the plurality of cameras. be able to.

  Further, the suspicious person determining means 26 stores face image information of a person determined to be a suspicious person in the storage means 28 and the like, generates feature data of the face of the person from the stored face image information, and Record facial feature data. As a result, the suspicious person determination means 26 checks whether the person determined to be a suspicious person using the facial feature data is the same person as the previously determined suspicious person, and has made suspicious actions in the past. It is also possible to change the degree of the suspicious person's determination result (attention required, attention, etc.) based on the determination result.

Further, when the suspicious person judging means 26 determines that the monitored person is a suspicious person, the notifying means 27 notifies the monitoring person, the security company, or the like. In this case, the notification unit 27, when the observer are in for example the monitoring zone, a lamp or the like is provided at the observer looks, by lighting the lamp when it is determined that it suspicious individual, the monitoring person You can be notified quickly.

  In addition, the notification unit 27 can notify information indicating that there is a suspicious person to a certain monitor or a resident near the suspicious person. In this case, contact information (email address, mobile number, etc.) of the person to be notified is stored in the storage means 28, and the notification means 27 can notify a predetermined person based on the contact information. Thereby, the presence of a suspicious person can be notified quickly. In addition, it is possible to reduce the monitoring burden on the supervisor.

  The information notified by the notification means 27 includes information indicating that there is a suspicious person, position information of the suspicious person, image information of the suspicious person, detailed information such as the reason for determining the suspicious person, and the like.

  The accumulating means 28 is used when image information obtained by the camera 11 (including image data, photographing time information, etc.), a suspicious person judgment result by the suspicious person judging means 26, and the monitored person are judged to be suspicious persons. Various kinds of information such as monitor information such as contact information of the monitor who gives notification to that effect and image information of the monitor determined to be a suspicious person are stored.

  The accumulating unit 28 accumulates parameters such as the accumulated number and accumulated time used when determining whether or not the person is a suspicious person.

  Further, the storage means 28 sets in advance position information corresponding to the cameras 11-1 to 11-n for specifying the position of the person, position information of at least one target object included in the shooting area of each camera, The position information etc. of the shield etc. to be stored is accumulated. As position information, for example, two-dimensional coordinates, three-dimensional coordinates, latitude / longitude information, and the like can be used. The storage unit 28 stores image information taken from each camera, a correspondence table that associates the image used in the above-described person position detection unit 23 with the real space, and the like.

  Various data stored in the storage means 28 can be read out as needed based on a control signal from the control means 30, and data can be updated, added, etc. written, deleted, etc. Can do.

  The transmission / reception means 29 includes an antenna, a communication line, etc., and transmits / receives data or control signals to / from a predetermined camera 11 by wireless or wired communication. Furthermore, the transmission / reception means 29 can contact a security company, a supervisor, etc., when it is determined that the photographed person is a suspicious person. Thereby, the supervisor etc. can acquire information on a suspicious person quickly and correctly.

  The control unit 30 controls the entire components of the target position specifying device 12. Specifically, the control means 30 detects the position of the person in the monitoring area from the person position detection means 23 based on the image taken by the camera 11 or the line of sight of the monitored person taken by the line-of-sight information acquisition means 24. Acquired, the attention position specifying means 25 specifies the attention position of the person included in the image, the suspicious person determination means 26 determines whether or not the monitored person is a suspicious person, Is controlled by the notification means 27 to a supervisor who is set in advance.

<Attention position identification method>
Next, a method for identifying a target position in the present invention will be described with reference to the drawings. FIG. 3 is a diagram illustrating an example of an outline of the attention position specifying method. In the present invention, when the attention position is specified, first, when at least one person (for example, two persons in FIG. 3) is included in the image captured by the camera 11 by the person position detecting unit 23. The position of the person (41-1, 41-2 in FIG. 3) is detected, and the line-of-sight information acquisition means 24 acquires the line-of-sight directions 42-1, 42-2.

  In the attention position specifying means 25, the position of the attention object is determined from the position information of the attention object previously stored with respect to the position, angle and line-of-sight direction acquired by the person position detection means 23 and the line-of-sight information acquisition means 24. 43 is specified.

Next, the identification of the attention position will be described in detail. FIG. 4 is a diagram illustrating an example of a method for identifying a target position using coordinates and angles. As shown in FIG. 4, for example, the position of the camera 11 is set to the origin (0, 0), and the coordinates (x ₁ , y ₁ ) of the position 51 of the person and the distance from the camera 11 to the person are detected by the person position detection unit 23. L ₁ (m) and a reference direction set for the camera 11, for example, an angle θ ₁ with respect to a straight line 52 in the front direction are detected. Further, the line-of-sight information acquisition unit 24 acquires an angle θ ₂ with reference to the straight line 53 that is translated so that the straight line 52 in the front direction of the camera 11 intersects the position 51 of the person.

Here, assuming that the coordinates of the position of the person are (x ₁ , y ₁ ) and the coordinates of the position of a certain target object are (x ₂ , y ₂ ), the position 51 of the person and the position of the target object The angle θ ₂ with respect to 54 can be calculated by “θ ₂ = tan ⁻¹ ((x ₂ −x ₁ ) / (y ₂ −y ₁ ))”. Therefore, if the face angle of the person obtained from the video of the camera 11 is θ ₁ + θ _2, it can be determined that the person is looking at the target object. Note that the reference direction with respect to the camera 11 is not limited to the front direction of the camera 11 and may be based on another direction.

In addition, in the identification of the position of interest, when there are a plurality of objects of interest in the direction of θ ₁ + θ ₂ described above, one of them may be selected (for example, the one closest to the person) It may be determined that all of the plurality of target objects are being viewed.

  Furthermore, in specifying the target position, if there is a shielding object such as a wall between the position of the person and the position of the target object, the person cannot see the target object. When the position information of the shielding object is stored in the storage means 28 in advance, the position information is compared with the position information. The attention position can be specified.

<Other examples of attention location method>
Here, when specifying the position of interest, for example, if the position information of the object of interest is a certain position such as the door knob or keyhole of the entrance door, the position can be specified with high accuracy. However, for example, when the object of interest is a window glass or the like in a house, it is not limited to the locking portion, and there are cases where both sides of the window glass are seen, and in that case, it has a certain size. In determining whether or not the position of the target object is being viewed, it is necessary to provide a certain allowable range (the range of the line-of-sight region).

  Here, the above-mentioned content is demonstrated using figures. FIG. 5 is a diagram illustrating an example of a person's line-of-sight direction when the line-of-sight area is included. FIG. 5 shows, as an example, a case where a person is in a residential premises that is a monitoring area. FIG. 5 (a) is a diagram showing a person entering the site from the side, and FIG. 5 (b) is a diagram showing the person entering the site from the top. is there.

  As shown in FIG. 5, for example, when a person 61 gets over a fence 62 and enters a house site that is a monitoring area, the person 61 is photographed by the camera 11. At this time, when the person 61 is looking at the window 64 provided on the wall 63 of the house, the position of interest can be specified by coordinates or the like if the locking part 65 of the window 64 is used. In practice, however, the entire window or frame portion may be viewed. Therefore, in the present invention, as shown in FIG. 5B, a certain range of the line-of-sight region is set.

  Here, FIG. 6 is a diagram illustrating an example of a determination method for determining whether a person is looking at a target object having a certain size. 6 includes the positional relationship among the camera, the person, and the target object corresponding to FIG. 4 described above. In the present invention, when a target object has a certain size, a plurality of feature points are extracted for the target object.

  Here, for example, as shown in FIG. 5B, the feature points can be the vertices when the target object is viewed from above. Therefore, as shown in FIG. 6, the respective angles between the positions of the vertices and the positions of the persons are obtained, and the range of the line-of-sight region is set based on these angles.

That is, when the target object has a certain size, for example, the coordinates of four points (x ₂₁ , y ₂₁ corresponding to each of the positions of the four target objects 71-1 to 71-4 shown in FIG. ) To (x ₂₄ , y ₂₄ ) and the coordinates (x ₁ , y ₁ ) of the position 51 of the person, the respective angles are calculated. Note that the feature point position information for each target object is stored in the storage unit 28.

Then, the smallest angle θ _2Min and the largest angle θ _2Max are extracted with respect to the straight line 53 shown in FIG. Further, it is determined whether or not the object of interest is being viewed based on whether or not a person's line of sight is within this angular range.

In this case, the minimum and maximum angles are different depending on the orientation of the camera 11, the orientation of the person, and the positional relationship of the target object. Specifically, in the example shown in FIG. _6A , a straight line connecting the coordinates (x ₁ , y ₁ ) of the person's position 51 and the coordinates (x ₂₃ , y ₂₃ ) of the point 71-3 is the angle θ _2Min On the other hand, in the example shown in FIG. 6B, a straight line connecting the coordinates (x ₁ , y ₁ ) of the person's position 51 and the coordinates (x ₂₂ , y ₂₂ ) of the point 71-2 is the angle θ _2Min Become. Furthermore, in FIG. 6B, the angle θ _2Min is a negative angle.

In other words, in the case of FIG. 6 (a), the viewing direction of the position 51 of a person if the range of "θ _2Min ~θ _2Max" relative to the straight line 53, is determined to have been looking at the object of interest, FIG. 6 (b in the case of) determines that if a straight line 53 in the range of "-θ _2Min ~θ _2Max" as a reference, was watching the object of interest.

  In this way, in the case of an object of interest having a certain size, a range of a predetermined line-of-sight region is set, and by performing determination processing based on the range, it is possible to determine where the human line of sight is directed. It can be specified with high accuracy. In the above description, the feature points of the target object are the vertices (four points) when the target object is viewed from above, but the present invention is not limited to this, and the shape of the target object is not limited thereto. A plurality of positions are set according to the above.

  Further, the above-described attention position specifying means 25 monitors the cumulative number of objects of interest viewed by the photographed person, thereby determining whether or not the photographed person is a suspicious person with high accuracy. I can grasp it. Further, the camera 11 itself can be set as the target object of attention, and in this case, it can be determined whether or not the person is a suspicious person based on the cumulative number of times the camera 11 has been viewed.

  With the functional configuration described above, it is possible to specify a person's attention position with high accuracy. Therefore, it can be determined with high accuracy whether or not the photographed person is a suspicious person. This not only detects whether the intruder who entered the site is a suspicious person from the photographed image, but also determines whether it is a suspicious person for previewing from outside the site. Can do. In the above-described embodiment, the position is specified using two-dimensional coordinates, but the same processing as described above can be performed using three-dimensional coordinates.

<Execution program>
Here, the attention position specifying device described above can perform the attention position specification and suspicious person determination processing according to the present invention using the above-described dedicated device configuration or the like, but to cause the computer to execute the processing in each configuration. The attention position specifying process according to the present invention can be realized by generating the execution program and installing the program in, for example, a general-purpose personal computer or server.

<Hardware configuration>
Here, a hardware configuration example of a computer capable of executing knowledge metadata generation according to the present invention will be described with reference to the drawings. FIG. 7 is a diagram illustrating an example of a hardware configuration capable of realizing the target position specifying process according to the present invention.

  7 includes an input device 81, an output device 82, a drive device 83, an auxiliary storage device 84, a memory device 85, a CPU (Central Processing Unit) 86 for performing various controls, and a network connection device. 87, which are connected to each other by a system bus B.

  The input device 81 includes a keyboard and a pointing device such as a mouse operated by a user, a voice input device, and the like, and inputs various operation signals and voice signals such as a program execution instruction from the user. The output device 82 has a display, a speaker, and the like that display various windows, data, and the like necessary for operating the computer main body for performing the processing in the present invention. Display or audio output is possible.

  Here, in the present invention, the execution program installed in the computer main body is provided by a recording medium 88 such as a CD-ROM. The recording medium 88 on which the program is recorded can be set in the drive device 83, and the execution program included in the recording medium 88 is installed in the auxiliary storage device 84 from the recording medium 88 via the drive device 83.

  The drive device 83 can record the execution program according to the present invention on the recording medium 88. Thereby, using the recording medium 88, it can be easily installed in a plurality of other computers, and the attention position specifying process can be easily realized.

  The auxiliary storage device 84 is a storage means such as a hard disk, and can store an execution program according to the present invention, a control program provided in a computer, and perform input / output as necessary. The auxiliary storage device 84 can also be used as a storage unit that stores the position information of the target object described above, the image information captured by the camera 11, the determination result of whether or not the person is a suspicious person, and the like.

  Based on a control program such as an OS (Operating System) and an execution program read from the auxiliary storage device 84 and stored in the memory device 85, the CPU 86 inputs various calculations and data to each hardware component. Each process in the target position specifying process can be realized by controlling the process of the entire computer such as output. Various information necessary during the execution of the program can be acquired from the auxiliary storage device 84 and can also be stored.

  The network connection device 87 obtains an execution program from another terminal connected to the communication network or executes the program by connecting to a communication network such as a telephone line or a LAN (Local Area Network) cable. The execution result obtained in this way or the execution program in the present invention can be provided to other terminals or the like.

  With the hardware configuration as described above, it is possible to realize the target position specifying process at a low cost without requiring a special device configuration. Further, the target position specifying process can be easily realized by installing the program.

<Attention position identification processing procedure>
Next, the attention position identification processing procedure in the present invention will be described using a flowchart. FIG. 8 is a flowchart illustrating an example of the attention position identification processing procedure. FIG. 8 shows a processing procedure for determining whether or not a person photographed by the camera is a suspicious person by using the attention position specifying method described above as an example.

  In FIG. 8, first, a surveillance area is photographed using a camera as an imaging means (S01), and an image photographed from the camera is acquired (S02). Here, the monitoring area refers to the vicinity of the house or the site if it is a house or the like, and includes the inside of the store if it is a convenience store or a bank.

  Note that if the captured image has already been acquired, the above-described processing may be omitted. Next, it is determined whether or not there is at least one person in the image acquired in S02 (S03). If there is a person in the image (YES in S03), at least one person included in the image. Is detected (S04). Note that it is possible to determine whether or not there is a person in the image by using an image processing technique. Further, after acquiring the position of the person, the line-of-sight information of the person is acquired (S05).

  Further, based on the position information of the person acquired in S04 and the line-of-sight information acquired in S05, the position focused on by the person is specified (S06). Further, after specifying the attention position, it is determined whether or not the person is a suspicious person (S07).

  If it is determined that the person is a suspicious person (YES in S07), the security company, the supervisor, the dweller of the house to be monitored, and the like are notified (S08). Note that as information to be notified in S08, in addition to information indicating that there is a suspicious person, the position of a person determined to be a suspicious person, a face image of the suspicious person, and the like can also be notified.

  In addition, suspicious person information is accumulated, and it is determined from the facial feature data whether the person who has been determined to be a suspicious person has been suspicious in the past, and the suspicious person determination result is based on the determination result. It is also possible to notify by changing the degree (needs attention, attention, etc.).

  Next, if there is no person in the image in the process of S03 (NO in S03), if the photographed person is not determined to be a suspicious person in the process of S07 (NO in S07), or in S08 After the notification process is completed, it is determined whether or not the monitoring of the monitoring area is ended (S09). When the monitoring is continued (NO in S09), the process returns to S01 and the above-described process is continued. If the monitoring is to be terminated (YES in S09), the process is terminated.

  Here, the suspicious person determination processing procedure in S07 will be described with reference to a flowchart.

<Suspicious person judgment processing>
FIG. 9 is a flowchart illustrating an example of a suspicious person determination processing procedure. In FIG. 9, first, as shown in S06 described above, after specifying the attention position of the photographed person (monitored person), the cumulative number of times the target object is present in the person's line of sight is measured (S11). ). Here, the cumulative number of times is counted continuously from the previous number of times when the person has once removed his line of sight from the object of interest and then is looking at the object of interest again. It is the number of times.

  Next, it is determined whether or not the predetermined number has passed (S12). If the predetermined number has passed (YES in S12), the person is determined to be a suspicious person (S13). . If the predetermined number has not elapsed (NO in S12), it is determined whether the person has moved out of the monitoring area (S14), and the person has moved out of the monitoring area (in S14). YES), the suspicious person determination process is terminated. In S14, if the person is photographed by, for example, another camera and has not moved outside the monitoring area (NO in S14), the process returns to S11 and the cumulative number of times is measured.

  In this way, it is possible to determine whether or not the person is a suspicious person with higher accuracy based on the number of times the photographed person views the target object.

<Other judgment procedures>
In addition, as the above-described suspicious person determination process, for example, the accumulated time of looking at the target object is measured with reference to the line-of-sight information of the photographed person (monitored person), and the accumulated time has passed a predetermined time. In this case, it can be determined that the person is a suspicious person. Accordingly, it is possible to determine whether or not the person is a suspicious person with higher accuracy based on time information when the photographed person is looking at the target object. Moreover, the suspicious person determination process in the present invention is not limited to this, and for example, the above-described cumulative number and cumulative time may be combined. Further, even if the target object has a certain size, it is possible to determine with high accuracy whether or not the person is looking at the target object by setting an appropriate range of the line-of-sight direction. Further, it is possible to determine with high accuracy whether or not the photographed person is a suspicious person based on the specific information of the attention position. Thereby, the monitoring burden of a supervisor etc. can be reduced. Furthermore, it is possible to determine whether or not the person is a suspicious person with higher accuracy based on the number of times the photographed person has viewed the target object.

  Note that the attention position specifying method in the present invention can detect a suspicious person with higher accuracy by applying it to a monitoring system or the like around a house as described above. In the above description, a house is used as an example of a monitoring area. However, the present invention can be applied to a building such as an office, a hospital, a bank, and the like.

  Further, the attention position specifying method in the present invention can be applied to the case where the attention level or interest level of a product or the like is checked by using the target object as a product, an advertisement, a signboard, or the like.

  Although the preferred embodiment of the present invention has been described in detail above, the present invention is not limited to the specific embodiment, and various modifications, within the scope of the gist of the present invention described in the claims, It can be changed.

It is a figure which shows an example of schematic structure of the attention position identification system in this invention. It is a figure which shows an example of a function structure of an attention position specific device. It is a figure of an example for demonstrating the outline | summary of an attention position identification method. It is a figure of an example for demonstrating the specific method of the attention position using a coordinate and an angle. It is an example for demonstrating the gaze direction of a person in case it has the range of a gaze area | region. It is a figure which shows an example of the determination method which determines whether the person is looking at the attention object which has a certain fixed magnitude | size. It is a figure which shows an example of the hardware constitutions which can implement | achieve the attention position specific process in this invention. It is a flowchart which shows an example of an attention position specific process sequence. It is a flowchart which shows an example of a suspicious person determination processing procedure.

Explanation of symbols

10 target location system 11 camera 12 focused localization equipment 21 input means 22 output means 23 person position detector 24 viewing information acquisition unit 25 the target position specifying means 26 suspicious individual judgment unit 27 notification unit 28 storage unit 29 transmitting and receiving means 30 control Means 41, 51 Position of person 42 Visual direction 43, 54, 71 Position of target object 52, 53 Straight line 61 Person 62 Fence 63 Wall 64 Window 65 Locking part 81 Input device 82 Output device 83 Drive device 84 Auxiliary storage device 85 Memory Equipment 86 CP U
87 Network connection device 88 Recording medium

Claims (9)

In a target position specifying system for specifying a target position of a person in a preset monitoring area,
One or a plurality of imaging means for capturing the monitoring area and acquiring image information;
Based on the image information obtained by the imaging means, the reference direction of the imaging means with, which is set in advance the position coordinates of the person positioned relative to the said imaging means and detecting the position information of the person Person position detecting means for detecting an angle from the person to the direction in which the person is present as angle information of the person;
Line-of-sight information acquisition means for acquiring line-of-sight information of the person from the image information;
Attention position specifying means for specifying the attention position of the person based on the position information and angle information of the person obtained by the person position detection means and the line-of-sight information obtained by the line-of-sight information acquisition means ;
Previously set one or more target objects included in the imaging area of the imaging means, have a storage means for storing position information of the object of interest,
The attention position specifying means includes
Based on the position information of the target object stored in the storage unit and the position information of the person, the angle information from the person to the target object with respect to the reference direction of the imaging unit is acquired and acquired. Based on the angle information from the person to the object of interest and the line-of-sight information obtained by the line-of-sight information acquisition means, identify the position of interest of the person,
Further, when the target object has a predetermined size, a plurality of feature points are extracted from the target object, and angle information at each feature point is obtained from the person with respect to the reference direction of the imaging means. Then, a maximum angle and a minimum angle are selected from the acquired plurality of angles, and the attention position of the person is specified based on the selected maximum angle and the minimum angle and the line-of-sight information. Attention position identification system. Based on the target position of the person obtained by the target position specifying means, a particular target position according to claim 1, characterized in that it has a suspicious individual judgment means the person to determine whether a suspicious person system. The suspicious person judging means is
3. The attention position according to claim 2 , wherein the number of times that the target object exists in the line-of-sight direction of the person is accumulated, and the person is determined to be a suspicious person when the accumulation number is a predetermined number or more. Specific system. The suspicious person judging means is
4. The time according to claim 2 , wherein the time when the target object is present in the line-of-sight direction of the person is accumulated, and the person is determined to be a suspicious person when the accumulated time is a predetermined time or more. Attention location system. In an attention position specifying method for specifying an attention position of a person in a preset monitoring area,
An imaging step of imaging the monitoring area by one or more imaging means;
Based on the image information obtained by the image pickup step, said image pickup means, wherein the position coordinates of the image of the person located relative to the imaging means and detects as the position information of the person is preset A person position detecting step of detecting an angle from the reference direction to the direction in which the person is present as angle information of the person;
A person position detecting step for detecting position information of the person and angle information of the person relative to a preset reference direction of the image pickup means based on the image information obtained by the imaging step;
A line-of-sight information acquisition step of acquiring line-of-sight information of the person from the image information;
Position information and angle information of the person obtained by the person position detection step, on the basis of the line of sight information obtained by the viewing information acquisition step, possess a target position specifying step of specifying a target position of the person ,
The attention position specifying step includes:
From the person to the target object with respect to the reference direction of the imaging unit based on the position information of one or more target objects included in the imaging region of the imaging unit set in advance and the position information of the person The position information of the person is identified based on the acquired angle information from the person to the target object and the line-of-sight information obtained by the line-of-sight information acquisition step,
Further, when the target object has a predetermined size, a plurality of feature points are extracted for the target object, and an angle at each feature point is acquired from the person with respect to a reference direction of the imaging unit. A maximum angle and a minimum angle are selected from the plurality of acquired angles, and the attention position of the person is specified based on the selected maximum angle and the minimum angle and the line-of-sight information. Attention position identification method. 6. The attention position specification according to claim 5 , further comprising a suspicious person determination step of determining whether or not the person is a suspicious person based on the attention position of the person obtained in the attention position specification step. Method. The suspicious person determination step includes
The attention position according to claim 6 , wherein the number of times that the target object exists in the line-of-sight direction of the person is accumulated, and the person is determined to be a suspicious person when the accumulation number is a predetermined number or more. Identification method. The suspicious person determination step includes
Accumulating the time during which the object of interest to the viewing direction of the person is present, when the accumulated time is equal to or longer than a predetermined time, according to claim 5 or 6, characterized in that determining the person to be suspicious person Attention position identification method. An attention position specifying program for causing a computer to function as each means included in the attention position specifying system according to any one of claims 1 to 4 .

Images