JP2020115608A - Sensor system, image sensor, and sensing method - Google Patents

Abstract

To further enhance availability of a sensor system.SOLUTION: According to an embodiment, a sensor system comprises: a plurality of image sensors; and means for transmitting feature quantity information on a moving body which is extracted from image data captured by the image sensors to other image sensors as attribute information of a target moving body. Further, each of the image sensors comprises: an imaging part for acquiring the image data; an extraction part for extracting the feature quantity information from the image data; a determination part for determining the target moving body on the basis of the feature quantity information; a data storage part for storing dictionary data relating to the determination; and a communication part for transmitting the extracted feature quantity information to the other image sensors.SELECTED DRAWING: Figure 5

Description

Embodiments of the present invention relate to a sensor system, an image sensor, and a sensing method.

The image sensor in recent years includes a CPU (Central Processing Unit) and a memory, and can be said to be an embedded computer with a lens. It also has a high-level image processing function, and can analyze photographed image data to calculate the presence/absence of a person or the number of people, for example.

Object recognition by image sensors has been studied for a long time. In particular, research has been advanced in recent years, such as finding obstacles during movement using an image sensor mounted on a robot that plays the role of a robot. Regarding object recognition, there is a document on obstacle detection using chamfer distance, for example. In addition, a method has also been proposed in which a moving body detected by using an image sensor is collated with a dictionary to confirm that the moving body is a person, and lighting control is performed.

There is a method using HOG (Histgrams of Oriented Gradients) as a method capable of extracting feature amount information having high discriminative ability with respect to a target moving object. This method called a brightness gradient direction histogram is a method of dividing the candidate region into a plurality of blocks and converting the brightness gradient direction of each block into a histogram. By using only the presence or absence of a gradient, it is said that it is less affected by changes in brightness and is robust against changes in shape. Further, a method called Co-HOG (Co-occurrence HOG), which has improved shape recognition ability, has also been developed.

JP, 2010-9847, A WO2016/031720 WO2011/114770 Patent No. 4759988

Lakshitha Dantanarayana*, Gamini Dissanayake, Ravindra Ranasinge, C-LOG: A Chamfer distance based algorithm for localisation in occupancy grid-maps, CAAI Transactions on Intelligence Technology 1 (2016) 272e284, http://www.journals.elsevier.com/ caai-transactions-on-intelligence-technology/ Navneet Dalal and Bill Triggs, Histograms of Oriented Gradients for Human Detection, INRIA Rh.one-Alps, 655 avenue de l'Europe, Montbonnot 38334, France

The needs for solutions using image sensors are increasing year by year. For example, from the captured image, an unexpected outing of a wandering elderly person from a health facility, a person who suddenly has difficulty walking due to poor physical condition, a drunken person, a lost child in a public place, or a monitor of a behavior suspicious person, etc. , There is a desire to discover a moving body (person) who behaves in an unusual manner. There is also a demand for grasping the flow lines of vehicles such as forklifts operating in a warehouse even with the same moving body. As described above, the image recognition technology has various needs, and expectations for the image sensor and the sensor system using the image sensor are great.

Therefore, it is an object to provide a sensor system, an image sensor, and a sensing method with higher availability.

According to the embodiment, the sensor system transmits the plurality of image sensors and the characteristic amount information regarding the moving body extracted from the image data captured by the image sensor to other image sensors as the attribute information of the target moving body. Further, each of the image sensors includes an imaging unit that acquires the image data, an extraction unit that extracts the characteristic amount information from the image data, and the target moving object based on the characteristic amount information. And a data storage unit that stores dictionary data relating to the determination, and a communication unit that transmits the extracted feature amount information to the other image sensor.

FIG. 1 is a diagram showing an example of a sensor system 1 according to the embodiment. FIG. 2 is a functional block diagram showing an example of the sensor system. FIG. 3 is a diagram showing an example of the arrangement of image sensors. FIG. 4 is a functional block diagram showing an example of the sensor system according to the embodiment. FIG. 5 is a functional block diagram showing another example of the sensor system according to the embodiment. FIG. 6 is a functional block diagram showing an example of the image sensor 10. FIG. 7 is a diagram for explaining an example of information transmission between image sensors in the embodiment. FIG. 8 is a diagram for explaining another example of information transmission between image sensors in the embodiment. FIG. 9 is a diagram showing an example of the next state of FIG. FIG. 10 is a diagram showing an example of the next state of FIG. 9. FIG. 11 is a diagram for explaining another example of information transmission between image sensors in the embodiment. FIG. 12 is a diagram for explaining an example of a method of determining the information transmission destination between the image sensors in the embodiment. FIG. 13 is a diagram showing an example of the next state of FIG. FIG. 14 is a flowchart showing an example of a feature quantity information transmission system of a target moving body in the embodiment. FIG. 15 is a flowchart showing a data processing procedure of an abnormal system in the feature quantity information transmission system of the target moving object in the embodiment. FIG. 16 is a diagram showing an example of setting information shared by the image sensors and the feature amount information to be transmitted in the embodiment. FIG. 17 is a diagram showing an example of the characteristic amount information collected by the image sensor on the transmitting side and transmitted as cumulative information in the embodiment. FIG. 18 is a flowchart illustrating an example of processing of target moving body attribute information regarding a specific moving body to be monitored. FIG. 19 is a diagram illustrating an example of simple motion detection according to the embodiment. FIG. 20 is a diagram showing an example of the entire trace map in the embodiment. FIG. 21 is a diagram showing an example of conditions for transmitting moving body feature amount information in the embodiment.

There is a case in which it is desired to grasp the path of movement of a moving body (for example, a person) imaged by an image sensor as a movement line and to know whether the movement line is a normal person moving with a purpose or not. This is done, for example, to check whether the moving body is a prowler or a lost child. If the target moving body (person) has a transmitter such as a mobile phone or a smartphone, it can be detected by GPS (Global Positioning System), etc. However, if the transmitter does not have a transmitter or indoors, this method is applied. I can't. In this case, for example, the flow line is grasped by the image sensor installed in the building.

A moving body that behaves in an unusual manner, such as an unexpected outing of a wandering elderly person from an old-age facility, sudden walking difficulty due to poor physical condition, a drunken person, a lost child in a public place, or a monitor for a suspicious individual. There is a demand to find out (person) from a captured image. There is also a demand for grasping the flow lines of vehicles such as forklifts operating in a warehouse even with the same moving body.

For example, image data is collected by using a person's face as a detection target, and the target moving object is specified by comparing it with separately registered dictionary information. Emotions such as tension, pleasure, and discomfort are estimated from minute differences in facial expressions. There is a way to do it. In this case, data acquisition is usually possible with one image sensor. On the other hand, there is a problem in carrying out face recognition at all times for the protection of personal information. Further, in order to improve the accuracy of shape recognition of a moving body including face recognition, it is necessary to learn a large amount of dictionary information and install a large number of image sensors in consideration of the direction of the moving body to be photographed. Further, the photographed side may feel uncomfortable because the camera is visible in the field of view. There is also a method of lowering the resolution and collecting the shape information of the moving body from the clothes color, shape, belongings, etc., but the above demands are not always satisfied. That is, the above requirements cannot be satisfied only by using the shape information of the moving body.

As another method, the motion of the moving body is monitored by one image sensor, and, for example, as a sign of shoplifting, there is no confirmation by a swiveling motion that there is no person in the surroundings, or there is no hand. There is a method to monitor natural movements. In such a case, usually, data can be collected by one image sensor. In a typical operation, a warning is sent to a monitor or a store clerk in the case of a store, whereby it is possible to prevent the occurrence or confirm the occurrence of shoplifting at an early stage. There is a problem that it cannot cover all the above requests.
The embodiment discloses a method of identifying a wandering elderly person, a lost child, a suspicious person, or simply the presence or absence of a moving body by using a plurality of image sensors in consideration of the flow line grasp.

[Constitution]
FIG. 1 is a schematic diagram showing an example of a sensor system 1 according to the embodiment. The sensor system 1 is a system for realizing monitoring of the target space 4 using a sensor. The target space 4 may be, for example, a manned space such as a building, a store, an office, etc., or a factory with loading/unloading of equipment, or an unmanned space such as a server room. The three-dimensional shape of the target space 4 does not have to be a rectangular parallelepiped as shown in the figure, and may be any spatial shape in which the sensor is installed. Further, it may be a multi-story, substantially hollow building including the entire building in which the sensor system 1 is installed.

The target space 4 is divided into, for example, areas a to f. The monitoring of the areas a to f by the sensors is realized by the corresponding sensor devices 9a to 9f and the communication units 20a to 20f. For example, the central server 30 collects and processes the information sent from each communication unit 20, whereby the air conditioning control in the entire target space 4 is realized. Areas a to f correspond to the sensor device 9 and the communication unit 20, which are distinguished by the characters (a to f) common to each. Hereinafter, when the areas a to f are not distinguished, they are simply referred to as “area”.

The sensor system 1 includes sensor devices 9a to 9f, communication units 20a to 20f, and a central server 30. The communication units 20a to 20f control the sensor devices that are distinguished by the common characters (a to f) among the sensor devices 9a to 9f, respectively. Hereinafter, when the sensor devices 9a to 9f are not distinguished, they are referred to as the sensor device 9, and when the communication units 20a to 20f are not distinguished, they are referred to as the communication unit 20.

The communication units 20 and the central server 30 are connected by a communication line 5 and are configured to be able to communicate with each other. The communication line 5 may be Ethernet (registered trademark), which is a typical wired LAN, or PLC (Power Line Communication), for example. Communication between each communication unit 20 and the central server 30 may be realized by wireless communication. The central server 30 totally controls each communication unit 20 based on the measurement data acquired from each communication unit 20.

FIG. 2 is a functional block diagram showing an example of the sensor system. As an example, the sensor devices 9a to 9d collect environmental information (sensor data) such as room temperature and illuminance. The sensor data is transmitted from the communication units 20a to 20d connected to the sensor devices 9a to 9d to the central server 30 via the primary communication system via the communication line 5.

The central server 30 analyzes the sensor data received via the communication unit 31 by the analysis unit 32, and determines the next processing by the determination unit 33. At the time of the determination, in addition to the comparison with the set value, the determination is made by comparing with the data in the past data 34, for example. The series of processes is controlled by the CPU 35 of the control/processing unit. That is, the CPU 35 issues a control command to the sensor device 9 or the air conditioner or lighting device (not shown) via the communication unit 31, and notifies it.

By the way, in the embodiment, a system in which an image sensor is applied as the sensor device 9 will be described. The image sensor can acquire various kinds of information as compared with a human sensor, a light sensor, an infrared sensor, or the like. If a fish-eye lens or a super wide-angle lens is used, the area that can be photographed by one image sensor can be enlarged, and image distortion can be corrected by calculation processing. There is also known an image sensor having a learning function and a function of setting a mask for an area in the field of view that is not desired to be sensed. Furthermore, by updating the dictionary data for detecting the dictionary data target, it is possible to optimize the detection performance of the target object.

A plurality of image sensors are provided, for example, on the ceiling of the target space 4 at predetermined intervals. The imaging range of each image sensor may be an image sensor placed in the corner of the ceiling in a substantially rectangular room, or a wide viewing angle lens such as a fisheye lens that can image 360 degrees around the center of the room is used. It may be an image sensor. Further, the image pickup area of each image sensor may partially overlap with an adjacent image sensor, or there may be a blind spot area.

FIG. 3 is a diagram showing an example of the arrangement of image sensors. In this example, there is no information path connecting the image sensors 10a to 10d, each image sensor functions independently, and data sharing between the image sensors is not assumed.

FIG. 4 is a functional block diagram showing an example of the sensor system according to the embodiment. Environmental sensors 8a-8d and image sensors 10a-10d are shown as sensor devices. Of these, the environment sensors 8a to 8d measure the environment of the target space such as temperature, humidity, illuminance, air volume, and volume. A motion sensor using infrared rays is also included in this. The environment sensors 8a to 8d are connected to the central server 30 via the communication line 5 by the primary communication system.

Further, image sensors 10a to 10d are installed separately from the environment sensors 8a to 8d. The image sensors 10a to 10d are connected to each other via the communication units 21a to 21d to form a secondary communication system. That is, while the communication units 20a to 20d (primary communication system) related to the environment sensor are directly connected to the central server 30, the communication units 21a to 21d (secondary communication system) connect the image sensors 10a to 10d. Connecting.

The communication method of the secondary communication system and the communication method of the primary communication system may be the same or different. The secondary communication system may be an event driven wireless temporary connection, for example, an ad hoc connection. Alternatively, the secondary communication system may be multiplexed with the primary communication system. This is because the requirement for the secondary communication system is that it is possible to transmit attribute information of a moving body from one image sensor to another image sensor.

In FIG. 4, the communication units 21a to 21d can communicate with each other. In the embodiment, the communication of the communication units 21a to 21d is executed based on the judgment of the image sensor itself. For example, the image sensor 10a extracts feature amount information from the image captured by itself, and the CPU inside the image sensor 10a determines whether or not communication with another image sensor is possible. When it is determined to be necessary, the moving object feature amount obtained by the image sensor 10a is transmitted to the peripheral image sensors 10b, 10c, and 10d via the communication unit 21a via the respective communication units 21b, 21c, and 21d. Information is transmitted. In the following description, the moving body characteristic amount information, the characteristic amount information, and the moving body information all have the same meaning.

The image sensors 10b, 10c, and 10d that have received the moving body feature amount information observe whether or not an object that matches the moving body feature amount information appears in the image being captured, by the CPUs provided in the image sensors 10b, 10c, and 10d. For example, when a moving body that matches the moving body feature amount information transmitted by the image sensor 10c is detected in the captured image, the transmitted moving body feature amount information and the moving body feature amount information collected by the corresponding image sensor are detected. In addition, the accumulated moving body feature amount information is further transmitted to the peripheral image sensor group. By such an operation, it is possible to continuously monitor the target moving object with immediacy.

FIG. 5 is a functional block diagram showing another example of the sensor system according to the embodiment. Even if the master 22 that controls the transfer of data from the communication units 21a to 21d is provided in the secondary communication system, communication between the image sensors can be realized regardless of the central server 30. The function of the master 22 may be provided in a device that exclusively controls the exchange of data between the image sensors, or may be shared by any of the image sensors.

Also in FIG. 5, the connection of the secondary communication system may be either one that is permanently maintained on the priority resource or one that is temporary by using the wireless resource. The communication system of the secondary communication system may be an event driven ad hoc communication system in which communication is started only when the image sensor detects a situation to be transmitted. The communication topology of the secondary communication system is not particularly limited, such as ring type, tree type, and mesh type. Even if a communication unit of any one of the image sensors fails due to a failure or a disaster, if the topology that can form the circuit is used, the connection with the other image sensor groups that form the sensor network can be maintained. You can A health check with the communication partner may be performed in advance.

In the configuration of FIG. 5 as well, by transmitting the moving body feature amount information between the image sensors, it is possible to generate cumulative feature amount information in the image sensors and make a final determination regarding the moving body determination.

It is necessary to provide a mechanism for time synchronization between the image sensors shown in FIGS. This is because the coordinate information and the time information are transmitted for the process of grasping the flow line. Regarding coordinate information, it has relative coordinates (camera coordinates) of each image sensor, but in the end, since an entire trace map is created for tracing a flow line, absolute coordinates (system coordinates) are required. Each image sensor may be converted into absolute coordinates and then passed to the next image sensor as transmission information. As a result, the final whole trace map can be easily created in any image sensor.

FIG. 6 is a functional block diagram showing an example of the image sensor 10. The image sensor 10 includes an image capturing camera 11 and a communication unit 21 connected to a secondary communication system. The image data acquired by the image capturing camera 11 is preprocessed by the image preprocessing unit 12 and converted into an image that is easy to determine. Next, the feature amount information extraction unit 13 extracts the feature amount information from the preprocessed image data, and the determination unit 14 determines the moving body. At this time, the dictionary data 16 stored in the data storage unit is referred to. The entire control is performed by the CPU 15. The imaged data after the determination and the extracted feature amount information are stored in the storage unit 17 as necessary. Next, the operation of the above configuration will be described.

[Action]
FIG. 7 is a diagram for explaining an example of information transmission between image sensors in the embodiment. It is assumed that the transmitted information is feature amount information generated from image data. As shown in FIG. 7, it is assumed that a large number of image sensors are arranged so as to capture images at different spatial positions. The layout shown in FIG. 7 represents an image sensor group arranged on one floor, for example. A number for identifying each other is given to each image sensor 10. FIG. 7 shows an example of numbering based on spatial position, which can be represented by xy coordinates.

There are several possible methods for transmitting the moving body feature amount information. First, there is a method of simultaneously transmitting the moving body feature amount information found by a certain image sensor and determined to be transmitted to all the image sensors in FIG. This method can be used when the adjacent relationship of each image sensor is registered as known information,
Here, instead of this method, a method of further reducing the transmission range to reduce the network load and the processing load of the image sensor will be sequentially disclosed.
Firstly, a method of concentrically transmitting moving body feature amount information from an image sensor that detects a moving body (for example, the central image sensor 10F41) to peripheral image sensors is conceivable. This method can be applied when the moving body moves linearly in the radial direction of the concentric circles with the image sensor 10F41 as the center. However, this method does not match when the flow line of the target moving body is not linear. Next, a method applicable even when the flow line is freely bent will be described.

8 to 10 are diagrams for explaining another example of information transmission between the image sensors in the embodiment. 8 is the first state, and time elapses from this state in the order of FIGS. 9 and 10. First, consider a case where the moving body feature amount information extracted from the image captured by the central image sensor 10F41 of the image sensor group in FIG. 8 exceeds a predetermined threshold value. That is, the image sensor 10F41 is the image sensor A1 that first detects the target moving body.

[Procedure (1)]
When the feature amount information that exceeds the threshold is detected (feature amount information>threshold is satisfied), the image sensor A1 notifies the peripheral image sensor groups B1 to B6 to continuously monitor the corresponding moving body. Next, the procedure (2) of FIG. 9 is implemented.

[Procedure (2)]
The moving body feature amount information is transmitted as a caution flag from the image sensor A1 to the predetermined peripheral image sensor groups B1 to B6. As shown in FIG. 9, it is assumed that the moving body feature amount information is transmitted from the image sensor A1 to the image sensors B1 to B6. The image sensors B1 to B6 that have received the caution flag monitor whether or not the target moving body is on the image based on the received moving body information. When the target moving body cannot be detected within a predetermined time, information indicating that the target moving body cannot be detected may be transmitted and shared to the image sensors B1 to B6 and A1 to be transmitted. Here, it is assumed that the image sensors B1 to B6 detect the target moving body.

[Procedure (3)]
The image sensors B6 and A1 detect that there is no moving object that matches the received attention flag information, and transmit it to the predetermined peripheral image sensor groups B1 to B6 and A1 (excluding the image sensor for transmission). Here, it is assumed that the target moving body is detected by the image sensor B2.

[Procedure (4)]
It is assumed that the image sensor B2 detects that a moving object that matches the received attention flag information is present in the visual field. When the target moving body is detected by the image sensor B2, the moving body characteristic amount information collected and analyzed by the image sensor B2 is added to the moving body characteristic amount information received from the image sensor A1 as temporal information to generate cumulative characteristic amount information. ..

It is expected that the determination accuracy can be further increased by using the flow line information collected via a plurality of image sensors. Therefore, for the determination of the cumulative feature amount information, a cumulative threshold value is defined, and the threshold value determination is performed using the value. The transmission destination of the accumulated characteristic amount information is not the image sensors B1 to B6 and A1 notified in the procedure (3), but a new image sensor centered on the image sensor B2 is determined. At that time, the information to be transmitted becomes the information of the target moving body including the accumulated characteristic amount information.

[Procedure (5)]
As shown in FIG. 10, the new peripheral image sensor groups C1 to C4 centering on the image sensor B2 are notified of the cumulative feature amount information together with the moving body feature amount information. That is, the peripheral image sensors C1 to C4 are selected around the image sensor B2, and the caution flag is transmitted. For example, the image sensor C1 corresponds to the image sensor 10F23, similarly, C2 corresponds to 10F53, C3 corresponds to 10F41, and C4 corresponds to 10F22.

The peripheral image sensors are determined around the image sensor (B2 in this case) that has detected the target moving object at the latest time. Since it is unknown in which direction the target moving body moves, the image sensor 10F41 in the direction returning to the original image sensor A1 is also newly selected as the transmission destination as the image sensor C3. For example, the image sensor C4 detects the target moving object, and the CPU in the image sensor C2 calculates the cumulative feature amount information.

[Procedure (6)]
There is a moving object that matches the received attention flag information, and the judgment of (cumulative feature amount information> cumulative final threshold value) is Yes, and there is a moving object that exceeds the cumulative final threshold value. Notify me first. For example, in a system including the central server 30, if the central server 30 is connected to the image sensor, the central server 30 may be the target notification destination. In this case, the supervisor is informed of the information via the central server 30.

As another method, it may be notified that all of the image sensor groups include a moving object having cumulative feature amount information that exceeds the final threshold value. Then, the appearance of such a moving body may be notified by a wired or wireless network connected to the image sensor. Alternatively, the notification can be made by lighting the LED or making a sound. The timing of transmitting moving body information between image sensors and the method of limiting the destination will be further described.

FIG. 11 is a diagram showing an example of a photographing region (a plane space to be detected) in the image sensor A1 of FIG. The photographing region is, for example, a square, and the square can be further divided into a lattice shape. In order to simplify the explanation, in the figure, it is assumed that the space is partitioned into a 4×4 square shape. Thus, it is possible to identify which of the 16 squares the moving body is detected.

The masses corresponding to the edges (edges) and corners of the imaging area of the image sensor A1 are denoted by reference numerals e1 to e12, and the masses in the center (Center) of the imaging area away from the edges are assigned numbers c1 to c4. And show it. By continuously observing in which square the moving body is detected, the timing and destination of transmitting the moving body information between the image sensors can be determined.

The first method of determining the transmission timing is to set the timing at which the moving body is detected in any of e1 to e12 as the transmission timing. That is, this is a case where a moving body is detected on a specific mass. This is because the moving body may be out of the visual field of the image sensor A1 when detected by the outer masses e1 to e12.

The second method of determining the timing of transmission is to set the timing of transmission as the time when the moving body moves from the central portion (c1 to c4) toward the edges (e1 to e12). Even in this case, the moving body is likely to go out of the imaging visual field of the image sensor. This is a method of determining the transmission timing based on the moving direction of the moving body, that is, the direction of the moving body vector. It should be noted that the timing of transmitting the time when the moving body is moving in a specific direction and the moving body comes on a specific mass may be used in combination with the first and second methods. By doing so, improvement in accuracy can be expected.
As a third method of determining the transmission timing, the transmission timing may be set after the time point when the moving body moves out of the visual field of the image sensor and the moving body cannot be detected. Next, a method of determining the transmission destination of the moving body information will be described.

The first method for determining the destination of the moving body information is to send the moving body feature amount information to all the image sensors in the target space.
A second method of determining the transmission destination of the moving body information is, for example, as shown in FIGS. 8 to 10, a method of determining the transmission destination concentrically (in the 360° direction) around the image sensor detecting the moving body. Is.

A third method of determining the transmission destination of the moving body information is to send the moving body information toward the image sensor in the edge direction depending on which edge of the imaging range of the image sensor A1 in FIG. Is. For example, when the imaging area is rectangular and a moving object is detected at the edge e5 or e6 of the imaging area, the moving object information is transmitted to the edge side image sensors 10F33 and 10F53. By doing so, the moving body information can be sent only to the sensor having a higher probability of appearance of the moving body, which is efficient in terms of saving the transmission band.

When a moving body is detected by e4 of the image sensor A1, the moving body information is transmitted to the image sensors 10F22 and 10F33 because it corresponds to a corner. When the moving body is detected at e7, the moving body information is transmitted to 10F53 and 10F62.

In addition, the imaging range of the image sensor may overlap with the adjacent image sensor at the edges. In this case, the timing of transmitting the moving body information or the transmission destination may be waited until after the time when the image sensor of the transmission destination detects some moving body. Details will be described later.

FIG. 12 is a diagram schematically showing that the transmission range is limited when a moving body is detected at the corner e4 of the image sensor A1. In the situation of FIG. 12, it is assumed that 10F22 (image sensor B1) and 10F33 (image sensor B2) detect a moving body. Therefore, the moving body information is transmitted to these image sensors. For example, the mass of the image sensor B2 that can arrive from the corner e4 of the image sensor A1 is e8 to e12.

The first method of detecting a moving body by the image sensor B2 is that the moving body is the same moving body if the moving body is detected within a predetermined time after the moving body information is sent from the image sensor A1 by the above-mentioned mass. Detection method.

A second method for detecting a moving body is to first identify information relating to the shape of the moving body, for example, a person, by using the dictionary information of each image sensor in FIG. 6 or the past imaging data and feature amount information. Is the way to do it.

A third method for detecting a moving object is to detect that the same moving object is detected based on the information related to the shape of the moving object in the moving object information sent from the image sensor A1, for example, HOG information or CoHOG information. , And it is a method to grasp the flow line. You may use combining these three moving body detection methods. In this way, the moving body can be detected or the target moving body can be identified.

In addition, it is assumed that the target moving body that has reached the edge portion is detected at the edge portions e5 and e6 after the elapse of the time when the target moving body has moved within the imaging range of the image sensor B2. The image sensors to which the image is moved from the edge are 10F23 and 10F63. In such a case, the search range may be further expanded and 10F53 may be added to the transmission destination of the moving body information.

FIG. 13 shows an example of the next state of FIG. FIG. 13 shows that the image sensor groups C1, C2, and C3 are selected as transmission destinations when a moving body is detected at the edges e5 and e6 of the image sensor B2. For example, in the image sensor C1, it is assumed that moving bodies appear from e1, e10, e11, and e12. When the target moving object is detected by the image sensor C1, it is received by adding the feature amount information of the image sensor C1 to the accumulated feature amount information from the image sensor B2 to which the feature amount information from the image sensor A1 is added. It is detected that there is a moving body that matches the target moving body information. Further, the judgment of (cumulative characteristic amount information>cumulative final threshold value) is Yes, and the fact that there is a moving object exceeding the cumulative final threshold value is notified to the administrator or the target notification destination. At this time, at the latest time, the information indicating in which square the target moving body is located may be sent. In this case, for example, it is indicated that there is a moving body in the mass C3 of the image sensor C1.

Regardless of whether the determination based on the cumulative final threshold value is Yes or No, the moving object information has a history of the time stamp and which image sensor the moving object information was sent to. In order to surely delete the data in the image sensor, a deletion instruction of unnecessary moving body information may be transmitted via the image sensor C1. Alternatively, if the determination by the cumulative final threshold value is Yes, the deletion instruction may be issued after the instruction to store the image data or to collect the image data in a predetermined location. Alternatively, a uniform processing rule may be adopted in which, when a predetermined time has elapsed from the start of storage, each image sensor preferentially deletes moving object information including the captured image itself as a deletion target. By performing any one of these processes, the limited storage capacity in the image sensor can be effectively utilized.

FIG. 14 is a flowchart showing an example of a feature quantity information transmission system of a target moving body in the embodiment. This flowchart is executed mainly by an arbitrary image sensor. In FIG. 14, S101 is a step which is a source of data collection in this flowchart. In S101, location setting information including moving body feature amount information from other image sensors is collected. This includes common information that is commonly held between image sensors, such as updated thresholds, the positional relationship of image sensor groups, and image preprocessing, and each image sensor dynamically collects information. The reception information of the moving body feature amount information is included. If there is image data captured by the image sensor itself, the information is also included.

Next, in S102, it is determined whether or not the image information collected by this image sensor is detected from the collected image as abnormal system data. If NO in S102, that is, if it is determined that the collected image is normal, the process proceeds to the next step S103.

In S103, it is determined whether or not there is the transferred moving body feature amount information. If there is no transferred information, the result is No and the process proceeds to S210. If there is the transferred information, the result is Yes, and the processing procedure moves to the next step S104.

In S104, image collection for feature amount information extraction is started. This is the step of collecting images with this image sensor. In S105, pre-processing of the acquired image is performed. This is a pre-process for an image in order to extract feature amount information from the collected image, and unnecessary information is deleted from the image, filtering or the like is performed.

Next, in step S106, feature amount information extraction is performed by image analysis. From the images, a brightness gradient direction histogram is generated, difference processing between images for determining whether or not a moving body is performed, and the like is converted into data as feature amount information. In this feature amount information, there is no face image remaining, and even if it is determined that they are the same moving body, it is information that does not usually know who. Further, since there is the transferred moving body feature amount information in step S103, the information generated in step S106 can be combined with the transferred moving body feature amount information so as to be cumulative moving body feature amount information, which will be described later. Be prepared.

In S107, it is determined whether or not the target moving object is found in the collected image in the image data obtained by the image sensor. If the answer is Yes, it means that the target moving object has been discovered, so the images are collected for a predetermined time.

In S108, the moving body characteristic amount information generated by this image sensor is generated as the accumulated characteristic amount information together with the time axis information in the transferred moving body characteristic amount information.
In S109, it is determined whether the cumulative feature amount information exceeds the cumulative threshold value. If the cumulative threshold is exceeded, the result is Yes.
In S110, it is determined whether the cumulative feature amount information exceeds the cumulative final threshold value. If it exceeds, the process proceeds to the next step S111.
In S111, the administrator/notification target device is notified that there is a moving object that exceeds the cumulative final threshold. Thus, by accumulating the moving body information, it can be notified that the target moving body does not form a flow line due to normal behavior such as wandering behavior or lost child.
S112 is a process after the generation, determination, and notification of a series of cumulative feature amount information for the target moving object is completed, and post-processing (deleting instruction, etc.) is performed. A deletion instruction is issued for information on each image sensor regarding the target moving object. The deletion instruction may be issued in this processing process or may be issued from the notification destination. Further, as described above, it may be an autonomous process in which the image information of the target moving object is targeted for deletion without a deletion instruction after a predetermined time has elapsed by the timer in each image sensor.

Next, the determination process including the Yes/No option will be described. When it is determined in S102 that abnormal system data is detected from the collected image due to system abnormality or the like, the determination result is Yes, and abnormal system processing is performed in steps S220 and S221. When the abnormal system processing ends, the abnormal system processing END of S221 is performed, and then the process returns to S101. Details of the abnormal system processing between S220 and S221 will be described later with reference to the drawings.

In S103, if there is no transferred moving body feature amount information, the determination result is No, and the processes from S210 are performed.
In S210, image collection for feature amount information extraction is started.
In S211, acquisition image pre-processing is performed.
In S212, feature amount information extraction is performed by image analysis.
In S213, the primary determination in the image sensor whether the feature amount information exceeds the threshold value is performed. If it exceeds the threshold value, the determination result is Yes, and the process proceeds to the next S214. If it does not exceed the threshold value, it is determined that there is no need to notify other image sensor groups, and the determination is No, and the process returns to S101.

In S214, the moving image feature amount information is notified to a predetermined image sensor group. This predetermined image sensor group is a value that is determined in advance for each image sensor from the setting information collected in step S101. After the notification, the process returns to step S101 in the initial state.
In S107, it is determined whether the target moving object is found in the collected image. If the target moving object is not found, the determination is No and the process proceeds to S215.

In step S215, a process of notifying a predetermined image sensor group that the moving body having the transferred moving body feature amount information is not detected is performed. After this process, the process returns to S101.
In S109, it is determined whether the cumulative feature amount information exceeds the cumulative threshold value. If the cumulative threshold is not exceeded, the determination is No and the process proceeds to S230.
In S230, the target moving object is detected, but the cumulative threshold value is not exceeded, and an instruction to delete from the target moving object is notified to a predetermined image sensor group. Although it is the moving object feature amount information of the target moving object that has been transferred by the primary judgment in the other image sensor, it is judged as the target moving object if there is no need for continuous monitoring by the judgment by the cumulative threshold value in this image sensor. doing. This is a process of notifying a predetermined image sensor group of a deletion instruction to remove from the target moving object. After the process of S230, the process returns to S101.

In S110, it is determined whether the cumulative feature amount information exceeds the cumulative final threshold value. If not exceeded, the determination is No and the process proceeds to the next step S240.
In S240, the moving body feature amount information of the target moving body and the deletion instruction of the image data information are transmitted to another predetermined image sensor group. This is the final determination, and if the relevant information and the image of the target moving object remain in each image sensor, it is also instructed to delete them.
Each image sensor operates according to a series of procedures in FIG. Accordingly, the moving body feature amount information is transmitted to another image sensor by the primary determination in each image sensor in the present embodiment. The image sensor that has received the moving body feature amount information creates its own moving body feature amount information (within this image sensor) and generates cumulative feature amount information. Further, if the accumulated characteristic amount information is accumulated for a sufficiently long time, a final judgment is made based on the accumulated final threshold value and the manager/notification target device is notified. Further, if it is determined that the cumulative threshold is not exceeded by another image sensor after the primary determination, that is, if it is determined that it is not necessary to continue monitoring as the target moving object, a moving object information deletion instruction is sent, and the CPU or the storage medium is sent. Resources are released. This allows continuous monitoring.

FIG. 15 is a flowchart showing a data processing procedure of an abnormal system in the feature quantity information transmission system of the target moving object in the embodiment. If YES in S102 of FIG. 14, this abnormal system processing is performed. When the abnormal system process is started in step S220, in S10, the location setting information including the moving body feature amount information from another image sensor is collected. As a result, a series of data including the image data captured by this image sensor and the moving body feature amount information from other image sensors may be judged as abnormal system data, or the latest judgment standard is acquired. It

In S11, it is determined whether the abnormal data may be processed. If YES, the process proceeds to the next S12. The abnormalities that result in YES are, for example, the following four cases.
Phenomena caused by abnormal data:
(1) The target moving object was divided into two hands.
(2) We have observed fixed images such as walls and floors that are usually reflected on the image sensor, and confirmed that the amount of feature information is within the specified range. The feature amount information exceeds.
(3) The result of contour extraction is an abnormal value.
(4) The target moving object cannot be detected by another image sensor and loses sight.

Processing as abnormal data:
(1) If there are moving objects with the same ID (IDentification), they cannot be distinguished.
(2) Assuming that an abnormal value is observed due to a failure of the image sensor or a significant decrease in illuminance, check the state of the image sensor and lighting. The data are ignored as specific outliers. Or report to the administrator.
(3) The brightness gradient histogram is calculated as feature amount information because it is unlikely to be affected by the difference in indoor brightness. If the brightness gradient histogram has already been used, the administrator is notified as an image sensor failure or the like.
(4) If the target moving object cannot be detected for some reason, the accumulated information of the target moving object is deleted after a predetermined time.
In S12, the abnormal system data is processed. The processing method instructs the target image sensor to delete the moving body feature amount information of the corresponding moving body. Alternatively, a method of notifying the administrator/notification target device before issuing the instruction and performing the ON/OFF processing of the image sensor or the deletion processing of the corresponding moving body information after receiving the instruction from the administrator side Good.

In S11, in the case of No, that is, when the process as abnormal system data is not performed, it is determined in S13 whether the abnormal system process starting in S220 has been performed a predetermined number of times or more for the same moving body. In the case of No, the process as abnormal system data is not performed, and the process proceeds to S221. In the case of YES, the abnormal system is in the abnormal system for some reason, and therefore the process as the normal data is once continued. That is, the processing shifts to S103 of FIG.
In S221, the abnormal system process is terminated, and the process returns to S101 in FIG.

FIG. 16 is a diagram showing an example of setting information shared by the image sensors and the feature amount information to be transmitted in the embodiment. An example of the format of the information collected in S101 of FIG. 14 is shown. The image sensor sharing setting information in the upper part of FIG. 16 is information that each image sensor 10 should commonly store. The image sensor sharing setting information 1 includes a positional relationship among the image sensors, a peripheral image sensor to be notified, threshold information, dictionary information, and a manager/notification target device. The image sensor sharing setting information 2 includes each updated and changed information. The central server information is information such as the address of the central server, which is added when the central server is connected.

The real-time transmission information (image sensor A1) between the image sensors in the lower part of FIG. 16 is information possessed by the image sensor A1 that first discovers the moving body that is the target moving body. The format of the moving body feature amount information is transmitted in the procedure of FIGS. 8 to 10 or the procedure of FIGS. 11 to 13. Each item is shown in the first line. Items start with a process ID unique to each image sensor, whether or not there is a caution flag indicating whether or not it should be transmitted to another image sensor, the target given by the image sensor A1 to the target moving object in the image captured by the image sensor A1. It is a moving body number.

The history is recorded over time from the second line to the fourth line. Here, it can be seen that two moving objects M0021 and M0022 are imaged. In the second line, the moving body M0021 is initially detected. Since M0021 is in the normal operation range and does not exceed the threshold value, the caution flag is "absent". Therefore, there is no data in the information column to be transmitted in the right column.

Then, in the third line, a moving body M0022 different from M0021 is detected. At first, since there is no operation that exceeds the threshold value, the caution flag is “none”. However, after the lapse of time, in the fourth line, the caution flag becomes “present” because M0022 has moving object feature amount information that exceeds the threshold value.

The information transmitted to the surrounding image sensor group is described as the item on the first line. It is information about the place of the transmission destination of the information regarding the imaging time of the image sensor, and information regarding the transmission destination (in this case, the group B). Information about the attribute information of the target moving object calculated by the image sensor A1 is included. The attribute information includes information about the shape of the moving body, information about a fine movement of the moving body, and information about a large movement formed by the moving body, that is, a moving line of the moving body. The information regarding the moving body flow line may be information regarding relative coordinates shown in FIG. 19 described later.

The table in the lower part of FIG. 16 shows an example of one image sensor as the peripheral image sensor information. At the stage (=time) at which the image sensor A1 collects the moving object feature amount information to be transmitted, the peripheral image sensors B1 to B6 do not yet have the target moving object attribute information of the moving object. The image sensor A1 sends the target moving object attribute information and the like to the peripheral image sensor groups B1 to B6 according to the above-mentioned flowchart.

FIG. 17 is a diagram showing an example of the characteristic amount information collected by the image sensor on the transmitting side and transmitted as cumulative information in the embodiment. 16 shows the data in the image sensor A1 that first detects the target moving object, while FIG. 17 shows the data on the peripheral image sensor group side to which the feature amount information is transmitted. That is, the data format of FIG. 17 is, for example, until the moving body feature amount information in the image sensor B4 that has detected the target moving body is received and until the data that is received and passed to the next image sensor group C1 to C4 is collected and transmitted. It shows information that changes over time.

In FIG. 17, the first line is the same as in FIG. 16, but the image sensor information (B4) is provided, and the peripheral image sensor information is the image sensor A1 that has sent the feature amount information. From the 2nd line to the 4th line, the image sensor B4 detects the moving body of M0001 and M0002, but since the moving body of the warning flag is not set, the data field is N/A (Not/Available). Is becoming

In the 4th line, since the moving body is not detected, the target moving body ID number is N/A. In the fifth line, the target moving object attribute information to be transferred is received from the image sensor A1 and the information is recorded as the peripheral image sensor information (A1). Next, from the sixth line onward, the information of the moving body ID number M0022 is described. In the sixth line, since the moving body ID number M0022 is detected, the column of the target moving body attribute information of the image sensor B4 is filled. In the seventh line, the moving body ID number M0022 is found to be moving body feature amount information that requires continuous monitoring in the peripheral image sensor groups C1 to C4 and exceeds the cumulative threshold, and should be transmitted as moving body feature amount information. , Information of the image sensor A1 and information of the image sensor B4 are recorded in time order. This is cumulative feature amount information that is the result of tracking by a plurality of image sensors, and since the value is larger than the cumulative threshold value, it is data to be transmitted to the next peripheral image sensor group C1 to C4.

FIG. 18 is a flowchart illustrating an example of processing of target moving body attribute information regarding a specific moving body to be monitored. In S20, processing of the target moving body attribute information received from another image sensor is started.

In S21, the reception information of the target moving object attribute information is received from another image sensor.
In S22, imaging by the main image sensor that has received the information and calculation of the target moving body attribute information are performed.
In S23, it is determined whether the received moving body shape information and the moving body in the image captured by this image sensor have been identified. If they are not the same moving body, the determination is No, and the image capturing in S22 and the calculation of the target moving body attribute information are continued. If YES, the process proceeds to S24.

In S24, it is determined whether or not the received moving body flow line information and the flow line information in the image picked up by this image sensor are matched (continuous). Here, the matching means determining whether the flow lines obtained by the two image sensors of the target moving body are approximately continuous lines or there is a break and the positions do not match exactly. If they do not match, the determination is No and the process proceeds to S25.

In S25, a comment is added to the transmission information (the continuity of the locus is (1) slightly/(2) greatly deviated). Then, the process proceeds to S27.

When the flow line information is matched in S24, YES is determined and the process proceeds to S26.
In S26, it is determined whether or not the received moving body motion information and the moving body motion information captured by this image sensor are stored as the accumulated feature amount information. Whether or not to save may be set in advance, or may be determined by a threshold value. If it is not saved, the process proceeds to S29 of processing the target moving object attribute information. If it is to be saved, YES is obtained and the process proceeds to S27.
In S27, cumulative feature amount information is generated.
In S28, the cumulative feature amount information is stored, or the target moving object attribute information is transmitted to the peripheral image sensor according to the determination unit result. By such a series of processing, the target moving object attribute information can be transmitted. It should be noted that the moving body movement information is not a flow line but information for detecting the presence or absence of a habit of the target moving body or a slight intentional movement, and is often the movement intended by the target moving body. This is information that is not normally required for grasping the normal flow line.

FIG. 19 is a diagram showing simple motion detection in the embodiment. As described above, sharing the moving body feature amount information among the image sensors and generating the cumulative moving body feature amount information is a method suitable for grasping the flow line while identifying the moving body. Apart from this, as a simpler method, there is also a method of notifying the administrator/notification target device when the moving object is detected by each image sensor without being accompanied by the shape information of the target moving object. Although it is not checked whether or not they are the same moving body, by notifying when the image sensor detects the movement, it becomes possible to call attention more quickly with a simpler system.

In FIG. 19, the vertical axis of the table represents the time history, the horizontal axis represents the sensor name, the coordinates of the detected moving body, and the fact that only the motion is detected as the detection target. The accumulated feature amount information is obtained from the movements detected by the plurality of image sensors.

FIG. 20 is a diagram showing an example of the entire trace map in the embodiment. According to the table of FIG. 17, there is shown an overall trace map in which maps formed from the image sensors A0, A1, B2, B3, C1 and C3 are connected.

It is assumed that the field of view of one image sensor is divided into 4×4 areas, for example. It is shown that the moving body first detected by the image sensor A1 moves with the coordinates x0 and y0 as the starting points and finally reaches the image sensor C1. It will be traced in one stroke with the movement route as the flow line. For example, the administrator who received the report can determine that the characteristic of the meandering flow line has the moving body feature amount information that exceeds the range of normal movement.

In this way, it is possible to determine to a certain degree that one particular moving object follows the moving line shown in FIG. 20 by only following the moving line of the moving object without making the primary judgment in each image sensor. You will be able to judge. In this case, since the flow line is only followed, a threshold line for determining the feature amount information is not required, and the flow line can be traced easily.

Further, when a moving body comes to the edge of the image pickup range of the image sensor A1, the feature amount information is calculated in the image sensor A1 and the moving body shape information is transmitted to another image sensor, so that, for example, the image sensor B2. In, it is possible to trace the flow line after confirming that they are the same moving body.

FIG. 21 shows a case where the image sensor A1 partially overlaps with the imaging area of the image sensor B2, which is the transmission destination of the feature amount information of the moving body, and a case where they do not overlap. As for the timing of transmission to the transmission destination, it may be assumed that the moving body is detected at the edge of the image sensor A1, or the detection is performed at the edge and the moving direction of the moving body is outward. You may add the case.

Further, when the image pickup areas partially overlap each other, when the moving object comes to the edge of the image pickup area in the image sensor A1, the caution flag is sent once, and the image sensor B2, which is the destination of the characteristic amount information, transmits the caution flag. The final feature amount information of the moving body detected by the image sensor A1 may be transmitted after waiting for the notification of the moving body detection. By setting the feature amount information of the moving body collected by the image sensor A1 as the longest time, the overlapping observation time of the image sensor A1 and the image sensor B2 becomes long, and the appearance of when the moving body appears in the mass and when it disappears. / It is possible to determine that they are the same moving body from the coincidence of the disappearance times.

In FIG. 21A, an example in which the moving image is detected by the image sensor A1 is shown. Observing in the order of 1) to 3), and when the condition of 3) is satisfied, the moving body feature amount information collected by the image sensor A1 is transmitted to the image sensor B2.
1) T1: the time when the moving body is detected in the divided area 2) T2: the time when the moving body is detected at the edge, and the time after T1 3) The moving body is at the edge and the movement is outward The time when it was confirmed to be. At this time, the feature amount information of the moving body is transmitted to the image sensor B2.

In FIG. 21B, the moving body feature amount information is transmitted at the timing when the moving body is detected by both the image sensor A1 and the image sensor B2.

Motion detection with image sensor A1
1) T1: Time when the moving body is detected in the divided area.
2) T2: the time when the moving body is detected at the edge, which is a time after T1.
3) The time when the moving object is detected by both the image sensor A1 and the image sensor B2. At this time, the feature amount information of the moving body is transmitted to the image sensor B2.

[effect]
As described above, in the embodiment, real-time monitoring and tracing are possible under constraint conditions in which information from a plurality of image sensors is more desirable for grasping a flow line. The CPU in the image sensor makes a primary determination and notifies it if necessary. Then, the moving body feature amount information is transmitted to the peripheral image sensor. The image sensor at the transmission destination notifies the presence or absence of the target moving object. If any of the image sensors that have received the notification detects the target moving body, the moving body characteristic amount information is calculated, and the final determination is made based on the moving body characteristic amount information accumulated in time series. If the image sensor to be notified is switched around the moving destination of the moving body, the movement line can be grasped in real time with the minimum communication amount. The communication target may be an image sensor group or an administrator, and it is possible to select a simple method of notifying only the presence or absence of a moving body that does not include the feature amount information in the communication content. Image data including the subject of portrait right does not always flow on the communication path. The flow line can be grasped by the digitized feature amount information of the moving body, and it is highly safe in terms of personal information protection. The system cost to cover the communication volume is low, and the flow line can be grasped only between the image sensors, so it can operate even in the event of a disaster.

The effects of the embodiment are listed below.

(1) The sensor system includes a plurality of image sensors and means for transmitting characteristic amount information relating to a moving body extracted from image data captured by the image sensor to other image sensors as attribute information of the target moving body. To have.

By doing so, it is possible to continuously capture images based on the feature amount information of the target moving body using a plurality of image sensors without the central server collecting the images.

(2) In the sensor system according to (1), each of the image sensors is based on an image capturing unit that acquires the image data, an extracting unit that extracts the feature amount information from the image data, and the feature amount information. A determination unit that determines the target moving object, a data storage unit that stores dictionary data related to the determination, and a communication unit that transmits the extracted feature amount information to the other image sensor. As a result, the image sensor can perform feature amount information extraction and primary determination by the image sensor.

(3) In the sensor system of (2), the communication unit transmits the accumulated feature amount information in which the attribute information of the target moving body is accumulated to another image sensor. In this way, since the plurality of image sensors have the attribute information of the target moving body, it is possible to transmit the accumulated characteristic amount information accumulated over time between the image sensors.

(4) In the sensor system according to (3), each of the image sensors changes the determination standard in the determination unit according to the accumulation of the accumulated feature amount information. By changing the determination criterion in accordance with the accumulation of the accumulated feature amount information, more accurate determination can be performed as the flow line of the target moving object extends.

(5) In the sensor system according to (3), each of the image sensors accumulates a route map so that the flow line of the target moving body can be connected with one stroke in accordance with the accumulation of the accumulated feature amount information. As a result, it is possible to form the entire map covered by the plurality of image sensors and generate the movement path of the flow line of the target moving body.

(6) In the sensor system of (2), when the determination unit determines that the value exceeds the threshold value to be notified, the communication unit notifies the predetermined information presentation destination of the characteristic amount information. In this way, each image sensor makes a primary determination and notifies each independently, so that the moving range of the moving body can be known in real time even without transmission of the moving body feature amount information.

(7) In the sensor system according to (2), the attribute information of the target moving object is feature amount information generated in the image sensor. Thereby, the attribute information of the target moving body can generate the feature amount information in each image sensor.

(8) In the sensor system according to (2), each of the image sensors has, as attribute information of the target moving body, coordinate information relating to flow line formation of the target moving body from relative coordinates in the image, and further, a space of the image sensor. It has a means for converting it as an absolute coordinate indicating a physical arrangement. By combining the flow line grasp based on the relative coordinates of each image sensor and the absolute coordinates indicating the spatial position of each image sensor, an overall map formed by a plurality of image sensors is formed even within each image sensor. can do.

(9) In the sensor system according to (3), when the cumulative feature amount information does not exceed a cumulative final threshold value, the communication unit includes the moving subject feature amount information of the target moving body and the image data information of the target moving body. The deletion instruction is transmitted to at least one other image sensor. In this way, since the moving object that was the monitoring target does not exceed the cumulative final threshold value, when it is removed from the monitoring target, by deleting the moving object attribute information, resources are released and continuous operation is possible. And

(10) In the sensor system according to (2), the attribute information includes image pickup time information of the plurality of image sensors. In this way, when the information is transmitted to the peripheral image sensor group, the information regarding the imaging time of each image sensor is also transmitted, so that the moving path of the moving body over time can be known.

(11) In the sensor system of (2), each of the image sensors performs abnormal system processing as abnormal system data when the attribute information exceeds a predetermined threshold range. In this way, when the attribute information exceeds the predetermined threshold value, it can be processed as abnormal data.

(12) In the sensor system according to (2), the attribute information includes a flow line of the target moving body. By using the attribute information of the moving body as the flow line, it is possible to obtain the flow line of the target moving body across a plurality of image sensors.

(13) In the sensor system according to (2), the communication unit causes at least one of the information that the target moving body has been detected and the information that the target moving body has not been detected to the other image sensor. introduce. By sharing not only the information that the target moving body has been detected but also the information that the target moving body has not been detected by the image sensor group, it is possible to more appropriately grasp the flow line.

(14) In the sensor system according to (2), each of the image sensors transmits the feature amount information of the target moving body to the other image sensor and stores the image data of the target moving body in itself, or It is transmitted to a predetermined destination. By collecting the attribute information that also includes the image data of the target moving object, the necessary image information can be collected without loss of information when the target moving object is finally determined.

(15) In the sensor system according to (2), the communication unit transmits the characteristic amount information of the target moving body and the captured image data to the plurality of image sensors. By transmitting not only the statistical data after analysis but also the captured image data as the feature amount information of the target moving object, there is an effect of facilitating aggregation and recalculation.

(16) In the sensor system according to (2), each of the image sensors transmits the fact to another image sensor when the target moving body cannot be detected, and accumulates the feature amount information of the target moving body. The accumulated feature amount information and the image data of the target moving body are acquired. As a result, the accumulated feature amount information can be generated and the image data can be totaled.

(17) In the sensor system according to (2), each of the image sensors lowers the priority of saving the attribute information of the target moving body with the passage of time after detecting the target moving body. In this way, the resources of the CPU of the image sensor and the resources of the storage medium can be released by lowering the storage priority when a predetermined time has elapsed after detecting the target moving object.

(18) In the sensor system of (2), each of the image sensors transmits the attribute information of the target moving body via the communication unit when the target moving body moves outside the visual field of the image sensor. .. This makes it possible to set the timing at which the image sensor transmits the information about the target moving object when the image sensor moves away from the visual field of the image sensor.

(19) In the sensor system according to (2), each of the image sensors obtains a response from the image sensor even after a predetermined time elapses after transmitting a connection request to another image sensor in a predetermined area. If not, the predetermined area is expanded and the connection request is transmitted again. The target transmission range can be changed by knowing whether or not the image sensor of the transmission destination is in an operable state by the confirmation by the connection request to the image sensor of the transmission destination of the attribute information of the target moving object, and thus more reliable. It is possible to grasp the flow line with high flexibility.

(20) In the sensor system according to (2), each of the image sensors holds the transmitted attribute information in the data storage unit until receiving an erasing instruction from another image sensor. By storing the image information of the target moving object in each image sensor that has been imaged for a certain period until the deletion instruction arrives, it becomes possible to collect it later as an image group accompanied by the time history.

(21) In the sensor system according to (2), each of the image sensors is connected to each other in a communication topology capable of forming a hollow circuit, and performs communication for confirming a pre-operation with a partner before communication. In the sensor network formed by each image sensor, even if one image sensor fails due to a disaster or failure, it has a network configuration that can collect information from other image sensors, thereby improving reliability. There is an advantage that you can.

(22) In the sensor system according to (2), each of the image sensors includes a unit that synchronizes time with each other. By synchronizing the time of the timers in each image sensor, it becomes possible to transmit and accumulate the moving body feature amount information including the time stamp based on the same time reference.

(23) In the sensor system according to (6), the determination unit detects that the moving body has entered the predetermined range of the imaging area, or the shape information of the moving body matches the predetermined shape information. In this case, it is determined that the threshold is exceeded. As a method of identifying the target moving object, a method of using a moving object as a criterion for judgment and a method of using the shape information of the moving object as a criterion can be used.

(24) In the sensor system according to (2), each of the image sensors is notified when the notification from the image sensor detecting that the cumulative threshold is exceeded or the storage period exceeds a predetermined value is transmitted. Delete attribute information. When the target moving objects are found one after another, they can be prioritized and deleted and the function can be maintained without overflowing the limited storage capacity in the sensor.

(25) In the sensor system of (2), each of the image sensors transmits its own camera coordinates or system coordinates to another image sensor together with the attribute information, and traces the flow line of the target moving object. The entire trace map is reconstructed based on the attribute information acquired from the group. By providing a means for converting camera coordinates to system coordinates when forming a flow line by a plurality of image sensors, it is possible to generate an overall trace map that traces the entire flow line.

(26) In the sensor system according to (2), each of the image sensors outputs the attribute information at the time when the target moving object appears at the edge of the imaging area of the image sensor that is the source of the attribute information. introduce. By setting the timing of transmitting the attribute information of the target moving object at the edge of the imaging area by the image sensor under observation, it becomes possible to continuously grasp the flow line by the plurality of image sensors.

(27) In the sensor system according to (26), each of the image sensors transmits the attribute information when it is detected that the moving direction of the moving body is from the center of the imaging area toward the edge. The timing of transmitting the attribute information of the target moving body, while the moving body reaches the edge of the imaging area by the image sensor under observation, when the trajectory of the moving body in the imaging area is out of the field of view of the image sensor, By setting the timing of transmitting the attribute information of the target moving body, it is possible to continuously grasp the flow line.

(28) In the sensor system according to (27), each of the image sensors sets the attribute information of the target moving body to an image sensor arranged in an area in the extension line direction from the edge. It becomes possible to accurately select the image sensor to which the attribute information is transferred, based on the information indicating which edge of the image sensor under observation the target moving object exists.

(29) In the sensor system according to (1), the attribute information includes brightness gradient information of the image data. By using brightness gradient information as feature amount information, even feature amount information from image sensors placed at different illuminances can be handled uniformly, and since it is numerical data, the transfer amount is small and the load on the network is reduced. , And personal information including portrait rights, there are advantages from the viewpoint of security.

From these things, according to an embodiment, there exists in providing a sensor system, an image sensor, and a sensing method which raised availability further.

Although the embodiment of the present invention has been described, the embodiment is presented as an example and is not intended to limit the scope of the invention. The novel embodiment can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. This embodiment and its modifications are included in the scope and gist of the invention, and are also included in the invention described in the claims and the scope equivalent thereto.

DESCRIPTION OF SYMBOLS 1... Sensor system, 4... Target space, 5... Communication line, 8... Environment sensor, 9a-9f... Sensor device, 10a-10d... Image sensor, 11... Image capturing camera, 12... Image pre-processing part, 13... Feature amount information extraction unit, 14... Judgment unit, 15... CPU, 16... Dictionary data, 17... Storage unit, 20... Communication unit, 20a-20d... Communication unit, 20a-20f... Communication unit, 22... Master, 30... Central server, 31... Communication unit, 32... Analysis unit, 33... Judgment unit, 34... Past data, 35... CPU, A0... Image sensor, A1... Image sensor, B1-B6... Image sensor, C1-C4... Image sensor , A to f... Area, e4... Corner, e5... Edge.

Claims (31)

Multiple image sensors,
A sensor system, comprising means for transmitting characteristic amount information relating to a moving body extracted from image data picked up by the image sensor to another image sensor as attribute information of the target moving body. Each of the image sensors is
An imaging unit for acquiring the image data,
An extraction unit that extracts the feature amount information from the image data,
A determination unit that determines the target moving object based on the feature amount information,
A data storage unit that stores dictionary data relating to the determination,
The sensor system according to claim 1, further comprising a communication unit that transmits the extracted feature amount information to the other image sensor. The sensor system according to claim 2, wherein the communication unit transmits, to another image sensor, accumulated characteristic amount information in which the attribute information of the target moving body is accumulated. The sensor system according to claim 3, wherein each of the image sensors changes a determination criterion in the determination unit according to the accumulation of the accumulated feature amount information. The sensor system according to claim 3, wherein each of the image sensors accumulates a route map so that the flow line of the target moving body can be connected with a single stroke in accordance with the accumulation of the accumulated feature amount information. The sensor system according to claim 2, wherein when the determination unit determines that the value exceeds a threshold value to be notified, the communication unit notifies a predetermined information presentation destination of the characteristic amount information. The sensor system according to claim 2, wherein the attribute information of the target moving object is feature amount information generated in the image sensor. Each of the image sensors converts, as attribute information of the target moving body, coordinate information relating to the flow line formation of the target moving body from relative coordinates in the image to absolute coordinates indicating a spatial arrangement of the image sensor. The sensor system according to claim 2, comprising: When the cumulative feature amount information does not exceed the cumulative final threshold value, the communication unit instructs the at least one other image sensor to delete the moving body feature amount information of the target moving body and the image data information of the target moving body. The sensor system according to claim 3, which transmits. The sensor system according to claim 2, wherein the attribute information includes image capturing time information of the plurality of image sensors. The sensor system according to claim 2, wherein each of the image sensors performs abnormal system processing as abnormal system data when the attribute information exceeds a predetermined threshold range. The sensor system according to claim 2, wherein the attribute information includes a flow line of the target moving body. The sensor system according to claim 2, wherein the communication unit transmits at least one of information that the target moving body is detected and information that the target moving body is not detected to the other image sensor. .. Each of the image sensors is
The sensor system according to claim 2, wherein the characteristic amount information of the target moving body is transmitted to the other image sensor, and the image data of the target moving body is stored in itself or transmitted to a predetermined destination. The sensor system according to claim 2, wherein the communication unit transmits the feature amount information of the target moving body and the captured image data to the plurality of image sensors. Each of the image sensors is
When the target moving body cannot be detected, the fact is transmitted to another image sensor, and accumulated feature amount information obtained by accumulating the feature amount information of the target moving body and image data of the target moving body are acquired, The sensor system according to claim 2. Each of the image sensors is
The sensor system according to claim 2, wherein after the target moving object is detected, the priority of saving the attribute information of the target moving object is lowered with the lapse of time. Each of the image sensors is
The sensor system according to claim 2, wherein when the target moving body moves outside the visual field of the image sensor, the attribute information of the target moving body is transmitted via the communication unit. Each of the image sensors is
After transmitting the connection request to another image sensor in the predetermined area, if the response from the image sensor is not obtained even after the predetermined time has passed, the predetermined area is expanded and the connection request is transmitted again. The sensor system according to claim 2, wherein Each of the image sensors is
The sensor system according to claim 2, wherein the transmitted attribute information is held in the data storage unit until a deletion instruction is received from another image sensor. 3. The sensor system according to claim 2, wherein each of the image sensors is connected to each other in a communication topology capable of forming a circuit and performs communication for confirming a pre-operation with the other party prior to the communication. The sensor system according to claim 2, wherein each of the image sensors includes a unit that synchronizes time with each other. The determination unit determines that the threshold value has been exceeded when detecting the entry of the moving body within a predetermined range of the imaging area, or when the shape information of the moving body matches the predetermined shape information. The sensor system according to claim 6, wherein The said each image sensor deletes the said transmitted attribute information, when the notification from the image sensor which detected that the accumulation threshold was exceeded, or when the retention period exceeded a predetermined value. Sensor system. Each of the image sensors transmits a unique camera coordinate or system coordinate to the other image sensor together with the attribute information, and based on the attribute information acquired from the image sensor group tracing the flow line of the target moving object, the entire image sensor. The sensor system according to claim 2, which reconstructs a trace map. The sensor system according to claim 2, wherein each of the image sensors transmits the attribute information when a target moving object appears at an edge of an imaging area of the image sensor that is a source of the attribute information. .. Each of the image sensors is
27. The sensor system according to claim 26, wherein the attribute information is transmitted when it is detected that the moving direction of the moving body is from the center of the imaging area toward the edge. Each of the image sensors is
28. The sensor system according to claim 27, wherein the attribute information of the target moving object is set to an image sensor arranged in an area in the extension line direction from the edge portion as a transmission destination. The sensor system according to claim 1, wherein the attribute information includes brightness gradient information of the image data. In the image sensor applied to a sensor system including a plurality of image sensors, means for transmitting characteristic amount information relating to a moving body extracted from image data to another image sensor as attribute information of the target moving body,
An imaging unit for acquiring the image data,
An extraction unit that extracts the feature amount information from the image data,
A determination unit that determines the target moving object based on the feature amount information,
A data storage unit that stores dictionary data relating to the determination,
An image sensor, comprising: a communication unit that transmits the extracted feature amount information to the other image sensor. A sensing method applied to a sensor system including a plurality of image sensors, wherein feature amount information about a moving body extracted from image data captured by each image sensor is used as another image sensor as attribute information of the target moving body. A sensing method, including a process of transmitting to.