KR20230155432A - Imaging device, tracking system and imaging method - Google Patents

Abstract

To provide an imaging device that can increase object detection accuracy while reducing the data bandwidth. The imaging device 12 includes an imaging unit 22 that captures an image of an object, a feature extraction unit 24 that extracts a plurality of feature quantities of the object imaged by the imaging unit 22, and a feature extraction unit 24. a priority determination unit 27 that determines the priority of the plurality of feature quantities extracted from the priority, a control unit 28 that determines the feature quantity according to the height of the priority and the allowable amount of the output destination, and a priority determined by the control unit 28. It is provided with an output unit 32 that outputs a characteristic quantity. The imaging device 12 further includes a movement direction calculation unit 25 that calculates the movement direction of the object, and the output unit 32 can output the feature quantity and the movement direction in relation to each other.

Description

Imaging device, tracking system and imaging method

The present technology relates to an imaging device, a tracking system, and an imaging method, and more particularly, to an imaging device, a tracking system, and an imaging method that identify an imaged object and track the movement of the imaged object.

Conventionally, a surveillance system has been known in which a person's movement trace is obtained and recorded from an image captured by a surveillance camera, and the person whose movement trace is obtained is subjected to facial recognition to identify the person from another person whose face is recognized.

For example, in Patent Document 1, an imaging means for surveillance, a human body detection means for detecting a person from an image captured by the imaging means, and a movement trace of the person detected by the human body detection means are analyzed on the image. Trajectory analysis means for obtaining a trace, and face recognition means for recognizing a face by obtaining face data that can be distinguished from another face from the face on the image of the person detected by the human body detection means and whose movement trace is obtained by the trace analysis means. and recording means for recording the face data and the movement trace corresponding to the face data by associating the face data of different people with an identification code that enables mutual identification, and in the face data recorded in the recording means. face determination means for comparing the face data obtained by the face recognition means and determining whether the face data satisfies a predetermined condition;

When the above conditions are met, the movement trace corresponding to the face data obtained by the face recognition means is associated with the identification code associated with the face data recorded in the recording means that matches the face data. The identification code that is recorded in the recording means and is not yet associated with the face data in the face data obtained by the face recognition means and the movement trace corresponding to the face data when the above conditions are not met. In connection with this, a surveillance information collection system is proposed that includes a face data collection means for recording in the recording means.

Japanese Patent Publication No. 2018-93283

However, in the technology of Patent Document 1, the amount of data recorded in the imaging means increases, and there is a possibility that the communication speed between the imaging means may become slow or data exchange may become impossible. Additionally, in the technology of Patent Document 1, it is not assumed to determine the imaging means for transmitting and receiving data according to the environmental conditions of the imaging means on the data receiving side, so depending on the circumstances, such as the environment, objects can be captured with high precision. There are cases where it cannot be detected.

Therefore, the main purpose of this technology is to provide an imaging device that can increase object detection accuracy while reducing the data bandwidth.

An imaging device according to the present technology includes an imaging unit for capturing an object, a feature extraction unit for extracting a plurality of feature quantities of an object imaged by the imaging unit, and determining the priority of the plurality of feature quantities extracted from the feature extraction unit. It has a priority determining unit, a control unit that determines a feature quantity according to the priority height and the allowable amount of the output destination, and an output unit that outputs the feature quantity determined by the control unit. The imaging unit may simultaneously image the object in the same area.

The imaging device according to the present technology further includes a movement direction calculation unit that calculates a movement direction of the object, and the output unit may output the correlation between the feature quantity and the movement direction. The priority determination unit may have a privacy level setting unit that sets a privacy level for each of the feature quantities, and may determine the priority according to the privacy level. The imaging device according to the present technology further includes an imaging position acquisition unit that acquires the imaging position and imaging direction of the imaging unit, and the output unit may output the imaging position and the imaging direction. The imaging device according to the present technology further includes an environmental information acquisition unit that acquires environmental information about the surroundings where the imaging device is installed, and the control unit may determine an output destination of the feature quantity based on the environmental information. . The imaging device according to the present technology further includes a state change information acquisition unit that acquires state change information of an output destination of the characteristic quantity, and the control unit may determine the output destination according to the state change information. The imaging device according to the present technology further includes an emotion information acquisition unit that acquires emotion information regarding the emotion of the object, and the output unit may output the emotion information in association with the feature quantity. The imaging unit further includes an object selection unit that captures a plurality of images of the objects and selects the object of interest from the plurality of objects imaged by the imaging unit, and the feature extraction unit is configured to select the object selected by the object selection unit. It is also possible to extract multiple feature quantities of an object.

In addition, another imaging device related to the present technology includes an input unit that inputs a plurality of characteristic quantities of the object according to the priority height and the allowable amount of input, and a tracking unit that tracks the object based on the characteristic quantities input by the input unit. Equipped with

In addition, the tracking system according to the present technology includes an imaging unit for capturing an object, a feature extraction unit for extracting a plurality of feature quantities of the object imaged by the imaging unit, and a plurality of the features extracted from the feature extraction unit. a priority determination unit that determines the priority of a quantity, a transmission unit that transmits the characteristic quantity according to the height of the priority and the allowance of the transmission destination, and a transmission unit that transmits the characteristic quantity according to the height of the priority and the allowance that can be received. It is provided with a plurality of imaging devices having a receiving unit for receiving and a tracking unit for tracking the object based on the characteristic quantity received by the receiving unit, and the plurality of imaging devices transmit and receive the characteristic quantity in conjunction with each other.

In addition, the imaging method according to the present technology includes steps of imaging an object, steps of extracting a plurality of feature quantities of the imaged object, steps of determining priorities of the plurality of extracted feature quantities, and steps of determining the priorities of the plurality of extracted feature quantities. It includes a step of determining the characteristic quantity according to the height and an allowance of the output destination, and a step of outputting the determined characteristic quantity. In addition, another imaging method related to the present technology includes steps of inputting a plurality of characteristic quantities of an object according to the priority height and input allowable amount, and tracking the object based on the inputted characteristic quantities. Includes.

According to the present technology, it is possible to provide an imaging device that can increase object detection accuracy while reducing the data bandwidth. In addition, the above-described effects are not necessarily limited, and any of the effects disclosed in this specification or other effects that can be understood from this specification may be exerted together with the above-described effects or instead of the above-described effects.

1 is a schematic diagram showing a configuration example of a tracking system according to a first embodiment of the present technology.
Fig. 2 is a block diagram showing a configuration example of an imaging device according to the first embodiment of the present technology.
Fig. 3 is a flowchart showing an example of operation of the imaging device according to the first embodiment of the present technology.
Figure 4 is an image for explaining a tracking method by a tracking system according to the first embodiment of the present technology.
Fig. 5 is a conceptual diagram showing list management by a tracking system according to the first embodiment of the present technology.
Fig. 6 is a sequence diagram showing an example of list distribution by the tracking system according to the first embodiment of the present technology.
Figure 7 is a conceptual diagram for explaining a tracking method by a tracking system according to the first embodiment of the present technology.
Fig. 8 is a sequence diagram showing an example of list distribution by the tracking system according to the second embodiment of the present technology.
Figure 9 is a conceptual diagram for explaining a tracking method by a tracking system according to the second embodiment of the present technology.
10 is a flowchart showing an example of a tracking method by a tracking system according to the second embodiment of the present technology.
Fig. 11 is a conceptual diagram showing list management by the tracking system according to the third embodiment of the present technology.
Fig. 12 is a sequence diagram showing an example of a tracking method by a tracking system according to the third embodiment of the present technology.
Figure 13 is a conceptual diagram for explaining a tracking method by a tracking system according to the fourth embodiment of the present technology.

Hereinafter, a suitable form for implementing the present technology will be described with reference to the drawings. The embodiment described below shows an example of a representative embodiment of the present technology, and any embodiment can be combined. Additionally, the scope of this technology is not interpreted narrowly by these factors. In addition, the explanation is carried out in the following order.

1. First embodiment

(1) Configuration example of tracking system 10

(2) Configuration example of camera 12

(3) Example of operation of camera 12

(4) Tracking method by tracking system 10

2. Second embodiment

3. Third embodiment

4. Fourth embodiment

1. First embodiment

(1) Configuration example of tracking system 10

First, with reference to FIG. 1, a configuration example of the tracking system 10 according to the first embodiment of the present technology will be described. Figure 1 is a schematic diagram showing an example of the configuration of the tracking system 10. The tracking system 10 can be applied to applications such as following suspicious people, traffic investigation, and demonstration detection.

As shown in FIG. 1, the tracking system 10 includes, as an example, a server 11 and a plurality of cameras 12, which are image capture devices, each of which is connected to the server 11 for communication. The server 11 can use a cloud server.

As an example, each camera 12 is provided with an RGB camera and a Depth camera and can capture one or more objects. In addition, in this embodiment, an example of using three cameras 12 (Cam1, Cam2, Cam3) is shown, but the number of cameras 12 is not limited to this and may be two, or four or more. You can continue.

The tracking system 10 captures images of the object to be tracked from each of the plurality of cameras 12 (Cam1, Cam2, and Cam3), calculates the characteristic quantity and three-dimensional movement direction vector of each object, and tracks the imaged camera. Calculation result information is transmitted from (12) to the server (11). Calculation of feature quantities and movement direction vectors can be performed, for example, using artificial intelligence (AI).

The server 11, which has received the calculation result information, specifies the tracking target by comparing the feature quantity based on the calculation result information with the registered feature quantity, and tracks the tracking target with each camera 12 along the moving direction of the tracking target. Send information. Each camera 12 tracks (tracks) the tracking object by transmitting and receiving characteristic quantities in conjunction with each other. Details of this tracking method will be described later.

(2) Configuration example of camera 12

Next, with reference to FIG. 2, a configuration example of the camera 12 will be described. FIG. 2 is a block diagram showing a configuration example of the camera 12.

As an example, an RGB camera, a depth camera (iTOF/dToF/Structured Light/Stereo/Active Stereo), a deflection camera, a DVS camera, a multispectral/IR camera, a high-speed vision camera, an infrared camera, etc. can be used as the camera 12. there is.

As shown in FIG. 2, the camera 12 includes, as an example, an input unit 21 that inputs information received from the outside in a required configuration, an imaging unit 22 that captures an object, and the imaging unit 22. An imaging position acquisition unit 23 that acquires the imaging position and imaging direction, a feature quantity extraction unit 24 that extracts a plurality of feature quantities of the object imaged by the imaging unit 22, and a movement unit that calculates the movement direction of the object. A direction calculation unit 25 is provided.

In addition, the camera 12 includes a priority determination unit 27 that determines the priority of a plurality of feature quantities extracted by the feature quantity extraction unit 24, and a priority determination unit 27 that determines the priority of the feature quantities according to the height of the priority and the allowable amount of the output destination. A control unit 28 that determines, an environmental information acquisition unit 29 that acquires environmental information about the surroundings where the camera 12 is installed, and a state change that acquires state change information of the output destination camera 12 of the characteristic quantity. It is provided with an information acquisition unit 30 and an output unit 32 that outputs the characteristic quantity determined by the control unit 28.

In addition, the camera 12 is provided with a tracking unit 33 that tracks the object based on the feature quantity input from the input unit 21, and a communication unit 34 that transmits and receives information to and from the outside. Additionally, the camera 12 may be provided with an emotion information acquisition unit 31 that acquires emotional information regarding the emotion of an object.

The input unit 21 can input a plurality of characteristic quantities of an object according to the priority height and the allowable input amount.

The imaging unit 22 may be provided with an object selection unit 41 that selects an object of interest from a plurality of objects imaged by the imaging unit 22. Additionally, the imaging unit 22 can simultaneously image objects in the same area. At this time, the feature quantity extraction unit 24 may extract a plurality of feature quantities of the object selected by the object selection unit 41. Additionally, the object selection unit 41 may be provided as a component other than the imaging unit 22 in the camera 12.

The priority determination unit 27 has a privacy level setting unit 26 that sets the level of privacy protection for each feature quantity. The priority determination unit 27 may determine the priority according to the privacy level set by the privacy level setting unit 26.

The privacy level setting unit 26 has a function to switch between “can handle” or “cannot handle” information with privacy concerns. Additionally, information indicating the privacy level is added to the feature quantity. When outputting feature quantities, the privacy level setting unit 26 also has a function of controlling output availability and the order according to the privacy level.

The control unit 28 determines the output destination of the characteristic quantity based on the environmental information acquired by the environmental information acquisition unit 29, or determines the output destination according to the state change information acquired by the state change information acquisition unit 30. You can decide or do it.

Environmental information acquired by the environmental information acquisition unit 29 includes latitude, longitude, time, direction, weather, temperature, humidity, air current, SO ₂ , CO, NOx, SPM, CO ₂ , and noise.

The state change information acquired by the state change information acquisition unit 30 is information when the feature quantity that can be acquired from the target camera 12 changes by the feature quantity detection algorithm replacement function, or the camera 12 Information in cases where characteristic quantities cannot be acquired due to environmental changes (failure, brightness, bad weather, battery capacity decline, etc.) is also included.

The output unit 32 can output the imaging position and imaging direction acquired by the imaging position acquisition unit 23. In addition, the output unit 32 outputs the feature quantity in association with the movement direction calculated by the movement direction calculation unit 25, or outputs the feature quantity acquired by the emotion information acquisition unit 31 in association with the emotion information. can do.

The communication unit 34 has a transmitting unit 42 that transmits the characteristic quantity according to the priority height and the allowable amount of the transmission destination, and a receiving unit 43 that receives the characteristic quantity according to the priority height and the allowable amount that can be received. there is.

Here, specific examples of characteristic quantities of objects will be described. Feature quantities extracted by an RGB camera include the static characteristics of a person, such as the position and movement amount of key parts such as face, hand, foot, fingerprint, physique, hair color, and eyes. Additionally, the dynamic characteristics of a person include gait, voice, behavior, blinking, and movement direction vector. Additionally, additional information may include the shape, color, pattern, movement amount, belongings, accessories, shopping cart, cart, etc. of clothes or objects.

Additionally, feature quantities extracted by the depth camera include the movement amount, shape, and reflectance (active method) of a person or object. In addition, an RGB camera can also acquire two-dimensional movement amount, but a depth camera that can acquire depth information can acquire three-dimensional movement amount.

In addition to the above, a deflection camera can extract the same feature quantities as an RGB camera, and can especially be used for subjects such as behind glass or underwater. In a DVS camera, feature quantities such as the appearance of a moving object and the change in position of a high-brightness or low-brightness object can be extracted. With multispectral/IR cameras, feature quantities such as plant vegetation index (NDVI) can be extracted. High-speed vision cameras can extract feature quantities such as the movement amount or vector of an object, or shapes or characters on a high-speed moving object. Additionally, an infrared camera can extract characteristic quantities such as the temperature of a person or object.

(3) Example of operation of camera 12

Next, with reference to FIG. 3, an operation example of the camera 12 will be described. FIG. 3 is a flowchart showing an example of operation of the camera 12. When the imaging unit 22 of the camera 12 captures an image including an object, the following operations are started.

As shown in FIG. 3, in step S1, the feature extraction unit 24 extracts an object from the RGB image captured by the imaging unit 22.

In step S2, the feature quantity extraction unit 24 extracts a plurality of feature quantities for each extracted object.

In step S3, the movement direction calculation unit 25 calculates the movement direction vector of each corresponding object from depth information.

In step S4, the control unit 28 determines a feature quantity from the extracted feature quantity according to the priority height and the allowable amount of the output destination, and packs the determined feature quantity and the calculated movement direction vector information for each object.

In step S5, the output unit 32 outputs data packed as the number of objects to the outside. That is, the output unit 32 can output the characteristic quantity determined by the control unit 28 to the outside.

(4) Tracking method by tracking system 10

Next, with reference to FIGS. 4 to 7 , a tracking method using the tracking system 10 according to the present embodiment will be described. FIG. 4 is an image for explaining the tracking method by the tracking system 10. A in FIG. 4 represents an image captured by Cam1 of the camera 12, B in FIG. 4 represents an image captured by Cam2 of the camera 12, and C in FIG. 4 represents an image captured by Cam3 of the camera 12. It shows a burn.

In this embodiment, as shown in FIG. 4B, the image of Person 1 and Person 2 moving in the Cam1 direction (left direction in the drawing) and Person 3 moving in the Cam3 direction (right direction in the drawing) is displayed as Cam2. A case where an image is captured, and Person 1 and Person 2 are tracked with Cam1 and Person 3 is tracked with Cam3 based on the captured image will be described.

With reference to FIGS. 5 and 6, list management by the tracking system 10 will be described. FIG. 5 is a conceptual diagram showing list management by the tracking system 10. A in FIG. 5 represents a list of feature quantities managed by the host server 11, B in FIG. 5 represents a list of feature quantities managed by Cam1, and C in FIG. 5 represents a list of feature quantities managed by Cam2. , and D in FIG. 5 shows a list of feature quantities managed by Cam3.

As shown in FIG. 5A, the server 11 manages, for example, a list of feature quantities 1 to 8, which are all feature quantities of the object to be tracked. As shown in FIG. 5B, Cam1 manages a list of feature quantities of the object to be tracked, for example, in order of priority, with feature quantities 1, 3, 6, 2, 4, and 5. As shown in FIG. 5C, Cam2, for example, manages a list of feature quantities of the object to be tracked in order of priority, with feature quantities 2, 1, 4, 5, 3, and 6. As shown in Figure 5D, Cam3, for example, manages a list of feature quantities of the object to be tracked in order of priority, with feature quantities 5, 2, 1, 3, 4, and 6.

FIG. 6 is a sequence diagram showing an example of list distribution by the tracking system 10. As shown in FIG. 6, when the server 11, which is the host, manages feature quantities, for example, the server 11 distributes the lists of Cam1 and Cam3 to Cam2, and distributes the list of Cam2 to Cam1 and Cam3, respectively. Distribute.

Here, one camera 12 can handle a plurality of feature quantities extracted by the feature quantity extraction unit 24. As described above, these plural feature quantities are managed in a list (feature table with priority) for each camera 12. Each camera 12 can independently determine the priority of a plurality of feature quantities in the list by the priority determination unit 27.

When outputting a characteristic quantity, the transmitting side (transmitting source) camera 12 compares the lists of the transmitting side and the receiving side (transmitting destination) and re-reads the transmitted data according to the priority of the receiving side camera 12. It can be arranged and printed.

When the transmission and reception capacity is limited due to constraints such as communication bandwidth or other systematic requests, each camera 12 sequentially discards the ones with the lowest priorities and adjusts the total capacity of the feature amount output for each object to fall within the restrictions. You can.

In addition, the list managed by each camera 12 is communicated between the server 11 and the camera 12 or between the cameras 12 (P2P) at regular intervals or event communication that occurs when a change occurs. It has a structure that can dynamically follow changes in the state of the camera 12.

Next, with reference to FIGS. 4 and 7 , a tracking method using the tracking system 10 through the server 11 will be described. FIG. 7 is a conceptual diagram for explaining a tracking method through the server 11 by the tracking system 10.

As shown in FIGS. 4 and 7 , the tracking system 10 captures images including Person 1, Person 2, and Person 3, which are objects to be tracked, with, for example, Cam2 of the camera 12. Next, Cam2 extracts each feature quantity of person 1, person 2, and person 3 from the captured image, calculates a three-dimensional movement direction vector, and packs the feature quantity and movement direction vector information for each person. Then, Cam2 transmits each person's packing information to the server 11. The server 11, which has received the packing information, compares the characteristic quantities in the packing information for each person with the characteristic quantities already registered in the server 11, and specifies the tracking target.

As shown in Fig. 7, when the result of the collation is that the characteristic quantity of person 1 is a new characteristic quantity that is not registered in the server 11, the server 11 registers the new characteristic quantity in the server 11, Based on the movement direction vector information in the packing information, it is determined that the movement destination of person 1 is in the Cam1 direction. When the characteristic quantities of person 2 and person 3 are characteristic quantities that have been registered in the server 11, the server 11 determines that their characteristic quantities are registered in the server 11 and adds them to the movement direction vector information in the packing information. Accordingly, it is determined that person 2's movement destination is in the Cam1 direction and person 3's movement destination is in the Cam3 direction.

Then, the server 11 transmits the information of person 1 and person 2 to Cam1, and the information of person 3 to Cam3, according to the moving direction of the tracking object, and transmits the information of person 3 to Cam3, and Cam1 tracks person 1 and person 2 (tracking ), and Cam3 tracks person 3.

Similar to the operation of Cam2, images containing person 1 and person 2 are captured with Cam1, and images containing person 3 are captured with Cam3. Next, Cam1 extracts each feature quantity of person 1 and person 2 from the captured image, calculates a three-dimensional movement direction vector, and packs the feature quantity and movement direction vector information for each person. Cam3 extracts the characteristic quantity of person 3 from the captured image, calculates a three-dimensional movement direction vector, and packs the characteristic quantity and movement direction vector information.

Then, Cam1 transmits packing information for each person to the server 11, and Cam3 transmits packing information for person 3 to the server 11. The server 11, which has received the packing information, compares the characteristic quantities in the packing information for each person with the characteristic quantities already registered in the server 11, and specifies the tracking target.

As shown in Fig. 7, as a result of the collation, the characteristic quantities of person 1 and person 2 have been registered in the server 11, so the server 11 has completed the registration of the characteristic quantities of person 1 and person 2 in the server 11 Then, based on the movement direction vector information in the packing information, it is determined that person 1's movement destination is in the CamX direction and person 2's movement destination is in the CamY direction.

Similarly, as a result of the collation, since the characteristic quantity of person 3 has been registered in the server 11, the server 11 determines that the characteristic quantity of person 3 has been registered in the server 11, and the movement direction vector information in the packing information Accordingly, it is determined that Person 3's moving destination is in the CamZ direction.

Then, the server 11 transmits the information of person 1, person 2, and person 3 to CamX, CamY, and CamZ, respectively, according to the moving direction of the tracking object, with CamX tracking person 1 and CamY tracking person 2. Tracks , and CamZ tracks person 3. The tracking system 10 can repeat these operations and track the tracking target by linking the plurality of cameras 12.

From the above, the tracking system 10 equipped with a plurality of cameras 12 according to the present embodiment can reduce the load by reducing the amount of data transmitted and received by using the characteristic quantity. Additionally, the tracking system 10 can acquire data by switching to the camera 12, which can recognize a person or object and acquire a plurality of appropriate feature quantities. Additionally, the tracking system 10 can dynamically change the amount of communication between cameras 12 based on the priority list. Therefore, according to the tracking system 10, the object detection accuracy can be increased while reducing the data bandwidth.

Additionally, the tracking system 10 can protect an individual's privacy because it does not directly output information about an individual's face or body with only feature quantities. Additionally, since the tracking system 10 has movement direction vector information for each object (person), when tracking multiple people captured by multiple cameras 12, it switches to which camera 12 for each object. The judgment process of what to do can be lightened. This makes it possible for the tracking system 10 to increase the number of people that can be tracked. Additionally, in order to collate people with a plurality of cameras 12 based on characteristic quantities, there is no need to overlap the imaging areas of the cameras 12.

2. Second embodiment

Next, with reference to FIGS. 8 to 10, a tracking method using a tracking system according to a second embodiment of the present technology will be described. Fig. 8 is a sequence diagram showing an example of list distribution by the tracking system according to the present embodiment. The tracking method according to the present embodiment differs from the tracking method according to the first embodiment in that information such as characteristic quantities is exchanged between the cameras 12. Other points of this embodiment are the same as those of the first embodiment.

As shown in Figure 8, when individual cameras 12 manage feature quantities, for example, Cam2 distributes the list of Cam1 and Cam3 to Cam1 and Cam3, respectively, and Cam1 and Cam3 distribute Cam2 to Cam2, respectively. Distribute the list of

Next, with reference to FIGS. 9 and 10 , a tracking method of automatic tracking by linking only the plurality of cameras 12 by the tracking system 10 will be described. Fig. 9 is a conceptual diagram for explaining a tracking method by the tracking system according to the present embodiment. Fig. 10 is a flowchart showing an example of a tracking method by the tracking system according to the present embodiment.

9 and 10, in step S11, Cam2 of the camera 12 captures images including Person 1, Person 2, and Person 3 as objects to be tracked, and captures Person 1, Person 2, and Person 3. Detect person 3. At this time, Cam2 extracts each feature quantity of person 1, person 2, and person 3 from the captured image, calculates a three-dimensional movement direction vector, and packs the feature quantity and movement direction vector information for each person. Then, Cam2 transmits each person's packing information to the server 11.

In step S12, the server 11 or Cam2 selects, for example, person 1 of interest as a tracking target. At this time, the server 11, which has received the packing information, compares the characteristic quantity in the packing information for each person with the characteristic quantity registered in the server 11, and specifies the tracking target.

As shown in Fig. 9, as a result of the collation, if the characteristic quantity of person 1 is a new characteristic quantity that is not registered in the server 11, the server 11 registers the new characteristic quantity in the server 11, Based on the movement direction vector information in the packing information, it is determined that the movement destination of person 1 is in the Cam1 direction. When the characteristic quantities of person 2 and person 3 are characteristic quantities that have been registered in the server 11, the server 11 determines that their characteristic quantities are registered in the server 11 and adds them to the movement direction vector information in the packing information. Accordingly, it is determined that person 2's movement destination is in the Cam1 direction and person 3's movement destination is in the Cam3 direction. Then, the server 11 transmits information about person 1 selected through Cam2.

In step S13, Cam2 marks and tracks person 1 based on information about person 1 received from the server 11.

In step S14, Cam2 determines that the movement destination of person 1 is in the Cam1 direction based on the movement direction vector information in the packing information. Then, Cam2 transmits instructions to switch the imaging camera to Cam1 and information about person 1 to Cam1.

In step S15, Cam1 continues tracking Person 1, similar to the operation of Cam2. At this time, Cam1 captures an image including person 1. Next, Cam1 extracts the characteristic quantity of person 1 from the captured image and calculates a three-dimensional movement direction vector. Cam1 determines a plurality of feature quantities from the extracted feature quantities according to the priority height and the allowable amount of the output destination, and packs the determined feature quantities of person 1 and the calculated movement direction vector information. Cam1 determines the camera 12 where person 1 is moving based on the movement direction vector information in the packing information. Then, based on the feature quantity in the packing information, person 1, which is an object, is tracked. After that, Cam1 transmits the instruction to switch the imaging camera to the determined camera 12 and information about person 1.

From the above, according to the tracking method according to the present embodiment, like the tracking method according to the first embodiment, object detection accuracy can be increased while reducing the data bandwidth. In addition, according to the tracking method according to the present embodiment, since the plurality of cameras 12 can transmit and receive characteristic quantities in conjunction with each other without going through the server 11, it is possible to automatically follow the highway tracking target.

3. Third embodiment

Next, with reference to FIGS. 11 and 12 , a tracking method using the tracking system 10 according to the third embodiment of the present technology will be described. Fig. 11 is a conceptual diagram showing list management by the tracking system 10 according to the present embodiment. A in FIG. 11 represents a list of feature quantities managed by the host server 11, B in FIG. 11 represents a list of feature quantities managed by Cam1, and C in FIG. 11 represents a list of feature quantities managed by Cam2. , and D in FIG. 11 shows a list of feature quantities managed by Cam3. The tracking method according to the present embodiment differs from the tracking method according to the first embodiment in that the priority of the feature quantity is determined according to the privacy level. Other points of this embodiment are the same as those of the first embodiment.

As shown in FIG. 11A, the server 11 manages, for example, a list in which feature quantities 1 to 8, which are all feature quantities of the object to be tracked, are associated with privacy levels. In this embodiment, feature quantities 1 to 3 and 6 to 8 are privacy level 0, feature quantity 4 is privacy level 1, and feature quantity 5 is privacy level 2.

As shown in FIG. 11B, Cam1 manages a list of feature quantities of the object to be tracked, for example, in order of priority, with feature quantities 1, 3, 6, 2, 4, and 5. As shown in FIG. 11C, Cam2, for example, manages a list of feature quantities of the object to be tracked in order of priority, with feature quantities 2, 1, 4, 5, 3, and 6. As shown in Figure 11D, Cam3, for example, manages a list of feature quantities of the object to be tracked in order of priority, with feature quantities 5, 2, 1, 3, 4, and 6. And feature quantity 4 of each of Cam1, Cam2, and Cam3 is privacy level 1, and feature quantity 5 is privacy level 2. Feature quantities other than feature quantities 4 and 5 of Cam1, Cam2, and Cam3 are privacy level 0.

Fig. 12 is a sequence diagram showing an example of a tracking method by the tracking system according to the present embodiment.

As shown in FIG. 12, for example, when transmitting information about person 1 from Cam1 of camera 12 to Cam2, the information about person 1 acquired by Cam1 is not transmitted in order of priority in Cam1's list, but is transmitted on the receiving side. The Cam2 list is rearranged according to the priority and privacy level and then transmitted. If there is a limit to the data transmission and reception capacity, output data is deleted and transmitted in order, starting from the lowest priority or the highest privacy level.

Cam2 compares the information about person 1 received from Cam1 with the information about person 1 acquired from Cam2, and determines whether they match. If they match, it is assumed that the person is person 1, and tracking of person 1 is performed with Cam2. In the case of a mismatch, it is assumed that the person is different from person 1, and a comparison is made with other possible people obtained from Cam2.

Afterwards, when Person 1 moves from the imaging range of Cam2 toward Cam3, Cam2 rearranges the list of Cam3 on the receiving side according to the priority and privacy level and then transmits Person 1's information.

Cam3, like Cam2, compares the information about person 1 received from Cam2 with the information about person 1 acquired from Cam3 and determines whether they match. If there is a match, it is assumed that the person is person 1, and tracking of person 1 continues with Cam3. In the case of a mismatch, it is assumed that the person is different from person 1, and a comparison is made with other possible people obtained from Cam3.

From the above, according to the tracking method according to the present embodiment, like the tracking method according to the first embodiment, object detection accuracy can be increased while reducing the data bandwidth, and judgment processing can be lightened. Additionally, according to the tracking method according to the present embodiment, since the feature amount to be transmitted is determined according to the privacy level, privacy can be further protected.

4. Fourth embodiment

Next, with reference to FIG. 13, a tracking method using the tracking system 10 according to the fourth embodiment of the present technology will be described. FIG. 13 is a conceptual diagram for explaining the tracking method by the tracking system 10 according to the present embodiment. The tracking method according to the present embodiment is different from the tracking method according to the first embodiment in that it tracks objects such as people by taking emotional information of multiple people into consideration. Other points of this embodiment are the same as those of the first embodiment.

As shown in FIG. 13, the tracking system 10 according to the present embodiment captures images including person 1, person 2, and person 3 with, for example, Cam1 of the camera 12, and detects the person from the captured images. Each feature quantity of person 1, person 2, and person 3 is extracted, a three-dimensional movement direction vector is calculated, and emotional information is acquired. In addition, images containing Person 4, Person 5, and Person 6 are captured with Cam2, the respective feature quantities of Person 4, Person 5, and Person 6 are extracted from the captured images, and a three-dimensional movement direction vector is calculated. , acquire emotional information. In addition, images containing Person 7, Person 8, and Person 9 are captured with Cam3, the respective feature quantities of Person 7, Person 8, and Person 9 are extracted from the captured images, and a three-dimensional movement direction vector is calculated. , acquire emotional information.

From the above, according to the tracking method according to the present embodiment, like the tracking method according to the first embodiment, object detection accuracy can be increased while reducing the data bandwidth. In addition, according to the tracking method according to the present embodiment, the flow of emotions of the entire place can be grasped from the emotional information of multiple people 1 to 9 obtained from Cam1, Cam2, and Cam3, so the flow of emotions of the entire place can be identified. Depending on the flow, you can track the tracking target.

Additionally, the present technology can have the following configuration.

(One)

An imaging unit that captures an image of an object,

a feature quantity extraction unit that extracts a plurality of feature quantities of the object imaged by the imaging unit;

a priority determination unit that determines priorities of the plurality of feature quantities extracted by the feature quantity extraction unit;

a control unit that determines the feature amount according to the height of the priority and the allowable amount of the output destination;

An output unit that outputs the characteristic quantity determined by the control unit.

An imaging device having a.

(2)

The imaging device according to (1), wherein the imaging unit simultaneously images the object in the same area.

(3)

Further comprising a movement direction calculation unit that calculates the movement direction of the object,

The imaging device according to (1) or (2), wherein the output unit outputs an output in association with the characteristic quantity and the moving direction.

(4)

The imaging device according to any one of (1) to (3), wherein the priority determination unit has a privacy level setting unit that sets a privacy level for each of the feature quantities, and determines the priority according to the privacy level.

(5)

further comprising an imaging position acquisition unit that acquires an imaging position and an imaging direction of the imaging unit;

The imaging device according to any one of (1) to (4), wherein the output unit outputs the imaging position and the imaging direction.

(6)

further comprising an environmental information acquisition unit that acquires environmental information about the surroundings where the imaging device is installed,

The imaging device according to any one of (1) to (5), wherein the control unit determines an output destination of the feature quantity based on the environmental information.

(7)

further comprising a state change information acquisition unit that acquires state change information of an output destination of the characteristic quantity,

The imaging device according to any one of (1) to (6), wherein the control unit determines the output destination according to the state change information.

(8)

Further comprising an emotion information acquisition unit that acquires emotion information regarding the emotion of the object,

The imaging device according to any one of (1) to (7), wherein the output unit outputs the feature quantity in association with the emotion information.

(9)

The imaging unit captures multiple images of the object,

further comprising an object selection unit that selects the object of interest from the plurality of objects imaged by the imaging unit;

The imaging device according to any one of (1) to (8), wherein the feature quantity extraction unit extracts a plurality of feature quantities of the object selected by the object selection unit.

(10)

an input unit for inputting a plurality of characteristic quantities of an object according to the priority height and the allowable input amount;

An imaging device comprising a tracking unit that tracks the object based on the characteristic quantity input from the input unit.

(11)

An imaging unit that captures an image of an object,

a feature quantity extraction unit that extracts a plurality of feature quantities of the object imaged by the imaging unit;

a priority determination unit that determines priorities of the plurality of feature quantities extracted by the feature quantity extraction unit;

a transmitting unit that transmits the characteristic quantity according to the height of the priority and the allowable amount of the transmission destination;

a receiving unit that receives the characteristic quantity according to the height of the priority and the allowable quantity that can be received;

Provided with a plurality of imaging devices having a tracking unit that tracks the object based on the characteristic quantity received by the receiving unit,

A tracking system wherein the plurality of imaging devices transmit and receive the characteristic quantity in conjunction with each other.

(12)

Steps for imaging an object,

A step of extracting a plurality of feature quantities of the imaged object;

A step of determining the priority of the plurality of extracted feature quantities;

a step of determining the feature quantity according to the height of the priority and the allowance of an output destination;

Step for outputting the determined characteristic quantity

An imaging method comprising:

(13)

Steps for inputting a plurality of characteristic quantities of an object according to the priority height and the allowable input amount,

A step of tracking the object based on the inputted feature quantity

An imaging method comprising:

10: Tracking system
11: Server
12: Camera (imaging device)
21: input unit
22: imaging unit
23: Imaging position acquisition unit
24: Feature extraction unit
25: Movement direction calculation unit
26: Privacy level setting section
27: Priority decision unit
28: control unit
29: Environmental Information Acquisition Department
30: State change information acquisition unit
31: Appraisal information acquisition unit
32: output unit
33: Tracking unit
34: Department of Communications
41: Target selection unit
42: Transmitting unit
43: Receiving unit

Claims (13)

An imaging unit that captures an image of an object,
a feature quantity extraction unit that extracts a plurality of feature quantities of the object imaged by the imaging unit;
a priority determination unit that determines priorities of the plurality of feature quantities extracted by the feature quantity extraction unit;
a control unit that determines the feature amount according to the height of the priority and the allowable amount of the output destination;
An output unit that outputs the characteristic quantity determined by the control unit.
An imaging device comprising: The imaging device according to claim 1, wherein the imaging unit simultaneously images the object in the same area. The method of claim 1, further comprising a movement direction calculation unit that calculates a movement direction of the object,
The imaging device wherein the output unit outputs an output in association with the characteristic quantity and the moving direction. The imaging device according to claim 1, wherein the priority determination unit has a privacy level setting unit that sets a privacy level for each of the feature quantities, and determines the priority according to the privacy level. The method of claim 1, further comprising an imaging position acquisition unit that acquires an imaging position and an imaging direction of the imaging unit,
The imaging device wherein the output unit outputs the imaging position and the imaging direction. The method of claim 1, further comprising an environmental information acquisition unit that acquires environmental information about the surroundings where the imaging device is installed,
The imaging device wherein the control unit determines an output destination of the feature quantity based on the environmental information. The method according to claim 1, further comprising a state change information acquisition unit that acquires state change information of an output destination of the characteristic quantity,
The imaging device, wherein the control unit determines the output destination according to the state change information. The method of claim 1, further comprising an emotion information acquisition unit that acquires emotion information regarding the emotion of the object,
The imaging device wherein the output unit outputs the feature quantity and the emotional information in association with each other. The method of claim 1, wherein the imaging unit captures multiple images of the object,
further comprising an object selection unit that selects the object of interest from the plurality of objects imaged by the imaging unit,
The imaging device wherein the feature quantity extraction unit extracts a plurality of feature quantities of the object selected by the object selection unit. an input unit for inputting a plurality of characteristic quantities of an object according to the priority height and the allowable input amount;
An imaging device comprising a tracking unit that tracks the object based on the characteristic quantity input from the input unit. An imaging unit that captures an image of an object,
a feature quantity extraction unit that extracts a plurality of feature quantities of the object imaged by the imaging unit;
a priority determination unit that determines priorities of the plurality of feature quantities extracted by the feature quantity extraction unit;
a transmitting unit that transmits the characteristic quantity according to the height of the priority and the allowable amount of the transmission destination;
a receiving unit that receives the characteristic quantity according to the height of the priority and the allowable quantity that can be received;
Provided with a plurality of imaging devices having a tracking unit that tracks the object based on the characteristic quantity received by the receiving unit,
A tracking system wherein the plurality of imaging devices transmit and receive the characteristic quantity in conjunction with each other. Steps for imaging an object,
A step of extracting a plurality of feature quantities of the imaged object;
A step of determining the priority of the plurality of extracted feature quantities;
a step of determining the feature quantity according to the height of the priority and the allowance of an output destination;
Step for outputting the determined characteristic quantity
Including an imaging method. Steps for inputting a plurality of characteristic quantities of an object according to the priority height and the allowable input amount,
A step of tracking the object based on the inputted feature quantity
Including an imaging method.

Images