JP2018139058A - Information processor and information processing method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information processor, an information processing method and a program making determination of mis-detection of an object more accurately.SOLUTION: An object is detected from a photographic image taken by photographic means. A feature quantity of the detected object and a direction of the object against the photographic means are obtained. A prediction value of a feature of the object is calculated based on those. In the next detection timing, the prediction value are compared with the feature quantity of a result of detection of the object by detection means, then whether the detection result is mis-detection is evaluated.SELECTED DRAWING: Figure 4

Description

  The present invention relates to an information processing apparatus, an information processing method, and a program.

  A technique for detecting a specific object such as a person or a car or a partial area thereof from an image is known. In addition, a technique for reducing erroneous detection when detecting a region where a specific object appears from an image is known. For example, Patent Document 1 discloses a method for determining whether or not each detection result is a false detection from the overlapping state when regions where objects are detected overlap. Patent Document 2 discloses a method of invalidating one of the detection results when an area where another head is detected is present within a predetermined distance from the area where the head is detected.

JP2013-61802A JP 2012-221968 A

However, the technique disclosed in Patent Document 1 has a problem in that it is impossible to determine whether or not an area where an object is detected is erroneously detected unless two or more areas where the object is detected overlap. Therefore, if the areas do not overlap, it cannot be determined that the detection is false even if no object is included in the area. Further, in the technique disclosed in Patent Document 2, since a detected head region within a predetermined range from the detection head region of interest is regarded as a false detection, a correct detected head existing within the predetermined range described above There was a problem that the area may be determined to be erroneously detected. As described above, in the conventional technique, there is a case where it is not possible to accurately determine erroneous detection of an object.
An object of the present invention is to make it possible to more accurately determine erroneous detection of an object.

  The information processing apparatus according to the present invention is based on a detection unit that detects an object from a captured image captured by the imaging unit, a feature amount of the object detected by the detection unit, and a direction of the object with respect to the imaging unit. Evaluation means for evaluating the detection result of the object by the detection means.

  According to the present invention, it is possible to more accurately determine erroneous detection of an object.

It is a figure showing an example of a system configuration of a detection system. It is a figure which shows an example of hardware constitutions, such as an imaging device. It is a figure which shows an example of functional structures, such as an imaging device. It is a flowchart which shows an example of a detection process. It is a figure which shows an example of imaging | photography environment. It is a figure which shows an example of the image | photographed image. It is a figure explaining an example of the relationship between coordinate systems. It is a figure which shows an example of the result of a detection. It is a figure which shows an example of imaging | photography environment. It is a figure which shows an example of the image | photographed image. It is a figure which shows an example of the positional relationship of an object and an imaging device.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.

<Embodiment 1>
In the present embodiment, the detection system performs the following processing. That is, the detection system predicts the value of the feature amount obtained from the detected area as a result of detecting the direction of the object as the subject viewed from the imaging apparatus and the object reflected in the direction. Thereafter, it is determined whether or not the feature value of the detected area actually detected deviates from the predicted value, and whether or not each detected area is erroneously detected based on the determined result. decide. In the present embodiment, the size of the detected area is used as the feature amount. However, the present invention is not limited to this, and feature quantities other than the size may be used.
FIG. 1 is a diagram illustrating an example of a system configuration of a detection system according to the present embodiment.
The detection system includes an imaging device 110 and a client device 120. The imaging device 110 and the client device 120 are connected via a network 150 so that they can communicate with each other. The client device 120 is connected to the input device 130 and the display device 140.
The imaging device 110 is an imaging device such as a network camera that performs imaging. The client device 120 is an information processing device such as a personal computer, a server device, or a tablet device that performs driving of the imaging device 110, acquisition of a captured image, detection of an object with respect to the acquired image, analysis, and the like. The input device 130 is an input device including a mouse, a keyboard, and the like. The display device 140 is a display device such as a monitor that displays an image output from the client device 120. In the present embodiment, the client device 120, the input device 130, and the display device 140 are independent devices. However, for example, the client device 120 and the display device 140 may be integrated, or the input device 130 and the display device 140 may be integrated. In addition, the client device 120, the input device 130, and the display device 140 may be integrated.
The network 150 is a network that connects the imaging device 110 and the client device 120. The network 150 includes a plurality of routers, switches, cables, and the like that satisfy a communication standard such as Ethernet (registered trademark). In the present embodiment, the network 150 only needs to be able to perform communication between the imaging device 110 and the client device 120, and the communication standard, scale, and configuration are not limited. For example, the network 150 may be configured by the Internet, a wired LAN (Local Area Network), a wireless LAN (Wireless LAN), a WAN (Wide Area Network), or the like.

FIG. 2A is a diagram illustrating an example of a hardware configuration of the imaging apparatus 110.
The imaging device 110 includes a CPU 211, a main storage device 212, an auxiliary storage device 213, a drive unit 214, an imaging unit 215, and a network I / F 216. Each element is connected to be communicable with each other via a system bus 217.
The CPU 211 is a central processing unit that controls the operation of the imaging device 110. The main storage device 212 is a storage device such as a random access memory (RAM) that functions as a work area for the CPU 211 and a temporary storage location for data. The auxiliary storage device 213 is a storage device such as a hard disk drive (HDD), a read only memory (ROM), or a solid state drive (SSD) that stores various programs, various setting data, and the like.
The drive unit 214 is a drive unit that drives the imaging device 110 to change the orientation and the like of the imaging device 110 and change the shooting direction and the angle of view of the imaging unit 215. The imaging unit 215 is an imaging unit that includes an imaging element and an optical system, and forms an image of a subject on the imaging element with an intersection of the optical axis of the optical system and the imaging element as an imaging center. Examples of the image pickup device include a complementary metal-oxide semiconductor (CMOS) and a charged coupled device (CCD). The network I / F 216 is an interface used for communication with an external device such as the client device 120 via the network 150.
The CPU 211 executes processing based on the program stored in the auxiliary storage device 213, thereby realizing the functions of the imaging device 110 and the processing of the imaging device 110, which will be described later with reference to FIG.

FIG. 2B is a diagram illustrating an example of a hardware configuration of the client device 120.
The client device 120 includes a CPU 221, a main storage device 222, an auxiliary storage device 223, an input I / F 224, an output I / F 225, and a network I / F 226. Each element is connected to be communicable with each other via a system bus 227.
The CPU 221 is a central processing unit that controls the operation of the client device 120. The main storage device 222 is a storage device such as a RAM that functions as a work area for the CPU 221 and a temporary storage location for data. The auxiliary storage device 223 is a storage device such as an HDD, ROM, or SSD that stores various programs, various setting data, and the like.
The input I / F 224 is an interface used when receiving an input from the input device 130 or the like. The output I / F 225 is an interface used for outputting information to the display device 140 or the like. The network I / F 216 is an interface used for communication with an external device such as the imaging device 110 via the network 150.
The CPU 221 executes processing based on the program stored in the auxiliary storage device 223, thereby realizing the processing of the client device 120 such as the function of the client device 120 described later in FIG. 3 and the processing of the flowchart described later in FIG. The

FIG. 3A is a diagram illustrating an example of a functional configuration of the imaging apparatus 110.
The imaging device 110 includes an imaging control unit 311, a signal processing unit 312, a drive control unit 313, and a communication control unit 314.
The imaging control unit 311 images the surrounding environment via the imaging unit 215. The signal processing unit 312 performs processing of an image captured by the imaging control unit 311. For example, the signal processing unit 312 encodes an image captured by the imaging control unit 311. In the case of a still image, the signal processing unit 312 performs image encoding using an encoding method such as JPEG (Joint Photographic Experts Group), for example. In the case of a moving image, the signal processing unit 312 performs the H.264 operation. An image is encoded using an encoding method such as H.264 / MPEG-4 AVC (hereinafter referred to as H.264) or HEVC (High Efficiency Video Coding method). In addition, the signal processing unit 312 encodes an image using, for example, a coding method selected by the user via the operation unit of the imaging device 110 from a plurality of preset coding methods. It may be.
The drive control unit 313 performs control to change the shooting direction and the angle of view of the imaging control unit 311 via the drive unit 214. However, the drive control unit 313 may change any one of the shooting direction and the angle of view by the imaging control unit 311. Further, the shooting direction and the angle of view of the imaging control unit 311 may be fixed. The communication control unit 314 transmits the image captured by the imaging control unit 311 processed by the signal processing unit 312 to the client device 120 via the network I / F 216. Further, the communication control unit 314 receives a control command for the imaging apparatus 110 from the client apparatus 120 via the network I / F 216.

FIG. 3B is a diagram illustrating an example of a functional configuration of the client device 120.
The client device 120 includes an input information acquisition unit 321, a communication control unit 322, an image acquisition unit 323, a detection unit 324, a prediction unit 325, an evaluation unit 326, and a display control unit 327.
The input information acquisition unit 321 receives input from the user via the input device 130. The communication control unit 322 receives the image transmitted from the imaging device 110 via the network 150. Further, the communication control unit 322 transmits a control command to the imaging device 110 via the network 150. The image acquisition unit 323 acquires an image photographed by the imaging device 110 from the imaging device 110 as an image that is a target of object detection processing via the communication control unit 322. The image acquisition unit 323 may acquire the image stored in the auxiliary storage device 223 as an image that is a target of the object detection process.
The detection unit 324 performs detection processing of the detection target object from the image acquired by the image acquisition unit 323. The prediction unit 325 predicts the feature amount of an object that can be detected in the image. The evaluation unit 326 erroneously detects the detection result of the detection unit 324 based on the feature amount of the object that can be detected in the image predicted by the prediction unit 325 and the feature amount of the object detected by the detection unit 324. It is determined whether or not. The display control unit 327 outputs an image to the display device 140 in accordance with an instruction from the CPU 221. For example, the display control unit 327 outputs, to the display device 140, an image indicating an object detected by the detection unit 324 excluding those determined to be erroneously detected by the evaluation unit 326.

FIG. 4 is a flowchart illustrating an example of an object detection process. Using FIG. 4, the client device 120 acquires an image from the imaging device 110, detects an object in the acquired image, and determines whether the detection result is a false detection for each detected region. Processing will be described.
In S <b> 400, the image acquisition unit 323 acquires an image captured by the imaging device 110 from the imaging device 110 as an object processing target image via the communication control unit 322. Hereinafter, the image captured by the imaging device 110 acquired in S400 is referred to as a captured image. In the present embodiment, it is assumed that the detection target object is a human body.
FIG. 5 is a diagram illustrating an example of an imaging environment in which the imaging apparatus 110 captures images according to the present embodiment. In FIG. 5, a range surrounded by a solid line extending from the imaging device 110 indicates a visible range of the imaging device 110. Objects 501 to 503 represent human bodies that are detection target objects in the present embodiment. In the present embodiment, the detection system detects a human body, but may detect an object other than the human body such as an automobile, a luggage, a drone, or an animal.

FIG. 6 is a diagram illustrating an example of a photographed image photographed by the imaging device 110 in the photographing environment illustrated in FIG. An image 600 shows a captured image obtained by capturing the imaging environment of FIG. In FIG. 6, objects 501 to 503 are the same objects as the objects 501 to 503 in FIG.
In the present embodiment, the client device 120 receives live video captured in real time from the imaging device 110, and performs the process of FIG. 4 on each frame of the received live video (moving image). However, the client device 120 may perform the process of FIG. 4 on each frame of a still image or a moving image stored in the auxiliary storage device 213 in the imaging device 110, for example. Further, the client device 120 may perform the process of FIG. 4 on each frame of a still image or a moving image stored in the auxiliary storage device 223 in the client device 120. Further, the client device 120 may access an external recording server and perform the processing shown in FIG. 4 on each frame of a still image or a moving image stored in the recording server.
In the present embodiment, the client device 120 is located in any direction from the imaging device 110 where arbitrary coordinates (x, y) of the captured image previously captured by the imaging device 110 before performing the processing of FIG. Ask for.

The Wx axis and Wy axis that go straight through the origin on a horizontal plane passing through the origin, and the direction perpendicular to the origin, with the viewpoint of the imaging device 110 (the pedestal point when the imaging unit 215 sees the environment to be photographed) as the origin And the Wz axis extending to A three-dimensional coordinate specified by the three axes of the Wx axis, the Wy axis, and the Wz axis is defined as a world coordinate system.
The coordinates on the world coordinate system can be shown in a form in which the respective positions on the three axes are arranged, for example, (x, y, z). The coordinates on the world coordinate system can also be expressed by spherical coordinates, for example (φ, θ, r). Here, φ indicates an angle formed by the Wx axis, a straight line passing through the origin and a point where the coordinates are projected on a plane including the Wx axis and the Wy axis. Θ is an angle formed by the Wz axis and a straight line passing through the origin and its coordinates. R is the length from the origin to the coordinates.
Here, φ and θ in coordinates on the world coordinate system displayed in spherical coordinates are information indicating in which direction the coordinates exist with respect to the imaging apparatus 110. In the present embodiment, the client device 120 obtains θ and φ in the coordinates on the world coordinate system from the coordinates on the screen coordinate system corresponding to the position of the object in the captured image. Identify the direction.

Also, with the viewpoint of the imaging device 110 as the origin, a Vz axis extending in the shooting direction (optical axis direction) of the imaging device 110 passing through the origin, and a Vx axis extending in the horizontal direction in an image taken by the imaging device 110 through the origin. Assuming Also, a Vy axis extending in the vertical direction in an image taken by the imaging device 110 through the origin is assumed. The three-dimensional coordinates specified by the three axes of the Vx axis, the Vy axis, and the Vz axis are set as a view coordinate system.
Further, an Sx axis that passes through the origin and extends in the horizontal direction in the image and an Sy axis that passes through the origin and extends in the vertical direction are assumed with the center of the image captured by the imaging device 110 as the origin. A two-dimensional coordinate specified by the two axes of the Sx axis and the Sy axis is defined as a screen coordinate system.

FIG. 7 is a diagram illustrating an example of the relationship among the world coordinate system, the view coordinate system, and the screen coordinate system. A rectangle in contact with Sx and Sy indicates an image (screen) photographed by the imaging device 110. Sw in FIG. 7 indicates the half length of the screen (half the length of the captured image). Further, d in FIG. 7 indicates the distance from the origin of the view coordinate system and the world coordinate system to the screen. Further, (x, y) in FIG. 7 indicates one point on the screen coordinates. (Φ, θ, r) are coordinates representing the coordinates of the point (x, y) in spherical coordinates.
That is, the client device 120 performs the following processing to obtain the direction (φ, θ) from arbitrary coordinates (x, y) in the screen coordinate system of the image captured by the imaging device 110. That is, the client device 120 may convert a point on the world coordinate system into a spherical coordinate representation after converting from the screen coordinate system to the view coordinate system and from the view coordinate system to the world coordinate system.
A method of converting from the screen coordinate system to the world coordinate system will be described. First, the client device 120 obtains the distance d from the view coordinate system origin to the screen based on the following formula (1).

Φmax in Expression (1) indicates a horizontal angle of view of the imaging apparatus 110. The client device 120 can acquire information on conditions at the time of shooting such as the angle of view of the imaging device 110 from the imaging device 110. In response to a request from the client device 120, the imaging device 110 transmits, to the client device 120, information on conditions at the time of shooting such as the angle of view of the imaging device 110 stored in the auxiliary storage device 213.
An arbitrary point (x, y) in the screen coordinate system can be represented by (x, y, d) on the view coordinate system. The client device 120 can convert the view coordinate system to the world coordinate system by applying a conversion matrix for converting the view coordinate system to the world coordinate system to the point (x, y, d) of the view coordinate system. it can. In the present embodiment, the client device 120 performs conversion from the screen coordinate system to the world coordinate system using a conversion matrix based on a quaternion. However, the client device 120 may use a rotation matrix around each of the x, y, and z axes, for example.
In order to convert from the view coordinate system to the world coordinate system, information on the optical axis direction of the imaging device 110 is necessary. In the present embodiment, the client device 120 obtains the attitude of the imaging device 110 based on information output from a gyro sensor in the imaging device 110. Further, the imaging apparatus 110 may obtain the attitude of the imaging apparatus 110 based on information output from a gyro sensor in the imaging apparatus 110 and transmit the obtained attitude information to the client apparatus 120. In addition, the client device 120 may acquire the posture information of the imaging device 110 by receiving input of posture information of the imaging device 110 based on a user operation via the input device 130, for example. In addition, the client device 120 may acquire information on pan and tilt driving performed by the imaging device 110 and may acquire information on the posture of the imaging device 110 from the acquired information on pan and tilt driving. Further, the client device 120 may perform camera calibration of the imaging device 110 based on a test chart placed on the floor.

In S401, the detection unit 324 performs a detection process of detecting a human body that is an object set as a detection target from the captured image acquired in S400.
In the present embodiment, the detection unit 324 first obtains enlarged / reduced images (hereinafter, scale images) of a plurality of photographed images by scaling the photographed images obtained in S400 with various sizes. To do. The detection unit 324 can detect objects of various sizes by performing object detection processing on the acquired enlarged / reduced images of the plurality of captured images.
Next, the detection unit 324 performs a raster scan on each scale image of the captured image using a detection window of a specific size. The auxiliary storage device 223 stores the feature amount of the detection target object calculated using the learning data in advance. When the error between the feature amount calculated from the detection window at the time of scanning and the feature amount based on the learning data stored in the auxiliary storage device 223 is smaller than the set threshold value, the detection unit 324 The area of the detection window in the captured image is determined to be an object. Information on this threshold is stored in advance in the auxiliary storage device 223. The detection unit 324 obtains the direction of the detected object with respect to the imaging device 110 from the center coordinates of the detected object region. More specifically, the detection unit 324 obtains the direction (φ, θ) of the detected object in the world coordinate system from the center coordinates of the detected object region.

FIG. 8 is a diagram illustrating an example of a result of detection processing performed by the detection unit 324 on a captured image. In FIG. 8, areas 801 to 803 indicate areas where the objects 501 to 503 are detected, respectively. In addition, areas 804 and 805 indicate misdetected areas, not objects. A region 804 shows a result of an erroneous detection that has a portion having characteristics close to the object to be detected inside. An area 805 indicates a result of an area that includes a plurality of dense objects having characteristics close to that of the object to be detected and appears as a false detection.
Hereinafter, the N (1 ≦ N) areas detected in S401 are referred to as detection areas n (n = 1 to N).

In S402, the prediction unit 325 determines whether or not to update the predicted value of the size of the detection area of the object that appears in each part of the captured image. In the present embodiment, the prediction unit 325 predicts the size of the detection area of the object that appears in each part of the captured image when the process of FIG. 4 is executed for the first time or when a certain period of time has elapsed since the previous process of FIG. It is determined that the value is updated, and the process proceeds to S403. In addition, when the process of FIG. 4 is not executed for the first time and when a certain period has not elapsed since the previous process of FIG. 4, the prediction unit 325 determines the size of the detection area of the object shown in each part of the captured image. Is determined not to be updated, and the process proceeds to S404.
In S403, the prediction unit 325 updates the predicted value of the object size in the direction of each detection area based on each detection area. In the present embodiment, the prediction unit 325 performs interpolation using a spherical harmonic function based on the direction and size of the detection area detected in S401, and predicts the size of the detection area of the object reflected in each direction. Therefore, the prediction unit 325 calculates the base coefficient c of the spherical search function based on the following equation (2).

  In the equation (2), φn and θn each represent the direction of the detection region n (n = 1 to N) detected in S401. Y represents the basis function of the spherical harmonic function, the first argument is the order, the second argument is the order, and the third and fourth arguments are the directions of the detection region. Sn represents the size of the detection region n. The prediction unit 325 can obtain the coefficient c (l, m) of the spherical harmonic function Y (l, m, φ, θ) using the equation (2). Then, the prediction unit 325 calculates a predicted value s (φ, θ) of the size of the detection region in an arbitrary direction (φ, θ) based on the following equation (3).

In the present embodiment, the order l is as small as possible. As a result, even if the spherical harmonic function coefficients are calculated using detection regions that are remarkably different in size from the surroundings, such as regions 804 and 805, high-frequency interpolation is possible, and these are regarded as outliers. be able to.
In addition, each base of the spherical harmonic function depends only on the value of θ when the order is 0, and does not depend on the value of φ. Therefore, in this embodiment, it is assumed that the size of the object region depends only on the depression angle in the world coordinate system from the imaging device 110 to the object, and does not depend on the azimuth angle, and the order m is set to zero.
As described above, the prediction unit 325 obtains a predicted value of the size of the object detection area using the spherical harmonic function. The prediction unit 325 can obtain a predicted value corresponding to the direction from the imaging device 110 in the world coordinate system by using a spherical harmonic function.
For example, it is assumed that the pan / tilt drive of the imaging device 110 occurs and the field of view of the imaging device 110 changes. However, the positional relationship between the imaging device 110 and the object in the world coordinate system does not change. Therefore, the client device 120 can use the predicted value of the detection area size of the object obtained using the spherical harmonic function in the world coordinate system before and after changing the field of view of the imaging device 110. Therefore, by using the spherical harmonic function, the prediction unit 325 does not have to calculate the predicted value of the size of the detection area of the object for each direction every time the field of view of the imaging device 110 changes, and the calculation process Can be reduced. Further, the prediction unit 325 uses a function that can approximate the size of the detection area based on the direction from the imaging device 110 in the world coordinate system, such as a radial basis function other than the spherical harmonic function, to predict the size of the detection area of the object. Even if it asks for, the same effect can be acquired.

In S404, the evaluation unit 326 creates a list of detection areas n and stores it in the auxiliary storage device 223 or the like. The evaluation unit 326 stores the size, center coordinates, and direction of each detection area in each node in the list to be created. In the example of FIG. 7, the evaluation unit 326 stores information on the rectangle size, center coordinates, and direction calculated from the center coordinates of the regions 801 to 805 in the list.
Next, the evaluation unit 326 determines whether each detection region in the list is erroneously detected while scanning the list created in S404. The flow of processing for determining whether or not it is erroneous detection is as follows.
In S405, the evaluation unit 326 selects the first detection area from the list created in S404. Hereinafter, the currently selected detection area is referred to as a detection area i.

In S406, the evaluation unit 326 performs the following processing based on the size, which is the feature amount of the detection area selected in S405 or S410 described later, and the direction of the detection area selected in S405 or S410. That is, the evaluation unit 326 performs a process of determining whether or not the detection region selected in S405 or S410 is erroneously detected. The evaluation unit 326, for example, the size s ′ (φi, θi) of the detection area i and the predicted value s (φi, θi) of the size of the detection area of the object corresponding to the direction (φi, θi) of the detection area i, , The detection result is evaluated for the detection region i.
In the present embodiment, the evaluation unit 326 evaluates the detection result related to the detection region i by determining whether or not the detection of the object for the detection region i is a false detection. However, for example, the evaluation unit 326 may evaluate the detection result related to the detection region i by obtaining a set likelihood indicating the probability that the detection region i is an object for the detection region i.
The evaluation unit 326 determines how much the size s ′ (φi, θi) of the detection region i is from the predicted value s (φi, θi) of the size predicted by the prediction unit 325 from the direction (φi, θi) of the detection region i. Based on the degree of deviation, the detection result relating to the detection region i is evaluated. In the present embodiment, when the size s ′ (φi, θi) of the detection area i satisfies the condition shown in the following expression (4), the evaluation unit 326 displays the detection result of the object related to the detection area i. It is determined that it is not a false detection. On the other hand, when the size s ′ (φi, θi) of the detection area i does not satisfy the condition of Expression (4), the evaluation unit 326 determines that the detection result of the object related to the detection area i is a false detection.

In the present embodiment, the values of α (φi, θi) and β (φi, θi) in equation (4) are both 10% of s (φi, θi). However, the evaluation unit 326 may obtain the values of α and β using a spherical harmonic function, for example. At this time, the evaluation unit 326 calculates α and β coefficients by randomly changing the coefficients of α and β, and calculates some of the data in the detection area in which whether or not erroneous detection has been performed in advance is represented by Equation (4). The coefficient value having the largest number of detection areas that have been correctly determined as erroneous detection may be employed.
In this way, the evaluation unit 326 determines whether or not a false detection is made based on the degree of deviation from the predicted value of the size of the detection region i.
In the present embodiment, the prediction unit 325 generates a prediction value by using a detection area obtained as a result of object detection for a captured image. However, for example, if the number of objects detected from the captured image is less than the set threshold value, the prediction unit 325 may use the detection result performed on the past captured image. The prediction unit 325 stores the past captured image in the auxiliary storage device 223 and the like by storing the size, the center coordinates, and the direction from the imaging device 110 in the detection area detected for the past captured image. Available.

In S407, the evaluation unit 326 stores information indicating that the detection area i is a false detection in the list created in S404.
In S408, the evaluation unit 326 stores information indicating that the detection area i is not erroneously detected in the list created in S404.
In S409, the evaluation unit 326 determines whether the detection area i is the detection area at the end of the list created in S404. If the evaluation unit 326 determines that the detection area i is the detection area at the end of the list created in S404, the evaluation unit 326 proceeds to the process of S411. If the evaluation unit 326 determines that the detection area i is not the detection area at the end of the list created in S404, the process proceeds to S410.

In S410, the evaluation unit 326 selects a detection area next to the current detection area i in the list created in S404 as a new detection area i, and the process proceeds to S406. In this way, the evaluation unit 326 sequentially determines whether or not the detection areas registered in the list are erroneously detected.
In S411, the display control unit 327 outputs the detection result by displaying the image 400, which is a captured image, on the display device 140 together with the detection result of the subject to be detected. For example, the display control unit 327 superimposes and displays a frame indicating each detection region that is determined not to be erroneously detected by the evaluation unit 326 on the captured image. For example, the display control unit 327 displays a screen obtained by removing the rectangular frames in the areas 804 to 805 from the screen in FIG. Note that the display control unit 327 may display a frame having a shape such as an ellipse other than a rectangle as a frame indicating the detection region.

As described above, in the present embodiment, the client device 120 detects an object from the image captured by the imaging device 110, and performs the following processing for each detected detection area. In other words, the client device 120 determines whether the detection region is erroneously detected based on the size of the detection region in the image and the predicted value of the size of the detection region set according to the direction of the detection region with respect to the imaging device 110. Decided whether or not.
Thereby, the detection system can reduce the possibility of erroneous detection. Further, even when a plurality of objects are densely populated, the detection system does not mean that a detection area near a detection area that is not erroneously detected is erroneously detected. As described above, the detection system can more accurately determine the erroneous detection of the object.

In the present embodiment, the detection system uses the size of the detection area as the feature amount of the object. However, the detection system may use, for example, the average brightness of the detection area as the feature amount of the object. For example, it is assumed that the shooting environment in which the imaging apparatus 110 captures includes a shaded area and a illuminated area. In that case, when the object to be detected is present in the shaded area, it can be assumed that the average brightness of the detected area in the image is lower than that in the case where the object is present in the certified area.
Therefore, the evaluation unit 326 determines that the detection area is false detection when the direction of the detection area is in the direction indicating the shaded area, and the average brightness of the detection area is equal to or greater than the set threshold, and is less than the set threshold. If it is, it may be determined that there is no false detection. In addition, the evaluation unit 326 determines that the detection area is erroneously detected when the direction of the detection area is in the direction indicating the illuminated area, and the average brightness of the detection area is less than the set threshold value. If it is equal to or greater than the threshold value, it may be determined that there is no false detection.

<Embodiment 2>
In the first embodiment, the prediction unit 325 determines the predicted value of the size of the detection area of an object appearing in an arbitrary direction when the process in FIG. 4 is executed for the first time, or when a certain period has elapsed since the previous update. It was updated. Further, the prediction unit 325 assumes that the size of the object detection area depends only on the depression angle of the imaging device 110 in the world coordinate system, and does not depend on the azimuth angle, and the spherical harmonic function used for prediction of the object detection area. The order m was limited to 0. In the present embodiment, an example will be described in which the size of the detection area of the object also depends on the azimuth angle of the imaging device 110 in the world coordinate system. In the present embodiment, a process for updating the predicted value of the size of the detection area as soon as the dependence of the size of the detection area of the object on the azimuth angle can be confirmed will be described.
The system configuration of the detection system of the present embodiment is the same as that of the first embodiment. The hardware configurations and functional configurations of the imaging device 110 and the client device 120 are the same as those in the first embodiment.
The processing of the client device 120 in this embodiment is the processing shown in FIG. 4 as in the first embodiment.

As a shooting environment in which the size of the detection area of the object depends on the azimuth angle of the imaging device 110 in the world coordinate system, for example, when a staircase appears in the captured image and there is a human body using the staircase, There may be a crouched human body. An example of such a case will be described with reference to FIGS.
FIG. 9 is a diagram illustrating an example of a shooting environment in which a plurality of objects exist in the same depression direction from the imaging device 110. In the example of FIG. 9, there are three objects 901 to 903. The object 901 is crouching on the floor. The object 902 stands upright under the stairs. The object 903 stands upright on the stairs. At this time, the image 1000 shown in FIG.

FIG. 10 is a diagram illustrating an example of an image captured by the imaging device 110 in the imaging environment of FIG. Objects 901 to 903 in FIG. 10 indicate objects 901 to 903 in the shooting environment in FIG. 9, respectively. It can be seen from FIG. 10 that even though each object is present in the same depression angle direction from the imaging device 110, there is a variation in size. In this case, in the process of the first embodiment, any of the objects 901 to 903 may be regarded as a false detection.
Therefore, in the present embodiment, the client device 120 performs the following processing.
When the client apparatus 120 repeats the process of FIG. 4 and erroneous detection frequently occurs in the set part of the photographed image, and the variation in the size of the erroneously detected detection area is lower than the set threshold, the following is performed. Perform proper processing. That is, as shown in FIG. 9, the client apparatus 120 determines that the size of the object detection area is a shooting environment that also depends on the azimuth angle, and generates a predicted value of the size of the object detection area reflected in an arbitrary direction. Try again.
For example, the evaluation unit 326 divides the captured image into an arbitrary number of areas when the process of FIG. 2 is repeated, counts frames in which false detections continuously occur in each area, and calculates the count number. This is stored in the auxiliary storage device 223. Further, the evaluation unit 326 stores information on the size of the erroneously detected detection area generated at that time in the auxiliary storage device 223. In S202, the prediction unit 325 predicts the size of the detection area of the object based on the count number stored in the auxiliary storage device 223 by the evaluation unit 326 and the size information of the detection area that has been erroneously detected. Determine whether to update the value. For example, when the number of counts is equal to or greater than a set threshold value, and the variance value of the size of the detected basin associated with a plurality of false detections that have occurred continuously is equal to or less than the set threshold value, the prediction unit 325 It is determined that the predicted value of the size of the detection area is updated. In other cases, the prediction unit 325 determines not to update the predicted value of the size of the detection area of the object. At this time, the prediction unit 325 uses a base whose order m of the spherical harmonic function is not 0, and cancels the restriction of the order m as in the first embodiment. Thereby, in this embodiment, the estimation part 325 can produce | generate the predicted value of the size of the detection area | region of the object reflected in arbitrary directions from which a size differs locally.

  As described above, in the present embodiment, the client device 120 determines the validity of the assumed assumption based on the position of erroneous detection that has occurred in the past and the size of the erroneously detected detection area. If the assumed assumption is incorrect, the client device 120 regenerates the predicted value of the size of the detection area of the object in the captured image based on the other assumptions, and generates the regenerated predicted value. Based on this, the detected object was evaluated. In other words, the client device 120 regenerates the predicted value of the size of the object detection area based on the past evaluation result by the evaluation unit 326. Thereby, the client apparatus 120 can further improve the accuracy of object detection as compared with the first embodiment.

<Embodiment 3>
In the first and second embodiments, the client device 120 performs object detection on the captured image captured by the imaging device 110. Based on the detection result, the client device 120 generates a predicted value of the size of the detection area of the object appearing in an arbitrary direction in the captured image. In the present embodiment, the client device 120 will be described with reference to processing for generating a predicted value of the size of the detection area of an object appearing in an arbitrary direction based on the installation position of the imaging device 110.
The system configuration of the detection system of the present embodiment is the same as that of the first embodiment. The hardware configurations and functional configurations of the imaging device 110 and the client device 120 are the same as those in the first embodiment.
The processing of the client device 120 in this embodiment is the processing shown in FIG. 4 as in the first embodiment.

In the present embodiment, the prediction unit 325 uses information on the installation position of the imaging device 110 when generating a predicted value of the size of the object detection area in S203.
FIG. 11 is a diagram illustrating an example of a positional relationship between an object and an imaging apparatus. The shooting environment in FIG. 11 shows a shooting environment in which the object 1101 exists in the space where the imaging device 110 is installed. In FIG. 11, wp1 (x1, y1, z1) indicates coordinates in the world coordinate system at the top of the object 1101. Further, wp2 (x2, y2, z2) indicates coordinates in the world coordinate system at the bottom of the object 901. In this case, z2 is obtained from the installation height of the imaging device 110. z1 is obtained from the height of the object. For example, two points wp1 and wp2 are converted from the world coordinate system to coordinates vp1 and vp2 on the view coordinate system. Thereafter, the view coordinate system is converted into points sp1 and sp2 on the screen coordinate system. At this time, the client device 120 calculates the distance between the points sp1 and sp2 on the captured image, and calculates the size when the subject existing at the point wp2 on the world coordinate system is detected based on the calculated distance. be able to.

In the present embodiment, the prediction unit 325 performs the following processing when performing the processing of FIG. 4 for the first time. The imaging device 110 captures the shooting environment of FIG. 11 while changing the location where the object 1101 exists, and prepares a plurality of patterns of captured images of the shooting environment of FIG. The prediction unit 325 obtains the size of the object 1101 for each of a plurality of patterns of captured images captured by the imaging device 110. Then, the predicting unit 325 generates a predicted value of the size of the detection area of the object appearing in an arbitrary direction in the captured image based on the obtained size. Thereby, the prediction unit 325 can generate a predicted value of the size of the detection area of the object before executing the object detection process.
In the present embodiment, the prediction unit 325 generates the prediction value only when the process of FIG. 2 is performed for the first time, but the prediction value may be dynamically updated.
In the present embodiment, as in the first embodiment, the prediction unit 325 limits the basis of the spherical harmonic function to be used to those having an order m of 0. However, the prediction unit 325 may release the restriction dynamically as in the second embodiment.

  As described above, in the present embodiment, the prediction unit 325 generates a predicted value of the size of the detection area of the object in the captured image captured by the imaging device 110 based on the installation position of the imaging device 110. Accordingly, the client device 120 can generate a predicted value of the size of the detection area of the object based on the position of the imaging device 110, not the direction of the object with respect to the imaging device 110.

<Other embodiments>
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in the computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

For example, a part or all of the functional configuration of the above-described detection system may be implemented in the imaging device 110 or the client device 120 as hardware.
As mentioned above, although preferable embodiment of this invention was explained in full detail, this invention is not limited to the specific embodiment which concerns. You may combine each embodiment mentioned above arbitrarily.

110 Imaging device 120 Client device 221 CPU

Claims (11)

Detecting means for detecting an object from a photographed image photographed by the imaging means;
Evaluation means for evaluating the detection result of the object by the detection means based on the feature amount of the object detected by the detection means and the direction of the object with respect to the imaging means;
An information processing apparatus.   The information processing apparatus according to claim 1, wherein the evaluation unit evaluates a detection result of the object and determines whether or not the detection of the object by the detection unit is a false detection.   The information processing apparatus according to claim 2, further comprising an output unit that outputs information indicating a result determined by the evaluation unit as not being erroneously detected among detection results by the detection unit.   The said evaluation means evaluates the detection result of the said object by the said detection means based on the said feature-value and the predicted value of the said feature-value according to the said direction. Information processing device. Further comprising generating means for generating the predicted value;
The information according to claim 4, wherein the evaluation unit evaluates a detection result of the object by the detection unit based on the feature amount and the predicted value corresponding to the direction generated by the generation unit. Processing equipment.   The information processing apparatus according to claim 5, wherein the generation unit generates the predicted value based on the direction.   The information processing apparatus according to claim 5, wherein the generation unit regenerates the predicted value based on a result of evaluation by the evaluation unit.   The information processing apparatus according to claim 5, wherein the generation unit generates the predicted value based on an installation position of the imaging unit.   The information processing apparatus according to claim 1, wherein the feature amount is a size of an area that can be detected as the object. An information processing method executed by an information processing apparatus,
A detection step of detecting an object from a captured image captured by the imaging means;
An evaluation step for evaluating a result of detection of the object in the detection step based on a feature amount of the object detected in the detection step and a direction of the object with respect to the imaging unit;
An information processing method including:   A program for causing a computer to function as each unit of the information processing apparatus according to any one of claims 1 to 9.

Images