JP6759400B2 - Information processing equipment, information processing methods, and programs - Google Patents

Description

The present invention relates to a technique for detecting an object from a camera image.

In a surveillance system or a monitoring camera system, there is a technique for detecting a moving object in an image by using an image recognition process. Further, a technique of recognizing a detected moving object so as to always catch it is known as a moving object tracking technique.

Further, Patent Document 1 describes that an object larger than a preset minimum detection size is detected by image recognition.

Japanese Unexamined Patent Publication No. 2012-242970

However, there is a risk that proper image recognition processing cannot be performed.

For example, if the maximum size and the minimum size of an object to be detected are changed by the recognition process, there is a risk that the object that should have been detected will not be detected.

The present invention has been made in view of the above problems, and an object of the present invention is to enable more suitable recognition processing for captured images.

To achieve the above object, the information processing apparatus of the present invention, for example, a first guide indicating the maximum size Ru engaging the objects that are detected from the captured image by the imaging means, the minimum size of the object a setting screen including the image of the second with the guide and is superimposed showing the, the maximum size and the setting screen capable of changing the setting of the minimum size and display control means for displaying on the Viewing device, wherein an area for determining the intrusion of an object and a setting means for setting in the image, wherein the display control unit, said object size between the previous SL maximum size and the minimum size has penetrated into the region In the case , the display device is made to execute a predetermined display , and the display control means causes the display device to display a predetermined message when the settings of the maximum size and the minimum size meet the predetermined conditions. It is a feature.
Further, in order to achieve the above object, the information processing apparatus of the present invention has, for example, a first guide indicating a maximum size of an object detected from an image captured by an imaging means, and a minimum size of the object. A display control means for displaying on a display device a setting screen including the image in which the second guide indicating the above is superimposed, and the setting of the maximum size and the minimum size can be changed, and the object. The display control means has a setting means for setting a line for determining the passage of the object in the video, and the display control means is used when the object having a size between the maximum size and the minimum size passes through the line. The display device is made to execute a predetermined display, and the display control means causes the display device to display a predetermined message when the settings of the maximum size and the minimum size correspond to a predetermined condition. To do.

According to the present invention, the user can set a setting that enables more suitable recognition processing for the image captured by the camera.

Configuration diagram of the video processing system. The block diagram which shows the configuration example of the control device 200. The figure which shows the composition example of the information managed by the locus management unit 207. The figure which shows the example of the correspondence between the object and the human body. The figure which shows an example of the screen which performs the setting process of the human body detection processing size. The figure which shows the structural example of the parameter for the image recognition processing. The flowchart which shows the processing procedure of the control device 200. A flowchart showing a processing procedure of parameter control.

Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. In addition, the embodiment described below shows an example when the present invention is concretely implemented, and is one of the specific embodiments of the configuration described in the claims.

FIG. 1 is a configuration diagram of a video processing system, and 101 and 108 are cameras equipped with an optical zoom mechanism. 102 is a network such as a LAN. 104 and 106 are PCs (personal computers). Reference numerals 105 and 107 are displays for displaying images based on image data from cameras 101 and 108 and processing results of PC 104 and PC 106. In addition, the displays 105 and 107 can also provide a user interface for setting the image recognition process according to the present embodiment.

A configuration example of the control device 200 according to the first embodiment will be described with reference to FIG. In the present embodiment, the control device 200 will be described as being incorporated in the camera (camera 101 or camera 108 in FIG. 1). However, the function of the control device 200 may be realized by, for example, a PC (personal computer) (corresponding to PCs 104 and 106 in FIG. 1), or by an image processing circuit mounted in a camera capable of capturing a moving image. It may be realized by other devices. The control device 200 has a function of displaying a parameter setting screen for performing image recognition processing (for example, human body detection) on the display screen of the display device 210 and setting parameters according to user instructions on the parameter setting screen. Has. In this embodiment, the display device 210 of FIG. 2 corresponds to the displays 105 and 107 of FIG.

The control device 200 includes an image acquisition unit 201, an object detection unit 202, an object tracking unit 203, a human body detection unit 204, a parameter setting unit 205, an object correspondence unit 206, a trajectory management unit 207, and trajectory information. A determination unit 208 and an external output unit 209 are provided. Further, the control device 200 includes a zoom control unit 211 for controlling the zoom mechanism of the camera and a parameter control unit 212 for controlling parameters according to the zoom magnification (zoom value). Further, the control device 200 is connected to a display device 210 composed of a CRT (Cathode Ray Tube), a liquid crystal screen, and the like. The display device 210 displays the processing result of the control device 200 as an image, characters, or the like. Hereinafter, a case where a moving image is displayed on the display screen of the display device 210 will be mainly described.

The image acquisition unit 201 acquires a moving image or a still image supplied from the internal image sensor or the outside, and sends the acquired moving image or the still image to the object detection unit 202.

When the image acquisition unit 201 acquires a moving image, the image of each frame constituting the moving image is sequentially sent to the object detection unit 202, and when the still image is acquired, the still image is sent to the object detection unit 202. Send to 202. The source of the moving image or the still image is not particularly limited, and may be a server device or an imaging device that supplies the moving image or the still image via a wired or wireless connection. Further, the present invention is not limited to the outside, and a moving image or a still image may be acquired from the memory in the control device. In the following description, a case where a certain image is input to the object detection unit 202 will be described regardless of whether the image acquisition unit 201 acquires a moving image or a still image. To do. In the former case, this one image corresponds to each frame constituting the moving image, and in the latter case, this one image corresponds to a still image.

The object detection unit 202 detects an object by the background subtraction method from the frame image acquired from the image acquisition unit 201. That is, the image acquisition unit 201 detects an object by comparing the frame image acquired by the image acquisition unit 201 with a predetermined frame image (background image). Then, the object detection unit 202 generates object information in response to the detection of the object. The object information includes information about the position of the object on the screen, the circumscribing rectangle, and the size of the object. The object detection unit 202 has a function of detecting an object from an image by the background subtraction method, but the method is not limited to this method.

The object tracking unit 203 associates an object detected from each of the plurality of frame images with the object based on the object information corresponding to the plurality of frame images. More specifically, when the object tracking unit 203 detects an object corresponding to an object detected from the image of the frame one frame before the frame of interest by the object detecting unit 202 from the image of the frame of interest, each frame Corresponds the objects in.

For example, it is assumed that the object tracking unit 203 assigns the object ID = A to the object detected by the object detecting unit 202 from the image of the frame one frame before the frame of interest. Then, when the object detection unit 202 detects an object corresponding to the object with the object ID = A from the image of the frame of interest, the object tracking unit 203 also assigns the object ID = A to this object. In this way, the object tracking unit 203 assigns the same object ID to each object when the corresponding object is detected over a plurality of frames. A new object ID is assigned to the object newly detected in the frame of interest.

In the object tracking unit 203, as a method of determining whether or not objects of different frames correspond to each other, the predicted movement position of the object specified by using the movement vector of the detected object and the position of the detected object are a fixed distance. If it is inside, there is a method of making it the same object. That is, the object tracking unit 203 specifies the distance between the position of the object detected from the frame image and the position of the object specified based on the movement vector information related to the frame image. Then, when the specified distance is less than the threshold value, the object tracking unit 203 associates the object with an object of another frame image.

However, the method of associating objects is not limited to the above method, and there is also a method of associating objects with high correlation between frames by using, for example, the color, shape, size (area) of the object. Further, the information of the movement vector and the information such as the color, shape, and size of the object may be combined to associate the objects. The object position tail 203 performs a process of associating and tracking an object detected over a plurality of frames according to a predetermined condition. For example, when the same person continues to exist over a plurality of frames, the object tracking unit 203 assigns the same object ID to the person. The method of associating objects is not limited to a specific method, and various methods for performing the same processing can be adopted.

The human body detection unit 204 detects a human body by performing a human body detection process on an area in which the object detection unit 202 has detected an object among the object detection areas set by the parameter setting unit 205 described later. The human body detection unit 204 of the present embodiment performs the human body detection process on the region in which the object is detected and the region including the surrounding area (human body detection region). For example, when an object having a height of 100 pixels and a width of 20 pixels centered on the coordinates (X, Y) in the frame is detected, the human body detection unit 204 centers on the coordinates (X, Y) of the frame. The human body detection process is performed on an area of 150 pixels in length and 30 pixels in width.

Further, the human body detection unit 204 can omit the human body detection process outside the range of the maximum size and the minimum size by referring to the maximum size and the minimum size of the human body detection set by the parameter setting unit 205. That is, the human body detection unit 204 can omit the human body detection process for the human body detection area larger than the maximum size and the human body detection area smaller than the minimum size in the human body detection area specified by the detection of the object. However, the human body detection unit 204 performs the human body detection process without omitting the human body detection process, and the detection result is obtained when the detected human body size is larger than the maximum size and smaller than the minimum size. Can be excluded from the processing result of the human body detection process.

The human body detection unit 204 detects the human body from the frame image by comparing the pattern image of the human body with the frame image, but the method for detecting the human body is not limited to this method. Further, in the present embodiment, the detection target is the human body, but the detection target is not limited to the human body. The detection target may be a person's face, a car, an animal, or the like. Further, it is also possible for the human body detection unit 204 to simultaneously execute a plurality of types of detection processes so that a plurality of types of specific objects are detected. That is, the human body detection unit 204 of the present embodiment can detect not only the human body but also various predetermined objects from the image data by the recognition process.

Further, it is not always necessary that the object detection unit 202 determines the human body detection area based on the area where the object is detected. For example, the human body detection unit 204 may specify the human body detection region from the parameters set by the parameter setting unit 205 and perform the human body detection process. In this case, it may be possible to omit the object detection process by the object detection unit 202.

The parameter setting unit 205 sets parameters related to the detection processing range (human body detection area) of the human body detection process in each frame, the maximum size and the minimum size of the human body detection, and the like.

Further, the parameter setting unit 205 may set not only the human body detection but also the object detection unit 202 with the same parameters related to the detection process. For example, the parameter setting unit 205 can set parameters related to the object detection area, the maximum size and the minimum size of the object detection, and the like as parameters for the object detection unit 202. However, in the description of this embodiment, it is assumed that the object detection area is the entire image acquired by the image acquisition unit 201. Generally, if the object detection area is narrowed, the processing speed increases.

The object-corresponding unit 206 associates the object detected by the object detection unit 202 with the human body detected by the human body detection unit 204. An example of associating an object with a human body will be described with reference to FIGS. 4A and 4B. FIG. 4A shows an example in which the external rectangle 401 of the object detected by the object detection unit 202 does not include the external rectangle 402 of the human body detected by the human body detection unit 204. The object detection unit 202 of the present embodiment performs object detection processing on the entire frame, and the human body detection unit 204 uses a region including the periphery of the circumscribing rectangle of the object detected by the object detection unit 202 as a human body detection region. Performs human body detection processing.

When an object and a human body are detected as shown in FIG. 4A, the object-corresponding portion 206 is set when the superposition ratio of the circumscribing rectangle 402 of the human body with respect to the circumscribing rectangle 401 of the object exceeds a preset threshold value. Make a mapping. That is, when the ratio of the area of the overlapping region where the circumscribing rectangle 401 of the object and the circumscribing rectangle 402 of the human body overlap to the area of the circumscribing rectangle 402 of the human body exceeds the threshold value, the object-corresponding portion 206 has the circumscribing rectangle 401. The object corresponding to is associated with the human body corresponding to the circumscribing rectangle 402.

On the other hand, FIG. 4B shows an example in which a plurality of human bodies are detected from the circumscribing rectangle 403 of the detected object. In this case, the object-corresponding portion 206 is the circumscribing rectangle when the ratio of the area of the overlapping region where the extrinsic rectangle 403 of the object and the extrinsic rectangle 404 of the human body overlap to the area of the circumscribing rectangle 404 of the human body exceeds the threshold value. The object corresponding to the 403 and the human body corresponding to the circumscribing rectangle 404 are associated with each other. Further, the object-corresponding portion 206 is set in the external rectangle 403 when the ratio of the area of the overlapping region where the external rectangular 403 of the object and the external rectangle 405 of the human body overlap to the area of the external rectangle 405 of the human body exceeds the threshold value. The object corresponding to is associated with the human body corresponding to the circumscribing rectangle 405. The object-corresponding portion 206 may associate one object with a plurality of human bodies. Further, the method of associating an object with a human body is not limited to the above method.

The locus management unit 207 acquires information about the object from the object detection unit 202, the object tracking unit 203, and the object correspondence unit 206, and manages each object as management information. An example of the management information 301 managed by the trajectory management unit 207 will be described with reference to FIG. As shown in FIG. 3, the locus management unit 207 of the present embodiment manages the object information 302 for each object ID. The object information 302 corresponding to one object ID includes a time stamp indicating the date and time when the object information was generated. Further, the object information 302 includes information 303 for each frame in which the object is detected. The information 303 includes a time stamp indicating the date and time when the information was generated, a coordinate position (Position) of the detected object, information indicating an circumscribing rectangle including the area of the detected object (Bounding box), and the size of the object. , Contains the attributes of the object. However, the information included in the information 303 is not limited to these, and the displays 105 and 107 may include the information. The management information 301 managed by the locus management unit 207 is used by the locus information determination unit 208.

The trajectory management unit 207 updates the object attribute (Attribute) according to the association result of the object correspondence unit 206. Further, the attribute (Attribute) of the past object may be updated according to the association result. Further, the attribute (Attribute) of the subsequent object may be set according to the association result. By performing such processing, the tracking results of objects having the same object ID can have the same attributes at any time.

The locus information determination unit 208 has a function as a passing object detection unit, and determines the passage of an object with respect to the detection line according to the parameters set by the parameter setting unit 205 and the management information managed by the locus management unit 207. Perform processing. The detection line can be set by the user, for example, by operating the user interface on the parameter setting screen of the display device 210. The parameter setting unit 205 of the present embodiment can set, for example, information for specifying a line segment set by the user in the locus information determination unit 208 as a parameter.

In the locus information determination unit 208, the movement vector from the circumscribing rectangle of the human body attribute object in the frame one frame before the frame of interest to the circumscribing rectangle of the human body attribute object in the frame of interest intersects the line segment for passing detection. Judge whether or not. The human body attribute object in the present embodiment is an object associated with the human body by the object corresponding unit 206. Further, the determination of whether or not they intersect corresponds to the determination of whether or not the human body attribute object has passed the line segment for passing detection. The determination result by the locus information determination unit 208 may be output to the outside (for example, the display device 210) via the external output unit 209. Further, when the external output unit 209 has a function of a display unit composed of a CRT, a liquid crystal screen, or the like, the determination result may be displayed by using the external output unit 209 instead of the display device 210.

Here, the locus information determination unit 208 in the present embodiment has described an example of detecting that the human body attribute object has passed a predetermined line segment, but the present invention is not limited to this example. For example, the locus information determination unit 208 can also detect that a human body attribute object has invaded the area when a predetermined area is set as a parameter. Further, the locus information determination unit 208 can also detect that an animal object has invaded instead of the human body attribute object. In addition, the locus information determination unit 208 can execute various detection processes using the event detection parameters and the locus information.

Next, the image recognition process according to the first embodiment will be described with reference to FIGS. 5 and 6.

FIG. 5 is a diagram for explaining the setting of the human body detection size. The parameter setting screen shown in FIG. 5 is displayed, for example, on the display device 210.

FIG. 5A is a diagram showing an example of a screen for setting the maximum size and the minimum size of human body detection.

In FIG. 5A, 500 is a setting screen for setting parameters for detecting a human body. The screen 500 shows that the road extends from the upper left of the screen to the lower right of the screen, and the human body 501 is shown in the upper left (far) and the human body 502 is shown in the lower right (near). The setting rectangle 503 is a UI (user interface) for setting the maximum size of human body detection. Similarly, the setting rectangle 504 is a UI for setting the minimum size of human body detection.

The human body detection unit 204 of the present embodiment detects the human body from the frame image by comparing the pattern image of the human body with the frame image. More specifically, the human body detection unit 204 scales the size of the frame image according to the sizes of the set rectangles 503 and 504, and compares the scaled frame image with the pattern image of the human body. Detects a person. For example, the human body detection unit 204 displays an image in which the size of the frame image is halved, an image in which the size is 1/3 times, and an image in which the size is 1/4 times, according to the sizes of the set rectangles 503 and 504. The human body is detected by generating and comparing each image with the pattern image of the human body.

In the above case, when the user performs a user operation to lower the zoom magnification, the human body detection unit 204 will detect the person detected before lowering the zoom magnification even after the zoom magnification is changed. , Control the scaling of the frame image. More specifically, the human body detection unit 204 generates, for example, an image in which the size of the frame image is 1/3 times larger, an image in which the size of the frame image is 1/4 times larger, and an image in which the size of the frame image is 1/6 times larger. The human body is detected by comparing the image with the pattern image of the human body.

The method for detecting the human body is not limited to the above method. For example, the pattern image of the human body can be scaled according to the sizes of the setting rectangles 503 and 504, and the human body can be detected by comparing with the frame image. is there.

In this way, if the human body detection process is performed so as to detect only the human body within the set range of the human body detection size, the processing speed or accuracy can be increased. The sizes of the setting rectangles 503 and 504 can be changed by a mouse operation such as dragging the operator on the sides and nodes of the setting rectangle. In the description of FIG. 5, an example of setting the maximum size and the minimum size of the human body detection has been described, but only the maximum size or the minimum size may be set.

FIG. 5B is an example of a screen display in which a part of the area of FIG. 5A is zoomed in. The zoom range 505 is a rectangle for representing the range on FIG. 5 (a) corresponding to FIG. 5 (b). Therefore, the result of zooming the zoom range 505 is the screen 510. In this figure, the zoom magnification is 2.5 times that of FIG. 5 (a).

FIG. 6 is a diagram showing a configuration example of a setting parameter set by the parameter setting unit 205. The control device 200 can display the parameter display screen shown in FIG. 6 on, for example, the display device 210.

FIG. 6A shows the set values of the setting rectangles 503 and 504.

In FIG. 6A, the maximum size (Max Size) of human body detection is set to the width height (900,900) pixels, and the minimum size (Min Size) is set to the width height (250,250) pixels. The resolution of the screen 500 is vertical and horizontal (1280, 1024). The zoom magnification at this time is set to 1x.

Here, consider a case where the setting value of FIG. 6A is applied as it is when zooming in as shown in FIG. 5B.

As shown in the screen 510, the human body imaged by the zoom-up operation becomes large. Therefore, after zooming in, it is possible that a human body larger than the setting rectangle 503 (Max Size in FIG. 6A) is imaged, but the maximum size (900,900) of human body detection in FIG. 6A remains unchanged. When applied, large human bodies may not be detected.

Further, an object that has not been subjected to human body detection processing because it is smaller than the minimum size for human body detection before zooming in may be subject to human body detection processing after zooming in. In such a case, the human body detection process may be performed unintentionally by the user, and the processing load may increase.

Problems such as those mentioned above can also occur when zooming out. That is, although the size of the human body in the frame is reduced by zooming out, if the parameter of the minimum size for detecting the human body does not change before and after zooming out, detection omission may occur. Also, if the maximum size of human body detection does not change before and after zooming out, it may take extra processing time because the detection processing is performed for the human body of a size that was not the detection target before zooming out. is there.

Therefore, the parameter control unit 212 of the present embodiment changes the parameters used for the image recognition process such as the minimum size and the maximum size of the human body detection according to the change of the zoom magnification. By doing so, suitable recognition processing can be performed even after the zoom magnification is changed. As described above, the human body detection unit 204 of the present embodiment detects the human body by comparing the pattern image of the human body with the plurality of scaled frame images. In this example, the magnification when scaling the frame image is changed according to the change in the zoom magnification. That is, the parameter control unit 212 changes the maximum size and the minimum size of the frame image used for human body detection according to the change in the zoom magnification.

Further, when the human body detection unit 204 scales the pattern image of the human body and detects the human body by comparing with the frame image, the magnification when scaling the pattern image of the human body changes according to the change in the zoom magnification. Will be done. That is, the parameter control unit 212 changes the maximum size and the minimum size of the pattern image used for detecting the human body according to the change in the zoom magnification.

FIG. 5C is an example of screen display when the setting value of FIG. 6A is changed according to the zoom magnification when the screen 500 is zoomed up. The parameter shown in FIG. 6A is changed to the parameter shown in FIG. 6B by the process described below, and the human body detection unit 204 executes the human body detection process based on the changed parameter.

Zooming from screen 500 to screen 520 in FIG. 5 changes the zoom magnification from 1x to 2.5x. The parameter control unit 212 of the present embodiment changes both the maximum size and the minimum size of the human body detection according to the change in the zoom magnification. As shown in FIG. 6 (b), the parameter control unit 212 sets the minimum size of the human body detection after the change of the zoom magnification to the minimum size (250, 250) of the human body detection in FIG. 6 (a). Change to 5 times the size (625, 625). The maximum size of human body detection should be 2.5 times as large as the minimum size, but since it exceeds the screen range here, the parameter control unit 212 of the present embodiment is vertically displayed on the screen 520. Change to a size (1024,1024) with the size as the upper limit.

The parameter control unit 212 of the present embodiment executes the parameter change process as described above each time it receives zoom magnification information from the zoom control unit 211. Further, each time the parameter control unit 212 changes the parameter, the UI (User Interface) is dynamically changed according to the change in the zoom magnification by notifying the parameter setting tool side of the changed parameter. ..

FIG. 5C shows the UI of the parameter setting tool after changing the maximum size and the minimum size of the human body detection according to the change of the zoom magnification. 522 in FIG. 5 (c) corresponds to the setting rectangle 503 in FIG. 5 (a), and 521 in FIG. 5 (c) corresponds to the setting rectangle 504 in FIG. 5 (a).

As described above, the parameter control unit 212 of the present embodiment zooms the maximum size and the minimum size when the zoom control for increasing the zoom magnification is performed after the maximum size and the minimum size of the human body detection are specified. Control is performed to make it larger than before control. Further, the parameter control unit 212 controls the maximum size and the minimum size to be smaller than those before the zoom control when the zoom control for lowering the zoom magnification is performed after the maximum size and the minimum size for human body detection are specified. ..

In addition, when the parameter is changed according to the change of the zoom magnification, the change is displayed, or the parameter change is notified on the screen before the parameter is changed, and the parameter is changed after obtaining the user's consent. You may do so.

Further, in the present embodiment, the parameters related to the maximum size and the minimum size of the human body detection are changed according to the zoom magnification, but when the changed size exceeds a predetermined threshold value, the human body detection process itself is stopped or stopped. It may be. Further, instead of stopping or stopping the human body detection process, a message such as a warning may be displayed on the display. By doing so, for example, there is an effect that an error of a parameter of recognition processing can be avoided.

Further, in the present embodiment, an example in which the human body detection size is set by a rectangle has been described, but other shapes such as a polygon and a circle may be used.

Further, in the present embodiment, the parameters changed by the parameter control unit 212 are set to the minimum size and the maximum size of the human body detection, but other parameters that depend on the captured image size and the position on the image may be used. For example, if the distance between the position of the object detected from the frame image and the position of the object specified from the movement vector related to the frame image is less than the threshold value, the object tracking unit 203 of the object and another frame image. It was explained that it is associated with an object. It is also possible to use this threshold value as a parameter that changes according to a change in the zoom magnification.

Further, for example, the parameters (line segments and regions for passing detection) used by the locus information determination unit 208 for event detection may be changed according to the change in the zoom magnification. More specifically, the parameter control unit 212 changes, for example, the position and length of the line segment used for the passage determination and the position and size of the area used for the intrusion determination according to the change in the zoom magnification. It is possible.

Further, for example, it may be a parameter that changes the object detection range according to the change in the zoom magnification. In this embodiment, the object detection range is described as the entire screen.

Further, for example, the object-corresponding portion 206 can be attached to an object when the ratio (superimposition ratio) of the area of the overlapping region where the extrinsic rectangle of the object and the outer rectangle of the human body overlap to the area of the extrinsic rectangle of the human body exceeds the threshold value. Explained to associate people. It is also possible to set the threshold value in this case as a parameter that changes according to the change in the zoom magnification. It should be noted that the superimposition ratio and the like are not the same magnification conversion according to the zoom magnification as in the human body detection size of the present embodiment because the areas are compared. The parameter control unit 212 of the present embodiment can change parameters that are not equal-magnification conversion by using a table in which the zoom magnification and the superposition rate are associated with each other.

Further, in the present embodiment, an example in which the parameters are relatively converted according to the change in the zoom magnification is described with the maximum size and the minimum size of the human body detection specified by using the user interface as reference values. That is, in the above description, for example, when the maximum size of human body detection (reference value) is set using the user interface and then the zoom magnification is doubled, the maximum size of human body detection is doubled of the reference value. An example has been explained, but it is not limited to this.

For example, an absolute range three-dimensional region in which a camera position and a position where a detection target object (human body in this case) can exist are specified, and an absolute size range that can be taken within the three-dimensional region of the detection target object. Can be obtained. Then, the parameter control unit 212 can specify appropriate parameters (here, the maximum size and the minimum size of the human body detection) for the zoom value. Further, the parameter control unit 212 can use the zoom value when the parameter is determined as described above as a reference value and change the recognition parameter according to the subsequent change in the zoom magnification.

Further, FIG. 5 is an angle of the camera such that the lower limit of the size of the person is finite. That is, in the present embodiment, an example has been described in which the upper and lower limits of the size of the human body projected on the captured image are the camera position, the depression angle, and the object to be detected. However, it is theoretically possible that the minimum size and the maximum size cannot be set depending on the camera position, the direction in which the camera is facing, and the type of the object to be detected, such as when the object is located at infinity. In such a case, if you do not change the object detection parameters according to the change in the zoom value (zoom magnification), or if you dare to set the minimum size, or change the camera installation location and direction, etc. Good. The control device 200 of the present embodiment can switch whether or not to change the parameters according to the change in the zoom magnification by a user operation.

Further, in the present embodiment, the zoom mechanism of the camera is an optical zoom, but a digital zoom may be used. The processing when the zoom mechanism of the camera is the digital zoom will be described below with reference to FIG.

It is assumed that the zoom range 505 on the screen 500 is displayed like the screen 510 due to the digital zoom. In that case, the parameter control unit 212 changes the setting rectangle 503 and the setting rectangle 504 according to the digital zoom magnification, as in the case of the optical zoom. For example, the parameter control unit 212 also changes the maximum size and the minimum size of person detection to 2 times when the zoom magnification is increased from 1 to 2 times by digital zoom. Then, the parameter control unit 212 notifies the display device 210 of the changed parameter to change the display of the user interface as shown in FIG. 5C. By linking the digital zoom and the user interface in this way, the user can recognize that the parameters have been changed according to the change in the zoom magnification. However, in the case of digital zoom, the image actually processed by the human body detection unit 204 may not be the image after digital zoom but the captured image before zooming. In that case, the parameters that accompany the change in zoom magnification should not be changed. That is, the parameter control unit 212 determines whether or not the image area to be processed for image recognition changes before and after the change of the zoom magnification, and if it is determined that the image area does not change, the zoom magnification is changed. The recognition processing parameters are not changed according to the change.

Hereinafter, the case where the image processed by the human body detection unit 204 is an image cut out after digital zooming will be described. When the maximum size of human body detection exceeds the clipped image range, the parameter control unit 212 may change the parameter so as to reduce the maximum size of human body detection to the clipped image range. For example, when the maximum size of human body detection before digital zoom is (1000,1000) and the size of the image cut out by digital zoom is (800,800), the parameter control unit 212 sets the maximum size to (800,800). You may. Alternatively, if the minimum size of the human body detection exceeds the clipped image range, the human body detection process itself may be stopped or stopped.

Next, the operation of the control device 200 according to the first embodiment will be described with reference to the flowchart of FIG. 7. The control device 200 of the present embodiment realizes the process of FIG. 7 by a CPU (not shown) reading a control program related to the process of FIG. 7 from the memory and executing the control program. Further, the control device 200 of the present embodiment is incorporated in the camera, and starts the process of FIG. 7 when the camera is activated. However, the control device 200 may be an independent device separate from the camera, or may be mounted on a PC, a mobile terminal, or the like that displays an image captured by the camera. A control unit (not shown) included in the control device 200 determines whether or not to continue the process of FIG. 7. For example, when the user instructs the end of the process, the control unit determines that the process of FIG. 7 is finished, and when there is no instruction from the user, the control unit determines that the process of FIG. 7 is continued. If it is determined that the process is to be continued (S701; YES), the process proceeds to S702. On the other hand, when it is determined that the processing is finished (S701; NO), the processing is finished.

In S702, the image acquisition unit 201 acquires the image data input to the control device 200. In S703, the object detection unit 202 performs the object detection process on the image acquired by the image acquisition unit 201. In S704, the object detection unit 202 determines whether or not the object has been detected in step S703. If it is determined that the object has been detected (S704; YES), the process proceeds to S705. On the other hand, when it is determined that the object is not detected (S704; NO), the process returns to S701.

In S705, the object tracking unit 203 performs the object tracking process. That is, the object tracking unit 203 performs a process of associating an object detected from a frame with an object detected from another frame according to a predetermined condition. By the tracking process, for example, when the same object exists over a plurality of frames, the objects are associated with each other.

In S706, the trajectory management unit 207 updates the trajectory information according to the result of the tracking process in S705. The update of the trajectory information corresponds to the addition of the information 303 shown in FIG.

In step S707, the human body detection unit 204 performs a human body detection process on the object detected by the object detection unit 202 and the area around the object using the parameters set by the parameter setting unit 205.

Here, the details of the human body detection process by the control device 200 of the present embodiment will be described with reference to the flowchart of FIG.

In S801, the parameter control unit 212 acquires the setting parameters (setting information such as the maximum size and the minimum size of the human body detection) set by the parameter setting unit 205. The setting parameters are not limited to the information on the maximum size and the minimum size of human body detection. That is, the parameter control unit 212 acquires the parameters for the image recognition process in S801. Further, the parameter control unit 212 acquires information regarding the current zoom magnification. The control device 200 of the present embodiment is incorporated in the camera, and the parameter control unit 212 acquires information on the zoom magnification from the storage unit in the camera, for example, zooming from the PC 104 connected to the camera. You may want to get information about the magnification.

In S802, the parameter control unit 212 determines whether or not the zoom magnification has been changed in the zoom control unit 211. That is, the parameter control unit 212 determines whether or not the information regarding the zoom magnification acquired last time and the information regarding the zoom magnification acquired this time are different. When a change in zoom magnification is detected (S802: YES), the process proceeds to S803. On the other hand, if it is determined that the zoom magnification has not been changed (S802; NO), the process proceeds to S804.

In S803, the parameter control unit 212 determines the parameter used in the human body detection process of S804 from the parameter acquired in S801 and the zoom magnification acquired in S801. For example, the parameter control unit 212 determines that the maximum size of person detection is also doubled when the zoom magnification is changed from 1x to 2x. However, it is not limited to this example. That is, the parameter control unit 212 changes the parameter acquired in S801 according to the change in the zoom magnification of the imaging unit.

Further, the parameter control unit 212 of the present embodiment transmits to the display device 210 a message indicating that the parameter for the recognition process has been changed according to the change in the zoom magnification and the parameter after the change. As a result, the display device displays a message on the parameter setting screen indicating that the parameters for the image recognition process have been changed, and a rectangle corresponding to the changed parameters (for example, the maximum size and the minimum size of the human body detection). ) Can be displayed.

In S804, the human body detection unit 204 performs the human body detection process using the parameters determined in S803 (parameters set by the user when the zoom magnification is not changed). When the human body detection process is completed in S804, the process proceeds to S708 in FIG.

In S708, the human body detection unit 204 determines whether or not the human body is detected in S707. If it is determined that the human body has been detected (S708; YES), the process proceeds to S709. On the other hand, if it is determined that the human body has not been detected (S708; NO), the process proceeds to S711.

In S709, the object-corresponding portion 206 performs an association process between the object detected in S703 and the human body detected in S707. As described above, the object-corresponding portion 206 associates the object with the human body according to the overlapping region of the extrinsic rectangle of the object and the extrinsic rectangle of the human body.

In S710, the locus management unit 207 updates the locus information based on the association processing result in S709. The update of the trajectory information corresponds to the addition of the information 303 shown in FIG. In S711, the locus information determination unit 208 performs the locus information determination process and determines whether or not the object has passed the detection line. The locus information determination unit 208 passes depending on whether or not the movement vector from the human body attribute object in the frame one frame before the focus frame to the human body attribute object in the focus frame intersects the line segment for passing detection. Judge the presence or absence of. The above-mentioned human body attribute object is an object that is determined by the object tracking unit 203 to be a corresponding object and is given the same object ID.

In S712, the external output unit 209 outputs the processing result related to the image recognition process to the outside, and returns to S712. The external output unit 209 outputs the position information of the circumscribing rectangle to the display device 210 so that the extrinsic rectangle of the detected object or human body is displayed on the display screen of the captured image, for example. Further, when the external output unit 209 detects, for example, the passage of a detection line or the intrusion into the detection area by a person, the detection result is displayed so that a message corresponding to the detection result is displayed on the display screen of the display device 210. Is output to the display device 210.

As described above, according to the present embodiment, the image captured by the photographing unit having a function of changing the zoom magnification by changing the parameters for the image recognition process according to the change of the zoom magnification. The recognition process can be performed more favorably. In the above description, an example in which the maximum size and the minimum size of the human body detection are used as parameters has been mainly described, but for example, the detection target may be a predetermined object other than the human body such as an automobile, a face, or an animal. When the detection target is an automobile, it is possible to set the maximum size and the minimum size of the automobile detection as parameters for the recognition process, and change the parameters according to the zoom magnification.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

Claims (43)

A setting screen including the image in which a first guide indicating the maximum size of an object detected from an image captured by the imaging means and a second guide indicating the minimum size of the object are superimposed. , A display control means for displaying a setting screen on which the settings of the maximum size and the minimum size can be changed are displayed on the display device.
It has a setting means for setting an area for determining the intrusion of the object in the video.
The display control means
When the object having a size between the maximum size and the minimum size invades the area, the display device is made to perform a predetermined display.
The display control means
An information processing device comprising displaying a predetermined message on the display device when the settings of the maximum size and the minimum size correspond to a predetermined condition. The display control means
The information according to claim 1, wherein as the predetermined display, information indicating that the object having a size between the maximum size and the minimum size has invaded the area is displayed on the display device. Processing equipment. The display control means
The invention according to claim 1 or 2, wherein as the predetermined display, a message indicating that the object having a size between the maximum size and the minimum size has invaded the area is displayed on the display device. Information processing equipment. A setting screen including the image in which a first guide indicating the maximum size of an object detected from an image captured by the imaging means and a second guide indicating the minimum size of the object are superimposed. , A display control means for displaying a setting screen on which the settings of the maximum size and the minimum size can be changed are displayed on the display device.
It has a setting means for setting a line for determining the passage of the object in the video.
The display control means
When the object having a size between the maximum size and the minimum size passes through the line, the display device is made to perform a predetermined display.
The display control means
An information processing device comprising displaying a predetermined message on the display device when the settings of the maximum size and the minimum size correspond to a predetermined condition. The display control means
The information according to claim 4, wherein as the predetermined display, information indicating that the object having a size between the maximum size and the minimum size has passed through the line is displayed on the display device. Processing equipment. The display control means
The fourth or fifth aspect of the present invention is characterized in that, as the predetermined display, a message indicating that the object having a size between the maximum size and the minimum size has passed through the line is displayed on the display device. Information processing equipment. Based on the operation received from the user while the setting screen including the image in which the first guide and the second guide are superimposed is displayed on the display device, the maximum size and the minimum size The information processing device according to any one of claims 1 to 6, wherein the setting can be changed. The information processing according to any one of claims 1 to 7, wherein the display control means causes the display device to display a first numerical value indicating the maximum size and a second numerical value indicating the minimum size. apparatus. The size of the first guide displayed on the setting screen is changed according to the change of the maximum size, and the size of the second guide displayed on the setting screen is changed according to the change of the minimum size. The information processing apparatus according to any one of claims 1 to 8, wherein the information processing device is characterized. The information processing apparatus according to any one of claims 1 to 9, wherein the first guide and the second guide are rectangular. The information processing apparatus according to any one of claims 1 to 10, further comprising a detecting means for detecting the object from an image captured by the imaging means. The information processing apparatus according to any one of claims 1 to 11, wherein the predetermined condition is a condition related to an error in the setting of the maximum size and the minimum size. The display control means
The information processing device according to any one of claims 1 to 12, wherein when the size indicated by the setting exceeds a predetermined value, the display device displays the predetermined message. The information processing device according to any one of claims 1 to 13, wherein the predetermined message is a message indicating a warning. A setting screen including the image in which the first guide indicating the maximum size of the object detected from the image captured by the imaging means and the second guide indicating the minimum size of the object are superimposed. The first display control step of displaying a setting screen on which the settings of the maximum size and the minimum size can be changed are displayed on the display device.
A setting step of setting an area for determining the intrusion of the object in the video, and
When the setting of the maximum size and the minimum size corresponds to a predetermined condition, a second display control step of displaying a predetermined message on the display device, and
An information processing method comprising a third display control step of causing the display device to execute a predetermined display when the object having a size between the maximum size and the minimum size invades the area. In the third display control step,
The information according to claim 15, wherein, as the predetermined display, information indicating that the object having a size between the maximum size and the minimum size has invaded the area is displayed on the display device. Processing method. In the third display control step,
15. The display according to claim 15 or 16, wherein as the predetermined display, a message indicating that the object having a size between the maximum size and the minimum size has invaded the area is displayed on the display device. Information processing method. A setting screen including the image in which the first guide indicating the maximum size of the object detected from the image captured by the imaging means and the second guide indicating the minimum size of the object are superimposed. The first display control step of displaying a setting screen on which the settings of the maximum size and the minimum size can be changed are displayed on the display device.
A setting process for setting a line for determining the passage of the object in the video, and
When the setting of the maximum size and the minimum size corresponds to a predetermined condition, a second display control step of displaying a predetermined message on the display device, and
An information processing method comprising a third display control step of causing the display device to execute a predetermined display when the object having a size between the maximum size and the minimum size passes through the line. In the third display control step,
The information according to claim 18, wherein, as the predetermined display, information indicating that the object having a size between the maximum size and the minimum size has passed through the line is displayed on the display device. Processing method. In the third display control step,
18. The display according to claim 18 or 19, wherein as the predetermined display, a message indicating that the object having a size between the maximum size and the minimum size has passed through the line is displayed on the display device. Information processing method. Based on the operation received from the user while the setting screen including the image in which the first guide and the second guide are superimposed is displayed on the display device, the maximum size and the minimum size The information processing method according to any one of claims 15 to 20, wherein the setting can be changed. The invention according to any one of claims 15 to 21, further comprising a fourth display control step of displaying the first numerical value indicating the maximum size and the second numerical value indicating the minimum size on the display device. Information processing method. The size of the first guide displayed on the setting screen is changed according to the change of the maximum size, and the size of the second guide displayed on the setting screen is changed according to the change of the minimum size. The information processing method according to any one of claims 15 to 22, wherein the information processing method is performed. The information processing method according to any one of claims 15 to 23, wherein the first guide and the second guide are rectangular. The information processing method according to any one of claims 15 to 24, further comprising a detection step of detecting the object from an image captured by the imaging means. The information processing method according to any one of claims 15 to 25, wherein the predetermined condition is a condition related to an error in the setting of the maximum size and the minimum size. In the second display control step ,
The information processing method according to any one of claims 15 to 26, wherein when the size indicated by the setting exceeds a predetermined value, the display device displays the predetermined message. The information processing method according to any one of claims 15 to 27, wherein the predetermined message is a message indicating a warning. Imaging means for capturing images and
A detection means for detecting an object from the video and
A setting screen including the image in which a first guide showing the maximum size of an object detected by the detection means and a second guide showing the minimum size of the object are superimposed, and the maximum size. And a display control means for displaying a setting screen on which the minimum size setting can be changed on the display device.
It has a setting means for setting an area for determining the intrusion of the object in the video.
The display control means
When the object having a size between the maximum size and the minimum size invades the area, the display device is made to perform a predetermined display.
The display control means
An information processing system characterized in that a predetermined message is displayed on the display device when the settings of the maximum size and the minimum size correspond to a predetermined condition. The display control means
29. The information according to claim 29, wherein, as the predetermined display, information indicating that the object having a size between the maximum size and the minimum size has invaded the area is displayed on the display device. Processing system. The display control means
29 or 30, wherein as the predetermined display, the display device displays a message indicating that the object having a size between the maximum size and the minimum size has invaded the area. Information processing system. Imaging means for capturing images and
A detection means for detecting an object from the video and
A setting screen including the image in which a first guide showing the maximum size of an object detected by the detection means and a second guide showing the minimum size of the object are superimposed, and the maximum size. And a display control means for displaying a setting screen on which the minimum size setting can be changed on the display device.
It has a setting means for setting a line for determining the passage of the object in the video.
The display control means
When the object having a size between the maximum size and the minimum size passes through the line, the display device is made to perform a predetermined display.
The display control means
An information processing system characterized in that when the settings of the maximum size and the minimum size correspond to a predetermined condition, a predetermined message is displayed on the display device. The display control means
32. The information according to claim 32, wherein, as the predetermined display, information indicating that the object having a size between the maximum size and the minimum size has passed through the line is displayed on the display device. Processing system. The display control means
32 or 33, wherein as the predetermined display, the display device displays a message indicating that the object having a size between the maximum size and the minimum size has passed the line. Information processing system. Based on the operation received from the user while the setting screen including the image in which the first guide and the second guide are superimposed is displayed on the display device, the maximum size and the minimum size The information processing system according to any one of claims 29 to 34, wherein the setting can be changed. The information processing according to any one of claims 29 to 35, wherein the display control means causes the display device to display a first numerical value indicating the maximum size and a second numerical value indicating the minimum size. system. The size of the first guide displayed on the setting screen is changed according to the change of the maximum size, and the size of the second guide displayed on the setting screen is changed according to the change of the minimum size. The information processing system according to any one of claims 29 to 36. The information processing system according to any one of claims 29 to 37, wherein the first guide and the second guide are rectangular. The information processing system according to any one of claims 29 to 38, further comprising a detecting means for detecting the object from an image captured by the imaging means. The information processing system according to any one of claims 29 to 39, wherein the predetermined condition is a condition related to an error in the setting of the maximum size and the minimum size. The display control means
The information processing system according to any one of claims 29 to 40, wherein when the size indicated by the setting exceeds a predetermined value, the display device displays the predetermined message. The information processing system according to any one of claims 29 to 41, wherein the predetermined message is a message indicating a warning. A program for causing a computer to function as each means of the information processing apparatus according to any one of claims 1 to 14.

Images