JP2019192155A - Image processing device, photographing device, image processing method, and program - Google Patents

Abstract

To provide an image processing device, a photographing device, an image processing method, and a program, capable of efficiently reducing a processing load for detecting a person on the basis of the passage amount of the person for each passage direction.SOLUTION: Detection processing for detecting an object from a photographed image is performed. The detected object is tracked to determine that the tracked object passes through a detection line and determine a passage direction and the number of objects determined to have passed the detection line is calculated for each passage direction. A detection area for executing the detection processing of the object in an image is set on the basis of the calculated result.SELECTED DRAWING: Figure 5

Description

  The present invention relates to an image processing technique for counting the number of people.

  For the purpose of utilization for marketing purposes, a technique is known in which a passage in a store or the like is photographed by a photographing device, and the number of people passing through the passage is counted for each passage direction from the photographed image.

  In Patent Document 1, the upper body of a person is detected by matching a predetermined matching pattern while scanning the entire image, and a human trajectory is generated based on the detection results of a plurality of frames and compared with a passing line. There is a statement that the number of people passing the line is counted.

JP 2009-211311 A

  However, even if the amount of passage of people in each passing direction is extremely biased, Patent Document 1 does not consider the tendency of the amount of passing in each passing direction, and does not consider people in the entire image area. Since the detection process is performed, the human detection load for the process of counting people is large.

  The present invention has been made in view of such a problem, and an object thereof is to efficiently reduce the processing load for detecting a person based on the passage amount of the person for each passing direction.

  In order to solve the above problems, an image processing apparatus of the present invention has the following configuration. That is, a detection unit that performs detection processing for detecting an object from an image, a tracking unit that tracks an object detected by the detection unit, and that an object tracked by the tracking unit has passed a detection line in the image; In the image, based on a result of the calculation unit, a determination unit that determines a passing direction, a calculation unit that calculates the number of objects that are determined to have passed through the detection line by the determination unit, for each passing direction Setting means for setting a detection region in which detection processing by the detection means is executed.

  According to the present invention, it is possible to efficiently reduce the processing load for detecting a person based on the passing amount of the person for each passing direction.

It is a schematic diagram which shows the structure of an image processing system. It is the schematic which shows the hardware constitutions of an image processing apparatus and an imaging device. It is a functional block diagram of an image processing apparatus and an imaging device. It is a schematic diagram for demonstrating the process which sets a detection area. It is a flowchart of the process which sets a passage count process and a detection area. It is an example of a data block diagram of locus information and passage information. It is a flowchart of the process which sets a detection area. It is a schematic diagram for demonstrating the process which sets a detection area. It is a functional block diagram of the imaging device and client device which have a part of function of an image processing device.

(Embodiment 1)
Embodiments according to the present invention will be described below with reference to the accompanying drawings. In addition, the structure shown in the following embodiment is only an example, and is not limited to the illustrated structure.

  FIG. 1 is a schematic diagram showing a system configuration according to the present embodiment. The image processing apparatus 100 is an apparatus that executes image processing to be described later. The image processing apparatus 100 is, for example, a personal computer in which a program for realizing an image processing function described later is installed.

  The display device 101 is connected to the image processing device 100 and includes a display that displays an image based on image data distributed from the imaging device 102, data output by image processing described later, and the like.

  The photographing apparatus 102 is an apparatus that captures an image, and is, for example, a network camera or a surveillance camera that can transmit image data of an image via a network. The recording device 103 records the image data transmitted from the imaging device 102.

  Further, the image processing apparatus 100, the imaging apparatus 102, and the recording apparatus 103 communicate with each other via the network 104. The network 104 includes a plurality of routers, switches, cables, and the like that satisfy a communication standard such as ETHERNET (registered trademark). In the present embodiment, any communication standard, scale, and configuration may be used as long as communication between the image processing apparatus 100, the imaging apparatus 102, and the recording apparatus 103 can be performed. For example, the network 104 may be configured by the Internet, a wired LAN (LOCAL AREA NETWORK), a wireless LAN (WIRELESS LAN), a WAN (WIDE AREA NETWORK), or the like.

  Further, in the configuration of the present embodiment shown in FIG. 1, image data (live video) of an image captured by the imaging device 102, image data (an image captured in the past) recorded by the recording device 103, and the like are processed. Transmitted to the device 100.

  Next, the hardware configuration of the image processing apparatus 100 and the photographing apparatus 102 will be described with reference to FIG. FIG. 2 is a schematic diagram illustrating a hardware configuration of the image processing apparatus 100 and the imaging apparatus 102.

  First, the hardware configuration of the image processing apparatus 100 will be described. A CPU (CENTRAL PROCESSING UNIT) 200 controls the image processing apparatus 100.

  The IF 201 communicates with the photographing apparatus 102 via the network 104 according to a protocol such as TCP / IP, HTTP, or ONVIF. The IF 201 receives image data of an image captured by the image capturing apparatus 102 and transmits various commands for controlling the image capturing apparatus 102.

  A RAM (RANDOM ACCESS MEMORY) 202 temporarily stores a computer program executed by the CPU 200. The RAM 202 temporarily stores data (commands and image data) acquired from the outside via the IF 201. The RAM 202 provides a work area used when the CPU 200 executes image processing to be described later. The RAM 202 functions as a frame memory or a buffer memory, for example.

  CPU 200 executes a computer program stored in RAM 202. In addition to the CPU, a processor such as a DSP (DIGITAL SIGNAL PROCESSOR) may be used.

  An HDD (HARD DISK DRIVE) 203 stores an operating system program and image data. Computer programs and data for executing the flowcharts of FIGS. 5 and 7 to be described later are stored in the HDD 203, and are appropriately loaded into the RAM 202 and executed by the CPU 200 under the control of the CPU 200. In addition to the HDD, other storage media such as a flash memory may be used.

  Note that the housing of the image processing apparatus 100 may be integrated with the display apparatus 101. A UI (USER INTERFACE) 204 is an input device such as a keyboard or a mouse.

  In the following description, an example in which the CPU 200 executes image processing will be described. However, at least a part of the processing of the CPU 200 may be performed by dedicated hardware. For example, processing for displaying GUI (GRAPHICAL USER INTERFACE) and image data on the display device 101 may be executed by a GPU (GRAPHICS PROCESSING UNIT). Further, the process of reading the program code from the HDD 203 and expanding it into the RAM 202 may be executed by a DMA (DIRECT MEMORY ACCESS) functioning as a transfer device.

  Next, the hardware configuration of the photographing apparatus 102 will be described. A CPU 210 is a central processing unit that performs overall control of the photographing apparatus 102.

  The IF 211 communicates with the image processing apparatus 100 via the network 104 according to a protocol such as TCP / IP, HTTP, or ONVIF. The IF 211 transmits image data of the image to the image processing apparatus 100 and receives various commands from the image processing apparatus 100 for controlling the photographing apparatus 102.

  The RAM 212 temporarily stores a computer program executed by the CPU 210. The RAM 212 temporarily stores data (commands, parameters relating to imaging conditions, etc.) acquired from the outside via the IF 211. The RAM 212 provides a work area used when the CPU 210 executes processing. The RAM 212 functions as, for example, a frame memory or a buffer memory.

  The ROM 213 stores a program for the CPU 210 to control the photographing apparatus 102 and the like. In addition to the ROM 213, a secondary storage device equivalent to the HDD 203 may be provided.

  The photographing device 214 is a photographing device such as a video camera or an infrared camera for photographing a moving image or a still image. Note that the housing of the photographing apparatus 102 and the housing of the photographing device 214 may not be integrated.

  Next, the image processing apparatus 100 according to the present embodiment will be described with reference to FIG. FIG. 3A is a functional block diagram of the image processing apparatus 100 according to the present embodiment. Each functional block shown in FIG. 3A may be realized by a software module or a hardware module. Further, the software module and the hardware module may be realized in cooperation. In the following description, each function of the image processing apparatus 100 is assumed to be executed by the CPU 200 of the image processing apparatus 100.

  The image processing apparatus 100 according to the present embodiment includes a communication unit 300, a detection unit 301, a tracking unit 302, a holding unit 303, a determination unit 304, a calculation unit 305, a setting unit 306, and an output control unit 307. , An operation receiving unit 308 and a storage unit 309.

  The communication unit 300 receives image data of an image captured by the image capturing device 102 via the network 104 and transmits various commands for controlling the image capturing device 102 to the image capturing device 102.

  The detection unit 301 performs detection processing for detecting a predetermined object (such as a person or a car) for a detection region in the image. The detection unit 301 according to the present embodiment performs a detection process for detecting a human body in a detection region of an image using a matching pattern (dictionary). The detection area is an area in the image that is a target for the detection unit 301 to perform a detection process, and the detection area is scanned with a matching pattern.

  In the case of detecting a specific object such as a human body from an image, improvement in detection accuracy can be expected by using different verification patterns depending on whether the human body is facing front or sideways. For example, a collation pattern for collating with an image of a human body facing the front (rear) and a collation pattern for collating with an image of a human body facing sideways are stored, and the installation state of the photographing apparatus 102 (direction of passage and photographing direction) Relationship) and user designation.

  The collation pattern may be prepared from another angle, such as from an oblique direction or an upward direction. Further, when detecting a human body, it is not necessary to create a dictionary of the features of the whole body, and a collation pattern may be prepared for a part of the human body such as the upper body, lower body, head, face, and feet. The verification pattern may be prepared for detecting a specific moving body such as a car or an animal in addition to the human body.

  The detection unit 301 only needs to have a function of detecting an object, which is a target that the user wants to detect, from an image, and may be a known moving object detection or skin color detection, and is not limited to pattern matching processing. . Further, in the following description, the detection unit 301 in the present embodiment will be described on the assumption that the detection processing target is a human body, but the detection processing target may be, for example, a human face, a car, an animal, or the like.

  The tracking unit 302 tracks the human body detected by the detection unit 301. In the present embodiment, the tracking unit 302, when the detection unit 301 detects the same human body as the human body detected by the detection unit 301 from the image of one or more frames before the target frame, from the image of the target frame, Associate human bodies with each other. That is, an object is tracked between images for a plurality of frames that are close in time.

  The tracking unit 302 assigns an ID to each human body to be tracked. For example, it is assumed that the tracking unit 302 assigns ID = 1 to the human body detected by the detection unit 301 from the image of one or more frames before the frame of interest. When the detection unit 301 detects this human body from the image of the frame of interest, the tracking unit 302 also assigns ID = 1 to this human body. Thus, when the same human body is detected over a plurality of frames of images, the same ID is assigned to each object. Note that the storage unit 309 stores, for each ID corresponding to the human body tracked by the tracking unit 302, trajectory information including the position of the human body in the image and the time when the image was captured. A more detailed description of the trajectory information will be described later with reference to FIG.

  As a method for determining that the tracking unit 302 is the same object over a plurality of frames of images, for example, using the movement vector of the detected object, the same if the predicted movement position of the object and the detected object position are within a certain distance. Suppose it is an object. Further, the tracking unit 302 may associate objects having high correlation between images of a plurality of frames using the color, shape, size (number of pixels) of the object, and the like. As described above, the tracking unit 302 is not limited to a specific method as long as it can perform processing for determining and tracking the same object over a plurality of frames of images.

  The holding unit 303 holds parameters for determining whether or not the human body tracked by the tracking unit 302 has passed the detection line, that is, information on a parameter that defines the detection line. For example, with respect to the image output to the display device 101 by the output control unit 307, the operation receiving unit 308 receives information on the position coordinates of two points on the image designated by the user. And the holding | maintenance part 303 hold | maintains the information of the position coordinate of the two points as a parameter which prescribes | regulates a detection line. In the present embodiment, (x, y) coordinates with the lower left end point of the image as the origin are determined as position coordinate information, and a line connecting the position coordinates of the two points is a detection line. The holding unit 303 may hold a parameter registered in advance, and a line defined by the parameter may be a detection line in the image.

  The determination unit 304 determines that the human body tracked by the tracking unit 302 has passed the detection line in the image and its passing direction. Note that the determination unit 304 in the present embodiment determines that the human body has passed the detection line and its passing direction based on the trajectory information for each human body stored in the storage unit 309 and the parameters that define the detection line. . Note that the storage unit 309 stores passage information including the time when it is determined by the determination unit 304 that it has passed, the ID of the human body that has passed, and the passing direction.

  The calculation unit 305 calculates the number of human bodies determined by the determination unit 304 as having passed the detection line for each passing direction. The storage unit 309 stores the number of human bodies for each passing direction calculated by the calculation unit 305.

  Based on the result of the calculation unit 305, the setting unit 306 sets a detection region that is a region where the detection unit 301 performs a detection process. Note that the setting unit 306 in the present embodiment defines the areas on the entrance side and the exit side in the passing direction in which the number of human bodies passing through the detection line is large when the number of human bodies in each passing direction satisfies a predetermined condition. At this time, the detection region is set so that the area of the detection region in the region on the exit side becomes small.

  The output control unit 307 outputs the image received by the communication unit 300, the result of image processing according to the present embodiment, and the like to the display device 101.

  The operation reception unit 308 receives information on an operation performed by the user via the UI 204. For example, the operation reception unit 308 receives information related to the position coordinates of the line specified in the image by the user.

  The storage unit 309 stores information related to the human body tracked by the tracking unit 302, information related to the human body determined to have passed by the determination unit 304, and the like.

  Next, the photographing apparatus 102 according to the present embodiment will be described with reference to FIG. FIG. 3B is a functional block diagram of the photographing apparatus 102. The photographing apparatus 102 includes a photographing unit 330, an image processing unit 331, a control unit 332, and a communication unit 333.

  The imaging unit 330 includes an image sensor such as a CCD (charge coupled device) sensor or a CMOS (complementary metal oxide semiconductor) sensor. Then, the photographing unit 330 photoelectrically converts the subject image formed through the lens of the photographing apparatus 102 to generate an electrical signal.

  The image processing unit 331 performs processing for converting the electrical signal photoelectrically converted in the photographing unit 330 into a digital signal, compression encoding processing, and the like, and generates image data.

  The control unit 332 controls the photographing apparatus 102 according to the command for controlling the photographing apparatus 102 received from the image processing apparatus 100. For example, the control unit 332 performs control to change the angle of view of the photographing apparatus 102.

  The communication unit 333 is an interface that performs communication with the image processing apparatus 100. For example, the communication unit 333 transmits the image data generated by the image processing unit 331 to the image processing apparatus 100. Further, the communication unit 333 receives a command for controlling the photographing apparatus 102 transmitted by the image processing apparatus 100.

  Next, image processing according to the present embodiment will be described with reference to FIGS. In the following description, the object that is the target of image processing according to the present embodiment is a human body. For example, the detection unit 301 will be described as performing detection processing for detecting a human body.

  FIG. 4 is a schematic diagram for explaining image processing according to the present embodiment. FIG. 4B is a schematic diagram showing a state where the human body shown in FIG. 4A is photographed after passing the detection line.

  In FIG. 4, an image 400 a and an image 400 b are images received by the communication unit 300. The image 400b is an image taken after a predetermined time has elapsed from the time when the image 400a was taken by the photographing apparatus 102. The region 401a and the region 401b are detection regions that are targets for the detection unit 301 to perform human body detection processing. The region 401a is the entire region in the image 400, and is a preset detection region. An area 401 b is a partial area in the image 400. The area 401c is an area where the human body detection process is not performed by the detection unit 301.

  A line 403 is a detection line for detecting a human body passing through the image. Note that the entire area indicated by the image received by the communication unit 300, such as the image 400a and the image 400b, is described as being divided into the area A and the area B on the basis of the line 403 for convenience. A frame 405 is a frame indicating the position and size of the human body 404 detected by the detection unit 301.

  For the sake of simplicity, the passing direction when the human body passes through the line 403 from the region A to the region B with the region A as a reference is referred to as “IN direction”. On the other hand, the passing direction when the human body passes through the line 403 from the region B to the region A is referred to as “OUT direction”. An arrow 402 is an arrow indicating the IN direction.

  Next, image processing in the present embodiment will be described with reference to a flowchart shown in FIG. FIG. 5A is a flowchart showing the flow of the pass count process according to the present embodiment. FIG. 5B is a flowchart showing the flow of detection area setting processing according to the present embodiment.

  In the present embodiment, the processing of the flowchart shown in FIG. 5 is executed by each functional block of FIG. 3 realized by the CPU 200 executing the computer program stored in the HDD 203 of the image processing apparatus 100. explain. However, a part of the processing of the flowchart shown in FIG. 5 may be executed by dedicated hardware.

  First, with reference to FIG. 5A, a flowchart of the passage count process according to the present embodiment will be described. In step S <b> 501, the communication unit 300 receives image data via the network 104.

  Next, in step S502, the detection unit 301 performs a detection process for detecting a human body with respect to a detection region in the image. It should be noted that in the initial stage before the processing described later of the setting unit 306 is performed, the detection unit 301 performs the detection process on the region 401a that is a detection region set in advance for the entire image. In FIG. 4A, the detection unit 301 detects a human body 404. Note that the output control unit 307 may display a frame on the display device 101 so that the detected position and size of the human body 404 can be superimposed on the image so as to circumscribe the human body 404.

  Next, in step S503, the tracking unit 302 tracks the human body detected by the detection unit 301. In FIG. 4A, the tracking unit 302 tracks the human body 404 and further assigns ID = 1 to the human body 404. Here, with reference to FIG. 6A, trajectory information, which is information on the trajectory of the human body tracked by the tracking unit 302, will be described.

  FIG. 6A is an example of a data configuration diagram showing the trajectory information of the human body. The data structure indicating the trajectory information in FIG. 6A includes the time detected by the detection unit 301 as the trajectory information of the human body and the position of the coordinates in the detected image for each ID given by the tracking unit 302. It is included. In the example of the data configuration shown in FIG. 6A, for the human body given ID = 1 by the tracking unit 302, the time detected by the detection unit 301 and the position of the human body in the image at that time are as follows. It is shown. Such trajectory information exists for each ID tracked by the tracking unit 302, that is, for each human body, and the trajectory information is stored in the storage unit 309. The position of the human body will be described as the coordinates of the center point of a frame (for example, the frame 405) indicating the position of the human body in the image.

  Next, in step S504, the determination unit 304 determines that the human body tracked by the tracking unit 302 has passed the detection line set in the image defined by the parameter and its passing direction. In FIG. 4, when the human body 404 moves from the region A to the region B in the process from the time when the image 400a is captured to the time when the image 400b is captured, the determination unit 304 causes the human body 404 to move the detection line in the IN direction. It is determined that it has passed. The storage unit 309 stores passage information that is information including the time when the human body passes the detection line by the determination unit 304, the ID assigned to the human body, and the passage direction. FIG. 6B is a data configuration diagram illustrating an example of passage information. The data configuration shown in FIG. 6B includes the time when it is determined by the determination unit 304 that it has passed, the ID assigned to the passed human body, and the passing direction of the passed human body.

  Next, processing for setting a detection area will be described with reference to FIG. In step S505, the calculation unit 305 calculates the number of human bodies determined by the determination unit 304 to have passed the detection line for each passing direction. That is, the calculation unit 305 calculates the passing amount of the human body for each passing direction. Note that the calculation unit 305 in the present embodiment calculates the passing amount of the human body for each passing direction based on the passing information stored in the storage unit 309.

  The storage unit 309 stores the number of human bodies for each passing direction calculated by the calculation unit 305. Note that the output control unit 307 may superimpose the number of human bodies for each passing direction calculated by the calculation unit 305 on the image and output the image to the display device 101.

  Next, in step S506, it is determined whether or not a detection area setting condition is satisfied. In the present embodiment, the condition is whether or not the number of people who have passed in one direction satisfies a condition equal to or greater than a predetermined value and the number of people who have passed in another direction satisfies a predetermined value or less in the latest certain period. In the following description, the condition whether or not the number of people who passed in one direction in the last 5 seconds is 10 or more and the number of people who passed in the other direction in the last 5 seconds is 1 or less is determined. The detection area setting condition is not limited to this. For example, in the latest fixed period, the condition of whether the difference in the number of passing human bodies in each passing direction is a predetermined value or more may be set as the detection region setting condition, and it can be determined whether or not the passing amount is biased. Any condition is acceptable. The detection area setting condition may be a condition registered in advance or may be set as appropriate by the user.

  If No in step S506, the process proceeds to step S507. In step S507, the setting unit 306 sets the first area as a detection area. In the present embodiment, the first area indicates the entire image area, and in the case of FIG. 4, the first area is the area 401a.

  If Yes in step S506, the process proceeds to step S508. In step S508, the setting unit 306 sets the second area as a detection area. The second region is a region set so as to reduce the area of the detection region in the region on the exit side when the region on the entrance side and the exit side in the passing direction in which the number of people who have passed is defined. For example, in FIG. 4, it is assumed that the number of people who passed the IN direction in the last 5 seconds is 10 or more and the number of people who passed the OUT direction in the last 5 seconds is 1 or less. At this time, the setting unit 306 sets the detection region so as to reduce the area of the detection region of the region B on the exit side in the IN direction, which is the direction in which the number of people who have passed is large. At this time, the setting unit 306 of the present embodiment targets the region B as a detection region in a direction orthogonal to the extension direction of the detection line, from the detection line to a distance 406 of a predetermined length, and sets the distance 406 from the detection line. The area of the detection region in the region separated by more than the length is reduced. In this case, the setting unit 306 sets an area 401b obtained by removing the area 401c from the area 401a as a detection area. Information on the size and position of the detection area set by the setting unit 306 is stored in the storage unit 309.

  Thus, when there is a deviation in the passage amount for each passage direction in the latest fixed period, it can be estimated that the deviation in the passage amount continues. Therefore, the size of the detection area is changed based on the passing direction in which the passing amount is biased. Specifically, when the area on the entrance side and the exit side in the passing direction with a large amount of passage is defined, it is determined that the detection line has already been passed by reducing the area for performing the detection process in the area on the exit side. It is possible to reduce the load of performing detection processing on a person who has been injured.

  Next, in step S509, the output control unit 307 superimposes the position of the detection region set by the setting unit 306 in step S507 or step S508 on the image and outputs the image to the display device 101. By doing so, the user can easily grasp the position of the detection region that is the target of the human body detection process.

  In step S508, the setting unit 306 may change the area of the region 401c according to the number of human bodies that pass in each passing direction. For example, the number of people who passed in one direction in the last 5 seconds is 10 or more, and the number of people who passed in the other direction in the last 5 seconds is 1 or less (S506). The larger the number of people, the larger the area 401c may be.

  Note that the setting unit 306 in the present embodiment changes the size of the detection region based on the amount of passage through the detection line, but is not limited thereto. The setting unit 306 may change the frequency with which the detection processing is performed by the detection unit 301 based on the passage amount passing through the detection line.

  For example, it is assumed that the detection unit 301 performs detection processing for each frame in the image received by the communication unit 300 in an initial stage, that is, at a frequency of once per frame. At this time, in step S507, that is, when the deviation of the passing amount passing through the detection line is small, the setting unit 306 sets the detection processing for each frame for the regions A and B.

  In step S508, that is, when the amount of passage through the detection line is uneven, when the entrance side and exit side regions in the passage direction with a large number of people passing are defined, the setting unit 306 In the area, the detection process is set to be performed once every five frames. For example, in FIG. 4, it is assumed that the number of people who passed the IN direction in the last 5 seconds is 10 or more and the number of people who passed the OUT direction in the last 5 seconds is 1 or less. At this time, the setting unit 306 sets the detection processing frequency for each frame in the region A, and the detection processing frequency by the detection unit 301 in the region B on the exit side in the IN direction, which is the direction in which many people have passed. Is set to once every 5 frames.

  In step S508, when the areas on the entrance side and the exit side in the passing direction with a large number of people passing are defined, the setting unit 306 performs detection processing based on the distance from the detection line in the area on the exit side. The frequency may be changed. For example, in FIG. 4, it is assumed that the number of people who passed the IN direction in the last 5 seconds is 10 or more and the number of people who passed the OUT direction in the last 5 seconds is 1 or less. At this time, the setting unit 306 sets the frequency of detection processing for each frame in the area A which is the area on the entrance side in the IN direction. Then, the setting unit 306 sets the frequency of detection processing for each frame from the detection line to the distance 406 in the direction orthogonal to the detection line for the area B that is the area on the exit side in the IN direction. In a more distant area, once every 5 frames is set as the frequency of detection processing.

  As described above, the setting unit 306 may change the frequency of the detection process based on the passage amount passing through the detection line. Specifically, the setting unit 306 reduces the frequency with which the detection processing is performed by the detection unit 301 in the region on the exit side when the region on the entrance side and the exit side in the passage direction in which the passage amount is large is defined. It may be. By doing so, it is possible to suppress a decrease in detection accuracy by the detection unit 301 for a person who passes through a passing direction with a small passing amount.

  In the present embodiment, the passage count process shown in FIG. 5A and the detection area setting process shown in FIG. 5B may be executed in parallel in time or executed separately in time. You may make it do.

  As described above, according to the present embodiment, it is possible to efficiently reduce the processing load for detecting a person for the process of counting people based on the passing amount of the person for each passing direction.

(Embodiment 2)
In the first embodiment, the process of changing the size of the detection region based on the passage amount of the person passing the detection line has been described, but the present invention is not limited to this. For example, the position of the rectangular area that the detection unit 301 cuts out from the image as the target of the detection process may be changed based on the passage amount of the person who passes the detection line. Image processing in this case will be described with reference to FIGS. 5 (a), 7 and 8. FIG. In the following description, components and processes that are the same as or equivalent to those in the first embodiment are denoted by the same reference numerals, and redundant descriptions are omitted as appropriate.

  Image processing according to the second embodiment will be described with reference to FIG. 5A, FIG. 7, and FIG. 8. FIG. 7 is a flowchart illustrating a flow of image processing according to the second embodiment. FIG. 8 is a diagram for explaining image processing according to the second embodiment.

  In step S <b> 501 in the passage count process illustrated in FIG. 5A, the communication unit 300 receives image data via the network 104.

  In step S502, the detection unit 301 performs a detection process for detecting a human body with respect to a detection region in the image. Note that the detection unit 301 according to the second embodiment sequentially cuts out a rectangular region having a predetermined size while scanning the image in the horizontal and vertical directions. And the detection part 301 performs the detection process of a human body using the collation pattern (dictionary) targeting the cut-out rectangular area. That is, the detection unit 301 performs detection processing for detecting a human body using the cut-out rectangular area as a detection area.

  In FIG. 8, the detection unit 301 scans a region 801 that is a rectangular region in the image in the horizontal and vertical directions, and cuts out the rectangular region. And the detection part 301 performs the detection process which detects a human body by making the cut-out rectangular area into a detection area.

  Since the processing in step S503 and step S504 is the same as that in the first embodiment, description thereof is omitted.

  Next, with reference to FIG. 7, a flowchart of detection area setting processing according to the present embodiment will be described. Note that the processing of the flowchart shown in FIG. 7 will be described as being executed by each functional block of FIG. 3 realized by the CPU 200 executing a computer program stored in the HDD 203 of the image processing apparatus 100. However, a part of the processing of the flowchart shown in FIG. 7 may be executed by dedicated hardware.

  In FIG. 7, the processes in S505 and S506 are the same as those in the first embodiment, and a description thereof will be omitted.

  If No in step S506, the process proceeds to step S707. In step S707, the setting unit 306 in the second embodiment sets the scanning interval in the horizontal direction of the rectangular region in the entire image region to the first interval. In the present embodiment, the first interval is assumed to be the length of the horizontal side in one rectangular area. In the case of FIG. 8, the first interval is the distance 802a. That is, through the processing in step S707, the detection unit 301 scans the rectangular area 800 in the horizontal direction according to the distance 802a that is the first interval, and cuts out the rectangular area. And the detection part 301 performs the detection process which detects a human body by making the cut-out rectangular area into a detection area. That is, in this case, the detection unit 301 performs detection processing using the entire image area as a detection area.

  If Yes in step S506, the process proceeds to step S708. In step S708, when the setting unit 306 in the second embodiment defines areas on the entrance side and the exit side in the passage direction in which the number of people who have passed is large, the horizontal scanning interval of the rectangular area in the area on the exit side. Is set to the second interval. In the case of this embodiment, the second interval is a length larger than the first interval by a predetermined value. For example, in FIG. 8, it is assumed that the number of people who passed the IN direction in the last 5 seconds is 10 or more and the number of people who passed the OUT direction in the last 5 seconds is 1 or less. At this time, as shown in FIG. 8B, the setting unit 306 sets the horizontal scanning interval to a second interval 802b in the region B on the exit side in the IN direction where a large number of people have passed. To do. Note that the length of the second distance 802b may be set in advance or may be set as appropriate by the user. As described above, the setting unit 306 according to the present embodiment is configured so that when the number of human bodies in each passing direction satisfies a predetermined condition, there are a large number of objects passing through the detection line on the entrance side and the exit side in the passing direction. Is defined, the interval for scanning the rectangular region in the horizontal direction is changed in the region on the exit side.

  If there is a bias in the passage amount of each human body in the latest fixed period, it can be estimated that the bias in the passage amount continues. Therefore, the interval for scanning the rectangular area is changed based on the passing direction in which the passing amount is biased. In a region where there is a person who has passed the detection line in the passing direction with a large amount of passage, the detection processing is performed on a person who has already been determined to pass by increasing the interval for scanning the rectangular region cut out by the detection unit 301. It is possible to reduce the load.

  Next, in step S509, the output control unit 307 generates an image of a detection region that is a target of detection processing of the detection unit 301 based on the interval for scanning the rectangular region set by the setting unit 306 in step S707 or step S708. And output to the display device 101. By doing so, the user can easily grasp the position of the detection region that is the target of the human body detection process.

  In step S708, the setting unit 306 may change the length of the second interval according to the number of human bodies that pass in each passing direction. For example, after satisfying the condition (S506) that the number of people passing in one direction in the last 5 seconds is 10 or more and the number of people passing in the other direction in the last 5 seconds is 1 or less (S506) The larger the number of people who have been, the longer the second distance 802b may be.

  In step S708, the setting unit 306 in the present embodiment increases the interval for scanning the rectangular area in the exit-side area in the passing direction where the number of people who have passed is large. For example, the following processing is performed. Also good. That is, in step S708, the setting unit 306 sets the first interval from the detection line to a predetermined length 803 in the direction orthogonal to the extension direction of the detection line, as shown in FIG. In the area away from the detection line from the distance 803, the second interval may be used. Note that the length of the distance 803 may be a pre-registered setting or may be set as appropriate by the user.

  In the present embodiment, the setting unit 306 changes the interval for scanning the rectangular area 801 in the horizontal direction in the image based on the amount of passage through the detection line, but the rectangular area 801 is changed in the vertical direction. The scanning interval may be changed. Further, the setting unit 306 may change the interval at which the rectangular area 801 is scanned in the horizontal direction and the vertical direction based on the amount of passage through the detection line.

  In the present embodiment, the passage count process shown in FIG. 5A and the detection area setting process shown in FIG. 7 may be executed in parallel in time, or may be executed separately in time. It may be.

  As described above, according to the present embodiment, it is possible to efficiently reduce the processing load for detecting a person for the process of counting people based on the passing amount of the person for each passing direction.

(Embodiment 3)
Although the image processing according to the above-described embodiment has been described as being performed by the image processing apparatus 100, the present invention is not limited to this. The imaging apparatus 102 may have one or more functions of the image processing apparatus 100 according to the first embodiment or the second embodiment. That is, the image processing according to the first or second embodiment is performed by the CPU 200 of the image processing apparatus 100, but may be realized by the cooperation of the CPU 210 of the photographing apparatus 102 and the CPU of the client apparatus.

  For example, even if the imaging apparatus 102 has the detection unit 301, the tracking unit 302, the holding unit 303, the determination unit 304, the calculation unit 305, the setting unit 306, and the storage unit 309 illustrated in FIG. Good. At this time, the output control unit 307 and the operation reception unit 308 are included in the client device 900. The client apparatus 900 is a personal computer having a hardware configuration equivalent to that of the image processing apparatus 100. This case will be described in more detail with reference to FIG. In addition, the same code | symbol shall be attached | subjected to the same or equivalent component and process as FIGS. 1-8, and the overlapping description is abbreviate | omitted suitably.

  FIG. 9 is an example of a functional block diagram of the image capturing apparatus 102 and the client apparatus 900 having some functions of the image processing apparatus 100. The imaging apparatus 102 and the client apparatus 900 communicate with each other via the network 104.

  The imaging unit 330 includes an image sensor such as a CCD sensor or a CMOS sensor, and generates an electrical signal by photoelectrically converting a subject image formed through the lens of the imaging device 102.

  The image processing unit 331 performs processing for converting the electrical signal generated by the photographing unit 330 into a digital signal, compression encoding processing, and the like, and generates image data.

  The image data generated by the image processing unit 331 is sent to the output control unit 307 via the communication unit 333 and the communication unit 901, and is output to the display device 101 by the output control unit 307. Note that the communication unit 901 in the client device 900 receives image data sent from the image capturing device 102 and transmits various commands for controlling the image capturing device 102.

  An operation reception unit 308 in the client device 900 receives an operation by a user via an input device such as a mouse or a keyboard. Note that the information received by the operation receiving unit 308 is sent to the photographing apparatus 102 via the communication unit 901 and the communication unit 333. For example, the operation reception unit 308 receives information regarding the position of the detection line set by the user with respect to the image output to the display device 101 by the output control unit 307. Information regarding the position of the detection region received by the operation receiving unit 300 is sent to the holding unit 303 of the imaging apparatus 102 via the communication unit 901 and the communication unit 333.

  The functions of the detection unit 301, the tracking unit 302, the holding unit 303, the determination unit 304, the calculation unit 305, the setting unit 306, and the storage unit 309 are the same as those described in the first or second embodiment. Omitted.

  Information on the position of the detection area set by the setting unit 306 is sent to the output control unit 307 of the client device 900 via the communication unit 333 and the communication unit 901. The output control unit 307 then superimposes the detection area set by the setting unit 306 on the image and outputs the image to the display device 101.

  As described above, the imaging apparatus 102 may have one or more functions of the image processing apparatus 100 according to the first embodiment or the second embodiment.

(Other embodiments)
The present invention can also be realized by a process in which one or more processors read and execute a program that realizes one or more functions of the above-described embodiment. The program may be supplied to a system or apparatus having a processor via a network or a storage medium. The present invention can also be realized by a circuit (for example, ASIC) that realizes one or more functions of the above-described embodiments.

301 Detection Unit 302 Tracking Unit 304 Determination Unit 305 Calculation Unit 306 Setting Unit

Claims (8)

Detection means for performing detection processing for detecting an object from an image;
Tracking means for tracking the object detected by the detection means;
A determination means for determining that an object tracked by the tracking means has passed a detection line in the image and a passing direction;
Calculating means for calculating the number of objects determined to have passed the detection line by the determining means for each passing direction;
An image processing apparatus comprising: a setting unit configured to set a detection region in the image based on a result of the calculation unit.   The setting means, when the number of objects for each passing direction calculated by the calculating means satisfies a predetermined condition, in the region on the exit side in the passing direction where the number of objects passing through the detection line is larger, The image processing apparatus according to claim 1, wherein the detection area is set so that an area of the detection area becomes small.   The setting means determines the number of objects passing through the detection line when the number of people who have passed in one direction in a certain period satisfies a condition equal to or greater than a predetermined value and the number of people who have passed in another direction satisfies a predetermined value. 3. The image processing apparatus according to claim 1, wherein the detection area is set so that an area of the detection area is small in a region on the exit side in a passing direction with more. The detection means is a detection means for detecting an object by sequentially cutting out a rectangular area of a predetermined size while scanning in the horizontal and vertical directions in the image, and using the cut-out rectangular area as a detection area,
The setting means, when the number of objects for each passing direction calculated by the calculating means satisfies a predetermined condition, the number of objects passing through the detection line is larger on the entrance side and the exit side in the passing direction. The image processing apparatus according to claim 1, wherein when the area is defined, an interval for scanning the rectangular area in the area on the exit side is changed. The detection means is a detection means for detecting an object by sequentially cutting out a rectangular area of a predetermined size while scanning in the horizontal and vertical directions in the image, and using the cut-out rectangular area as a detection area,
The setting means is configured such that the number of objects passing through the detection line when the number of people who have passed in one direction in a certain period satisfies a condition equal to or larger than a predetermined value and the number of people who have passed in another direction satisfies a predetermined value or less. 5. The image processing apparatus according to claim 1, wherein when an entrance-side area and an exit-side area in a passing direction with a larger number of lines are defined, an interval for scanning a rectangular area in the exit-side area is changed. . Photographing means for photographing an image;
Detection means for performing detection processing for detecting an object from an image photographed by the photographing means;
Tracking means for tracking the object detected by the detection means;
A determination means for determining that an object tracked by the tracking means has passed a detection line in the image and a passing direction;
Calculating means for calculating the number of objects determined to have passed the detection line by the determining means for each passing direction;
An imaging apparatus, comprising: a setting unit that sets a detection region in the image based on a result of the calculation unit for executing detection processing by the detection unit. A detection step for performing detection processing for detecting an object from an image;
A tracking step of tracking the object detected by the detection step;
A determination step of determining that an object to be tracked by the tracking step has passed a detection line in the image and a passing direction;
A calculation step of calculating the number of objects determined to have passed through the detection line by the determination step for each of the passing directions;
An image processing method comprising: a setting step for setting a detection region in the image based on a result of the calculation step, in which detection processing by the detection step is executed. Detection means for performing detection processing for detecting an object from an image;
Tracking means for tracking the object detected by the detection means;
A determination means for determining that an object tracked by the tracking means has passed a detection line in the image and a passing direction;
Calculating means for calculating the number of objects determined to have passed the detection line by the determining means for each passing direction;
A program for causing a computer to function as a setting unit that sets a detection region for executing detection processing by the detection unit in the image based on a result of the calculation unit.

Images