WO2016139868A1

WO2016139868A1 - Image analysis device, image analysis method, and image analysis program

Info

Publication number: WO2016139868A1
Application number: PCT/JP2015/085600
Authority: WO
Inventors: 靖和田中; 安川　徹; 伸生中嶋
Original assignee: ノ－リツプレシジョン株式会社
Priority date: 2015-03-04
Filing date: 2015-12-21
Publication date: 2016-09-09
Also published as: JP6638723B2; JPWO2016139868A1

Abstract

The purpose of the present invention is to provide a technology which allows accurate detection of moving objectsusing background subtraction. An image analysis device according to an aspect of the present invention comprises: an image acquisition unit which acquires a photographic image which is photographed by a photography device; a background subtraction computation unit which, by computing the difference between a background image and the photographic image on the basis of background subtraction, extracts a foreground region of the photographic image; a moving object detection unit which detects from the extracted foreground region, among objects appearing in the extracted foreground region, moving objects which move in the field of view of the photographic image; and a background update unit which determines whether the number of detected moving objects matches the number of objects appearing in the foreground region, and if it is determined that the number of detected moving objects does not match the number of objects appearing in the foreground region, updates the background image, using the acquired photographic image, for the region excluding the region in which the moving objects appear.

Description

Image analysis apparatus, image analysis method, and image analysis program

The present invention relates to an image analysis apparatus, an image analysis method, and an image analysis program.

The background subtraction method is generally well known as a method for detecting a moving object in a photographed image photographed by a photographing device. The background difference method is a method of extracting a region (foreground region) that is different from the background image by calculating the difference between the background image acquired in advance and the captured image (input image). When there is a moving object in the captured image, the pixel value of the area where the moving object is captured changes from the background image. Therefore, according to the background subtraction method, the area where the moving object is captured can be extracted as the foreground area, and the presence of the moving object can be detected.

In recent years, detection of moving objects by the background subtraction method has been used in various fields. For example, Patent Document 1 proposes a method of detecting an area in which a person to be watched is captured using a background difference method. Specifically, each estimation condition is set on the assumption that the foreground region extracted by the background subtraction method is related to the behavior of the person being watched over, and by determining whether or not each of these estimation conditions is satisfied A method for estimating the behavior of the watching target person has been proposed.

JP 2014-236896 A

However, the present inventors have found that the following problems occur when a moving object is detected based on a general background subtraction method. That is, it is assumed that a moving object such as a human moves a stationary object such as furniture that exists within the angle of view of the photographing apparatus. For example, this applies to a case where a target person appearing in a photographed image is sitting with a chair that is within the angle of view of the photographing apparatus.

In such a case, after calculating the difference between the captured image acquired by the imaging apparatus and the background image after the moving object moves the stationary object, the area where the moving stationary object is captured in addition to the area where the moving object is captured It will be detected as a foreground area. Therefore, as a result of the moving object having some effect on the background within the angle of view of the imaging device, such as moving a stationary object that is a part of the background, the general background subtraction method cannot properly detect the moving object. The present inventors have found that this problem arises.

In one aspect, the present invention has been made in consideration of such points, and an object of the present invention is to provide a technique that can appropriately detect a moving object in the background subtraction method.

The present invention adopts the following configuration in order to solve the above-described problems.

That is, an image analysis device according to one aspect of the present invention includes an image acquisition unit that acquires a captured image captured by a capturing device, a background image set as a background of the captured image based on a background difference method, and the By calculating a difference from the acquired captured image, a background difference calculation unit that extracts a foreground region of the acquired captured image, and among the target objects that appear in the extracted foreground region, A moving object detector that detects a moving object that moves from the foreground area, and determines whether or not the number of detected moving objects matches the number of target objects that appear in the foreground area. When it is determined that the number of moving objects does not match the number of target objects appearing in the foreground area, the background image is updated using the acquired photographed image for the target area excluding the area where the moving object appears. It includes a scene update section.

The image analysis apparatus according to the above configuration extracts the foreground area of the captured image based on the background difference method, and detects a moving object from the extracted foreground area. Then, the image processing apparatus determines whether or not the number of detected moving objects matches the number of target objects appearing in the foreground region, and the number of detected moving objects matches the number of target objects appearing in the foreground region. If it is determined not to, the background image is updated using the acquired captured image for the target region excluding the region where the moving object is captured.

Here, when the number of detected moving objects does not match the number of target objects appearing in the foreground area, in other words, the moving object has some effect on the background within the angle of view of the imaging device, and changes to the background. Is the case. In such a case, the image analysis apparatus according to the above configuration uses a background image to be used for the background subtraction method in a captured image obtained after a change in the background for a target area including a background area in which a change has occurred. Update.

That is, according to the above configuration, when the background changes, the background image can be updated for the region. Therefore, in the background subtraction method, it is possible to prevent the background area where the change has occurred from being extracted as the foreground area. Therefore, according to the above configuration, it is possible to appropriately detect the moving object in the background difference method.

Note that the moving object may be an object that moves within the angle of view of the photographing apparatus, and is, for example, a living organism such as a human being. In addition to the moving object, the target object shown in the foreground area may include a stationary object moved by the moving object, a coating applied by the moving object so that a change occurs in the background, and the like. The stationary object is, for example, furniture. Moreover, the coated material is, for example, paint. The target object shown in the foreground region can include any target object that can be extracted as a difference by the background difference method. Furthermore, the target object that appears in the foreground area includes an object that is temporarily or permanently brought into the imaging range by the moving object. For example, as an example of an object that is temporarily brought in, there may be a baggage that a person, who is a moving object, left unattended immediately after returning home, a laundry that is loaded immediately after being taken in, or the like. Moreover, furniture, a figurine, etc. can be mentioned as an example of the object brought in permanently.

As another form of the image analysis device according to the above aspect, the image acquisition unit may continuously acquire a captured image captured by the imaging device, and the moving object detection unit may continuously The moving object may be continuously detected in the captured image by tracking the moving object once detected in the captured image. The background update unit is configured to detect when the moving object is not detected when the moving object is not detected in the captured image due to the moving object moving outside the angle of view of the imaging device. The background image may be updated with the acquired image.

According to this configuration, when there is no moving object within the angle of view of the photographing apparatus, the background image can be updated with the photographed image acquired at that time. Therefore, even if the moving object causes a change in the background, the entire background image can be updated in a lump without the moving object thereafter. That is, after a moving object has moved outside the angle of view of the imaging device, when the moving object enters again within the angle of view of the imaging device, the moving object that has entered again can be properly detected by the background subtraction method. it can. Therefore, according to the said structure, it becomes possible to detect a moving object appropriately in the background subtraction method.

As another form of the image analysis device according to the above aspect, the image acquisition unit may acquire a captured image including depth data indicating the depth of each pixel in the captured image. The moving object detection unit analyzes a state of the target object in the foreground area in real space based on a depth of each pixel in the foreground area obtained by referring to the depth data. Thus, the moving object may be detected from the foreground region.

When the acquired captured image is a two-dimensional image, there is a possibility that the acquired captured image hardly changes even if the moving object moves depending on the viewpoint of the imaging apparatus. In such a case, it is difficult to distinguish and detect a moving object among the target objects shown in the foreground area in the acquired captured image. If the moving object cannot be detected, the background image cannot be updated, and there is a possibility that the moving object cannot be properly detected by the background difference method.

On the other hand, according to the configuration, the acquired captured image includes depth data indicating the depth of each pixel. The depth of each pixel indicates the depth from the photographing apparatus to the subject. Therefore, if this depth data is used, the state of the subject in real space (three-dimensional space) can be analyzed.

Therefore, according to this configuration, it is possible to analyze the state of each target object in the foreground area in the real space and detect the moving object from among the target objects regardless of the viewpoint of the photographing apparatus. . Therefore, according to this configuration, it is possible to update the background image used for the background difference method so that the moving object can be appropriately detected by the background difference method regardless of the viewpoint of the photographing apparatus. That is, it is possible to provide a background difference method that is robust against differences in the viewpoints of the photographing devices.

As another form of the image analysis apparatus according to each of the above embodiments, an information processing system that realizes each of the above-described configurations, an information processing method, or a program may be used. It may be a storage medium that can be read by a computer, other devices, machines, or the like in which such a program is recorded. Here, the computer-readable recording medium is a medium that stores information such as programs by electrical, magnetic, optical, mechanical, or chemical action. The information processing system may be realized by one or a plurality of information processing devices.

For example, in an image analysis method according to one aspect of the present invention, a computer acquires a captured image captured by an imaging device, and a background image set as a background of the captured image based on a background difference method The step of extracting a foreground area of the acquired captured image by calculating a difference from the acquired captured image, and moving within the angle of view of the imaging apparatus among the target objects reflected in the extracted foreground area Detecting a moving object to be detected from the foreground region, determining whether or not the number of detected moving objects matches the number of target objects appearing in the foreground region, When it is determined that the number does not match the number of target objects appearing in the foreground area, the background image is obtained using the acquired captured image for the target area excluding the area where the moving object appears. And updating the an information processing method for execution.

Further, for example, an image analysis program according to one aspect of the present invention includes a step of acquiring a captured image captured by a capturing apparatus in a computer, and a background set as a background of the captured image based on a background difference method A step of extracting a foreground area of the acquired captured image by calculating a difference between the image and the acquired captured image; and within an angle of view of the imaging apparatus among the target objects reflected in the extracted foreground area Detecting from the foreground area, determining whether the number of detected moving objects matches the number of target objects appearing in the foreground area, and detecting the detected movement When it is determined that the number of objects does not match the number of target objects appearing in the foreground area, the acquired captured image is used for the target area excluding the area where the moving object appears. Is a program for executing the steps of: updating the background image.

According to the present invention, it is possible to properly detect a moving object in the background subtraction method.

FIG. 1A schematically illustrates a scene to which the present invention is applied (a time point before a person enters the angle of view of the camera). FIG. 1B schematically illustrates a scene to which the present invention is applied (when a person enters the angle of view of the camera). FIG. 1C schematically illustrates a scene where the present invention is applied (when a person sits on a chair). FIG. 1D schematically illustrates a scene where the present invention is applied (when a person leaves the chair). FIG. 2 illustrates a hardware configuration of the image analysis apparatus according to the embodiment. FIG. 3 illustrates the relationship between the depth acquired by the camera according to the embodiment and the subject. FIG. 4 illustrates a functional configuration of the image analysis apparatus according to the embodiment. FIG. 5 illustrates a processing procedure related to the update of the background image in the image analysis apparatus according to the embodiment. FIG. 6A illustrates a captured image (at the time when a person enters the angle of view of the camera) acquired by the camera according to the embodiment. FIG. 6B illustrates a captured image (when a person sits on a chair) acquired by the camera according to the embodiment. FIG. 6C illustrates a captured image (at the time when the person leaves the chair) acquired by the camera according to the embodiment. FIG. 7 illustrates the coordinate relationship in the captured image according to the embodiment. FIG. 8 illustrates the positional relationship between an arbitrary point (pixel) of the captured image and the camera in the real space according to the embodiment. FIG. 9 illustrates a background image (before update) according to the embodiment. FIG. 10A illustrates the difference (foreground region) between the captured image and the background image of FIG. 6A. FIG. 10B illustrates the difference (foreground region) between the captured image and the background image of FIG. 6B. FIG. 10C illustrates the difference (foreground region) between the captured image and the background image of FIG. 6C. FIG. 11 illustrates a background image (after update) according to the embodiment. FIG. 12 illustrates another scene in which the image analysis apparatus according to the embodiment updates the background image. FIG. 13A illustrates a scene in which a plurality of moving objects enter within the angle of view of the imaging apparatus according to the embodiment. FIG. 13B illustrates a scene in which a plurality of moving objects enter within the angle of view of the imaging apparatus according to the embodiment.

Hereinafter, an embodiment according to one aspect of the present invention (hereinafter also referred to as “this embodiment”) will be described with reference to the drawings. However, this embodiment described below is only an illustration of the present invention in all respects. It goes without saying that various improvements and modifications can be made without departing from the scope of the present invention. That is, in implementing the present invention, a specific configuration according to the embodiment may be adopted as appropriate. Although data appearing in the present embodiment is described in a natural language, more specifically, it is specified by a pseudo language, a command, a parameter, a machine language, or the like that can be recognized by a computer.

§1 Application Scenes First, scenes to which the present invention is applied will be described with reference to FIGS. 1A to 1D. 1A to 1D show an example of a scene in which the image analysis apparatus 1 according to the present embodiment is used. A person who enters a room sits by pulling a chair placed in the room, and then the person leaves the room. The scene to go is illustrated. The image analysis apparatus 1 according to the present embodiment is an information processing apparatus that detects a moving object in a captured image based on a background difference method. Therefore, in the scene, the image analysis apparatus 1 according to the present embodiment detects a person who has entered the room as a moving object.

Specifically, FIG. 1A schematically illustrates a scene before a person appears within the angle of view of the camera 2. The image analysis apparatus 1 according to the present embodiment is connected to a camera 2 and acquires a captured image 3 captured by the camera 2. In the present embodiment, a table and a chair are arranged as part of the background within the angle of view of the camera 2. Each of the table and the chair is a stationary object, and is an example of a target object other than a moving object. The image analysis apparatus 1 acquires a captured image 3 obtained by capturing this scene before a person appears within the angle of view of the camera 2 as a background image 4. However, the background image 4 is not limited to such an example, and the image analysis apparatus 1 may acquire the background image 4 at an arbitrary timing.

Next, FIG. 1B schematically illustrates a scene in which a person enters the angle of view of the camera 2 after the scene illustrated in FIG. 1A. This person is an example of the moving object of the present invention. The image analysis apparatus 1 according to the present embodiment calculates the difference between the background image 4 set as the background and the acquired captured image 3 based on the background difference method, thereby obtaining the foreground region of the acquired captured image 3. Extract. In this scene, the portion where the change occurs between the background image 4 and the captured image 3 is an area where a person is captured. For this reason, an area in which a person is captured is extracted as a foreground area.

Subsequently, FIG. 1C schematically illustrates a scene in which a person pulls a chair existing within the angle of view of the camera 2 after the scene illustrated in FIG. 1B. From the scene of FIG. 1B to the scene of FIG. 1C, the person as the moving object moves from the left end of the captured image 3 to the region where the chair is reflected. In each of the captured images 3 acquired during this time, the image analysis apparatus 1 extracts an area in which the person is captured as a foreground area based on the background difference method.

Then, as illustrated in FIG. 1C, when the person pulls the chair and sits down, the chair moves together with the person from the original position. Therefore, in the scene illustrated in FIG. 1C, the image analysis apparatus 1 integrally extracts an area where the person and the chair are captured as a foreground area.

Furthermore, FIG. 1D schematically illustrates a scene in which the target person leaves the chair after the scene illustrated in FIG. 1C. In this scene, the person is away from the chair, and the chair is offset from its original position. Therefore, in the scene illustrated in FIG. 1D, the image analysis apparatus 1 extracts two areas, that is, the area where the person appears and the area where the chair appears as the foreground area.

Here, the image analysis apparatus 1 detects, from the foreground area, a moving object that moves within the angle of view of the camera 2 among the extracted target objects that appear in the foreground area, and the number of detected moving objects is in the foreground area. It is determined whether or not the number matches the number of target objects. In one example, the image analysis apparatus 1 recognizes a lump area having a size equal to or larger than a threshold value as an object.

Therefore, in the scene of FIG. 1C, the image analysis apparatus 1 recognizes a region where a person and a chair are captured as one object, and detects a person who is a moving object in this region. That is, in the scene illustrated in FIG. 1C, the image analysis apparatus 1 determines that the number of detected moving objects matches the number of target objects that appear in the foreground region.

On the other hand, in the scene of FIG. 1D, the image analysis apparatus 1 detects a person who is a moving object in the area where the person is captured while recognizing the area where the person is captured and the area where the chair is captured as separate objects. That is, in the scene illustrated in FIG. 1D, the image analysis apparatus 1 determines that the number of detected moving objects does not match the number of target objects that appear in the foreground region.

As illustrated in FIG. 1D, a scene in which the number of detected moving objects is determined not to match the number of target objects in the foreground area is a scene in which at least a part of the background has been altered. . That is, in this scene, due to the influence of the modification of at least a part of the background, the modified area is extracted independently as the foreground area in addition to the area where the moving object appears.

In such a case, the image analysis apparatus 1 according to the present embodiment updates the background image 4 using the acquired captured image 3 for the target region excluding the region where the moving object is captured. That is, the image analysis apparatus 1 updates the background image 4 by using the captured image 3 obtained by capturing the state of the area where the chair is extracted as the foreground area as a result of moving from the original position.

That is, according to the image analysis apparatus 1 according to the present embodiment, when a change occurs in at least a part of the background, the photographed image 3 photographed after the change has occurred with respect to the changed region. The background image 4 can be updated. Therefore, in the background subtraction method, it is possible to prevent the background area where the change has occurred from being extracted as the foreground area. Therefore, according to the present embodiment, it is possible to provide a technique that can appropriately detect a moving object in the background subtraction method.

In this embodiment, for convenience of explanation, a scene in which a person (moving object) entering the room sits by pulling a chair (stationary object), then stands up from the chair and exits the room is illustrated. However, the image analysis apparatus 1 according to the present embodiment is not limited to such a scene, but is used to detect a moving object in a scene where at least part of the background may be altered. Widely applicable.

In the present embodiment, a person moving within the angle of view of the camera 2 is illustrated as an example of the moving object. However, the moving object is not limited to such an example, and may be other than a person as long as the object moves within the angle of view of the camera 2.

Further, in this embodiment, as an example of the target object shown in the foreground area, a table and a chair, which are stationary objects, are illustrated in addition to a person. However, the target object appearing in the foreground area is not limited to such an example, and can include any object that can be extracted as the foreground area by the background subtraction method.

Further, the location of the image analysis device 1 can be determined as appropriate according to the embodiment as long as the captured image 3 can be acquired from the camera 2. For example, the image analysis apparatus 1 may be disposed so as to be close to the camera 2 as illustrated in FIGS. 1A to 1D. In addition, the image analysis apparatus 1 may be connected to the camera 2 via a network, or may be disposed at a place completely different from the camera 2.

§2 Configuration example <Hardware configuration>
Next, the hardware configuration of the image analysis apparatus 1 will be described with reference to FIG. FIG. 2 illustrates the hardware configuration of the image analysis apparatus 1 according to the present embodiment. As illustrated in FIG. 2, the image analysis apparatus 1 stores a control unit 11 including a CPU, a RAM (Random Access Memory), a ROM (Read Only Memory), and the like, a program 5 executed by the control unit 11, and the like. Unit 12, a touch panel display 13 for displaying and inputting images, a speaker 14 for outputting sound, an external interface 15 for connecting to an external device, a communication interface 16 for communicating via a network, and This is a computer to which a drive 17 for reading a program stored in the storage medium 6 is electrically connected. In FIG. 2, the communication interface and the external interface are described as “communication I / F” and “external I / F”, respectively.

It should be noted that regarding the specific hardware configuration of the image analysis apparatus 1, the components can be omitted, replaced, and added as appropriate according to the embodiment. For example, the control unit 11 may include a plurality of processors. In addition, for example, the touch panel display 13 may be replaced with an input device and a display device that are separately connected independently. For example, the speaker 14 may be omitted. Further, for example, the speaker 14 may be connected to the image analysis device 1 as an external device instead of as an internal device of the image analysis device 1. Further, the image analysis apparatus 1 may incorporate a camera 2. Furthermore, the image analysis device 1 may include a plurality of external interfaces 15 and may be connected to a plurality of external devices.

The camera 2 according to the present embodiment is connected to the image analysis apparatus 1 via the external interface 15 and is installed to photograph a person who has entered the room. However, the installation purpose of the camera 2 is not limited to such an example, and can be selected as appropriate according to the embodiment. This camera 2 corresponds to the photographing apparatus of the present invention.

In this embodiment, the camera 2 includes a depth sensor 21 for measuring the depth of the subject. The type and measurement method of the depth sensor 21 may be appropriately selected according to the embodiment. For example, the depth sensor 21 may be a sensor of TOF (TimeＦOf Flight) method or the like.

However, the configuration of the camera 2 is not limited to such an example, and can be appropriately selected according to the embodiment. For example, the camera 2 may be a known imaging device that captures a two-dimensional image (for example, an RGB image) without acquiring the depth. Further, when the camera 2 is configured so that the depth can be measured, the camera 2 may be a stereo camera. Since the stereo camera shoots the subject within the shooting range from a plurality of different directions, the depth of the subject can be recorded. The camera 2 may be replaced with the depth sensor 21 alone.

Note that the place where pedestrians are photographed may be dark. Therefore, the depth sensor 21 may be an infrared depth sensor that measures the depth based on infrared irradiation so that the depth can be acquired without being affected by the brightness of the shooting location. Examples of relatively inexpensive imaging apparatuses including such an infrared depth sensor include Kinect from Microsoft, Xtion from ASUS, and CARMINE from PrimeSense.

Here, the depth measured by the depth sensor 21 according to the present embodiment will be described in detail with reference to FIG. FIG. 3 shows an example of a distance that can be handled as the depth according to the present embodiment. The depth represents the depth of the subject. As exemplified in FIG. 3, the depth of the subject may be expressed by, for example, a straight line distance A between the camera 2 and the object, or a perpendicular distance B from the horizontal axis with respect to the subject of the camera 2. It may be expressed as

That is, the depth according to the present embodiment may be the distance A or the distance B. In the present embodiment, the distance B is treated as the depth. However, the distance A and the distance B can be converted into each other by using, for example, the three-square theorem. Therefore, the following description using the distance B can be applied to the distance A as it is. By using such a depth, the image analysis apparatus 1 according to the present embodiment can specify the position of the subject in the real space.

In addition, the storage unit 12 of the image analysis apparatus 1 according to the present embodiment stores the background image 4 used for the background difference method. The background image 4 is an image set as the background of the captured image 3 and can be appropriately acquired according to the embodiment. For example, as described above, the storage unit 12 may hold the captured image 3 acquired before the person who is the moving object enters the angle of view of the camera 2 as the background image 4.

In FIG. 2, the background image 4 is stored in the storage unit 12 in advance. However, the storage location of the background image 4 may not be limited to such an example. For example, the background image 4 may be held in another information processing apparatus or the like. In this case, the image analysis apparatus 1 may access the other information processing apparatus via a network or the like to acquire the background image 4 used for the background difference method processing.

The storage unit 12 further stores the program 5. The program 5 is a program for causing the image analysis apparatus 1 to execute each process related to the background image update described later, and corresponds to the “image analysis program” of the present invention. The program 5 may be recorded on the storage medium 6.

The storage medium 6 stores information such as a program by an electrical, magnetic, optical, mechanical, or chemical action so that information such as a program recorded by a computer or other device or machine can be read. It is a medium to do. The storage medium 6 corresponds to the “storage medium” of the present invention. 2 illustrates a disk-type storage medium such as a CD (Compact Disk) or a DVD (Digital Versatile Disk) as an example of the storage medium 6. However, the type of the storage medium 6 is not limited to the disk type and may be other than the disk type. Examples of the storage medium other than the disk type include a semiconductor memory such as a flash memory.

Further, such an image analysis device 1 may be, for example, a device designed exclusively for the provided service, or a general-purpose device such as a PC (Personal Computer) or a tablet terminal. Furthermore, the image analysis apparatus 1 may be implemented by one or a plurality of computers.

<Functional configuration example>
Next, the functional configuration of the image analysis apparatus 1 will be described with reference to FIG. FIG. 4 illustrates a functional configuration of the image analysis apparatus 1 according to the present embodiment. In the present embodiment, the control unit 11 of the image analysis device 1 expands the program 5 stored in the storage unit 12 in the RAM. And the control part 11 interprets and runs the program 5 expand | deployed by RAM by CPU, and controls each component. Accordingly, the image analysis device 1 functions as a computer including the image acquisition unit 31, the background difference calculation unit 32, the moving object detection unit 33, and the background update unit 34.

The image acquisition unit 31 acquires a captured image 3 captured by the camera 2. The background difference calculation unit 32 extracts the foreground region of the acquired captured image 3 by calculating the difference between the background image 4 stored in the storage unit 12 and the acquired captured image 3 based on the background difference method. To do. As described above, the foreground area may include an area where a background change occurs in addition to a person who is a moving object.

Therefore, the moving object detection unit 33 detects, from the foreground area, a moving object that moves within the angle of view of the camera 2 among the extracted target objects reflected in the foreground area, and the background update unit 34 detects the detected moving object. It is determined whether or not the number of matches the number of target objects in the foreground area. When the background update unit 34 determines that the number of detected moving objects does not match the number of target objects appearing in the foreground area, the background update unit 34 acquires the captured image 3 for the target area excluding the area where the moving object appears. Is used to update the background image 4.

In the present embodiment, an example is described in which all of these functions are realized by a general-purpose CPU. However, some or all of these functions may be realized by one or more dedicated processors. In addition, regarding the functional configuration of the image analysis apparatus 1, functions may be omitted, replaced, and added as appropriate according to the embodiment. Each function will be described in detail in an operation example described later.

§3 Operation Example Next, an operation example of the image analysis apparatus 1 will be described with reference to FIG. FIG. 5 illustrates a processing procedure related to the update of the background image of the image analysis apparatus 1. The processing procedure related to the background image update described below corresponds to the “image analysis method” of the present invention. However, the processing procedure related to the background image update described below is merely an example, and each processing may be changed as much as possible. Further, in the processing procedure described below, steps can be omitted, replaced, and added as appropriate according to the embodiment.

(Step S101)
In step S <b> 101, the control unit 11 functions as the image acquisition unit 31 and acquires the captured image 3 captured by the camera 2. Then, the control part 11 advances a process to following step S102.

Here, the captured image 3 acquired in step S101 will be described with reference to FIGS. 6A to 6C. 6A to 6C illustrate the captured images 3a to 3c acquired in this step S101. As illustrated in FIGS. 6A to 6C, the control unit 11 according to the present embodiment continuously acquires the captured image 3 captured by the camera 2 as, for example, a moving image.

Specifically, the captured image 3 a in FIG. 6A is a captured image 3 that was captured when a person entered the angle of view of the camera 2. A photographed image 3b in FIG. 6B is a photographed image 3 photographed when a person sits with a chair pulled within the angle of view of the camera 2 after the photographed image 3a in FIG. 6A is photographed. Furthermore, the captured image 3c in FIG. 6C is a captured image 3 that is captured when the target person leaves the chair after the captured image 3b in FIG. 6B is captured.

The control unit 11 may acquire such captured images 3a to 3c in synchronization with the video signal of the camera 2. Then, at the stage where one or a plurality of photographed images 3 are acquired, the control unit 11 can immediately execute the processing from steps S102 to S105 described later to the one or a plurality of photographed images 3 that have been acquired. Good. The image analysis apparatus 1 can perform real-time image processing by continuously executing such an operation continuously, and can detect a moving object existing in the shooting range of the camera 2 in real time.

In the present embodiment, the camera 2 includes a depth sensor 21. For this reason, the captured images 3a to 3c acquired in step S101 include depth data indicating the depth of each pixel. Specifically, each of the captured images 3a to 3c illustrated in FIGS. 6A to 6C is the captured image 3 in which the gray value of each pixel is determined according to the depth of each pixel.

6A to 6C show that the black pixels are closer to the camera 2. On the other hand, a white pixel is farther from the camera 2. Based on the depth data, the control unit 11 can specify the position of each pixel in the real space. That is, the control unit 11 can specify the position in the three-dimensional space (real space) of the subject captured in each pixel from the coordinates (two-dimensional information) and the depth of each pixel in the captured image 3. . Hereinafter, a calculation example in which the control unit 11 specifies the position of each pixel in the real space will be described with reference to FIGS. 7 and 8.

FIG. 7 illustrates the coordinate relationship in the captured image 3. FIG. 8 illustrates the positional relationship between an arbitrary pixel (point s) of the captured image 3 and the camera 2 in the real space. 7 corresponds to a direction perpendicular to the paper surface of FIG. That is, the length of the captured image 3 shown in FIG. 8 corresponds to the length in the vertical direction (H pixels) illustrated in FIG. Further, the length in the horizontal direction (W pixels) illustrated in FIG. 7 corresponds to the length in the vertical direction of the photographed image 3 that does not appear in FIG.

As illustrated in FIG. 7, the coordinates of an arbitrary pixel (point s) of the captured image 3 are (x _s , y _s ), the horizontal angle of view of the camera 2 is V _x , and the vertical image Assume that the corner is V _y . Further, it is assumed that the number of pixels in the horizontal direction of the captured image 3 is W, the number of pixels in the vertical direction is H, and the coordinates of the center point (pixel) of the captured image 3 are (0, 0).

The control unit 11 can acquire information indicating the angle of view (V _x , V _y ) of the camera 2 from the camera 2. However, the method for acquiring information indicating the angle of view (V _x , V _y ) of the camera 2 is not limited to such an example, and the control unit 11 is information indicating the angle of view (V _x , V _y ) of the camera 2. May be acquired based on user input, or may be acquired as a preset setting value. Further, the control unit 11 can acquire the coordinates (x _s , y _s ) of the point s and the number of pixels (W × H) of the captured image 3 from the captured image 3. Furthermore, the control unit 11 can acquire the depth Ds of the point s by referring to the depth data included in the captured image 3.

The control unit 11 can specify the position of each pixel (point s) in the real space by using these pieces of information. For example, the control unit 11 performs vector S (S _x , S _y , S _z) from the camera 2 to the point s in the camera coordinate system illustrated in FIG. , 1) can be calculated. Thereby, the position of the point s in the two-dimensional coordinate system in the captured image 3 and the position of the point s in the camera coordinate system can be mutually converted.

However, the vector S is a vector of a three-dimensional coordinate system centered on the camera 2. As illustrated in FIG. 8, the camera 2 may be tilted with respect to the horizontal direction. That is, the camera coordinate system may be tilted from the world coordinate system in the three-dimensional space (real space). Therefore, the control unit 11 applies the projective transformation using the roll angle, pitch angle (α in FIG. 8), and yaw angle of the camera 2 to the vector S, so that the vector S of the camera coordinate system is converted to the world coordinate system. And the position of the point s in the world coordinate system may be calculated.

In addition, the data format of the captured image 3 including the depth data may not be limited to such an example, and may be appropriately selected according to the embodiment. For example, the captured image 3 may be data (for example, a depth map) in which the depth of the subject within the imaging range is two-dimensionally distributed. For example, the captured image 3 may include an RGB image together with the depth data. Such a captured image 3 may be a moving image or one or a plurality of still images.

(Step S102)
Returning to FIG. 5, in the next step S102, the control unit 11 functions as the background difference calculation unit 32, and based on the background difference method, the background image 4 stored in the storage unit 12 and each acquired in step S101. The difference from the captured images 3a to 3c is calculated. Thereby, the control unit 11 extracts the foreground area of each of the captured images 3a to 3c acquired in step S101. When the foreground area of each of the captured images 3a to 3c is extracted, the control unit 11 proceeds to the next step S103.

Here, the background image 4 stored in the storage unit 12 will be described with reference to FIG. FIG. 9 illustrates the background image 4 a stored in the storage unit 12. The background image 4a is the background image 4 acquired before the photographed image 3a of FIG. 6A is photographed, that is, before a person enters the angle of view of the camera 2. For example, the control unit 11 acquires, as the background image 4, the captured image 3 at the time when there is no moving object before starting the processing of this operation example. For this reason, in the present embodiment, the background image 4 also includes depth data in the same manner as the captured image 3. However, the method of acquiring the background image 4 is not limited to such an example, and can be set as appropriate according to the embodiment.

In step S102, the control unit 11 calculates a difference between each of the captured images 3a to 3c acquired in step S101 and the background image 4. For example, the control unit 11 calculates a pixel value difference between corresponding pixels of each of the captured images 3a to 3c and the background image 4, and when the calculated difference exceeds a predetermined threshold value, The pixel is recognized as a pixel in the foreground area. However, the method for extracting the foreground region is not limited to such an example, and can be appropriately set based on various background subtraction methods.

FIGS. 10A to 10C illustrate foreground regions extracted in the captured images 3a to 3c by such processing. FIG. 10A illustrates a difference area (foreground area) between the captured image 3a in FIG. 6A and the background image 4a in FIG. FIG. 10B illustrates a difference area (foreground area) between the captured image 3b in FIG. 6B and the background image 4a in FIG. FIG. 10C illustrates a difference area (foreground area) between the captured image 3c in FIG. 6C and the background image 4a in FIG. 10A to 10C are views of the shooting range of the camera 2 as viewed from above. That is, the vertical direction of each of FIGS. 10A to 10C corresponds to the direction perpendicular to the paper surface of FIGS. 6A to 6C.

As described above, the gray value of each pixel of each of the captured images 3a to 3c and the background image 4a is determined according to the depth of each pixel. Therefore, the difference in pixel value between corresponding pixels of each of the captured images 3a to 3c and the background image 4 corresponds to the difference in depth of each pixel. Therefore, as illustrated in FIGS. 10A to 10C, in the present embodiment, based on the background subtraction method, it is possible to extract a region where the background has changed in real space as a foreground region.

More specifically, in the scene of FIG. 6A, as illustrated in FIG. 10A, an area in which a person is captured is extracted as a foreground area. Next, in the scene of FIG. 6B, as illustrated in FIG. 10B, the area where the person and the chair are captured is extracted as a foreground area. In the scene of FIG. 6C, the area where the person is photographed and the area where the chair is photographed are extracted as separate foreground areas. Since each of the captured images 3a to 3c and the background image 4a includes depth data, in this step S102, an area where the background has changed in real space can be extracted as a foreground area.

(Step S103)
Returning to FIG. 5, in the next step S <b> 103, the control unit 11 functions as the moving object detection unit 33, and moves to move within the angle of view of the camera 2 among the target objects captured in the foreground area extracted in step S <b> 102. An object is detected from the foreground region. Then, when the detection of the moving object is completed, the control unit 11 advances the processing to the next step S104.

In the present embodiment, for example, the control unit 11 recognizes a block of foreground areas having a size equal to or larger than a predetermined threshold as one target object. In this case, in the scenes of FIGS. 6A and 6B, the control unit 11 has a single foreground object in the foreground area because the foreground area appears in one place as illustrated in FIGS. 10A and 10B. Recognize. On the other hand, in the scene of FIG. 6C, the control unit 11 recognizes that there are two target objects in the foreground area because the foreground area appears apart in two places as illustrated in FIG. 10C. However, the method for recognizing the number of target objects appearing in the foreground region is not limited to such an example, and may be appropriately selected according to the embodiment.

The control unit 11 recognizes that the target object is a moving object when it is determined that the target object in the foreground region is an object moving in real space. For example, as illustrated in FIGS. 10A to 10C, the control unit 11 can acquire the depth of each pixel in the foreground region by referring to the depth data. As described above, the depth of each pixel indicates the position of each pixel in the real space.

Therefore, the control unit 11 can analyze the state of the target object in the foreground area in the real space based on the depth of each pixel in the foreground area. Specifically, the control unit 11 can determine whether or not the position of the foreground region varies in real space based on the depth of each pixel in the foreground region.

Therefore, when the control unit 11 determines that the position of the foreground region is fluctuating in the real space, the control unit 11 recognizes that the target object in the foreground region is moving in the real space, and the target object is You may recognize that it is a moving object. That is, in this case, the control unit 11 can detect a moving object from the foreground area. On the other hand, when the position of the foreground area has not changed in real space, the control unit 11 may recognize that the target object appearing in the foreground area is an object other than a moving object (for example, a stationary object). . Note that such a change in the foreground region can also be determined based on an optical flow or the like.

For example, the control unit 11 may recognize a moving object as follows. That is, when a moving object enters within the angle of view of the camera 2, a foreground region appears on the periphery of the captured image 3 as illustrated in FIGS. 6A and 10A. Therefore, when the foreground area appears on the periphery of the captured image 3, the control unit 11 may recognize that the target object appearing in the foreground area is a moving object and increment the number of moving objects.

Next, the control unit 11 may continuously detect the moving object in the captured image 3 by tracking (tracking) the moving object once detected in the continuously acquired captured image 3. Such tracking can be performed based on an optical flow or the like. That is, as illustrated in FIGS. 10A to 10C, the foreground area in which a person is photographed varies in a series of captured images 3. Therefore, the control unit 11 may identify a foreground area where a person is captured from among the foreground areas appearing in each captured image 3 by tracking the fluctuating foreground area based on an optical flow or the like.

When the moving object moves out of the angle of view of the camera 2, the foreground area moves toward the periphery of the captured image 3 as illustrated in FIGS. 6C and 10C, and then the foreground area Disappear. Therefore, when the foreground area being tracked moves to the periphery of the captured image 3 and disappears, the control unit 11 recognizes that the moving object has moved out of the angle of view, and decrements the number of moving objects. Good. Accordingly, the control unit 11 can manage the number of moving objects that appear in the captured image 3 acquired in series.

The control unit 11 can detect a moving object as described above, for example. Specifically, in the scenes of FIGS. 6A and 6B, the control unit 11 recognizes a group of foreground areas illustrated in FIGS. 10A and 10B as moving objects. Further, in the scene of FIG. 6C, the control unit 11 recognizes the left foreground region illustrated in FIG. 10C as a moving object, and the right foreground region illustrated in FIG. 10C is an object other than the moving object. Recognize.

Note that the method of recognizing a moving object is not limited to these examples, and may be appropriately selected according to the embodiment. In addition, the method for recognizing each state in the image analysis apparatus 1 is not limited to such an example. If the state in which the moving object is detected can be recognized, the method is appropriately determined according to the embodiment. May be set.

(Step S104)
In the next step S104, the control unit 11 functions as the background update unit 34, and determines whether or not the number of moving objects detected in step S103 matches the number of target objects appearing in the foreground region. If the control unit 11 determines that the number of moving objects detected in step S103 does not match the number of target objects in the foreground area, the control unit 11 proceeds to the next step S105. On the other hand, if it is determined that the number of moving objects detected in step S103 matches the number of target objects that appear in the foreground area, the control unit 11 omits the process of the next step S105 and performs this operation example. This process is terminated.

The method for determining whether or not the number of moving objects detected in step S103 matches the number of target objects appearing in the foreground region may be appropriately set according to the embodiment. For example, when there is no foreground area corresponding to the target object other than the moving object, in other words, when all the foreground areas correspond to the moving objects, the control unit 11 detects the moving object detected in step S103. Can be determined to match the number of target objects in the foreground area. On the other hand, when there is at least one foreground area corresponding to the target object other than the moving object, it can be determined that the number of moving objects detected in step S103 does not match the number of target objects appearing in the foreground area. .

Specifically, in the scenes of FIGS. 6A and 6B, the control unit 11 recognizes that the number of target objects and the number of moving objects are one each, and therefore the number of moving objects detected in step S103 is the foreground. It is determined that the number matches the number of target objects appearing in the region, and the processing according to this operation example is terminated. On the other hand, in the scene of FIG. 6C, the control unit 11 recognizes that the number of target objects is two while the number of moving objects is one, and thus the moving object detected in step S103. Is determined not to coincide with the number of target objects in the foreground area. And the control part 11 advances a process to following step S105.

(Step S105)
In the next step S105, the control unit 11 functions as the background update unit 34 and updates the background image 4 using the acquired captured image 3 for the target region excluding the region where the moving object is captured. Then, when the update of the background image 4 is completed in step S105, the control unit 11 ends the process according to this operation example.

Here, as described above, the process of step S105 is executed in the scene illustrated in FIG. 6C among the scenes illustrated in FIGS. 6A to 6C. Therefore, an example of the process of step S105 will be described using the captured image 3c illustrated in FIG. 6C.

First, the control unit 11 determines a target area (hereinafter, also referred to as “update area”) used for updating the background image 4a from an area excluding the area 301 in which a person as a moving object is captured in the captured image 3c. . The method for determining the update area can be appropriately selected according to the embodiment. However, it is preferable that the update region is determined so as to include a foreground region (for example, region 302) in which a target object other than a moving object is captured.

For example, the control unit 11 may determine all the regions except the region where the moving object is captured (for example, the region 301) as the update region. In addition, for example, the control unit 11 may determine all regions except the region where the moving object is captured and a predetermined range around the region as the update region. For example, the captured image 3 may be divided into a predetermined number of blocks. And the control part 11 may determine the block which does not contain the area | region where a moving object is reflected among the predetermined number of blocks as an update area | region. With these methods, in the captured image 3c, for example, the region 303 that does not include the region 301 but includes the region 302 is determined as the update region.

Next, the control unit 11 updates the background image 4a using the captured image 3c for the determined update region (region 303). The method of updating the background image 4 using the captured image 3 can be set as appropriate according to the embodiment. For example, for the region 303, the control unit 11 may update the background image 4a by replacing the pixel value of each pixel of the background image 4a with the pixel value of each pixel of the captured image 3c. For example, for the region 303, the control unit 11 calculates the pixel value of each pixel of the background image 4a from the plurality of captured images 3 acquired during a predetermined time including the time when the captured image 3c is acquired. The background image 4a may be updated by replacing the average value of each pixel. Thus, the control unit 11 generates a new background image 4b illustrated in FIG.

FIG. 11 exemplifies a new background image 4b updated by the process of step S105. In the background image 4b illustrated in FIG. 11, the region where the chair moved from the original position is illustrated in FIG. 9 using each pixel included in the region 302 of the captured image 3c illustrated in FIG. 6C. Updated from the background image 4a. Therefore, in the captured image 3 acquired thereafter, even if the foreground region extraction process based on the background difference method in step S102 is applied, the region in which the chair appears is not extracted as the foreground region.

In the image analysis apparatus 1 according to the present embodiment, in this way, when the number of moving objects detected in step S103 does not match the number of target objects appearing in the foreground area, the target excluding the area where the moving object appears For the region, the background image 4 is updated using the captured image 3. That is, when a foreground area in which a target object other than a moving object is captured, a new background image that does not extract the foreground area using the captured image 3 and the original background image 4 acquired at that time. 4 is generated.

The control unit 11 stores the newly generated background image 4 in the storage unit 12. At this time, the control unit 11 may delete the original background image 4 from the storage unit 12, that is, may replace the original background image 4 with a new background image 4 in the storage unit 12. Further, the control unit 11 may leave the original background image 4 in the storage unit 12 as it is. The treatment of the original background image 4 can be appropriately selected according to the embodiment.

<Others>
(1) Update of background image In addition to the said timing, the control part 11 may update the background image 4. FIG. For example, the control unit 11 may continuously detect the moving object in the captured image 3 by tracking the moving object once detected in the captured image 3 continuously acquired in step S103. . Then, when the moving object is not detected in the captured image 3 due to the moving object moving outside the angle of view of the camera 2, the control unit 11 captures the captured image acquired when the moving object is not detected. 3 may update the background image 4. Hereinafter, this update process will be described with reference to FIG.

FIG. 12 illustrates a scene where a moving object no longer exists within the angle of view of the camera 2. In step S103, the control unit 11 may increment the number of moving objects when the moving object enters the angle of view of the camera 2, and when the moving object moves out of the angle of view of the camera 2, the control unit 11 You may decrement the number. Then, as illustrated in FIG. 12, when the number of moving objects becomes zero during this decrement, the control unit 11 updates the entire background image 4 with the captured image 3 acquired at this time. Also good.

Note that the method of updating the entire background image 4 can be selected as appropriate according to the embodiment. For example, the control unit 11 may store the captured image 3 acquired when the moving object no longer exists in the storage unit 12 as a new background image 4. Further, for example, the control unit 11 may generate a new background image 4 by averaging the captured images 3 acquired within a predetermined time after the moving object no longer exists.

According to this method, when there is no moving object within the angle of view of the camera 2, the background image 4 can be updated with the captured image 3 acquired at that time. Therefore, even if the moving object causes a change in the background, the entire area of the background image 4 can be updated at a time in the absence of the moving object thereafter. That is, after a moving object moves outside the angle of view of the camera 2, when the moving object enters again within the angle of view of the camera 2, the moving object that has re-entered can be properly detected by the background subtraction method. it can. Therefore, according to the method, it is possible to appropriately detect the moving object in the background difference method.

(2) Multiple Moving Objects In the above embodiment, an example in which one moving object enters within the angle of view of the camera 2 has been described. However, the number of moving objects entering the angle of view of the camera 2 is not limited to one and may be plural. The case where a plurality of moving objects enter can be explained in substantially the same manner as in the above embodiment. Hereinafter, the process of the control unit 11 when a plurality of moving objects enter will be described with reference to FIGS. 13A and 13B.

FIG. 13A exemplifies a scene in which two persons have entered into the angle of view of the camera 2 and have entered. FIG. 13B illustrates a scene in which two persons entering the angle of view of the camera 2 are separated from each other after the scene illustrated in FIG. 13A. In the above embodiment, the control unit 11 recognizes a group of foreground areas having a size equal to or larger than a predetermined threshold as one target object. Therefore, in the scene illustrated in FIG. 13A, the control unit 11 recognizes that one moving object exists in the captured image 3. Thereafter, in the scene illustrated in FIG. 13B, the control unit 11 recognizes that there are two moving objects in the captured image 3.

That is, when a plurality of moving objects enter the angle of view of the camera 2, the situation recognized by the control unit 11 and the actual situation in the captured image 3 can be different. However, according to the processing in the above embodiment, in each scene, the number of moving objects detected in step S103 is an object to be reflected in the foreground area unless the background is altered in the area excluding the area where the person is captured. It matches the number of objects. Therefore, even if the situation recognized by the control unit 11 and the actual situation in the captured image 3 deviate, the control unit 11 can execute the process according to the above operation example without any problem.

(Action / Effect)
As described above, the image analysis apparatus 1 according to the present embodiment updates the background image 4 using the acquired captured image 3 for the target region excluding the region where the moving object is captured. More specifically, when the number of foreground regions to be extracted is larger than the number of moving objects captured in the captured image 3, the image analysis apparatus 1 according to the present embodiment includes the captured image 3 acquired at that time. Using the original background image 4, a new background image 4 is generated so that a foreground region that captures an object other than the moving object is not extracted.

Therefore, according to the image analysis apparatus 1 according to the present embodiment, when a change occurs in at least a part of the background, the photographed image 3 photographed after the change has occurred with respect to the changed region. The background image 4 can be updated. Therefore, according to the present embodiment, it is possible to prevent a region that captures an object other than a moving object from being extracted as a foreground region in the processing based on the background difference method, thereby appropriately moving a moving object based on the background difference method. It can be made detectable.

In the above embodiment, the acquired captured image 3 includes depth data indicating the depth of each pixel. Therefore, as illustrated in FIGS. 10A to 10C, the position of the foreground region in the real space can be specified by using this depth data. Therefore, according to the configuration, regardless of the viewpoint of the camera 2, the state of the target object in the foreground area in the real space is analyzed, and based on the result of the analysis, whether the foreground area corresponds to the moving object. It can be determined whether or not. Therefore, according to the configuration, the background image used for the background difference method can be updated so that the moving object based on the background difference method can be appropriately detected regardless of the viewpoint of the camera 2. That is, it is possible to provide a background difference method that is robust against differences in the viewpoint of the camera 2.

Specifically, when the moving object moves closer to or away from the camera 2 side, the movement of the moving object is detected because there is no significant change in the area where the moving object appears in the two-dimensional image. It ’s difficult. On the other hand, according to this embodiment, the movement of the moving object can be detected based on the depth. In addition, when a moving object overlaps a stationary object in a captured image, it is difficult to separate these objects in a two-dimensional image. On the other hand, according to this embodiment, since the position of each object can be specified based on the depth, when a moving object and a stationary object are separated, these objects can be separated. Therefore, according to this configuration, the background image used for the background subtraction method can be updated regardless of the viewpoint of the camera 2.

Note that the image analysis apparatus 1 according to the present embodiment can detect a moving object in the captured image 3 captured by the camera 2. Therefore, the image analysis apparatus 1 according to the present embodiment can be used in various systems that involve detection of moving objects.

For example, the image analysis apparatus 1 according to the present embodiment can be used in a system that detects a watching target person as a moving object. Here, in this embodiment, since the captured image 3 includes depth data, it is possible to analyze the state of the person being watched over in real space based on the depth data. And when the control part 11 analyzes that it is in the state where danger is approaching a watching target person, you may alert | report that the monitoring target person is approaching danger via the speaker 14 grade | etc.,. .

Also, for example, the image analysis apparatus 1 according to the present embodiment can be used in a system that detects a suspicious person entering a building as a moving object. In this case, the camera 2 is installed on a route through which a suspicious person can enter. At this time, the control unit 11 may display the moving object on the touch panel display 13 while being color-coded with other target objects. As a result, the manager of the building can instantly recognize the moving object from the captured image 3 displayed on the touch panel display 13, and can easily find the suspicious person.

§4 Modifications Embodiments of the present invention have been described in detail above, but the above description is merely an illustration of the present invention in all respects. It goes without saying that various improvements and modifications can be made without departing from the scope of the present invention.

For example, in the above embodiment, the camera 2 includes the depth sensor 21 so that the depth of each pixel of the captured image 3 can be acquired. However, the camera 2 may not be limited to such an example, and may not be configured to be able to acquire the depth. For example, the camera 2 may be a known imaging device that can acquire a two-dimensional image such as an RGB image. Even in this case, the image analysis apparatus 1 can extract the foreground region based on the background difference method, detect the moving object, and update the background image 4 in the same manner as described above.

1 ... Image analysis device,
2 ... Camera, 21 ... Depth sensor,
3 (3a-3c) ... taken image, 4 (4a, 4b) ... background image,
5 ... Program, 6 ... Storage medium,
11 ... Control unit, 12 ... Storage unit, 13 ... Touch panel display,
14 ... Speaker, 15 ... External interface, 16 ... Communication interface,
17 ... drive,
31 ... Image acquisition unit, 32 ... Background difference calculation unit, 33 ... Moving object detection unit,
34 ... Background update section

Claims

An image acquisition unit for acquiring a photographed image photographed by the photographing device;
Based on a background difference method, a background difference calculation unit that extracts a foreground region of the acquired captured image by calculating a difference between the background image set as the background of the captured image and the acquired captured image;
A moving object detection unit that detects, from the foreground area, a moving object that moves within the angle of view of the imaging device among the extracted target objects that are reflected in the foreground area;
It is determined whether or not the number of detected moving objects matches the number of target objects appearing in the foreground region, and if the number of detected moving objects does not match the number of target objects appearing in the foreground region A background update unit that updates the background image using the acquired captured image for the target region excluding the region where the moving object is captured,
Comprising
Image analysis device.
The image acquisition unit continuously acquires captured images captured by the imaging device,
The moving object detection unit continuously detects the moving object in the captured image by tracking the moving object once detected in the captured image continuously acquired,
The background update unit is acquired when the moving object is not detected when the moving object is not detected in the captured image due to the moving object moving outside the angle of view of the imaging device. Update the background image with the captured image,
The image analysis apparatus according to claim 1.
The image acquisition unit acquires a captured image including depth data indicating the depth of each pixel in the captured image,
The moving object detection unit, based on the depth of each pixel in the foreground region obtained by referring to the depth data, by analyzing the state in real space of the target object that appears in the foreground region, Detecting the moving object from the foreground region;
The image analysis apparatus according to claim 1 or 2.
Computer
Obtaining a photographed image photographed by the photographing device;
Extracting a foreground region of the acquired captured image by calculating a difference between a background image set as a background of the captured image and the acquired captured image based on a background difference method;
Detecting, from the foreground area, a moving object that moves within the angle of view of the imaging device among the extracted target objects in the foreground area;
Determining whether the number of detected moving objects matches the number of target objects in the foreground region;
When it is determined that the number of detected moving objects does not match the number of target objects appearing in the foreground area, the background is obtained using the acquired captured image with respect to the target area excluding the area where the moving object appears. Updating the image;
Image analysis method to execute.
On the computer,
Obtaining a photographed image photographed by the photographing device;
Extracting a foreground region of the acquired captured image by calculating a difference between a background image set as a background of the captured image and the acquired captured image based on a background difference method;
Detecting, from the foreground area, a moving object that moves within the angle of view of the imaging device among the extracted target objects in the foreground area;
Determining whether the number of detected moving objects matches the number of target objects in the foreground region;
When it is determined that the number of detected moving objects does not match the number of target objects appearing in the foreground area, the background is obtained using the acquired captured image with respect to the target area excluding the area where the moving object appears. Updating the image;
Image analysis program for executing