US20110243385A1 - Moving object detection apparatus, moving object detection method, and program - Google Patents

Moving object detection apparatus, moving object detection method, and program Download PDF

Info

Publication number
US20110243385A1
US20110243385A1 US13/065,374 US201113065374A US2011243385A1 US 20110243385 A1 US20110243385 A1 US 20110243385A1 US 201113065374 A US201113065374 A US 201113065374A US 2011243385 A1 US2011243385 A1 US 2011243385A1
Authority
US
United States
Prior art keywords
image
moving object
moving
determination
range
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US13/065,374
Other languages
English (en)
Inventor
Katsuaki NISHINO
Nobuhiro Tsunashima
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Corp
Original Assignee
Sony Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Corp filed Critical Sony Corp
Assigned to SONY CORPORATION reassignment SONY CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: NISHINO, KATSUAKI, TSUNASHIMA, NOBUHIRO
Publication of US20110243385A1 publication Critical patent/US20110243385A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/223—Analysis of motion using block-matching

Definitions

  • the present invention relates to a moving object detection apparatus, a moving object detection method, and a program. More particularly, the invention relates to a moving object detection apparatus, a moving object detection method, and a program for ensuring moving object detection with sufficient accuracy even in dark places, typically at night.
  • surveillance systems that monitor a predetermined space using surveillance cameras. Such surveillance systems typically use the image taken by each surveillance camera as an analysis image of which the data is to be analyzed, thereby detecting the image of a moving object moving in a designated region inside the analysis image of interest.
  • Traditionally most of these surveillance systems utilize the technique for detecting images of the moving object using a moving vector (e.g., see Japanese Patent Laid-open No. 2006-260049), or the technique for detecting the moving object by use of correlations between current and past images (e.g., see Japanese Patent No. 3506934 and Japanese Patent Laid-open No. 2007-251721).
  • the surveillance system is required to ensure detection with at least a certain level of accuracy so that objects being monitored will not be missed or erroneously detected even in dark places, typically at night.
  • the requirement has yet to be met sufficiently by traditional techniques for moving object detection, including those cited above.
  • the present invention has been made in view of the above circumstances and provides innovative arrangements for ensuring moving object detection with sufficient accuracy even in dark places, typically at night.
  • a moving object detection apparatus including: image input processing means for inputting an analysis image composed of an image taken by a camera in order to establish a designated region inside the analysis image; first detection processing means for detecting an image of a moving object which moves within the designated region established by the image input processing means and which is at a distance in a first range from the camera; and second detection processing means for detecting an image of the moving object which moves within the designated region established by the image input processing means and which is at a distance in a second range from the camera, the second range being farther than the first range from the camera.
  • the second detection processing means selectively uses either moving vector determination or correlation determination as a processing technique for detecting the image of the moving object at the distance in the second range, the moving vector determination involving determining whether there exists the moving object using a moving vector, the correlation determination involving determining whether there exists the moving object using correlations between past and current images.
  • the second detection processing means may include: processing technique selection means for selecting either the moving vector determination or the correlation determination as the processing technique based on predetermined parameter; moving vector determination means configured such that if the moving vector determination is determined to be the processing technique by the processing technique selection means, then the moving vector determination means may detect the image of the moving object at the distance in the second range in accordance with the moving vector determination; and correlation determination means configured such that if the correlation determination is determined to be the processing technique by the processing technique selection means, then the correlation determination means may detect the image of the moving object at the distance in the second range in accordance with the correlation determination.
  • the second detection processing means may include brightness determination means as the predetermined parameter for use by the processing technique selection means for determining whether the brightness of the designated region is below a predetermined level; wherein, if the brightness of the designated region is determined to be above the predetermined level by the brightness determination means, then the processing technique selection means may select the moving vector determination as the processing technique; and wherein, if the brightness of the designated region is determined to be below the predetermined level by the brightness determination means, then the processing technique selection means may select the correlation determination as the processing technique.
  • brightness determination means as the predetermined parameter for use by the processing technique selection means for determining whether the brightness of the designated region is below a predetermined level
  • the moving object detection apparatus of the invention may further include external input means for inputting externally the parameter for use by the processing technique selection means; wherein, based on the parameter input by the external input means, the processing technique selection means may select either the moving vector determination or the correlation determination as the processing technique.
  • the second detection processing means may have a plurality of ranges established for the distance to the moving object to be detected; and independently in each of the plurality of ranges, the second detection processing means may select either the moving vector determination or the correlation determination as the processing technique to be used.
  • a moving object detection method representing the functionality of the above-outlined moving object detection apparatus of the invention, as well as a program functionally equivalent to the inventive moving object detection method.
  • an analysis image taken by a camera is input in order to establish a designated region inside the analysis image.
  • An image is detected of a moving object moving within the established designated region at a distance in a first range from the camera.
  • An image is also detected of the moving object moving within the established designated region at a distance in a second range from the camera, the second range being farther than the first range.
  • Either moving vector determination or correlation determination is used selectively as a processing technique for detecting the image of the moving object at the distance in the second range, the moving vector determination involving determining the presence of the moving object using a moving vector, the correlation determination involving determining the presence of the moving object using correlations between past and current images.
  • FIG. 1 is a block diagram showing a functional structure of an image analysis apparatus
  • FIG. 2 is a schematic view showing a typical analysis image subject to an image analysis process
  • FIG. 3 is a schematic view showing typically divided search ranges
  • FIG. 4 is a schematic view showing a typical moving vector search range
  • FIG. 5 is a schematic view showing a typical detection region
  • FIG. 6 is a flowchart explanatory of a typical image analysis process
  • FIG. 7 is a flowchart explanatory of a typical short-distance detection process
  • FIG. 8 is a flowchart explanatory of a typical long-distance detection process
  • FIG. 9 is a block diagram showing a typical functional configuration of a surveillance system including as one of its components an image analysis apparatus embodying the present invention.
  • FIG. 10 is a block diagram showing a typical functional configuration of a system including as one of its components the image analysis apparatus embodying the present invention.
  • FIG. 11 is a block diagram showing a typical functional configuration of another system including as one of its components the image analysis apparatus embodying the present invention.
  • FIG. 12 is a schematic view showing a typical processing technique applicable to each of four segmented search ranges
  • FIG. 13 is a block diagram showing another functional structure of the image analysis apparatus.
  • FIG. 14 is a schematic view explanatory of a specific example in which processing techniques are switched by use of an external input section.
  • FIG. 15 is a block diagram showing a typical hardware structure of a moving object detection apparatus to which the present invention is applied.
  • First embodiment an example in which the processing techniques are switched based on the brightness of the analysis image.
  • Second embodiment an example in which the processing techniques are switched using a switching instruction from the outside).
  • FIG. 1 is a block diagram showing a functional structure of an image analysis apparatus 1 as an embodiment of the moving object detection apparatus to which the present invention is applied.
  • the image analysis apparatus 1 of FIG. 1 uses an image taken by a surveillance camera of the surveillance system as an analysis image of which the data is to be analyzed, in order to determine whether an image of a moving object has moved in a region designated inside the analysis image of interest (the region will be called the designated region hereunder).
  • the series of these steps above will be called the image analysis process hereunder.
  • the image analysis apparatus 1 is made up of an image input processing section 11 , a short-distance detection processing section 12 , a long-distance detection processing section 13 , a result integration section 14 , and a result output section 15 .
  • the image input processing section 11 inputs analysis image data from the outside. Inside the analysis image in question, the region designated typically by a user as the object under surveillance is established as the designated region FS.
  • FIG. 2 is a schematic view showing a typical analysis image 41 subject to the image analysis process.
  • the designated region FS is established to the right of the center.
  • the image analysis apparatus 1 adopts a moving image as the subject of surveillance for moving object detection.
  • the moving image is made up of a plurality of unit images such as frames or fields, the unit images being arrayed in a predetermined order to constitute the moving image.
  • the data representing such unit images is input to the image input processing section 11 as analysis image data. That is, every time a unit image constituting part of a moving image is input, the first embodiment performs the image analysis process to determine whether there exists an image of the moving object.
  • the short-distance detection processing section 12 and long-distance detection processing section 13 detect whether a moving object image has moved in the designated region FS inside the analysis image of interest.
  • FIG. 3 shows typically divided search ranges taken on by the short-distance detection processing section 12 and long-distance detection processing section 13 .
  • the short-distance detection processing section 12 may adopt advantageously a technique for detecting the image of the moving object in question by determining whether the moving object image exists using correlations between current and past images. This technique will be called correlation determination hereunder.
  • the long-distance detection processing section 13 may advantageously adopt a technique for detecting the image of the moving object in question by determining whether the moving object image exists using a moving vector. This technique will be called moving vector determination hereunder. The reasons why these techniques may be utilized advantageously are explained below.
  • correlation determination entails a reduced possibility of such missed detection taking place because correlation determination involves utilizing correlations between past and current images inside the designated region FS. For these reasons, correlation determination is suitable for, and adopted by, the short-distance detection processing section 12 of the first embodiment.
  • the short-distance detection processing section 12 includes a correlation determination block 21 for detecting an image of a moving object through correlation determination.
  • the correlation determination block 21 calculates the value Rzncc in the following expression (1) using data of the analysis image currently input by the image input processing section 11 (the image is called the current image) and data of the analysis image previously input by the image input processing section 11 (the image is called the past image):
  • Rzncc ⁇ ( O - O avg ) ⁇ ( P - P avg ) ⁇ ( O - O avg ) 2 ⁇ ⁇ ( P - P avg ) 2 ( 1 )
  • the value Rzncc denotes the coefficient of normalized cross correlation.
  • the value O represents each pixel value inside the detection region of the current image.
  • the value O indicates each pixel value in the designated region FS of the current image.
  • the value Oavg stands for an average of the pixel values inside the detection region of the current image.
  • the value Oavg denotes the average of the pixel values in the designated region FS of the current image.
  • the value P represents each pixel value inside the detection region of the past image.
  • the value P indicates each pixel value in the designated region FS of the past image.
  • the value Pavg stands for an average of the pixel values inside the detection region of the past image.
  • the value Pavg denotes the average of the pixel values in the designated region FS of the past image.
  • the coefficient Rzncc of normalized cross correlation is small when the moving object is included in the detection region and is large when the moving object is not included in the detection region.
  • the correlation determination block 21 determines that an image of the moving object exists in the designated region FS if the coefficient Rzncc of normalized cross correlation is found smaller than, say, a predetermined threshold value, and that the image of the moving object does not exist in the designated region FS if the coefficient Rzncc of normalized cross correlation is found larger than the threshold value.
  • correlation determination is not appropriate where the moving object to be detected is at a long distance from the surveillance camera 61 . That is because the image of the moving object becomes smaller in size in the designated region FS the farther away from the surveillance camera 61 . In such a case, the size of the moving object inside the designated region FS becomes about the same as that of the image of, say, the trees swaying near the surveillance camera 61 . Thus with correlation determination, it is difficult to determine whether the moving object image included in the designated region FS is the image of the moving object to be detected or the image of the disturbances caused by the swaying trees or the like. Correlation determination may then result in faulty detection.
  • moving vector determination entails a reduced possibility of such erroneous detection taking place because this technique involves acquiring the moving vector of the moving object image included in the designated region FS thereby making it easy to find the moving speed of the moving object image and its moving direction inside the designated region FS. For this reason, moving vector determination is principally suitable for, and adopted by, the long-distance detection processing section 13 of the first embodiment as the processing technique.
  • the wording “moving vector determination is principally suitable” means that it is suitable as the processing technique assuming that at least a certain amount of light is secured as in the daylight.
  • the image taken by the surveillance camera 61 with an insufficient amount of light typically at night has a low level of brightness.
  • the use of the data derived from such images at low levels of brightness worsens the accuracy of calculating the moving vector of the moving object image. As a result, the accuracy of detecting the moving object image declines.
  • the long-distance detection processing section 13 adopts as its processing technique moving vector determination where the brightness of the analysis image is above a predetermined reference level, or takes up correlation determination where the brightness of the analysis image is below the reference level.
  • the images of disturbances such as swaying trees are either not included in the analysis image or may be included but at far lower levels of brightness than the image of the moving object to be detected. It follows that where the brightness of the analysis image is below the reference level, there is practically no possibility of erroneously regarding the disturbances typically caused by swaying trees or the like as the image of the moving object. Thus correlation determination may be adopted by the long-distance detection processing section 13 with little fear of faulty detection as long as the brightness of the analysis image is below the reference level.
  • the long-distance detection processing section 13 selects either moving vector determination or correlation determination as its processing technique based on the brightness of the analysis image. By use of the processing technique thus selected, the long-distance detection processing section 13 detects the image of the moving object.
  • the long-distance detection processing section 13 is made up of a brightness determination block 31 , a processing technique selection block 32 , a moving vector determination block 33 , and a correlation determination block 34 , as illustrated in FIG. 1 .
  • the brightness determination block 31 determines whether the brightness of the analysis image of interest in the designated region FS is below a predetermined reference level on the basis of the analysis image data output from the image input processing section 11 . More specifically, the brightness determination block 31 counts the number of the pixels whose brightness values are below the reference level from among the pixels inside the designated region FS. If the pixel count thus obtained is above a predetermined threshold value, the brightness determination block 31 determines that the brightness in the designated region FS is below the reference level. If the pixel count is below the threshold value, then the brightness determination block 31 determines that the brightness in the designated region FS is above the reference level.
  • the processing technique selection block 32 selects either moving vector determination or correlation determination as the processing technique. That is, if the brightness determination block 31 determines that the brightness in the designated region FS is above the reference level, the processing technique selection block 32 selects moving vector determination as the processing technique and allows the analysis image data output from the image input processing section 11 to be supplied to the moving vector determination block 33 . On the other hand, if the brightness determination block 31 determines that the brightness in the designated region FS is below the reference level, then the processing technique selection block 32 selects correlation determination as the processing technique and allows the analysis image data output from the image input processing section 11 to be sent to the correlation determination block 34 .
  • the moving vector determination block 33 detects the image of the moving object in accordance with moving vector determination. For example, the moving vector determination block 33 establishes each of the pixels of the current image in the designated region FS successively as the pixel of interest so as to establish a block surrounding the pixels of interest in the current image (the block is called the block of interest). The moving vector determination block 33 then searches a past image for a block corresponding to the block of interest (the searched-for block is called the corresponding block). The moving vector determination block 33 proceeds to detect a vector ranging from the corresponding block to the block of interest through the current and past images being overlaid (in the same coordinate system) as the moving vector of the pixels of interest.
  • FIG. 4 shows a typical search range for the corresponding block, i.e., a typical moving vector search range.
  • a search range FV surrounds the designated region FS in the analysis image 41 and is set to be larger than the designated region FS. It should be noted that the search range FV in FIG. 4 is only an example. Any other desired range in the analysis image 41 may be adopted as the search range instead.
  • block-matching algorithm is only an example and other desired techniques such as gradient methods may be adopted instead.
  • the correlation determination block 34 detects the image of the moving object in accordance with correlation determination. That is, the correlation determination block performs basically the same process as the correlation determination block 21 of the short-distance detection processing section 12 . It should be noted, however, that the detection region used to find the coefficient Rzncc of normalized cross correlation using the expression (1) above is different between the correlation determination block 34 of the long-distance detection processing section 13 and the correlation determination block 21 of the short-distance detection processing section 12 .
  • FIG. 5 shows a typical detection region used to find the coefficient Rzncc of normalized cross correlation.
  • the correlation determination block 21 of the short-distance detection processing section 12 uses the designated region FS unchanged as the detection region.
  • the correlation determination block 34 of the long-distance detection processing section 13 utilizes the region FF smaller in size than the designated region FS as the detection region.
  • the detection region is made smaller in size, the image of the moving object to be detected becomes larger in size in reverse proportion. Then it is that much easier to detect the image of the moving object farther in the distance.
  • the correlation determination block 34 of the long-distance detection processing section 13 functions as long as the brightness is below the reference level. For this reason, there is virtually no possibility of faulty detection taking place due to the disturbances such as swaying trees.
  • the functional structure of the long-distance detection processing section 13 capable of choosing between moving vector determination and correlation determination. As shown in FIG. 1 , the result of the detection performed by the long-distance detection processing section 13 is supplied to the result integration section 14 along with the result of the detection made by the short-distance detection processing section 12 .
  • the result integration section 14 integrates the detection result coming from the short-distance detection processing section 12 with the detection result from the long-distance detection processing section 13 , and sends the integrated result to the result output section 15 .
  • the result output section 15 outputs the integrated result as the definitive result of the detection carried out by the image analysis apparatus 1 .
  • the result integration section 14 acquires the integrated result indicating that there is a moving object and causes the result to be output by the result output section 15 .
  • the result integration section 14 acquires the integrated result indicating that there is no moving object and has the result output by the result output section 15 .
  • FIG. 6 is a flowchart explanatory of a typical image analysis process.
  • the image analysis apparatus 1 adopts the moving image as the subject of surveillance for moving object detection. That moving image is composed of a plurality of unit images taken at predetermined intervals by the surveillance camera 61 in FIG. 3 or the like. Thus every time the data of each of these unit images constituting the moving image is output from the surveillance camera 61 in FIG. 3 or the like, the image analysis process is carried out.
  • step S 1 the image input processing section 11 of the image analysis apparatus 1 in FIG. 1 inputs the data of a unit image as the analysis image output from the surveillance camera 61 or the like, and establishes the designated region inside the analysis image in question.
  • steps S 2 and S 3 the short-distance detection processing section 12 and long-distance detection processing section 13 perform a short-distance detection process and a long-distance detection process, respectively, in parallel.
  • the short-distance detection process refers to a series of steps carried out by the short-distance detection processing section 12 until a moving object image is detected.
  • the short-distance detection process will be discussed later in detail in reference to the flowchart of FIG. 7 .
  • the long-distance detection process refers to a series of steps performed by the long-distance detection processing section 13 until a moving object image is detected. The long-distance detection process will be explained later in detail in reference to the flowchart of FIG. 8 .
  • step S 4 the result integration section 14 integrates the results of the short-distance detection process and long-distance detection process. That is, if the results of both the short-distance detection process and the long-distance detection process indicate that there is no moving object, the integrated result says there exists no moving object. If at least either the detection result from the short-distance detection processing section 12 or the detection result from the long-distance detection processing section 13 indicates that there is a moving object, then the integrated result says there exists a moving object.
  • step S 5 the result output section 15 outputs the integrated result obtained in step S 4 as the definitive result of the detection performed by the image analysis apparatus 1 . This step concludes the image analysis process.
  • FIG. 7 is a flowchart explanatory of a typical short-distance detection process.
  • step S 21 the correlation determination block 21 of the short-distance detection processing section 12 performs correlation determination on the data of the analysis image input in step S 1 of FIG. 6 .
  • Performing correlation determination means detecting a moving object image in accordance with correlation determination.
  • step S 22 the correlation determination block 21 outputs the result of the process of correlation determination in step S 21 .
  • step S 21 If no moving object image is detected in the process of correlation determination in step S 21 , then the correlation determination block 21 goes to step S 22 and outputs the result indicating that there is no moving object.
  • This step concludes the short-distance detection process.
  • the definitive result of the image analysis process in FIG. 6 is output in step S 5 indicating that there is no moving object.
  • the definitive result of the image analysis process in FIG. 6 is output in step S 5 indicating that there exists a moving object.
  • step S 21 if a moving object image is detected in the process of correlation determination in step S 21 , then the correlation determination block 21 goes to step S 22 and outputs the result of the detection indicating that there is a moving object.
  • This step concludes the short-distance detection process.
  • the definitive result of the image analysis process in FIG. 6 is output in step S 5 indicating that there exists a moving object.
  • step S 21 After the result of the process in step S 21 is output in step S 22 , control is transferred to step S 4 in FIG. 6 .
  • FIG. 8 is a flowchart explanatory of a typical long-distance detection process.
  • step S 41 based on the data of the analysis image input in step S 1 of FIG. 6 , the brightness determination block of the long-distance detection processing section 13 determines whether the brightness of the analysis image in question is above a predetermined reference level.
  • the processing technique selection block 32 selects moving vector determination as the processing technique.
  • the processing technique selection block 32 proceeds to supply the moving vector determination block 33 with the analysis image data input in step S 1 .
  • the result of the determination in step S 41 is negative (“NO”), and control is transferred to step S 42 .
  • step S 42 the moving vector determination block 33 performs moving vector determination on the analysis image data.
  • moving vector determination means detecting a moving object image in accordance with moving vector determination.
  • step S 44 the moving vector determination block outputs the result of the process of moving vector determination in step S 42 .
  • step S 42 If no moving image data is detected in the process of moving vector determination in step S 42 , then the moving vector determination block 33 goes to step S 44 and outputs the result indicating that there is no moving object.
  • This step concludes the long-distance detection process.
  • the definitive result of the image analysis process in FIG. 6 is output in step; S 5 indicating that there is no moving object.
  • the definitive result of the image analysis process in FIG. 6 is output in step S 5 indicating that there exists a moving object.
  • step S 42 if a moving object image is detected in the process of moving vector determination in step S 42 , then the moving vector determination block 33 goes to step S 44 and outputs the result of the detection indicating that there is a moving object.
  • This step concludes the long-distance detection process.
  • the definitive result of the image analysis process in FIG. 6 is output in step S 5 indicating that there exists a moving object.
  • the processing technique selection block 32 selects correlation determination as the processing technique.
  • the processing technique selection block 32 proceeds to supply the correlation determination block 34 with the analysis image data input in step S 1 .
  • the result of the determination in step S 41 is affirmative (“YES”), and control is transferred to step S 43 .
  • step S 43 the correlation determination block 34 performs correlation determination on the data of the analysis image.
  • Performing correlation determination means detecting a moving object image in accordance with correlation determination.
  • step S 44 the correlation determination block 34 outputs the result of the process of correlation determination performed in step S 43 .
  • step S 43 If no moving image data is detected in the process of correlation determination in step S 43 , then the correlation determination block 34 goes to step S 44 and outputs the result indicating that there is no moving object.
  • This step concludes the long-distance detection process.
  • the definitive result of the image analysis process in FIG. 6 is output in step S 5 indicating that there is no moving object.
  • the definitive result of the image analysis process in FIG. 6 is output in step S 5 indicating that there exists a moving object.
  • step S 43 if a moving object image is detected in the process of correlation determination in step S 43 , then the correlation determination block 34 goes to step S 44 and outputs the result of the detection indicating that there is a moving object.
  • This step concludes the long-distance detection process.
  • the definitive result of the image analysis process in FIG. 6 is output in step S 5 indicating that there exists a moving object.
  • step S 44 the result of the process in step S 42 or S 43 is output. Control is then transferred to step S 4 in FIG. 6 .
  • the image analysis apparatus 1 can distinguish between the moving object image at a short distance and the moving object image at a long distance when detecting the image of the moving object moving inside the designated region FS of the analysis image. More specifically, when detecting the image of the moving object at a short distance, the image analysis apparatus 1 always performs correlation determination. On the other hand, if the brightness of the analysis image is above the predetermined reference level, then the image analysis apparatus 1 detects the image of the moving object at the long distance in accordance with moving vector determination. This allows the image analysis apparatus 1 to remain robust against disturbances such as the images of swaying trees in a bright environment as in the daylight, thereby enabling the apparatus 1 to detect moving object images in stable fashion.
  • the image analysis apparatus 1 switches the processing technique for detecting moving object images at long distances from moving vector determination to correlation determination. Correlation determination is more robust against brightness fluctuation than moving vector determination. Meanwhile, the disturbances such as swaying trees to which correlation determination is vulnerable are not imaged at all or may be imaged but at a very low level of brightness where the brightness of the analysis image is below the reference level in a dark environment such as at night. This provides for little possibility of faulty detection. Thus even if the brightness of the analysis image drops below the reference level in a dark environment such as at night, it is possible to secure stable detection of moving object images.
  • the image analysis apparatus 1 can be applied not only to the surveillance system discussed above but also to other diverse fields. A few typical applications of the image analysis apparatus 1 will be explained below in reference to FIGS. 9 through 11 .
  • FIG. 9 is a block diagram showing a typical functional configuration of a surveillance system 81 including as one of its components the image analysis apparatus embodying the present invention.
  • the surveillance system 81 in FIG. 9 is made up of an imaging unit 91 , an imaging signal processing unit 92 , an imaging data processing unit 93 , an image analysis unit 94 composed of the above-described image analysis apparatus 1 embodying the invention, and a transmission unit 95 .
  • the imaging unit 91 is composed of an image pickup device such as CCD (charge coupled device) or CMOS (complementary metal oxide semiconductor) and lenses. Typically, the imaging unit 91 may be the surveillance camera 61 in FIG. 3 . The imaging unit 91 takes images of the moving object or the like and outputs the resulting imaging signal.
  • CCD charge coupled device
  • CMOS complementary metal oxide semiconductor
  • the imaging signal processing unit 92 performs various image-related processes such as image correction permitting appropriate gradations, noise removal, and colorization on the imaging signal. Consequently, the imaging signal processing unit 92 outputs a digitized imaging signal, i.e., imaging data that is supplied to the imaging data processing unit 93 and image analysis unit 94 .
  • the imaging data processing unit 93 performs a process for converting the imaging data into a format in which to distribute the data of the taken image over a network. For example, the imaging data processing unit 93 carries out a compression coding process on the imaging data.
  • the image analysis unit 94 analyzes the imaging data output from the imaging signal processing unit 92 as the data of the analysis image, thereby detecting an image of a moving object moving inside the designated region FS in the analysis image of interest.
  • the transmission unit 95 multiplexes the image data encoded by the imaging data processing unit 93 with the result of the analysis performed by the image analysis unit 94 .
  • the transmission unit 95 proceeds to transmit the multiplexed result onto the network.
  • FIG. 10 is a block diagram showing a typical functional configuration of a system 111 including as one of its components the image analysis apparatus embodying the present invention, the system 111 changing the image signal coming from an external source other than the surveillance camera into a stream for distribution over the network.
  • the system 111 in FIG. 10 is made up of an image input unit 121 , an image signal processing unit 122 , an image data processing unit 123 , an image analysis unit 124 composed of the above-described image analysis apparatus 1 embodying the invention, and a transmission unit 125 .
  • the image input unit 121 inputs the image signal coming from an external source other than the surveillance camera, such as an analog camera.
  • the image signal processing unit 122 performs various image-related processes such as image correction permitting appropriate gradations, noise removal, and colorization on the image signal. As a result, the image signal processing unit 122 outputs a digitized image signal, i.e., image data that is supplied to the image data processing unit 123 and image analysis unit 124 .
  • the image data processing unit 123 like the imaging data processing unit 93 , performs a process for converting the image data into a format in which to distribute the data of the image over the network. For example, the image data processing unit 123 carries out a compression coding process on the image data.
  • the image analysis unit 124 analyzes the image data output from the image signal processing unit 122 as the data of the analysis image, thereby detecting an image of a moving object moving inside the designated region FS in the analysis image of interest.
  • the transmission unit 125 multiplexes the image data encoded by the image data processing unit 123 with the result of the analysis performed by the image analysis unit 124 .
  • the transmission unit 125 proceeds to transmit the multiplexed result onto the network.
  • FIG. 11 is a block diagram showing a typical functional configuration of another system 141 including as one of its components the image analysis apparatus embodying the present invention.
  • the system 141 in FIG. 11 is a system that includes a recorder for storing processed signals regardless of the difference in format between analog and digital signals (data), a dedicated device for outputting an alarm based on the processed signals, and a personal computer.
  • the system 141 in FIG. 11 is made up of an image input unit 151 , an image signal processing unit 152 , an image analysis unit 153 composed of the above-described image analysis apparatus 1 embodying the invention, and a transmission unit 154 .
  • the image input unit 151 inputs the image signal coming from an external source other than the surveillance camera.
  • the image signal processing unit 152 performs various image-related processes such as image correction permitting appropriate gradations, noise removal, and colorization on the image signal. As a result, the image signal processing unit 152 outputs a digitized image signal, i.e., image data that is supplied to the image analysis unit 153 .
  • the image analysis unit 153 analyzes the image data output from the image signal processing unit 152 as the data of the analysis image, thereby detecting an image of a moving object moving inside the designated region FS in the analysis image of interest.
  • the transmission unit 154 transmits the result of the analysis coming from the image analysis unit 153 onto the network.
  • the first embodiment of the present invention can thus be applied to diverse fields and may also be practiced in many applications other than those described above.
  • search ranges D 1 and D 2 there are two search ranges D 1 and D 2 in which to search for the image of the moving object as illustrated in FIG. 3 .
  • the number of search ranges for the detection of the moving object image is not limited to two but may be selected as desired.
  • FIG. 12 is a schematic view showing a typical processing technique applicable to each of four segmented search ranges.
  • the search range D 1 at a short distance within a predetermined distance from the surveillance camera 61 is the same as the search range D 1 indicated in FIG. 3 .
  • the search range D 2 at a long distance shown in FIG. 3 is segmented into three search ranges, i.e., a second search range D 21 , a third search range D 22 , and a fourth search range D 23 in order of increasing distance from the surveillance camera 61 .
  • a search is made independently for an image of a moving object.
  • each of the second search range D 21 , the third search range D 22 and the fourth search range D 23 it is determined independently whether the brightness of the analysis image is below a predetermined reference level.
  • moving vector determination is adopted as the processing technique.
  • correlation determination is adopted as the processing technique.
  • the designated region FS is used unchanged as the detection region in the short-distance search range D 1 .
  • the region FF smaller in size than the designated region FS is used as the detection region.
  • the region FF for use in the second search range D 21 , the third search range D 22 and the fourth search range D 23 as the detection region takes the form of a region FF 1 , a region FF 2 and a region FF 3 , respectively.
  • the regions FF 1 , FF 2 and FF 3 each serving as the detection region diminish progressively in size in that order. That is, the farther away from the surveillance camera 61 , the smaller the detection region is arranged to be in size. This arrangement permits detection of the image of a moving object that is farther away from the surveillance camera than ever.
  • function blocks not shown, that are each functionally and structurally equivalent to the long-distance detection processing section 13 in FIG. 1 in principle are provided independently to deal with the second search range D 21 , the third search range D 22 and the fourth search range D 23 , each of the function blocks permitting detection of the moving object in the corresponding search range.
  • the technique of changing the resolution of the image depending on the search range may be adopted. If this technique is utilized, it is possible to apply low resolution to, say, the second search range D 21 while using high resolution in the third search range D 22 and the fourth search range D 23 .
  • the technique of varying frame rate depending on the search range may be adopted. If this technique is utilized, it is possible to apply high frame rate to, say, the second and the third search ranges D 21 and D 22 while using low frame rate in the fourth search range D 23 .
  • the above-described first embodiment was shown switching the processing techniques for detecting the image of the moving object that is at a long distance in accordance with the brightness of the analysis image of interest.
  • the brightness of the analysis image is not the only parameter for use with the processing technique for detecting the image of a long-distance moving object. Any other suitable parameter may be adopted instead.
  • FIG. 13 is a block diagram showing a functional structure of another image analysis apparatus 161 serving as the moving object detection apparatus embodying the present invention, the image analysis apparatus 161 utilizing a parameter different from that which is used by the image analysis apparatus 1 in FIG. 1 when switching the processing techniques for detecting the image of a moving object at a long distance.
  • the image analysis apparatus 161 in FIG. 13 is made up of an image input processing section 181 , a short-distance detection processing section 182 , an external input section 183 , a long-distance detection processing section 184 , a result integration section 185 , and a result output section 186 .
  • the short-distance detection processing section 182 is composed of a correlation determination block 191 .
  • the long-distance detection processing section 184 is constituted by a processing technique selection block 201 , a moving vector determination block 202 , and a correlation determination block 203 .
  • processing technique selection block 201 i.e., the processing technique selection block 201 , moving vector determination block 202 and correlation determination block 203
  • processing technique selection block 32 moving vector determination block 33
  • correlation determination block 34 of the long-distance detection processing section 13 in FIG. 1 , respectively. That is, those components of the image analysis apparatus 161 in FIG. 13 which are described in this paragraph match the components of the image analysis apparatus 1 in FIG. 1 . The matching components will not be discussed further in order to avoid redundancy.
  • the image analysis apparatus 161 in FIG. 13 is different from the image analysis apparatus 1 in FIG. 1 in the following points: that the parameter by which the processing technique selection block 32 in FIG. 1 selects the processing technique is given by the brightness determination block 31 , whereas the parameter by which the processing technique selection block 201 in FIG. 13 selects the processing technique is supplied by the external input section 183 .
  • the image analysis apparatus 161 in FIG. 13 differs from its counterpart in FIG. 1 in that it has the external input section 183 replacing the brightness determination block 31 shown in FIG. 1 .
  • the external input section 183 inputs a processing technique switching instruction from the outside and notifies the processing technique selection block 201 of the input instruction.
  • the processing technique selection block 201 selects either moving vector determination or correlation determination as the processing technique. That is, when notified of the instruction for switching to moving vector determination by the external input section 183 , the processing technique selection block 201 selects moving vector determination as the processing technique and supplies the moving vector determination block 202 with the analysis image data output from the image input processing section 181 . On the other hand, when notified of the instruction for switching to correlation determination by the external input section 183 , the processing technique selection block 201 selects correlation determination as the processing technique and supplies the correlation determination block 203 with the analysis image data output from the image input processing section 181 .
  • FIG. 14 is a schematic view explanatory of a specific example in which the processing techniques are switched by use of the external input section 183 described above.
  • a light source 211 such as a light is located in front of the lens of the surveillance camera 61 . That is, the light source 211 is supposed to be positioned in a manner facing the surveillance camera 61 from far away.
  • moving vector determination is used as the processing technique of the long-distance detection processing section 13 in the image analysis apparatus 1 of FIG. 1 .
  • the image analysis apparatus 161 of FIG. 13 causes an instruction for switching to correlation determination to be input to the external input section 183 in order to use correlation determination as the processing technique of the long-distance detection processing section 184 .
  • the image analysis apparatus 161 of FIG. 13 causes the instruction for switching to correlation determination to be input to the external input section 183 so that correlation determination will be used as the processing technique of the long-distance detection processing section 184 .
  • a personal computer such as one shown in FIG. 15 may be used at least as part of the above-described moving object detection apparatus.
  • a CPU (central processing unit) 301 performs various processes in accordance with programs recorded in a ROM (read only memory) 302 or in keeping with programs that are loaded from a storage section 308 into a RAM (random access memory) 303 .
  • the RAM 303 may also accommodate data necessary for the CPU 301 to carry out its diverse processing.
  • the CPU 301 , ROM 302 , and RAM 303 are interconnected via a bus 304 .
  • An input/output interface 305 is also connected to the bus 304 .
  • the input/output interface 305 is connected with an input section 306 typically made up of a keyboard and a mouse and with an output section 307 usually composed of a display.
  • the input/output interface 305 is also connected with the storage section 308 such as a hard disk and with a communication section 309 generally made up of a modem and a terminal adapter.
  • the communication section 309 controls communications conducted with other apparatuses (not shown) over networks including the Internet.
  • a drive 310 is connected as needed to the input/output interface 305 .
  • a piece of removable media 311 such as magnetic disks, optical disks, magneto-optical disks or semiconductor memory may be loaded into the drive 310 .
  • the computer programs retrieved from the loaded removable medium are installed as needed into the storage section 308 .
  • the programs constituting the software may be either retrieved from dedicated hardware of the computer in use or installed over networks or from a suitable recording medium into a general-purpose computer or like equipment capable of executing diverse functions based on the installed programs.
  • the program recording medium carrying these programs is offered to users not only as the removable media (package media) 311 apart from their apparatuses and constituted by magnetic disks (including floppy disks), optical disks (including CD-ROM (compact disk-read only memory), DVD (digital versatile disk) and Blu-ray disks), magneto-optical disks (including MD (Mini-disk)), or semiconductor memories; but also in the form of the ROM 302 or the hard disk in the storage device 308 , each accommodating the programs and incorporated beforehand in the users' apparatuses.
  • magnetic disks including floppy disks
  • optical disks including CD-ROM (compact disk-read only memory), DVD (digital versatile disk) and Blu-ray disks
  • magneto-optical disks including MD (Mini-disk)
  • semiconductor memories but also in the form of the ROM 302 or the hard disk in the storage device 308 , each accommodating the programs and incorporated beforehand in the users' apparatuses.
  • the steps describing the programs stored on the storage medium represent not only the processes that are to be carried out in the depicted sequence (i.e., on a time series basis) but also processes that may be performed parallelly or individually and not necessarily chronologically.
  • the present invention can be applied to apparatuses which include an analysis section for analyzing image data, such as a surveillance camera, a personal computer, or a dedicated alarm output device and which are capable of detecting images of a moving object.
  • an analysis section for analyzing image data such as a surveillance camera, a personal computer, or a dedicated alarm output device and which are capable of detecting images of a moving object.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)
  • Closed-Circuit Television Systems (AREA)
US13/065,374 2010-03-30 2011-03-21 Moving object detection apparatus, moving object detection method, and program Abandoned US20110243385A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2010079652A JP2011210179A (ja) 2010-03-30 2010-03-30 移動物体検出装置及び方法、並びにプログラム
JPP2010-079652 2010-03-30

Publications (1)

Publication Number Publication Date
US20110243385A1 true US20110243385A1 (en) 2011-10-06

Family

ID=44697868

Family Applications (1)

Application Number Title Priority Date Filing Date
US13/065,374 Abandoned US20110243385A1 (en) 2010-03-30 2011-03-21 Moving object detection apparatus, moving object detection method, and program

Country Status (3)

Country Link
US (1) US20110243385A1 (enExample)
JP (1) JP2011210179A (enExample)
CN (1) CN102209233A (enExample)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110135158A1 (en) * 2009-12-08 2011-06-09 Nishino Katsuaki Image processing device, image processing method and program
CN103177454A (zh) * 2011-12-24 2013-06-26 南京理工大学常熟研究院有限公司 一种动态图像运动目标检测方法
EP2699021A1 (en) 2012-08-13 2014-02-19 Starkey Laboratories, Inc. Method and apparatus for own-voice sensing in a hearing assistance device
CN104376580A (zh) * 2014-11-21 2015-02-25 西安理工大学 一种视频摘要中非关注区域事件的处理方法
US9245187B1 (en) 2014-07-07 2016-01-26 Geo Semiconductor Inc. System and method for robust motion detection
US20160036882A1 (en) * 2013-10-29 2016-02-04 Hua Zhong University Of Science Technology Simulataneous metadata extraction of moving objects
US20160225154A1 (en) * 2015-01-29 2016-08-04 Samsung Electronics Co., Ltd. Method and apparatus for determining eye position information
US9635475B2 (en) 2013-05-01 2017-04-25 Starkey Laboratories, Inc. Hearing assistance device with balanced feed-line for antenna

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103096185B (zh) * 2012-12-30 2016-04-20 信帧电子技术（北京）有限公司 一种视频摘要生成方法和装置
CN103092925B (zh) * 2012-12-30 2016-02-17 信帧电子技术（北京）有限公司 一种视频摘要生成方法和装置

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5812193A (en) * 1992-11-07 1998-09-22 Sony Corporation Video camera system which automatically follows subject changes
US20040101162A1 (en) * 2002-11-19 2004-05-27 Honda Motor Co., Ltd. Moving object detection device, moving object detection method, and moving object detection program
US6812835B2 (en) * 2000-02-28 2004-11-02 Hitachi Kokusai Electric Inc. Intruding object monitoring method and intruding object monitoring system
US20080137913A1 (en) * 2005-01-07 2008-06-12 Gesture Tek, Inc. Optical Flow Based Tilt Sensor
US20080252723A1 (en) * 2007-02-23 2008-10-16 Johnson Controls Technology Company Video processing systems and methods
US20100265331A1 (en) * 2005-09-20 2010-10-21 Fujinon Corporation Surveillance camera apparatus and surveillance camera system
US20110228987A1 (en) * 2008-10-27 2011-09-22 Masahiro Iwasaki Moving object detection method and moving object detection apparatus

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3926572B2 (ja) * 2001-03-02 2007-06-06 株式会社日立製作所 画像監視方法、画像監視装置及び記憶媒体
JP4404822B2 (ja) * 2004-08-31 2010-01-27 三洋電機株式会社 手ぶれ補正装置および撮像機器
JP4623368B2 (ja) * 2005-03-16 2011-02-02 ソニー株式会社 移動物体検出装置、移動物体検出方法及びプログラム
JP2007064894A (ja) * 2005-09-01 2007-03-15 Fujitsu Ten Ltd 物体検出装置、物体検出方法および物体検出プログラム
JP2007078358A (ja) * 2005-09-09 2007-03-29 Sumitomo Electric Ind Ltd 移動体計測装置、信号制御システム、移動体計測方法及びコンピュータプログラム
CN1937766A (zh) * 2005-09-20 2007-03-28 富士能株式会社 监视摄像机装置及监视摄像机系统
JP4544334B2 (ja) * 2008-04-15 2010-09-15 ソニー株式会社 画像処理装置および画像処理方法

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5812193A (en) * 1992-11-07 1998-09-22 Sony Corporation Video camera system which automatically follows subject changes
US6812835B2 (en) * 2000-02-28 2004-11-02 Hitachi Kokusai Electric Inc. Intruding object monitoring method and intruding object monitoring system
US20040101162A1 (en) * 2002-11-19 2004-05-27 Honda Motor Co., Ltd. Moving object detection device, moving object detection method, and moving object detection program
US20080137913A1 (en) * 2005-01-07 2008-06-12 Gesture Tek, Inc. Optical Flow Based Tilt Sensor
US20100265331A1 (en) * 2005-09-20 2010-10-21 Fujinon Corporation Surveillance camera apparatus and surveillance camera system
US20080252723A1 (en) * 2007-02-23 2008-10-16 Johnson Controls Technology Company Video processing systems and methods
US8358342B2 (en) * 2007-02-23 2013-01-22 Johnson Controls Technology Company Video processing systems and methods
US20110228987A1 (en) * 2008-10-27 2011-09-22 Masahiro Iwasaki Moving object detection method and moving object detection apparatus

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110135158A1 (en) * 2009-12-08 2011-06-09 Nishino Katsuaki Image processing device, image processing method and program
US8630453B2 (en) * 2009-12-08 2014-01-14 Sony Corporation Image processing device, image processing method and program
CN103177454A (zh) * 2011-12-24 2013-06-26 南京理工大学常熟研究院有限公司 一种动态图像运动目标检测方法
EP2699021A1 (en) 2012-08-13 2014-02-19 Starkey Laboratories, Inc. Method and apparatus for own-voice sensing in a hearing assistance device
US10231066B2 (en) 2013-05-01 2019-03-12 Starkey Laboratories, Inc. Hearing assistance device with balanced feed-line for antenna
US9635475B2 (en) 2013-05-01 2017-04-25 Starkey Laboratories, Inc. Hearing assistance device with balanced feed-line for antenna
US20160036882A1 (en) * 2013-10-29 2016-02-04 Hua Zhong University Of Science Technology Simulataneous metadata extraction of moving objects
US9390513B2 (en) * 2013-10-29 2016-07-12 Hua Zhong University Of Science Technology Simultaneous metadata extraction of moving objects
US9390333B2 (en) 2014-07-07 2016-07-12 Geo Semiconductor Inc. System and method for robust motion detection
US9245187B1 (en) 2014-07-07 2016-01-26 Geo Semiconductor Inc. System and method for robust motion detection
CN104376580A (zh) * 2014-11-21 2015-02-25 西安理工大学 一种视频摘要中非关注区域事件的处理方法
US20160225154A1 (en) * 2015-01-29 2016-08-04 Samsung Electronics Co., Ltd. Method and apparatus for determining eye position information
US9953247B2 (en) * 2015-01-29 2018-04-24 Samsung Electronics Co., Ltd. Method and apparatus for determining eye position information

Also Published As

Publication number Publication date
CN102209233A (zh) 2011-10-05
JP2011210179A (ja) 2011-10-20

Similar Documents

Publication Publication Date Title
US20110243385A1 (en) Moving object detection apparatus, moving object detection method, and program
KR101861722B1 (ko) 비디오 데이터를 처리하는 방법과 이미지 처리 회로
JP4847165B2 (ja) 映像記録再生方法及び映像記録再生装置
EP2293540B1 (en) Image processing apparatus and method for detection and correction of camera shake
KR100252080B1 (ko) 비트 플레인 정합을 이용한 입력영상의 움직임 보정을 통한 영상안정화 장치 및 그에 따른 영상 안정화방법
US20190230269A1 (en) Monitoring camera, method of controlling monitoring camera, and non-transitory computer-readable storage medium
JP5644097B2 (ja) 画像処理装置、画像処理方法及びプログラム
US8792681B2 (en) Imaging system and imaging method
US8355063B2 (en) Camera noise reduction for machine vision systems
KR20170038144A (ko) 어텐션 검출 장치 및 어텐션 검출 방법
US8436906B2 (en) Image processing apparatus, image processing method, program, and recording medium
EP2261858B1 (en) Method and apparatus for determining presence of user's hand tremor or intentional motion
US20120327254A1 (en) Image stabilization system
JP5155110B2 (ja) 監視装置
EP1503578A1 (en) Motion detector, image processing system, motion detecting method, program, and recording medium
US8160151B2 (en) Motion vector detection apparatus, motion vector processing method and program
EP2827580B1 (en) Flicker detection method and flicker detection apparatus
JPWO2018037479A1 (ja) 画像処理装置、ステレオカメラ装置及び画像処理方法
US20210243351A1 (en) Image processing device, image processing method, and program
CN106454066B (zh) 图像处理设备及其控制方法
JP2018180714A (ja) 動体検知システム、動体検知装置、動体検知方法、およびプログラム
KR102369615B1 (ko) 영상의 사전 장애검출 시스템
JP4541335B2 (ja) 画像処理装置
JP2011123562A (ja) 画像処理装置、画像処理方法及びプログラム
KR102131903B1 (ko) 모션 벡터 추정 방법 및 모션 벡터 추정 장치

Legal Events

Date Code Title Description
AS Assignment

Owner name: SONY CORPORATION, JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:NISHINO, KATSUAKI;TSUNASHIMA, NOBUHIRO;REEL/FRAME:026040/0578

Effective date: 20110301

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO PAY ISSUE FEE