US20200175692A9 - Low-complexity motion detection based on image edges - Google Patents
Low-complexity motion detection based on image edges Download PDFInfo
- Publication number
- US20200175692A9 US20200175692A9 US14/828,733 US201514828733A US2020175692A9 US 20200175692 A9 US20200175692 A9 US 20200175692A9 US 201514828733 A US201514828733 A US 201514828733A US 2020175692 A9 US2020175692 A9 US 2020175692A9
- Authority
- US
- United States
- Prior art keywords
- image
- pixel value
- pixels
- difference
- edge
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001514 detection method Methods 0.000 title description 10
- 238000000034 method Methods 0.000 claims description 52
- 230000008859 change Effects 0.000 claims description 25
- 230000015654 memory Effects 0.000 claims description 13
- 230000003628 erosive effect Effects 0.000 claims description 8
- 238000012544 monitoring process Methods 0.000 claims description 5
- 230000008569 process Effects 0.000 description 18
- 238000005516 engineering process Methods 0.000 description 9
- 238000012545 processing Methods 0.000 description 9
- 230000003287 optical effect Effects 0.000 description 8
- 238000004891 communication Methods 0.000 description 4
- 239000013598 vector Substances 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000002123 temporal effect Effects 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000009472 formulation Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012806 monitoring device Methods 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000011410 subtraction method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G06K9/4604—
-
- G06K9/6215—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/40—Analysis of texture
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30232—Surveillance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30242—Counting objects in image
Definitions
- the present disclosure generally relates to the field of digital images and, more specifically, to detecting motion based on data at the edges of digital images.
- Video monitoring devices monitor premises for various purposes, including, e.g., security monitoring, infant or elderly monitoring, videoconferencing, etc.
- the video feed may be constantly or periodically monitored by security personnel, and/or recorded for later review.
- motion detection technology that can automatically detect moving events in the video may decrease monitoring costs and improve the efficiency and sensitivity of monitoring by alerting or otherwise providing notice of significant events.
- the motion detector may begin a video feed, and/or begin recording the video, or otherwise alert security personnel of the potential intrusion.
- Motion detection technology has been playing an increasingly important role in the technology of video recording. Recently, small-sized, portable, lightweight smart video recorders that incorporate motion detection technology have been developed.
- Traditional motion detection methods can generally be categorized into two groups: methods based on pixel values and methods based on the optical flow of pixels.
- the methods based on pixel values detect motion information in the video frames by detecting changes in values of a pixel or group of pixels over time.
- This type of method may include, for example, background subtraction or temporal frame differencing. Background subtraction methods first generate a background image based on historical image data and then detect motion by comparing the current frame with the background image. Temporal frame differencing methods calculate pixel value differences between two or three successive frames and detect motion areas in which the calculated pixel value differences meets or exceeds a threshold.
- a motion vector (i.e., a flow vector) is assigned to each pixel of the image. If there is no moving object, the change in the motion vectors in the whole image is continuous. On the other hand, if there is at least one moving object, the motion vectors of the moving object(s) are different from that of neighboring area. The area(s) of the moving object(s) may be detected based on such differences.
- the methods based on pixel values generally rely on less complex algorithms than the methods based on optical flow, but may deliver lower accuracy.
- the methods based on optical flow may provide better performance, they usually require hardware with high computational capacity in order to analyze images in real time, which is difficult implemented on small-sized, portable, lightweight video recorders.
- optical flow methods Another advantage of the optical flow methods is that false detections from light interference may be reduced or avoided.
- Methods based on pixel values can be very sensitive to image changes due to some irrelevant motion, such as wind, false motion due to light reflections, or other interference, as described below.
- optical flow methods may be very sensitive to image noise.
- they may require extensive computational resources because of the high complexity of the algorithm. Accordingly, real-time processing video frames using an optical flow method may require specialized hardware.
- a false alarm may be generated when the ambient light in the view of the video recorder is suddenly changed, for example, when room lighting is turned on or off.
- traditional technology may determine the areas of the image in which pixel values have changed; if all (or a majority of) the pixel values in the image have changed, such change may be attributed to the ambient lighting and an alert may not be generated.
- a problem with this approach is that a motion event causing changes in majority of the pixels (e.g., motion close to the video camera) may be missed.
- Traditional motion detection technology may give false alarms for image changes due to irrelevant movement, rather than movement in which the monitor may be interested. For example, images may change due to the motion of trees or plants due to wind. To reduce this type of false alarm, traditional motion detection technology may determine the number of the pixels whose values have changed; if the calculated number is lower than a threshold, an alarm may not be generated. A problem with this approach is that it may miss some relevant motion that causes only a small change in the image, such as a moving object that is far away from the video camera.
- an image sensor While capturing images, an image sensor may generate noise due to many factors, including environmental factors (e.g., light conditions etc.) and hardware limitations (e.g., sensor temperature, etc.). Noise may be distributed across the whole image, and may cause changes that can be misinterpreted as motion. Traditional motion detection technology may generate a false alarm due to such noise.
- environmental factors e.g., light conditions etc.
- hardware limitations e.g., sensor temperature, etc.
- a method for detecting a motion event may include acquiring a first image; obtaining a comparison image; identifying edge pixels in the first image; generating a first edge image by assigning a first pixel value to the edge pixels of the first image and a second pixel value to non-edge pixels of the first image, the first pixel value being different with the second pixel value; identifying edge pixels in the comparison image; generating a comparison edge image by assigning the first pixel value to the edge pixels of the comparison image and the second pixel value to non-edge pixels of the comparison image; determining difference between the first edge image and the comparison edge image; determining whether the difference meets or exceeds a first threshold; and indicating that a motion event occurs if the difference meets or exceeds the first threshold.
- a device or system for detecting a motion event includes a processor and a memory storing programming for execution by the processor.
- the processor is configured for the device or system to acquire a first image; obtain a comparison image; identify edge pixels in the first image; generate a first edge image by assigning a first pixel value to the edge pixels of the first image and a second pixel value to non-edge pixels of the first image, the first pixel value being different with the second pixel value; identify edge pixels in the comparison image; generate a comparison edge image by assigning the first pixel value to the edge pixels of the comparison image and the second pixel value to non-edge pixels of the comparison image; determine difference between the first edge image and the comparison edge image; determine whether the difference meets or exceeds a first threshold; and indicate that a motion event occurs if the difference meets or exceeds the first threshold.
- non-transitory computer-readable storage media may store program instructions, which are executed by at least one processor and perform any of the methods described herein.
- FIG. 1 is a flowchart of an exemplary process for detecting a motion event according to some embodiments
- FIG. 2 is a flowchart of an exemplary process for acquiring and processing images according to some embodiments
- FIGS. 3(A)-3(F) are exemplary images according to some embodiments.
- FIG. 4 is a flowchart of an exemplary process for generating an edge image according to some embodiments
- FIG. 5 is a diagram illustrating a pixel and its neighboring pixels according to some embodiments.
- FIG. 6 is a flowchart of an exemplary erosion process according to some embodiments.
- FIG. 7 is a flowchart of an exemplary process for detecting differences between images according to some embodiments.
- FIG. 8 illustrates an exemplary process for computing the distance between pixels
- FIG. 9 is a block diagram of an exemplary system for detecting a motion event according to some embodiments.
- Embodiments of the present disclosure provide methods, systems, and articles of manufacture, such as non-transitory computer-readable media, for detecting a motion event based on image edges.
- Edges are lines of points in a digital image where discontinuities occur, such as a sharp change in color or brightness.
- FIG. 1 is a flowchart of an exemplary process for detecting a motion event.
- a first image and a comparison image may be acquired.
- the first image may be an image or image frame captured by an image sensor in real time.
- the first image may be an image or age frame stored in memory.
- the comparison image may be an image or image frame captured by the image sensor earlier than the first image.
- the comparison image may be a reference image or image frame stored in memory.
- the comparison image may be a processed image as disclosed below.
- the comparison image may be acquired by, for example, the methods discussed later with regard to step 107 .
- the first image and the comparison image may be obtained by extracting them from a video file.
- the first image and the comparison image may have a same resolution. In some embodiments, only a portion of the first image and a corresponding portion of the comparison image may be acquired.
- FIG. 3(A) is an exemplary image of the first image
- FIG. 3(B) is an exemplary image of the comparison image.
- the first image i.e., FIG. 3(A) is an image frame captured after the comparison image, i.e., FIG. 3(B) .
- the first image and comparison image may be processed. Alternatively, only a portion of the first image and the corresponding portion of the comparison image may be processed. In some embodiments, the processed first image and processed comparison image may be saved in the memory for future use. In some embodiments, the first image and the comparison image may be upscaled or downscaled. Additionally or alternatively, certain pixel information (e.g., color information) may be removed from the first image and comparison image to reduce computing requirement.
- pixel information e.g., color information
- the processed first image and the processed comparison image may be edge images generated using an edge-identifying process.
- edge pixels in the first image and comparison image may be identified by a convolution of a 2-D filter/kernel such as Roberts Cross or a Sobel formulation. Pixel information of non-edge pixels of the images may be removed.
- the first image and comparison images may be processed to generate the processed first image and comparison image, based on a process for detecting edge pixels illustrated in FIG. 4 .
- FIG. 4 is an exemplary flowchart of a process for detecting edge pixels and generating an edge image (i.e., a processed image).
- an edge image i.e., a processed image.
- the pixel values of its neighboring pixels may be obtained. For example, for pixel 501 (with the pixel value p(x, y)) shown in FIG.
- the pixel values of the above pixel 503 and the bottom pixel 508 i.e., p(x, y ⁇ 1) and p(x, y+1), respectively
- the pixel values of the left pixel 505 and the right pixel 506 i.e., p(x ⁇ 1, y) and p(x+1, y), respectively
- the difference in pixel value between any two of the neighboring pixels may be computed.
- the difference in pixel value between the neighboring pixels along the vertical direction, and/or the difference in pixel value between the neighboring pixels along the horizontal direction may be computed.
- the differences Delta h and Delta v between the neighboring pixels may be computed based on Equations 1 and 2 (in connection to FIG. 5 ) as follows:
- the threshold may be pre-determined. In some embodiments, the threshold may be adjusted by a user. The threshold can be any number between the minimum and maximum image pixel values. In other embodiments, the threshold may be any number between the lowest and highest possible pixel values for the image. In some embodiments, the minimum or lowest possible pixel value is 0 (black), and the maximum or highest possible pixel value is 255 (white). In some embodiments, the threshold may be any number between 1 and 255. In other embodiments, the threshold may be restricted to subranges 1 to 50, 51 to 100, 101 to 150, 151 to 200, or 201 to 255.
- the pixel e.g., pixel 501
- a new pixel value (a first pixel value) may then be assigned to this identified edge pixel.
- the first pixel value may be any number between the minimum pixel value and the maximum pixel value among the pixels in the image. In other embodiments, the first pixel value may be any number between lowest and highest possible pixel values for the image. In some embodiments, the first pixel value may be the maximum or the highest possible pixel value.
- the pixel e.g., pixel 501
- the pixel may be identified as a non-edge pixel.
- a second pixel value may then be assigned to this non-edge pixel.
- the second pixel value may be any number between the minimum pixel value and the maximum pixel among the pixels in the image.
- the first pixel value may be any number between lowest and highest possible pixel values for the image.
- the second pixel value may be the minimum or lowest possible pixel value. In some embodiments, the minimum or lowest possible pixel value is 0.
- FIG. 3(C) and FIG. 3(D) show examples of the processed first image (i.e., the first edge image) and processed comparison image (i.e., the comparison edge image), respectively, generated according to some embodiments disclosed herein, with the threshold being 50, the first pixel value being 255 (white), and the second pixel value being 0 (black).
- the process of FIG. 1 may be modified as illustrated in FIG. 2 .
- the first image is acquired.
- the acquired first image may be processed, as described elsewhere in this disclosure.
- the threshold period of time may be pre-determined, or may be adjusted by a user. In some embodiments, the threshold may be any period of time between 0 and 1000 milliseconds. In other embodiments, depending upon the application, the threshold may be restricted to subranges of 0 to 200 milliseconds, 200 to 400 milliseconds, 400 to 600 milliseconds, 600 to 800 milliseconds, 800 to 1000 milliseconds, 0 to 1 second, 1 to 20 seconds, 20 to 40 seconds, 40 to 60 seconds, 0 to 1 minute, 1 to 5 minutes, 5 to 10 minutes, etc. If the time that has elapsed since acquiring the first image (e.g., the image shown in FIG.
- a comparison image or a processed comparison image (e.g., the image shown in FIG. 3(B) or the image shown in FIG. 3(D) ), which was acquired by the camera before the first image may not be obtained until the time interval (t 2 -t 1 ) reaches or exceeds the threshold period of time.
- a comparison image may be obtained (at 101 b ) and processed (at 102 b ), as described elsewhere in this disclosure with respect to steps 101 and 102 , respectively. Processing may then continue as described below with respect to step 103 .
- FIG. 3(E) is an exemplary difference image generated based on the processed first image ( FIG. 3(D) ) and processed comparison image ( FIG. 3(C) ) according to some embodiments disclosed herein.
- the processed comparison image may be subtracted from the processed first image (or vice versa) to obtain a difference image.
- a difference image may be generated by assigning a first pixel value to the pixels in the processed first image that have a different pixel value from the corresponding pixels in the processed comparison image, and by assigning a second pixel value to the pixels in the processed first image that have the same pixel value as the corresponding pixels in the processed comparison image.
- the first pixel value and second pixel value may be any number between the minimum and maximum pixel values among the pixels of the images.
- the first pixel value and second pixel value may be any number between the lowest possible and highest possible pixel values for the images.
- the first pixel value may be the maximum or highest possible pixel value
- the second pixel value may be the minimum or lowest possible pixel value.
- the maximum or highest possible pixel value may be 255 (white), and the minimum or lowest possible pixel value may be 0 (black).
- noise from the processed first image and the processed comparison image may be reduced before the difference image is generated.
- FIG. 3(F) is an exemplary denoised difference image generated by reducing the noise from the difference image ( FIG. 3(E) ), in accordance with some embodiments disclosed herein.
- an erosion operation may be performed on the difference image to reduce noise as illustrated in FIG. 6 .
- the difference image may be a denoised by a noise reducing process as described elsewhere in this disclosure.
- a third party e.g., the police or responsible security personnel
- the previously processed first image (“old” first image) or the previously processed comparison image (“old” comparison image) may be assigned as the processed “new” comparison image.
- Steps 103 through 106 may also be performed, as described elsewhere in this disclosure, using the new first image.
- steps 104 and 105 may be performed based on a process illustrated in FIG. 7 .
- a difference image is generated based on the processed first image and the processed comparison images, as described elsewhere in this disclosure.
- Noise from the processed first image and comparison image may be reduced before the difference image is generated and the difference is determined.
- an erosion process may be performed on the difference image as illustrated in FIG. 6 to reduce noise.
- FIG. 6 is a flowchart of an exemplary erosion process.
- the pixel values of the pixel and a plurality of its neighboring pixels may be obtained. For instance, for pixel 501 as shown in FIG. 5 , the pixel values of pixels 501 , 502 , 503 , and 504 (i.e., p(x, y), p(x ⁇ 1, y ⁇ 1), p(x, y ⁇ 1), and p(x ⁇ 1, y), respectively) may be obtained.
- a minimum value among the obtained pixel values may be determined. For example, the minimum value (or pe(x, y)) may be determined based on the following equation:
- pe ( x, y ) min ⁇ p ( x, y ), p ( x ⁇ 1, y ⁇ 1), p ( x, y ⁇ 1), p ( x ⁇ 1, y ) ⁇ (3)
- the minimum value pe(x, y) may be assigned to the pixel 501 .
- a denoised difference image may also be generated at the end of the erosion operation. In some embodiments, the erosion process on the difference image may be performed more than once to obtain a denoised difference image.
- a number count of the pixels in the difference image (or a portion thereof) that have a pixel value equal to or greater than a threshold pixel value may be determined.
- the threshold pixel value may be in the range or subranges described above in relation to FIG. 1 .
- the threshold may be pre-determined. In other embodiments, the threshold may be adjusted by a user.
- the threshold may be any number. Depending on the application, for example, the threshold may be chosen to lie in the subrange between 1 10, 11 to 20, 21 to 30, 31 to 40, or 41 to 50, etc. If the number count does not exceed the threshold, it may be indicated that no motion event has been detected. Alternatively, no indication will be given unless the number count meets or exceeds the threshold. If the number count meets or exceeds the threshold, an indication that a motion event has occurred may be given at 105 . An alert indicating the motion event may be also generated, as described above.
- steps 702 and 703 may be skipped. Processing may then continue as described elsewhere in this disclosure, with respect to step 106 .
- a change area in the difference image may be determined at 704 .
- the change may be determined by calculating an area comprising the pixels having the first pixel value, assigned above in step 404 .
- the change area may be estimated based on a distance (e.g., the maximum distance) between the pixels with the maximum and minimum pixel values that are greater than 0, among all pixels in the difference image (or a portion thereof).
- FIG. 8 illustrates an exemplary process for computing the distance in the vertical and/or horizontal direction between the pixels with maximum and minimum pixel values.
- the pixels in the difference image that have a pixel value greater than 0 may be obtained.
- the coordinates of the pixels with the maximum and minimum pixel values that are greater than 0 may be determined.
- the coordinates e.g. max_x and min_x in the horizontal direction and max_y and min_y in the vertical direction in the following equations
- the coordinates e.g. max_x and min_x in the horizontal direction and max_y and min_y in the vertical direction
- X ⁇ the coordinates in the horizontal direction of the pixels with a pixel value greater than 0 ⁇
- Y ⁇ the coordinates of the pixels in the vertical direction of the pixels with a pixel value greater than 0 ⁇ .
- the distances in the horizontal and vertical directions between the pixels having the maximum pixel value and the minimum pixel value may be computed.
- the distance dx in the horizontal direction and distance dy in the horizontal direction may be determined based on the following equations:
- step 705 it is determined whether the change meets or exceeds a threshold. If the change does not exceed the threshold, it may be indicated that no motion event has been detected. Alternatively, no indication will be given unless the threshold is met or exceeded. For example, when the change meets or exceeds the threshold, an indication that a motion event occurred may be given as described elsewhere in this disclosure with respect to step 105 . Processing may then continue at step 106 , as described elsewhere in this disclosure,
- the threshold distance may be pre-determined. In some embodiments, the threshold distance may be any number between 1 to 10, 11 to 20, 21 to 30, 31 to 40, or 41 to 50, etc. If neither of distances dx and dy meets or exceeds the threshold distance, an indication of no motion event detected will be given. Alternatively, no indication will be givers unless the threshold is met or exceeded. For example, if either of the distances dx and dy meets or exceeds the threshold distance, an indication that a motion event occurred may be given at 105 as shown in FIG. 1 . An alert indicating the motion event may also be generated and transmitted to the user, a third party (e.g., the police), or service provider, etc. Whether there is a new image that needs analyzing is also determined at 106 , as described elsewhere in this disclosure.
- a third party e.g., the police
- a number count of the pixels in the difference image (or a portion thereof) with a pixel value that is equal to or greater than a threshold pixel value is determined at 702 .
- whether the number count meets or exceeds a threshold is also determined, as described elsewhere in this disclosure. If the number count does not exceed the threshold, an indication of no motion event detected will be given. Alternatively, no indication will be given unless the threshold is met or exceeded. For example, when the number count meets or exceeds the threshold, a change area in the difference image is determined at 704 and whether the change area meets or exceeds a threshold is determined at 705 , as described elsewhere in this disclosure.
- an indication of no motion event detected will be given.
- no indication will be given unless the threshold is met or exceeded.
- An alert indicating the motion event may also be generated and transmitted to the user, a third party (e.g., the police), or service provider, etc. Whether there is a new image that needs analyzing is also determined at 106 , as described elsewhere in this disclosure.
- a maximum distance (or “distance”) between the pixels with the maximum and minimum pixel values that are greater than 0, among the all pixels in the difference image (or a portion thereof) may be further determined at 704 .
- Whether any of the distances calculated meets or exceeds a threshold distance may be also determined at 705 , as described elsewhere in this disclosure. If neither of the calculated distances meets or exceeds the threshold distance, an indication of no motion event detected will be given. Alternatively, no indication will be given unless the threshold is met or exceeded. For example, if any of the calculated distances exceed the threshold distance, an indication that a motion event occurred will be given at 105 as shown in FIG. 1 . Processing may then continue at step 106 , as described elsewhere in this disclosure.
- the alert may include image and/or video data associated with the detected motion event.
- the alert may be transmitted to the user, a third-party (e.g., the police), or a service provider, etc.
- FIG. 9 is a block diagram of an exemplary system 900 for detecting a motion event.
- System 900 may include one or more processors 910 , memory 920 , image sensor 930 , communication port 940 , input 950 , and output 960 .
- processor 910 may execute computer instructions (program code) and perform functions (e.g., one or more steps in FIGS. 1, 2, 4, 6, and 7 ) in accordance with techniques described herein.
- processor 910 may receive and analyze the captured image data by image sensor 930 and determine whether the image data shows any moving objects.
- Memory 920 may store executable computer instructions (program code) and captured data such as still images and videos.
- Image sensor 930 may capture image data, such as still images and videos.
- Image sensor 930 may include, e.g., one or more CCD sensors or CMOS sensors configured to capture visible and/or invisible (e.g., infrared or ultraviolet) light.
- Communication port 940 may be a device configured to facilitate communications between system 100 and the user, third parties, or service providers, etc.
- Input 950 may be a device (e.g., a keyboard, keypad, touchscreen, and/or a mouse or other pointing device) configured to allow the user to enter input various types of information
- output 960 may be a device (e.g., a screen or display) configured to communicate information to the user.
- Processor 910 may include or be part of one or more known processing devices such as, for example, a microprocessor. In some embodiments, processor 910 may include any type of single or multi-core processor, mobile device microcontroller, central processing unit, etc.
- Memory 920 may include electronic memories such as random access memory (RAM), read-only memory (ROM), or other types of memory, in any combination. Memory 920 may also be viewed as what is more generally referred to as a “computer program product” having executable computer instructions (program code) in accordance with discussed techniques embodied therein. Examples of computer program products embodying aspects of the invention may include non-transitory computer-readable storage media such as hard drives, optical or magnetic disks, solid state memory, or other computer-readable media.
- System 900 may be communicated with one or more user devices, third-parties (such as the police or fire station, etc.), or service providers (not shown) through network 970 .
- Network 970 may include, or be part of any one or more of a variety of networks or other types of communication connections known to those skilled in the art.
- Network 970 may include a network connection, bus, or other type of data link, such as a hardwire or other connection known in the art.
- network 970 may include or be part of the Internet, an intranet network, a local area network, or other wireless or other hardwired connection or connections (e.g., Bluetooth, WiFi, 4G, LTE cellular data networks, etc.) through which the components of system 900 may communicate.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Image Analysis (AREA)
Abstract
Description
- This application is based upon and claims the benefit of priority from Chinese Patent Application No. 201510441049.5, filed on Jul. 4, 2015, the disclosure of which is expressly incorporated herein by reference to its entirety.
- The present disclosure generally relates to the field of digital images and, more specifically, to detecting motion based on data at the edges of digital images.
- Video monitoring devices monitor premises for various purposes, including, e.g., security monitoring, infant or elderly monitoring, videoconferencing, etc. The video feed may be constantly or periodically monitored by security personnel, and/or recorded for later review.
- The use of motion detection technology that can automatically detect moving events in the video may decrease monitoring costs and improve the efficiency and sensitivity of monitoring by alerting or otherwise providing notice of significant events. When a motion event, such as a person entering a secure area, is detected, the motion detector may begin a video feed, and/or begin recording the video, or otherwise alert security personnel of the potential intrusion.
- Motion detection technology has been playing an increasingly important role in the technology of video recording. Recently, small-sized, portable, lightweight smart video recorders that incorporate motion detection technology have been developed.
- Traditional motion detection methods can generally be categorized into two groups: methods based on pixel values and methods based on the optical flow of pixels. The methods based on pixel values detect motion information in the video frames by detecting changes in values of a pixel or group of pixels over time. This type of method may include, for example, background subtraction or temporal frame differencing. Background subtraction methods first generate a background image based on historical image data and then detect motion by comparing the current frame with the background image. Temporal frame differencing methods calculate pixel value differences between two or three successive frames and detect motion areas in which the calculated pixel value differences meets or exceeds a threshold.
- In methods based on optical flow, a motion vector (i.e., a flow vector) is assigned to each pixel of the image. If there is no moving object, the change in the motion vectors in the whole image is continuous. On the other hand, if there is at least one moving object, the motion vectors of the moving object(s) are different from that of neighboring area. The area(s) of the moving object(s) may be detected based on such differences.
- The methods based on pixel values generally rely on less complex algorithms than the methods based on optical flow, but may deliver lower accuracy. On the other hand, although the methods based on optical flow may provide better performance, they usually require hardware with high computational capacity in order to analyze images in real time, which is difficult implemented on small-sized, portable, lightweight video recorders.
- Another advantage of the optical flow methods is that false detections from light interference may be reduced or avoided. Methods based on pixel values (including both background subtraction and temporal frame differencing methods) can be very sensitive to image changes due to some irrelevant motion, such as wind, false motion due to light reflections, or other interference, as described below. On the other hand, as described below, optical flow methods may be very sensitive to image noise. In addition, they may require extensive computational resources because of the high complexity of the algorithm. Accordingly, real-time processing video frames using an optical flow method may require specialized hardware.
- (1) Changes in Ambient Light
- A false alarm may be generated when the ambient light in the view of the video recorder is suddenly changed, for example, when room lighting is turned on or off. To reduce this type of false alarm, traditional technology may determine the areas of the image in which pixel values have changed; if all (or a majority of) the pixel values in the image have changed, such change may be attributed to the ambient lighting and an alert may not be generated. A problem with this approach is that a motion event causing changes in majority of the pixels (e.g., motion close to the video camera) may be missed.
- (2) Irrelevant Movement
- Traditional motion detection technology may give false alarms for image changes due to irrelevant movement, rather than movement in which the monitor may be interested. For example, images may change due to the motion of trees or plants due to wind. To reduce this type of false alarm, traditional motion detection technology may determine the number of the pixels whose values have changed; if the calculated number is lower than a threshold, an alarm may not be generated. A problem with this approach is that it may miss some relevant motion that causes only a small change in the image, such as a moving object that is far away from the video camera.
- (3) Image Noise
- While capturing images, an image sensor may generate noise due to many factors, including environmental factors (e.g., light conditions etc.) and hardware limitations (e.g., sensor temperature, etc.). Noise may be distributed across the whole image, and may cause changes that can be misinterpreted as motion. Traditional motion detection technology may generate a false alarm due to such noise.
- There remains a need for motion detection technology that can be implemented on small-sized, portable, lightweight video recorders to meet the challenges discussed above.
- Consistent with embodiments of this disclosure, there is provided a method for detecting a motion event. The method may include acquiring a first image; obtaining a comparison image; identifying edge pixels in the first image; generating a first edge image by assigning a first pixel value to the edge pixels of the first image and a second pixel value to non-edge pixels of the first image, the first pixel value being different with the second pixel value; identifying edge pixels in the comparison image; generating a comparison edge image by assigning the first pixel value to the edge pixels of the comparison image and the second pixel value to non-edge pixels of the comparison image; determining difference between the first edge image and the comparison edge image; determining whether the difference meets or exceeds a first threshold; and indicating that a motion event occurs if the difference meets or exceeds the first threshold.
- Consistent with embodiments of this disclosure, there is further provided a device or system for detecting a motion event. The device or system includes a processor and a memory storing programming for execution by the processor. The processor is configured for the device or system to acquire a first image; obtain a comparison image; identify edge pixels in the first image; generate a first edge image by assigning a first pixel value to the edge pixels of the first image and a second pixel value to non-edge pixels of the first image, the first pixel value being different with the second pixel value; identify edge pixels in the comparison image; generate a comparison edge image by assigning the first pixel value to the edge pixels of the comparison image and the second pixel value to non-edge pixels of the comparison image; determine difference between the first edge image and the comparison edge image; determine whether the difference meets or exceeds a first threshold; and indicate that a motion event occurs if the difference meets or exceeds the first threshold.
- Consistent with other disclosed embodiments, non-transitory computer-readable storage media may store program instructions, which are executed by at least one processor and perform any of the methods described herein.
- The foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the claims.
- The methods, systems, and/or programming described herein are further described in terms of exemplary embodiments. These non-limiting exemplary embodiments are described in detail with reference to the drawings, wherein:
-
FIG. 1 is a flowchart of an exemplary process for detecting a motion event according to some embodiments; -
FIG. 2 is a flowchart of an exemplary process for acquiring and processing images according to some embodiments; -
FIGS. 3(A)-3(F) are exemplary images according to some embodiments; -
FIG. 4 is a flowchart of an exemplary process for generating an edge image according to some embodiments; -
FIG. 5 is a diagram illustrating a pixel and its neighboring pixels according to some embodiments; -
FIG. 6 is a flowchart of an exemplary erosion process according to some embodiments; -
FIG. 7 is a flowchart of an exemplary process for detecting differences between images according to some embodiments; -
FIG. 8 illustrates an exemplary process for computing the distance between pixels; and -
FIG. 9 is a block diagram of an exemplary system for detecting a motion event according to some embodiments. - Reference will now be made in detail to the disclosed embodiments, examples of which are illustrated in the accompanying drawings. Wherever convenient, the same reference numbers will be used throughout the drawings to refer to the same or like parts.
- The objects, features, and characteristics of the present disclosure, as well as the methods of operation and functions of the related elements of structure and the combination of parts and economies of manufacture, may become more apparent upon consideration of the following description with reference to the accompanying drawing(s), all of which form a part of this specification. It is to be expressly understood, however, that the drawing(s) are for the purpose of illustration and description only and are not intended as a definition of the limits of the invention. As used in the specification and in the claims, the singular form of “a,” “an” and “the” include plural referents unless the context clearly dictates otherwise.
- Embodiments of the present disclosure provide methods, systems, and articles of manufacture, such as non-transitory computer-readable media, for detecting a motion event based on image edges. Edges are lines of points in a digital image where discontinuities occur, such as a sharp change in color or brightness.
-
FIG. 1 is a flowchart of an exemplary process for detecting a motion event. At 101, a first image and a comparison image may be acquired. The first image may be an image or image frame captured by an image sensor in real time. In other embodiments, the first image may be an image or age frame stored in memory. The comparison image may be an image or image frame captured by the image sensor earlier than the first image. In other embodiments, the comparison image may be a reference image or image frame stored in memory. In some embodiments, the comparison image may be a processed image as disclosed below. For example, the comparison image may be acquired by, for example, the methods discussed later with regard to step 107. In some embodiments, the first image and the comparison image may be obtained by extracting them from a video file. - In some embodiments, the first image and the comparison image may have a same resolution. In some embodiments, only a portion of the first image and a corresponding portion of the comparison image may be acquired.
-
FIG. 3(A) is an exemplary image of the first image, andFIG. 3(B) is an exemplary image of the comparison image. In some embodiments, the first image, i.e.,FIG. 3(A) is an image frame captured after the comparison image, i.e.,FIG. 3(B) . - Returning to
FIG. 1 , at 102, the first image and comparison image may be processed. Alternatively, only a portion of the first image and the corresponding portion of the comparison image may be processed. In some embodiments, the processed first image and processed comparison image may be saved in the memory for future use. In some embodiments, the first image and the comparison image may be upscaled or downscaled. Additionally or alternatively, certain pixel information (e.g., color information) may be removed from the first image and comparison image to reduce computing requirement. - In some embodiments, the processed first image and the processed comparison image may be edge images generated using an edge-identifying process. For example, edge pixels in the first image and comparison image may be identified by a convolution of a 2-D filter/kernel such as Roberts Cross or a Sobel formulation. Pixel information of non-edge pixels of the images may be removed. In some embodiments, the first image and comparison images may be processed to generate the processed first image and comparison image, based on a process for detecting edge pixels illustrated in
FIG. 4 . -
FIG. 4 is an exemplary flowchart of a process for detecting edge pixels and generating an edge image (i.e., a processed image). At 401, for each pixel of an image (e.g., the first image acquired at 101), the pixel values of its neighboring pixels may be obtained. For example, for pixel 501(with the pixel value p(x, y)) shown inFIG. 5 , the pixel values of theabove pixel 503 and the bottom pixel 508 (i.e., p(x, y−1) and p(x, y+1), respectively) and the pixel values of theleft pixel 505 and the right pixel 506 (i.e., p(x−1, y) and p(x+1, y), respectively) may be obtained. At 402, the difference in pixel value between any two of the neighboring pixels may be computed. In some embodiments, the difference in pixel value between the neighboring pixels along the vertical direction, and/or the difference in pixel value between the neighboring pixels along the horizontal direction may be computed. For example, the differences Deltah and Deltav between the neighboring pixels may be computed based onEquations 1 and 2 (in connection toFIG. 5 ) as follows: -
Deltah =|p(x, y−1)−p(x, y+1)| (1) -
Deltav =|p(x−1, y)−p(x+1, y)| (2) - At 403, it may be determined whether any of the computed differences meets or exceeds a threshold. For example, whether the computed Deltah or Deltav meets or exceeds a threshold may be determined. In some embodiments, the threshold may be pre-determined. In some embodiments, the threshold may be adjusted by a user. The threshold can be any number between the minimum and maximum image pixel values. In other embodiments, the threshold may be any number between the lowest and highest possible pixel values for the image. In some embodiments, the minimum or lowest possible pixel value is 0 (black), and the maximum or highest possible pixel value is 255 (white). In some embodiments, the threshold may be any number between 1 and 255. In other embodiments, the threshold may be restricted to
subranges 1 to 50, 51 to 100, 101 to 150, 151 to 200, or 201 to 255. - At 404 if any of computed difference(s) meets or exceeds the threshold (403, “YES”), the pixel (e.g., pixel 501) may be identified as an edge pixel. A new pixel value (a first pixel value) may then be assigned to this identified edge pixel. In some embodiments, the first pixel value may be any number between the minimum pixel value and the maximum pixel value among the pixels in the image. In other embodiments, the first pixel value may be any number between lowest and highest possible pixel values for the image. In some embodiments, the first pixel value may be the maximum or the highest possible pixel value. On the other hand, if the computed difference does not exceed the threshold (403, “NO”), at 405, the pixel (e.g., pixel 501) may be identified as a non-edge pixel. A second pixel value may then be assigned to this non-edge pixel. In some embodiments, the second pixel value may be any number between the minimum pixel value and the maximum pixel among the pixels in the image. In other embodiments, the first pixel value may be any number between lowest and highest possible pixel values for the image. In some embodiments, the second pixel value may be the minimum or lowest possible pixel value. In some embodiments, the minimum or lowest possible pixel value is 0.
- At 406, it is determined whether the process for all pixels in the image (or a portion thereof) is completed.
FIG. 3(C) andFIG. 3(D) show examples of the processed first image (i.e., the first edge image) and processed comparison image (i.e., the comparison edge image), respectively, generated according to some embodiments disclosed herein, with the threshold being 50, the first pixel value being 255 (white), and the second pixel value being 0 (black). - In some embodiments, the process of
FIG. 1 may be modified as illustrated inFIG. 2 . At 101 a, the first image is acquired. At 102 a, the acquired first image may be processed, as described elsewhere in this disclosure. At 201, it is determined whether the time that has elapsed since acquiring the first image meets or exceeds a threshold period of time. For example, if the first image is acquired at time t1, and the processed first image is obtained at time t2, the interval between time t1 and time t2 may be computed as t2-t1. It may then be determined whether the interval (t2-t1) meets or exceeds a threshold period of time. The threshold period of time may be pre-determined, or may be adjusted by a user. In some embodiments, the threshold may be any period of time between 0 and 1000 milliseconds. In other embodiments, depending upon the application, the threshold may be restricted to subranges of 0 to 200 milliseconds, 200 to 400 milliseconds, 400 to 600 milliseconds, 600 to 800 milliseconds, 800 to 1000 milliseconds, 0 to 1 second, 1 to 20 seconds, 20 to 40 seconds, 40 to 60 seconds, 0 to 1 minute, 1 to 5 minutes, 5 to 10 minutes, etc. If the time that has elapsed since acquiring the first image (e.g., the image shown inFIG. 3(A) ) does not exceed the threshold, a comparison image or a processed comparison image (e.g., the image shown inFIG. 3(B) or the image shown inFIG. 3(D) ), which was acquired by the camera before the first image may not be obtained until the time interval (t2-t1) reaches or exceeds the threshold period of time. On the other hand, if the time that has elapsed since acquiring the first image meets or exceeds the threshold, a comparison image may be obtained (at 101 b) and processed (at 102 b), as described elsewhere in this disclosure with respect tosteps - Now returning to
FIG. 1 , at 103, the difference between the processed first image and the processed comparison image may be determined, and a difference image based on the processed first image and the processed comparison image may be generated.FIG. 3(E) is an exemplary difference image generated based on the processed first image (FIG. 3(D) ) and processed comparison image (FIG. 3(C) ) according to some embodiments disclosed herein. - For example, the processed comparison image may be subtracted from the processed first image (or vice versa) to obtain a difference image. In some embodiments, a difference image may be generated by assigning a first pixel value to the pixels in the processed first image that have a different pixel value from the corresponding pixels in the processed comparison image, and by assigning a second pixel value to the pixels in the processed first image that have the same pixel value as the corresponding pixels in the processed comparison image. The first pixel value and second pixel value may be any number between the minimum and maximum pixel values among the pixels of the images. In some embodiments, the first pixel value and second pixel value may be any number between the lowest possible and highest possible pixel values for the images. Alternatively, the first pixel value may be the maximum or highest possible pixel value, and the second pixel value may be the minimum or lowest possible pixel value. The maximum or highest possible pixel value may be 255 (white), and the minimum or lowest possible pixel value may be 0 (black).
- In some embodiments, noise from the processed first image and the processed comparison image may be reduced before the difference image is generated.
FIG. 3(F) is an exemplary denoised difference image generated by reducing the noise from the difference image (FIG. 3(E) ), in accordance with some embodiments disclosed herein. - In some embodiments, for example, an erosion operation may be performed on the difference image to reduce noise as illustrated in
FIG. 6 . In some embodiments, the difference image may be a denoised by a noise reducing process as described elsewhere in this disclosure. - At 104, it may be determined whether the difference meets or exceeds a threshold. If the difference does not exceed the threshold, it may be indicated that no motion event has been detected. Alternatively, no indication may be given, unless the difference meets or exceeds the threshold. If the difference meets or exceeds the threshold, an indication that a motion event occurred may be given at 105. An alert indicating the motion event may be also generated and transmitted to a user, a third party (e.g., the police or responsible security personnel), or service provider, etc., as disclosed elsewhere in this disclosure.
- At 106, it may be determined whether there is a new image available for analyzing. For example, it may be determined whether to acquire a new image captured by the image sensor in real time. If there is a new image to be analyzed, the new image may be acquired as the first image, and a new comparison image may also be obtained, as described elsewhere in this disclosure with respect to step 101.
Steps 102 through 106 (if applicable) may also be performed, as disclosed elsewhere in this disclosure. In other embodiments, the new first image may be acquired at 101 and processed at 102, as described elsewhere in this disclosure, - At 107, the previously processed first image (“old” first image) or the previously processed comparison image (“old” comparison image) may be assigned as the processed “new” comparison image.
Steps 103 through 106 (if applicable) may also be performed, as described elsewhere in this disclosure, using the new first image. - In some embodiments,
steps FIG. 7 . At 701, a difference image is generated based on the processed first image and the processed comparison images, as described elsewhere in this disclosure. Noise from the processed first image and comparison image may be reduced before the difference image is generated and the difference is determined. For example, an erosion process may be performed on the difference image as illustrated inFIG. 6 to reduce noise. -
FIG. 6 is a flowchart of an exemplary erosion process. At 601, for each of pixels of the difference image (or a portion thereof), the pixel values of the pixel and a plurality of its neighboring pixels may be obtained. For instance, forpixel 501 as shown inFIG. 5 , the pixel values ofpixels -
pe(x, y)=min{p(x, y), p(x−1, y−1), p(x, y−1), p(x−1, y)} (3) - At 603, the minimum value pe(x, y) may be assigned to the
pixel 501. At 604, it may be determined whether there are more pixels that need processing. A denoised difference image may also be generated at the end of the erosion operation. In some embodiments, the erosion process on the difference image may be performed more than once to obtain a denoised difference image. - Returning to
FIG. 7 , at 702, a number count of the pixels in the difference image (or a portion thereof) that have a pixel value equal to or greater than a threshold pixel value may be determined. The threshold pixel value may be in the range or subranges described above in relation toFIG. 1 . - At 703, it may be determined whether the number count meets or exceeds a threshold. The threshold may be pre-determined. In other embodiments, the threshold may be adjusted by a user. The threshold may be any number. Depending on the application, for example, the threshold may be chosen to lie in the subrange between 1 10, 11 to 20, 21 to 30, 31 to 40, or 41 to 50, etc. If the number count does not exceed the threshold, it may be indicated that no motion event has been detected. Alternatively, no indication will be given unless the number count meets or exceeds the threshold. If the number count meets or exceeds the threshold, an indication that a motion event has occurred may be given at 105. An alert indicating the motion event may be also generated, as described above.
- In some embodiments, after the difference image is generated,
steps - A change area in the difference image may be determined at 704. The change may be determined by calculating an area comprising the pixels having the first pixel value, assigned above in
step 404. In other embodiments, the change area may be estimated based on a distance (e.g., the maximum distance) between the pixels with the maximum and minimum pixel values that are greater than 0, among all pixels in the difference image (or a portion thereof).FIG. 8 illustrates an exemplary process for computing the distance in the vertical and/or horizontal direction between the pixels with maximum and minimum pixel values. At 801, the pixels in the difference image that have a pixel value greater than 0 may be obtained. At 802, the coordinates of the pixels with the maximum and minimum pixel values that are greater than 0 may be determined. For example, the coordinates (e.g. max_x and min_x in the horizontal direction and max_y and min_y in the vertical direction in the following equations) may be determined based on the following equations: -
max_x=max(X) (4) -
min_x=min(X) (5) -
max_y=max(Y) (6) -
min_y=min(Y) (7) - Wherein X ∈ {the coordinates in the horizontal direction of the pixels with a pixel value greater than 0} and Y ∈ {the coordinates of the pixels in the vertical direction of the pixels with a pixel value greater than 0}. At 803 and 804, the distances in the horizontal and vertical directions between the pixels having the maximum pixel value and the minimum pixel value may be computed. For example, the distance dx in the horizontal direction and distance dy in the horizontal direction may be determined based on the following equations:
-
dx=max_x−min_x (8) -
dy=max_y−min_y (9) - After the distances are computed as described above, at 705, it is determined whether the change meets or exceeds a threshold. If the change does not exceed the threshold, it may be indicated that no motion event has been detected. Alternatively, no indication will be given unless the threshold is met or exceeded. For example, when the change meets or exceeds the threshold, an indication that a motion event occurred may be given as described elsewhere in this disclosure with respect to step 105. Processing may then continue at
step 106, as described elsewhere in this disclosure, - Alternatively or additionally, at 705, whether either of the distances dx and dy calculated meets or exceeds a threshold distance is determined. In some embodiments, the threshold distance may be pre-determined. In some embodiments, the threshold distance may be any number between 1 to 10, 11 to 20, 21 to 30, 31 to 40, or 41 to 50, etc. If neither of distances dx and dy meets or exceeds the threshold distance, an indication of no motion event detected will be given. Alternatively, no indication will be givers unless the threshold is met or exceeded. For example, if either of the distances dx and dy meets or exceeds the threshold distance, an indication that a motion event occurred may be given at 105 as shown in
FIG. 1 . An alert indicating the motion event may also be generated and transmitted to the user, a third party (e.g., the police), or service provider, etc. Whether there is a new image that needs analyzing is also determined at 106, as described elsewhere in this disclosure. - Now returning again to 701, after the difference image is generated, a number count of the pixels in the difference image (or a portion thereof) with a pixel value that is equal to or greater than a threshold pixel value is determined at 702. At 703, whether the number count meets or exceeds a threshold is also determined, as described elsewhere in this disclosure. If the number count does not exceed the threshold, an indication of no motion event detected will be given. Alternatively, no indication will be given unless the threshold is met or exceeded. For example, when the number count meets or exceeds the threshold, a change area in the difference image is determined at 704 and whether the change area meets or exceeds a threshold is determined at 705, as described elsewhere in this disclosure. If the change area does not exceed the threshold, an indication of no motion event detected will be given. Alternatively, no indication will be given unless the threshold is met or exceeded. For example, if the change area meets or exceeds the threshold, an indication that a motion event occurred will be given at 105 as shown in
FIG. 1 . An alert indicating the motion event may also be generated and transmitted to the user, a third party (e.g., the police), or service provider, etc. Whether there is a new image that needs analyzing is also determined at 106, as described elsewhere in this disclosure. - In some embodiments, if the number count meets or exceeds the threshold (703), a maximum distance (or “distance”) between the pixels with the maximum and minimum pixel values that are greater than 0, among the all pixels in the difference image (or a portion thereof) may be further determined at 704. Whether any of the distances calculated meets or exceeds a threshold distance may be also determined at 705, as described elsewhere in this disclosure. If neither of the calculated distances meets or exceeds the threshold distance, an indication of no motion event detected will be given. Alternatively, no indication will be given unless the threshold is met or exceeded. For example, if any of the calculated distances exceed the threshold distance, an indication that a motion event occurred will be given at 105 as shown in
FIG. 1 . Processing may then continue atstep 106, as described elsewhere in this disclosure. - Now returning to step 105, the alert may include image and/or video data associated with the detected motion event. The alert may be transmitted to the user, a third-party (e.g., the police), or a service provider, etc.
-
FIG. 9 is a block diagram of anexemplary system 900 for detecting a motion event.System 900 may include one ormore processors 910,memory 920,image sensor 930,communication port 940,input 950, andoutput 960. In operation,processor 910 may execute computer instructions (program code) and perform functions (e.g., one or more steps inFIGS. 1, 2, 4, 6, and 7 ) in accordance with techniques described herein. For example,processor 910 may receive and analyze the captured image data byimage sensor 930 and determine whether the image data shows any moving objects.Memory 920 may store executable computer instructions (program code) and captured data such as still images and videos.Image sensor 930 may capture image data, such as still images and videos.Image sensor 930 may include, e.g., one or more CCD sensors or CMOS sensors configured to capture visible and/or invisible (e.g., infrared or ultraviolet) light.Communication port 940 may be a device configured to facilitate communications between system 100 and the user, third parties, or service providers, etc. Input 950 may be a device (e.g., a keyboard, keypad, touchscreen, and/or a mouse or other pointing device) configured to allow the user to enter input various types of information, andoutput 960 may be a device (e.g., a screen or display) configured to communicate information to the user. -
Processor 910 may include or be part of one or more known processing devices such as, for example, a microprocessor. In some embodiments,processor 910 may include any type of single or multi-core processor, mobile device microcontroller, central processing unit, etc. -
Memory 920 may include electronic memories such as random access memory (RAM), read-only memory (ROM), or other types of memory, in any combination.Memory 920 may also be viewed as what is more generally referred to as a “computer program product” having executable computer instructions (program code) in accordance with discussed techniques embodied therein. Examples of computer program products embodying aspects of the invention may include non-transitory computer-readable storage media such as hard drives, optical or magnetic disks, solid state memory, or other computer-readable media. -
System 900 may be communicated with one or more user devices, third-parties (such as the police or fire station, etc.), or service providers (not shown) throughnetwork 970.Network 970 may include, or be part of any one or more of a variety of networks or other types of communication connections known to those skilled in the art.Network 970 may include a network connection, bus, or other type of data link, such as a hardwire or other connection known in the art. For example,network 970 may include or be part of the Internet, an intranet network, a local area network, or other wireless or other hardwired connection or connections (e.g., Bluetooth, WiFi, 4G, LTE cellular data networks, etc.) through which the components ofsystem 900 may communicate. - While illustrative embodiments have been described herein, the scope of any and all embodiments having equivalent elements, modifications, omissions, combinations (e.g., of aspects across various embodiments), adaptations and/or alterations as would be appreciated by those skilled in the art based on the present disclosure. The limitations in the claims are to be interpreted broadly based on the language employed in the claims and not limited to examples described in the present specification or during the prosecution of the disclosure. The examples are to be construed as non-exclusive. Furthermore, the steps of the disclosed routines may be modified in any manner, including by reordering steps and/or inserting or deleting steps. In particular, non-dependent steps may be performed in any order, or in parallel. It is intended, therefore, that the specification and examples be considered as illustrative only, with a true scope and spirit being indicated by the following claims and their full scope of equivalents.
Claims (20)
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201510441049 | 2015-07-24 | ||
CN201510441049.5 | 2015-07-24 | ||
CN201510441049.5A CN105096321B (en) | 2015-07-24 | 2015-07-24 | A kind of low complex degree Motion detection method based on image border |
Publications (3)
Publication Number | Publication Date |
---|---|
US20170004629A1 US20170004629A1 (en) | 2017-01-05 |
US20200175692A9 true US20200175692A9 (en) | 2020-06-04 |
US10713798B2 US10713798B2 (en) | 2020-07-14 |
Family
ID=54576660
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/828,733 Active 2035-10-04 US10713798B2 (en) | 2015-07-24 | 2015-08-18 | Low-complexity motion detection based on image edges |
Country Status (2)
Country | Link |
---|---|
US (1) | US10713798B2 (en) |
CN (1) | CN105096321B (en) |
Families Citing this family (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9805567B2 (en) * | 2015-09-14 | 2017-10-31 | Logitech Europe S.A. | Temporal video streaming and summaries |
US10299017B2 (en) | 2015-09-14 | 2019-05-21 | Logitech Europe S.A. | Video searching for filtered and tagged motion |
US10762708B2 (en) * | 2016-06-23 | 2020-09-01 | Intel Corporation | Presentation of scenes for binocular rivalry perception |
CN106991418B (en) * | 2017-03-09 | 2020-08-04 | 上海小蚁科技有限公司 | Winged insect detection method and device and terminal |
US10430671B2 (en) | 2017-04-22 | 2019-10-01 | Honda Motor Co., Ltd. | System and method for remapping surface areas of a vehicle environment |
CN108198208B (en) * | 2017-12-27 | 2021-08-24 | 浩云科技股份有限公司 | Movement detection method based on target tracking |
US11373315B2 (en) * | 2019-08-30 | 2022-06-28 | Tata Consultancy Services Limited | Method and system for tracking motion of subjects in three dimensional scene |
CN110992389A (en) * | 2019-11-08 | 2020-04-10 | 浙江大华技术股份有限公司 | Termite monitoring method, termite monitoring device and termite monitoring storage device |
US10904446B1 (en) | 2020-03-30 | 2021-01-26 | Logitech Europe S.A. | Advanced video conferencing systems and methods |
US10965908B1 (en) | 2020-03-30 | 2021-03-30 | Logitech Europe S.A. | Advanced video conferencing systems and methods |
US10951858B1 (en) | 2020-03-30 | 2021-03-16 | Logitech Europe S.A. | Advanced video conferencing systems and methods |
US10972655B1 (en) | 2020-03-30 | 2021-04-06 | Logitech Europe S.A. | Advanced video conferencing systems and methods |
KR20220015964A (en) * | 2020-07-31 | 2022-02-08 | 삼성전자주식회사 | Methods and systems for restoration of lost image features for visual odometry applications |
CN114157855A (en) * | 2020-09-07 | 2022-03-08 | 苏州佳世达电通有限公司 | Imaging system and diagnostic method thereof |
CN113435287A (en) * | 2021-06-21 | 2021-09-24 | 深圳拓邦股份有限公司 | Lawn obstacle recognition method and device, mowing robot and readable storage medium |
Family Cites Families (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5815199A (en) * | 1991-01-31 | 1998-09-29 | Matsushita Electric Works, Ltd. | Interphone with television |
DE69232732T2 (en) * | 1991-03-19 | 2009-09-10 | Mitsubishi Denki K.K. | Measuring device for moving bodies and image processing device for measuring traffic flow |
US6128047A (en) * | 1998-05-20 | 2000-10-03 | Sony Corporation | Motion estimation process and system using sparse search block-matching and integral projection |
AU2001290608A1 (en) * | 2000-08-31 | 2002-03-13 | Rytec Corporation | Sensor and imaging system |
US6798342B2 (en) * | 2002-06-26 | 2004-09-28 | Honeywell International Inc. | Security system with remote indication device |
US20060190750A1 (en) * | 2005-02-22 | 2006-08-24 | Logitech Europe S.A. | System power management based on motion detection |
US7796780B2 (en) * | 2005-06-24 | 2010-09-14 | Objectvideo, Inc. | Target detection and tracking from overhead video streams |
US20070070243A1 (en) * | 2005-09-28 | 2007-03-29 | Ali Corporation | Adaptive vertical temporal flitering method of de-interlacing |
EP1969560B1 (en) * | 2005-12-30 | 2017-04-05 | Telecom Italia S.p.A. | Edge-controlled morphological closing in segmentation of video sequences |
US8405780B1 (en) * | 2007-08-22 | 2013-03-26 | Adobe Systems Incorporated | Generating a clean reference image |
JP4471039B2 (en) * | 2008-02-14 | 2010-06-02 | 日本電気株式会社 | Moving vector detection device |
JP5301239B2 (en) * | 2008-08-09 | 2013-09-25 | 株式会社キーエンス | Pattern model positioning method, image processing apparatus, image processing program, and computer-readable recording medium in image processing |
JP2010097438A (en) * | 2008-10-16 | 2010-04-30 | Keyence Corp | Outline information extraction method using image processing, creation method for pattern model in image processing, positioning method for pattern model in image processing, image processor, image processing program and computer-readable recording medium |
US9367927B2 (en) * | 2011-07-13 | 2016-06-14 | Nec Corporation | Moving image region detection device |
US9413951B2 (en) * | 2014-07-31 | 2016-08-09 | Apple Inc. | Dynamic motion estimation and compensation for temporal filtering |
-
2015
- 2015-07-24 CN CN201510441049.5A patent/CN105096321B/en not_active Expired - Fee Related
- 2015-08-18 US US14/828,733 patent/US10713798B2/en active Active
Also Published As
Publication number | Publication date |
---|---|
US20170004629A1 (en) | 2017-01-05 |
CN105096321B (en) | 2018-05-18 |
CN105096321A (en) | 2015-11-25 |
US10713798B2 (en) | 2020-07-14 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10713798B2 (en) | Low-complexity motion detection based on image edges | |
KR101223424B1 (en) | Video motion detection | |
US10372995B2 (en) | System and method for previewing video | |
US9497425B2 (en) | Motion detection method and device using the same | |
US9818277B1 (en) | Systems and methods for smoke detection | |
US20180144476A1 (en) | Cascaded-time-scale background modeling | |
US20160117837A1 (en) | Modification of at least one parameter used by a video processing algorithm for monitoring of a scene | |
US20070047811A1 (en) | Method for background generation and its system for video surveillance | |
JP6414066B2 (en) | Image processing system, image processing method, and program | |
CN111654700B (en) | Privacy mask processing method and device, electronic equipment and monitoring system | |
WO2016021147A1 (en) | Image processing system for detecting retention of moving body from image, image processing method, and recording medium | |
JP4999794B2 (en) | Still region detection method and apparatus, program and recording medium | |
JP2017117244A (en) | Image analysis device for detecting a person displayed in photographic image of camera | |
KR20220015964A (en) | Methods and systems for restoration of lost image features for visual odometry applications | |
US9818178B1 (en) | Method and system for detection of ghosting artifact in a video | |
CN104202533B (en) | Motion detection device and movement detection method | |
US10916016B2 (en) | Image processing apparatus and method and monitoring system | |
US10339660B2 (en) | Video fingerprint system and method thereof | |
WO2014038924A2 (en) | A method for producing a background model | |
KR101248154B1 (en) | Method and apparatus for estimating moving object using variable area size, and computer readable record medium with moving estimating program for the same | |
CN113108919B (en) | Human body temperature detection method, device and storage medium | |
US11270442B2 (en) | Motion image integration method and motion image integration system capable of merging motion object images | |
JP6124739B2 (en) | Image sensor | |
JP5599228B2 (en) | Busy detection system and busy detection program | |
TWI512638B (en) | Intelligent area method and automatic camera state judgment method |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: XIAOYI TECHNOLOGY CO., LTD., CHINA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:ZHAO, LILI;REEL/FRAME:036348/0428 Effective date: 20150817 |
|
AS | Assignment |
Owner name: SHANGHAI XIAOYI TECHNOLOGY CO., LTD., CHINA Free format text: CORRECTIVE ASSIGNMENT TO CORRECT THE RECEIVING PARTY'S NAME PREVIOUSLY RECORDED AT REEL: 036348 FRAME: 0428. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT;ASSIGNOR:ZHAO, LILI;REEL/FRAME:045194/0162 Effective date: 20150817 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: ADVISORY ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
FEPP | Fee payment procedure |
Free format text: PETITION RELATED TO MAINTENANCE FEES GRANTED (ORIGINAL EVENT CODE: PTGR); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
AS | Assignment |
Owner name: KAMI VISION INC., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:SHANGHAI XIAOYI TECHNOLOGY CO., LTD.;REEL/FRAME:059275/0652 Effective date: 20210813 |
|
AS | Assignment |
Owner name: EAST WEST BANK, CALIFORNIA Free format text: INTELLECTUAL PROPERTY SECURITY AGREEMENT;ASSIGNOR:KAMI VISION INCORPORATED;REEL/FRAME:059512/0101 Effective date: 20220325 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YR, SMALL ENTITY (ORIGINAL EVENT CODE: M2551); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY Year of fee payment: 4 |