CN114827450B - Analog image sensor circuit, image sensor device and method - Google Patents
Analog image sensor circuit, image sensor device and method Download PDFInfo
- Publication number
- CN114827450B CN114827450B CN202111397742.9A CN202111397742A CN114827450B CN 114827450 B CN114827450 B CN 114827450B CN 202111397742 A CN202111397742 A CN 202111397742A CN 114827450 B CN114827450 B CN 114827450B
- Authority
- CN
- China
- Prior art keywords
- pixel
- value
- pixel value
- image
- digital processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 35
- 238000012545 processing Methods 0.000 claims abstract description 92
- 230000033001 locomotion Effects 0.000 claims description 154
- 238000001514 detection method Methods 0.000 claims description 99
- 239000000872 buffer Substances 0.000 claims description 29
- 230000008859 change Effects 0.000 claims description 24
- 230000008878 coupling Effects 0.000 claims 2
- 238000010168 coupling process Methods 0.000 claims 2
- 238000005859 coupling reaction Methods 0.000 claims 2
- 230000001960 triggered effect Effects 0.000 abstract description 3
- 238000012544 monitoring process Methods 0.000 description 65
- 238000010586 diagram Methods 0.000 description 18
- 238000004364 calculation method Methods 0.000 description 9
- 238000010191 image analysis Methods 0.000 description 9
- 230000006870 function Effects 0.000 description 8
- 230000003068 static effect Effects 0.000 description 6
- 230000008569 process Effects 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 230000004913 activation Effects 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000002618 waking effect Effects 0.000 description 2
- 244000025254 Cannabis sativa Species 0.000 description 1
- 241001465754 Metazoa Species 0.000 description 1
- 230000005856 abnormality Effects 0.000 description 1
- 230000003213 activating effect Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000010219 correlation analysis Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005265 energy consumption Methods 0.000 description 1
- 230000005284 excitation Effects 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/65—Control of camera operation in relation to power supply
- H04N23/651—Control of camera operation in relation to power supply for reducing power consumption by affecting camera operations, e.g. sleep mode, hibernation mode or power off of selective parts of the camera
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B13/00—Burglar, theft or intruder alarms
- G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
- G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
- G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
- G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/667—Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Closed-Circuit Television Systems (AREA)
Abstract
The invention discloses a method for simulating an image sensor circuit, which comprises the following steps: providing an event camera comprising pixel units; sensing a current pixel value of the pixel unit by using the event camera to detect whether the pixel value is changed; when the pixel value changes, the digital processing circuit in the power saving mode is triggered by the event camera, and information of the pixel value is transmitted to the digital processing circuit. The invention can save more power to avoid being consumed, can avoid losing the image at the same time, and reduce the calculated amount. The invention also provides a related analog image sensor circuit, an image sensor device and a method.
Description
Technical Field
The present disclosure relates to image monitoring mechanisms, and more particularly to analog image sensor circuits, image sensor devices, and corresponding methods.
Background
Referring to fig. 1, fig. 1 is a schematic diagram of a monitoring system 50 in the prior art. The monitoring system 50 includes a passive sensor 52 electrically connected to an external host 56 and an image sensing device 54. The passive sensor 52 can send a trigger signal to the external host 56 when detecting a temperature change, and the external host 56 is awakened and starts the image sensor 54 under the trigger of the trigger signal, so that the image sensor 54 can perform exposure adjustment after starting, and then start to acquire a monitoring image or record a monitoring video. Therefore, even if the passive sensor 52 senses the temperature change, the image sensor 54 still needs to complete the transmission of the trigger signal, wake-up waiting time of the external host 56 and the image sensor 54, and the exposure adjustment time of the image sensor 54, so that the monitoring system 50 cannot immediately record the monitoring video when the passive sensor 52 senses the abnormal situation.
Disclosure of Invention
It is therefore an object of the present application to disclose an analog image sensor circuit, an image sensor device and a corresponding method to solve the above-mentioned problems.
According to an embodiment of the present application, an analog image sensor circuit is disclosed. The analog image sensor circuit is used for being externally coupled with a digital processing circuit, and the digital processing circuit is used for executing movement detection. The analog image sensor circuit comprises an event camera, wherein the event camera comprises at least one pixel unit. The event camera is used for sensing at least one current pixel value of the at least one pixel unit to detect whether the at least one pixel value is changed or not. When the at least one pixel value is changed, the event camera triggers the digital processing circuit in a power saving mode and transmits information of the at least one pixel value to the digital processing circuit.
According to an embodiment of the present application, an image sensor device is disclosed. The image sensor device comprises the analog image sensor circuit and the digital processing circuit. The digital processing circuit uses the information of the at least one pixel value to generate a current data frame by using the last data frame stored in the digital processing circuit.
According to an embodiment of the present application, a method of an analog image sensor circuit is disclosed, the analog image sensor circuit is used for being externally coupled to a digital processing circuit, and the digital processing circuit is used for performing a motion detection. The method comprises the following steps: providing an event camera comprising at least one pixel unit; using the event camera to sense at least one current pixel value of the at least one pixel unit to detect whether the at least one pixel value is changed; when the at least one pixel value is changed, the event camera is used to trigger the digital processing circuit in a power saving mode and transmit information of the at least one pixel value to the digital processing circuit.
According to an embodiment of the present application, a method of an image sensor device is disclosed. The method comprises the following steps: providing a digital processing circuit externally coupled to the image sensor device for performing a motion detection; and generating a target data frame by using the information of the at least one pixel value and the last data frame stored in the digital processing circuit.
Drawings
Fig. 1 is a schematic diagram of a prior art monitoring system.
Fig. 2 is a schematic diagram of a motion detection device according to an embodiment of the invention.
Fig. 3 is a flowchart of a motion detection method applicable to a motion detection device according to an embodiment of the present invention.
Fig. 4 is a flowchart of a motion detection method applied to a motion detection device according to another embodiment of the present invention.
Fig. 5 is a schematic diagram showing the change of the frame rate of the image capturing unit according to the embodiment of the present invention.
FIG. 6 is a functional block diagram of an intelligent motion detection device according to a first embodiment of the present invention.
Fig. 7 is a schematic program diagram of an intelligent motion detection device according to a first embodiment of the present invention.
FIG. 8 is a functional block diagram of an intelligent motion detection device according to a second embodiment of the present invention.
Fig. 9 is a schematic program diagram of an intelligent motion detection device according to a second embodiment of the invention.
FIG. 10 is a functional block diagram of an intelligent motion detection device according to a third embodiment of the present invention.
Fig. 11 is a schematic program diagram of an intelligent motion detection device according to a third embodiment of the invention.
Fig. 12 is a flowchart of a judging method according to an embodiment of the present invention.
FIG. 13 is a block diagram of a monitoring system according to an embodiment of the present application.
Fig. 14 is a schematic diagram of three different operational scenarios of portions or elements of the monitoring system of fig. 13.
FIG. 15 is a schematic diagram showing two different operations of the digital processing circuit of FIG. 13 for updating and generating data of a current frame.
FIG. 16 is a flowchart illustrating the operation of the smart movement detector that has just entered the normal mode according to one embodiment of the present application.
Wherein reference numerals are as follows:
50. monitoring system of the prior art
52. Prior art passive sensors
54. Image sensing device of the prior art
56. External host of the prior art
60. 60' movement detection device
62. Passive sensor
64. External host
66. Image acquisition unit
68. Arithmetic processor
70. Memory device
72. Light-emitting unit
80. 80', 80' intelligent motion detection device
82. Memory module
84. Processor and method for controlling the same
86. 86', 86 "sensor module
88. External storage module
90. Comparator with a comparator circuit
92. Passive sensor
I1 Pre-storing images
I2 Real-time image
1300. Monitoring system
1301. Image sensor device
1305. Analog image sensor
1306. Event camera
1310. Digital processing circuit
1311. 1316 image buffer
1312. Intelligent movement detector
1315. Back-end system device
1317. External processor
Detailed Description
Referring to fig. 2, fig. 2 is a schematic diagram of a motion detection device 60 according to an embodiment of the invention. The motion detection device 60 can be coupled to the passive sensor 62 and the external host 64 to provide a preferred intelligent motion detection function. The motion detection device 60 is electrically connected between the passive sensor 62 and the external host 64. The passive sensor 62 is used for sensing whether a specific situation occurs, for example, a living body is opened through a monitoring area or a door panel in the monitoring area, so as to trigger the motion detection device 60 to analyze whether an event conforming to a standard exists in the specific situation, for example, the event sensed by the passive sensor 62 may be confirmed as an expected object. After the event is confirmed, the mobile detection device 60 transmits the relevant data to the external host 64 to determine whether to activate the security alarm.
In a possible implementation, the passive sensor 62 may be a temperature sensor, such as an infrared sensor, and the motion detection device 60 may be selectively switched between a power saving mode and an awake mode. The passive sensor 62 does not sense a temperature change when the monitoring area is in a normal state, and the movement detection device 60 is kept in the power saving mode; when an abnormal condition occurs in the monitoring area, such as a living body passing through, the passive sensor 62 can detect a temperature change and generate a trigger signal for switching the motion detection device 60 from the power saving mode to the wake-up mode.
The motion detection device 60 may include an image acquisition unit 66, an operation processor 68, a memory 70, and a light emitting unit 72. The operation processor 68 may drive the image acquisition unit 66 to remain in the power saving mode or the awake mode, and further may drive the image acquisition unit 66 to selectively acquire the low-quality as well as high-quality monitoring images. In a possible implementation, the light emitting unit 72 is only activated when the image capturing unit 66 captures an image to provide light filling, which can save energy consumption and improve the quality of the image captured by the image capturing unit 66.
The image acquisition unit 66 may operate at a low grid rate to acquire a background image in the power saving mode and at a Gao Yingge rate to acquire a plurality of monitoring images in the wake-up mode. The background image may be a low quality image and serve as an automatic exposure adjustment basis for the image acquisition unit 66. The monitoring images may include a low quality first monitoring image and a high quality second monitoring image, wherein the first monitoring image is provided to the operation processor 68 identifying whether the event occurred; the second monitoring image is provided to the external host 64 to determine whether to activate a security alarm. The monitoring image acquired by the image acquisition unit 66 may be stored in the memory 70, and the high-quality monitoring image may be additionally transmitted to the external host 64.
In this embodiment, the monitoring system first detects whether an object passes through the monitoring area by using the passive sensor 62, and then analyzes whether the passing object meets a default condition (e.g., an event that meets a standard) by using the movement detection device 60. If a passing object is in the visual field of the passive sensor 62 and the passing object is identified to be in accordance with a specific condition, the passive sensor 62 switches the motion detection device 60 to the wake-up mode, and the motion detection device 60 determines whether the passing object is an expected object (e.g. a pedestrian); if the passing object is determined to be a pedestrian, the motion detection device 60 starts the external host 64, and the external host 64 starts to identify the object in the monitored image, and performs one or more of the following operations: selecting whether to switch the motion detection device 60 to the video mode, requesting the motion detection device 60 to send the surveillance video out, instructing the motion detection device 60 to alert, turning off the motion detection device 60, or waking up another motion detection device 60' electrically connected to the external host 64.
Referring to fig. 3, fig. 3 is a flowchart illustrating a motion detection method applicable to a motion detection apparatus 60 according to an embodiment of the invention. First, steps S200 and S202 are performed to start the monitoring system, and the passive sensor 62 is used to detect objects within the field of view. If the passive sensor 62 does not detect the temperature change, step S204 is performed to maintain the image acquisition unit 66 in the power saving mode; if the passive sensor 62 detects a temperature change, step S206 is performed to enable the passive sensor 62 to transmit a trigger signal to switch the image acquisition unit 66 from the power saving mode to the wake-up mode. Next, steps S208 and S210 are performed, the light emitting unit 72 is started according to the ambient brightness, the image obtaining unit 66 obtains the (low quality) first monitor image, and the operation processor 68 simply analyzes the first monitor image to determine whether to start the external host 64.
In one embodiment, the image acquisition unit 66 acquires a low-quality monitor image using a part of pixels, for example, groups the number of pixels into a plurality of pixel blocks of 2×2, and acquires an image using one pixel in each pixel block. In other possible embodiments, the image acquisition unit 66 acquires an image using all pixels, divides all pixels into a plurality of pixel blocks (e.g., 2×2 pixel blocks), and then combines the values of all pixels in each pixel block into a block value to generate a low-quality monitoring image according to the plurality of block values.
In step S210, the operation processor 68 preferably analyzes a specific region of interest in the first monitoring image to determine whether to activate the external host 64, wherein the size of the specific region of interest is smaller than that of the first monitoring image, so that the operation processor 68 can quickly obtain the image analysis result due to the small data processing amount of the region of interest. Setting the first monitoring image to a low quality monitoring image helps to speed up the image analysis of a specific region of interest. The location and size of the region of interest is preferably preset by the user; for example, when the first monitoring image has a gate and a window, and the region of interest only covers the gate pattern, the image analysis result can be prevented from being influenced by the swing of the outdoor leaf shadow of the window, or the region of interest can cover the edge of the window, so as to detect whether a thief climbs the window or not, and prevent the image analysis result from being influenced by the door shadow. The location and size of the region of interest may further vary as a result of the image analysis. However, it is also possible for the operation processor 68 to analyze the whole area in the first monitoring image to execute step S210, which varies depending on the design requirement. The image analysis technique can be completed by identifying the pattern contour in the monitoring image, comparing the characteristic points of the monitoring image, and selectively analyzing the intensity variation of the monitoring image.
When the object does not meet the default condition, for example, the passing object in the monitored image is an animal, but not a human, step S212 is performed without activating the external host 64, and the image acquisition unit 66 may be turned off actively (time-to-automatic) or passively (according to the external command generated by the analysis result of the monitored image) to return to the power saving mode. If the object meets the default condition, i.e. the passing object in the monitoring image is an unauthorized human, step S214 is performed to start the external host 64 and the image acquisition unit 66 starts to acquire a high quality second monitoring image; the second monitoring image may be in a still image format or a continuous video format, and may be stored in the memory 70. Next, step S216 is executed to enable the external host 64 to receive the second monitoring image, and the external host 64 uses the image recognition algorithm to accurately recognize the object in the second monitoring image.
When the second monitored image does not meet the predetermined threshold, i.e. the object is not an unauthorized person, step S218 is performed to actively or passively turn off the motion detection device 60 to save energy. If the second monitored image meets the predetermined threshold, the object is defined as an unauthorized person, and step S220 is performed to enable the external host 64 to switch the mobile detection device 60 to the video mode, the mobile detection device 60 can backup the video, and the other mobile detection devices 60' can wake up at the same time to provide comprehensive monitoring. Therefore, the passive sensor 62 does not directly activate the external host 64 when detecting the object, the mobile detection device 60 wakes up to obtain the first monitoring image by the triggering of the passive sensor 62, and then the external host 64 determines whether to activate according to the low-quality image analysis result of the first monitoring image obtained by the mobile detection device 60.
The motion detection device 60 starts to acquire the second monitoring image after the external host 64 is started. The external host 64 must wait for a period of time before the other mobile detection device wakes up to wake up the other mobile detection device, and the second monitoring image can record any suspicious object occurring in the monitoring area before the other mobile detection device wakes up, that is, the monitoring system will not leak the suspicious object during the period of time before the other mobile detection device wakes up after the passive sensor 62 detects the abnormality. The motion detection device 60 uses the low-quality first monitoring image to determine the existence of the object, and the correlation analysis determination of the existence is simply operation and may be affected by noise interference; the external host 64 further utilizes the high quality second monitoring image to analyze the accurate motion detection of the object, such as the object motion detection using facial recognition techniques.
The present invention further provides a real-time exposure adjustment function to enable the motion detection device 60 to have a preferred operation performance. Please refer to fig. 4 and fig. 5. Fig. 4 is a flowchart of a motion detection method applied to the motion detection device 60 according to another embodiment of the present invention, and fig. 5 is a schematic diagram of a frame rate change shown by the image acquisition unit 66 according to the foregoing embodiment of the present invention. In this embodiment, the steps having the same numbers as those of the previous embodiment have the same contents, and thus are not described in detail. If the passive sensor 62 does not wake up the motion detection device 60, step S205 may be performed after step S202, and the image acquisition unit 66 may be periodically switched to the wake-up mode to operate at a low frame rate, so that the image acquisition unit 66 in the wake-up mode may perform exposure adjustment while acquiring a low quality background image. If the motion detection device 60 is awakened, step S207 may be performed after step S206, and the image acquisition unit 66 is switched to the awake mode to operate at Gao Yingge; at this time, the image acquisition unit 66 can still acquire a low-quality monitoring image for judging whether to activate the external host 64 as compared with the background image.
For example, as shown in fig. 5, the image capturing unit 66 may obtain a background image every second and perform the exposure adjustment function when the passive sensor 62 has not triggered the motion detection device 60, i.e. the background images are obtained at the time points T1, T2, T3 and T4, respectively, and the exposure parameters of the image capturing unit 66 can be adjusted accordingly in real time. When the passive sensor 62 triggers the motion detection device 60 at the time point T5 and enters the wake-up mode, the motion detection device 60 can acquire the first monitor image at a frame rate of 30 frames per second, and since the exposure parameter of the latest background image (acquired at the time point T4) is quite similar to the exposure parameter of the first monitor image acquired at the time point T5, the image acquisition unit 66 in the wake-up mode can acquire the preferred monitor image with the proper exposure parameter in real time without performing exposure adjustment.
In summary, the mobile detection device of the present invention is electrically connected between the passive sensor and the external host, and the mobile detection device can activate the external host after the passive sensor switches the mobile detection device from the power saving mode to the wake-up mode. When the mobile detection device is in the power saving mode, the mobile detection device can be awakened at intervals in the mode of low image rate or can adjust exposure parameters in the power saving mode to acquire a background image; when the motion detection device is in the wake-up mode, the motion detection device is operated at Gao Yingge to obtain low-quality monitoring images. The mobile detection device firstly uses the interested area of the low-quality monitoring image to execute simple image analysis and judges whether to start an external host; after the external host is started, the mobile detection device acquires and stores the high-quality monitoring image, so that the external host can perform accurate image analysis according to the high-quality monitoring image so as to start related application programs. The mobile detection device can effectively shorten the starting time of the monitoring system, and does not need to wait for the wakeup time of an external host and the exposure adjustment time of the mobile detection device in a time-consuming manner.
Referring to fig. 6 and 7, fig. 6 is a functional block diagram of an intelligent motion detection device 80 according to a first embodiment of the present invention, and fig. 7 is a program diagram of the intelligent motion detection device 80 according to the first embodiment of the present invention. The intelligent motion detection device 80 may include a memory module 82, a processor 84, and a sensing module 86. The memory module 82, the processor 84 and the sensing module 86 may be three separate components or one or two integrated components. The sensing module 86 may be directly coupled to the memory module 82 and further electrically connected to the processor 84. The sensor module 86 may include a plurality of photo-detecting pixels arranged in a two-dimensional manner for capturing an image. The processor 84 can switch between sleep mode and wake mode for image processing the images acquired by the sensor module 86 to identify specific events within the acquired images, such as unexpected objects appearing in the acquired images.
The sensing module 86 can pre-store (i.e. read/write) the acquired image in the memory module 82 or directly transmit the acquired image to the processor 84 according to the operation mode of the processor 84 or the warning signal generated by the motion detection result. The image capacity of the memory module 82 has a default value, and if the memory module 82 is full and still has a new image to be pre-stored, all or part of the previous image is removed to take space for storing the new image. In addition, the image processed by the processor 84 and the pre-stored image stored in the memory module 82 may be transferred to the external storage module 88, and the external storage module 88 is electrically connected to the intelligent motion detection device 80.
As shown in the first embodiment of fig. 7, the processor 84 operates in the sleep mode when the intelligent motion detection device 80 has not been activated. The sensing module 86 may include a comparator 90 for generating a warning signal when movement of the object is monitored. When the processor 84 is operating in the sleep mode, the sensing module 86 may continuously or intermittently acquire a plurality of images, such as five images per second, which are all pre-stored in the memory module 82. At the same time, the comparator 90 reads one or several pre-stored images I1 from the plurality of pre-stored images I1 and compares them with the reference image. If the change in intensity between the pre-stored image I1 and the reference image is below a default value, the processor 84 remains in sleep mode and the comparator 90 reads the next pre-stored image I1 and compares it to the reference image. If the intensity variation between the pre-stored image I1 and the reference image exceeds a predetermined value, the comparator 90 may generate a warning signal to wake up the processor 84 and pre-store the image acquired by the sensing module 86 to the memory module 82. Thus, the alert signal is used to switch the processor 84 from the sleep mode to the wake mode.
The comparator 90 of the present invention can compare the pre-stored image I1 with the reference image in a variety of ways, for example, the comparator 90 can compare the pre-stored image I1 with the entire image range of the reference image or only a portion of the image range. The comparator 90 may compare the intensity sum of all pixels or the intensity sum of part of pixels; alternatively still, the comparator 90 may compare with each pixel within the entire image or with only the pixel intensities of the internal regions within the image.
When the processor 84 is operating in the wake-up mode, the real-time image I2 obtained by the sensing module 86 can be directly transmitted to the processor 84 for digital image processing, and is not stored in the memory module 82. The processor 84 operating in the wake-up mode may alternatively perform image processing on the real-time image I2 and receive the pre-stored image I1 from the memory module 82, or may receive the pre-stored image I1 after the image processing of the real-time image I2 is completed. The image processing of the real-time image I2 may take precedence over the image processing of the pre-stored image I1, so that the intelligent motion detection device 80 can focus on processing real-time conditions within the monitoring range. The image processing of the pre-stored image I1 may be started when the image processing of the real-time image I2 is completed or paused. If the processor 84 has sufficient operation performance to cope with the huge amount of data, the real-time image I2 and the pre-stored image I1 can be alternately processed, i.e. the intelligent motion detection device 80 can provide the detection results of the current time period and the previous time period simultaneously.
In some possible embodiments, the pre-stored image acquired by the sensing module 86 when the processor 84 is operating in the sleep mode is pre-stored to the memory module 82, and the image acquired by the sensing module 86 when the processor 84 is operating in the wake mode may be transferred to the processor 84. In other possible embodiments, the processor 84 and the sensing module 86 may be turned off in the non-operating mode context; when the intelligent motion detection device 80 receives the trigger signal, the sensing module 86 can acquire the image and directly transmit the image to the memory module 82, and then the processor 84 can send a request to the sensing module 86 to receive the acquired image. The trigger signal may be an alarm notification generated by an external unit or an alarm notification generated by a built-in unit of the intelligent motion detection apparatus 80.
In addition, either or both of the image quality and the picture update rate characteristics of the sensing module 86 may vary as the processor 84 operates in either sleep mode or wake mode. For example, when the processor 84 is operating in the sleep mode, the sensor module 86 can obtain a low quality or low refresh rate image for comparison with the reference image, thereby helping to save transmission bandwidth and storage capacity. If the intensity change between the low quality or low picture update rate image and the reference image exceeds a predetermined value, a warning signal is generated so that the sensor module 86 can start to acquire the high quality or high picture update rate image for pre-storing to the memory module 82 and can also switch the processor 84 to the wake-up mode at the same time. Then, the pre-stored high quality image or the pre-stored high refresh rate image in the memory module 82 can be transferred to the processor 84 when the processor 84 is operating in the wake-up mode, so that the intelligent motion detection device 80 will not lose important image information before the processor 84 switches to the wake-up mode.
Referring to fig. 8 to 11, fig. 8 is a functional block diagram of an intelligent motion detection device 80 'according to a second embodiment of the present invention, fig. 9 is a program diagram of the intelligent motion detection device 80' according to the second embodiment of the present invention, fig. 10 is a functional block diagram of an intelligent motion detection device 80″ according to a third embodiment of the present invention, and fig. 11 is a program diagram of the intelligent motion detection device 80″ according to the third embodiment of the present invention. In the second embodiment and the third embodiment, elements having the same numbers as those in the first embodiment have the same structures and functions, and the description thereof will not be repeated.
In a possible embodiment, the intelligent motion detection device 80 'may include a memory module 82, a processor 84, a sensing module 86', and a passive sensor 92. The passive sensor 92 may electrically connect the processor 84 with the sensing module 86'. When no anomalies are detected by the passive sensor 92, the sensor module 86' is turned off and the processor 84 remains in sleep mode. When the passive sensor 92 detects movement of the object, the passive sensor 92 generates a warning signal that can be used to activate the sensor module 86' and switch the processor 84 from the sleep mode to the wake mode. While the processor 84 is still operating in the sleep mode, the sensing module 86' can acquire the pre-stored image I1 and transmit the pre-stored image I1 to the memory module 82. If the processor 84 is operating in the wake-up mode, the sensing module 86' can acquire the real-time image I2 and transmit the real-time image I2 to the processor 84, and the pre-stored image I1 in the memory module 82 can also be correspondingly transmitted to the processor 84.
The intelligent motion detection device 80 'may have a non-operational mode in which the processor 84 and the sensor module 86' may be turned off. When the passive sensor 92 detects the movement of the object and generates a warning signal, the warning signal triggers the sensor module 86', so that the sensor module 86' starts to acquire the pre-stored image and transmits the pre-stored image to the memory module 82. The processor 84 may then be switched to the wake-up mode and transmit a request to the sensor module 86' for subsequent receipt of the pre-stored image.
In other possible embodiments, the intelligent motion detection device 80″ may include a memory module 82, a processor 84, a sensing module 86″ having a comparator 90, and a passive sensor 92. The passive sensor 92 may trigger the sensing module 86 "when an anomaly is detected. The triggered sensing module 86″ can acquire the pre-stored image I1 and transmit the pre-stored image I1 to the memory module 82, and the comparator 90 can compare the pre-stored image I1 with the reference image to determine whether to switch the mode of the processor 84. The comparator 90 is used to identify anomalies. If the intensity variation between the pre-stored image I1 and the reference image is below the default value, the anomaly may be caused by noise, so that the processor 84 is not awakened; if the intensity variation between the pre-stored image I1 and the reference image exceeds a default value, an anomaly may represent that someone or an object is invading the monitoring range of the intelligent motion detection device, so that the processor 84 may be switched to the wake-up mode for recording. When the processor 84 is operating in the wake-up mode, the real-time image I2 acquired by the sensing module 86″ and the pre-stored image I1 in the memory module 82 may be transferred to the processor 84 and then further transferred to the external storage module 88 to perform digital image processing.
Referring to fig. 12, fig. 12 is a flowchart of a determining method according to an embodiment of the invention. The judging method shown in fig. 12 is applicable to the intelligent motion detecting device shown in fig. 6 to 11. First, step S800 and step S802 are performed to start the determining method to monitor the movement of the object, and the monitoring function can be performed by the sensor modules 86, 86' and 86″ or the passive sensor 92. If no anomaly is detected, step S804 is performed to maintain the processor 84 in the sleep mode. If movement of the object is detected, steps S806 and S808 are performed to generate a warning signal to enable the processor 84 and to acquire images via the sensor modules 86, 86' and 86″. When the processor 84 is not operating in the wake-up mode, step S810 is performed to enable the sensing modules 86, 86' and 86″ to generate the pre-stored image I1 in the memory module 82. When the processor 84 is operating in the wake-up mode, the steps S812 and S814 are performed, the sensing modules 86, 86' and 86″ generate the real-time image I2, and the pre-stored image I1 and the real-time image I2 can be transmitted to the processor 84.
Next, after executing step S816 and triggering the capturing function of the sensing module 86, 86 'or 86", the processor 84 may analyze the real-time image I2 acquired by the sensing module 86, 86' or 86". Perhaps because the object suddenly disappears or otherwise is special, the sensing modules 86, 86' and 86 "are not activated, and step S818 may be performed to analyze the pre-stored image I1 in the memory module 82 by the processor 84. It should be noted that the processor 84 may not only perform the image processing of the real-time image I2 before the pre-storing the image I1, but also alternatively perform the image processing of the pre-storing image I1 and the real-time image I2 alternately according to the actual requirement of the user and the effective operation performance.
In summary, the warning signal may be generated by a sensing module or a passive sensor (e.g., a thermal sensor, an accelerometer, or a gyroscope). The warning signal is used for triggering a pre-storage function of the sensing module and a mode switching function of the processor. Upon receiving the warning signal, the sensing module may thus trigger and acquire the pre-stored image at the first time, and the pre-stored image may be transmitted to the memory module. After a period of time has elapsed, after the processor has switched from sleep mode to wake mode, the processor receiving the alert signal may send a request to the sensing module at a second time associated with the real-time image and the pre-stored image. The second time is later than the first time, the pre-stored image in the memory module is subjected to image processing after the first time, and the real-time image is directly transmitted to the processor for image processing and is not stored in the memory module. Compared with the prior art, the intelligent motion detection device and the related judging method thereof can acquire the detection image without waiting for waking up the processor, and can effectively shorten the starting time of the intelligent motion detection device.
Referring to fig. 13 and 14 in combination, fig. 13 is a block diagram of a monitoring system 1300 according to an embodiment of the present application, and fig. 14 is a schematic diagram of three different operation schemes of various parts or elements of the monitoring system 1300 in fig. 13. As shown in fig. 13, the monitoring system 1300 includes three parts, namely an analog integrated circuit (integrated circuit, IC) part or component, a digital integrated circuit part or component, and a back-end system device. In other embodiments, the analog integrated circuit portion or element and the digital integrated circuit portion or element may be integrated together to implement an image sensor device 1301, wherein the image sensor device 1301 may also be a single integrated circuit. That is, the image sensor device 1301 may be divided into an analog portion (i.e., the analog integrated circuit portion or element) and a digital portion (i.e., the digital integrated circuit portion or element).
The analog integrated circuit portion or component is a plurality of analog circuits or a collection of components, such as may be (or include) an analog image sensor circuit, such as an analog image sensor 1305 including an event camera 1306, the event camera 1306 may also be referred to as an event sensor and includes at least one pixel element (e.g., at least one active pixel element), that is, one or more pixel elements (e.g., a plurality of pixels or a plurality of sub-pixels). It should be noted that in practice, an event camera unit/circuit may include a pixel unit and be configured to report a change in brightness when the brightness is changed and not when the brightness is not changed. That is, the event camera 1306 is configured to detect whether one or more brightness changes have occurred in one or more pixel units.
The digital integrated circuit portion or component is a collection of digital circuits or components that may be (or include) a digital processing circuit including a first image buffer 1311 and a motion detector, such as a smart motion detector (smart motion detector, SMD) 1312. When the event camera 1306 transmits information of a change in a pixel image or its value, the intelligent motion detector 1312 may use the information of the pixel level to generate frame data (i.e., an image of a frame level) and may detect whether motion has occurred. If it is determined that a movement has occurred, the smart movement detector 1312 may generate a warning signal to the backend system 1315. Otherwise, if no movement occurs, the smart movement detector 1312 does not generate the warning signal. The intelligent motion detector 1312 can accurately detect whether actual motion is occurring and can filter out some unwanted image disturbances, such as moving images of swaying leaves or swaying grass; this is not a limitation of the present application.
Upon receiving a warning signal sent from the digital processing circuit 1310, the back-end system 1315 is configured to receive the image stream (i.e., frames) from the digital processing circuit 1310 and, for example, to initiate performance of a video recording operation. The back-end system 1315 includes a second image buffer 1316 and an external processor 1317 coupled externally to the image sensor device (i.e., the analog integrated circuit portion or component and the digital integrated circuit portion or component).
The start-up speed or time of the event camera 1306 (or the analog image sensor circuit 1305) is substantially faster than the start-up speed or time of the smart movement detector 1312 (or the digital processing circuit 1310) and is also substantially faster than the start-up speed or time of the external processor 1317 (or the back-end system device 1315). In one embodiment, if there is no pixel change, only the power of the event camera 1306 (or only the analog image sensor circuit 1305) is powered, and the power of the other circuits (i.e., the digital processing circuit 1310 and the back-end system device 1315) are turned off or left in a power saving mode, which may operate at a low operating frequency, for example, to save more power. Once the event camera 1306 (or the analog image sensor circuit 1305) detects a pixel value change, the smart motion detector 1312 wakes up by a trigger signal sent from the analog image sensor circuit 1305 to perform the motion detection described above. Only when the result of the motion detection indicates that an actual motion has occurred, the external processor 1317 in the back-end system device 1315 is awakened by a trigger signal sent from the digital processing circuit 1310 to perform additional image processing and/or video recording operations. In other embodiments, if the event camera 1306 includes a plurality of pixels, then "no pixel change" may refer to a situation where the number of pixels that actually change in pixel value is below a first particular threshold, and "change in pixel value" may refer to a situation where the number of pixels that actually change in pixel value exceeds a second particular threshold.
If the pixel values change before the smart movement detector 1312 is fully awake, then the one or more pixel values sensed by the image sensor circuit 1305 are stored in the first image buffer 1311, which are transferred or forwarded to the smart movement detector 1312 once the smart movement detector 1312 is fully awake to be able to receive the sensed one or more pixel values. That is, when the event camera 1306 is about to send one or more pixel values to the digital processing circuit 1310, the one or more pixel values are stored in the first image buffer 1311 before the smart movement detector 1312 is awakened, and the first image buffer 1311 is used to store a plurality of pixel values from the event camera 1306 before the smart movement detector 1312 is awakened. Then, similarly, before the external processor 1317 is fully awakened, when it is determined that the frame data formed by the sensed one or more pixel values is related to an actual movement event, one or more frames or one or more image streams processed by the digital processing circuit 1310 are stored in the second image buffer 1316, and then once the external processor 1317 is fully awakened to be able to receive one or more frames or one or more image streams, the one or more frames or one or more image streams are transferred or forwarded to the external processor 1317. Furthermore, when the digital processing circuit 1310 is to send one or more frames to the back-end system device 1315, the one or more frames are stored in the second image buffer 1316 before the external processor 1317 is woken up, and the second image buffer 1316 can be used to store frames from the digital processing circuit 1310 before the external processor 1317 is woken up. By using the above mechanism and the first and second image buffers, more power can be saved from being consumed, and image loss can be avoided.
As shown in fig. 14, for example, the analog image sensor circuit 1305 is permanently awake (i.e., in an awake state or in a normal mode other than a power saving mode) when powered to continuously or periodically detect whether one or more pixel values of one or more pixel units are changed. In a first operational scenario (but not limited to), if no pixel change is determined, the analog image sensor circuit 1305 does not send a trigger signal to wake up the digital processing circuit 1310 and the back-end system device 1315. Thus, the digital processing circuit 1310 and the back-end system device 1315 remain asleep or in the power-saving mode, while the analog image sensor circuit 1305 does not transmit pixel data to the digital processing circuit 1310 and the back-end system device 1315.
In a second operational scenario, but not limited to, if at least one pixel value is determined to be changed, the analog image sensor circuit 1305 is configured to send a trigger signal to wake up the digital processing circuit 1310 and also send sensed pixel data to the digital processing circuit 1310. In the case where the activation speed/time of the first image buffer 1311 is substantially faster than the activation speed/time of the smart movement detector 1312, the sensed pixel data may be temporarily stored in the first image buffer 1311 by using an image freeze operation before the smart movement detector 1312 wakes up completely. After the digital processing circuit 1310 completely leaves the power saving mode and enters the normal mode, the pixel data sensed by the analog image sensor circuit 1305 may be directly transferred to the smart motion detector 1312 without using the first image buffer 1311. In this case, if it is determined that no actual movement has occurred, the digital processing circuit 1310 does not send a warning signal to wake up the back-end system device 1315, and does not transmit the generated frames or image streams to the back-end system device 1315.
It should be noted that once the smart motion detector 1312 has completely entered the normal mode, the smart motion detector 1312 is able to simultaneously receive and process the incoming pixel data and the stored pixel data of the first image buffer 1311. For example and without limitation, the intelligent motion detector 1312 can employ an image processing frequency of a super high frame rate or a higher transmission frame number to rapidly process the accumulated pixel values stored in the first image buffer 1311 and then synchronize the processed pixel values with the analog values of the incoming pixel data. For example, each time the analog image sensor circuit 1305 is called to sense a plurality of pixel units, the intelligent motion detector 1312 is used to process the accumulated pixel values stored in the first image buffer 1311 and process the analog values of the incoming pixel data in parallel until the first image buffer 1311 becomes empty.
In a third operational aspect (but not limited to), the first image buffer 1311 is capable of collecting and storing pixel information (pixel values or pixel differences) for a plurality of pixel units transmitted from the analog image sensor circuit 1305, and is capable of processing, forming and generating data for one or more complete frames. The intelligent motion detector 1312 can determine whether an actual motion has occurred based on the generated frame data. If it is determined that an actual movement has occurred, the digital processing circuit 1310 sends a warning signal to wake up the back-end system device 1315 and also sends the sensed frames or streams of images to the back-end system device 1315. In this case, the sensed frames or streams of images may be temporarily stored in the second image buffer 1316 using the image freeze operation before the back-end system device 1315 wakes up completely. When the back-end system device 1315 completely exits the power-saving mode and enters the normal mode, the sensed frames or streams of images may be transferred directly to the external processor 1317 without being buffered in the second image buffer 1316.
Please refer to fig. 13 again. Specifically, for determining whether a pixel value (one or each) has changed, the event camera 1306 is configured to sense or capture the current pixel value of a corresponding pixel unit to detect whether the pixel value has changed. The event camera 1306, for example, senses or captures the current pixel value of the pixel unit (S13051), and then calculates a pixel difference Diff between the sensed current pixel value and a reference pixel value of the pixel unit (S13052), wherein the reference pixel value of the pixel unit may be a previous pixel value sensed by the event camera for the pixel unit at an earlier point in time, or may be an average of a plurality of pixel values sensed by the event camera for the pixel unit at an earlier point in time. Thus, as such, for a plurality (or all) of pixel units, the event camera 1306 is capable of capturing a plurality of current pixel values and then calculating or generating a plurality of pixel differences corresponding to the plurality of pixel units, respectively.
Next, for the pixel value, the event camera 1306 determines whether the pixel value has changed by comparing the pixel difference with a pixel threshold TH (S13053). If the pixel difference Diff is higher than the pixel threshold TH, the event camera 1306 can determine that the pixel value has changed. Conversely, if the pixel difference Diff is not higher than the pixel threshold TH, the event camera 1306 determines that the pixel value has not changed. It should be noted that in the present embodiment, the pixel value is changed to mean that the pixel value is changed greatly, and the pixel value is not changed to mean that the pixel value is not changed at all or is changed only slightly.
When the pixel difference Diff is equal to or higher than the pixel threshold TH, the event camera 1306 is configured to generate and send a trigger signal to wake up the digital processing circuit 1310, send information/data related to the currently captured pixel value and a currently counted value of a counter value N to the digital processing circuit 1310, update the reference pixel value by using the currently captured pixel value, and reset the counter value N to zero. The counter value N is initially set to zero. If the smart motion detector 1312 is not in the normal mode, the value of the currently counted counter value N and the information/data related to the currently captured pixel value are temporarily stored in the first image buffer 1311. It should be noted that the pixel difference Diff of the pixel cell is recalculated and updated after each exposure operation performed on the pixel cell is completed to obtain the pixel value it is currently sensing. When the pixel difference Diff is not higher than the pixel threshold TH, the event camera 1306 adds one or more to the counter value, and neither of the event camera 1306 sends the trigger signal, the currently counted value of the counter value N, and information/data related to the currently captured pixel value.
The value of the counter N is used to indicate the number of frames in which the pixel value of a pixel unit has not changed within the time points of a plurality of consecutive frames, or equivalently, the counter N used by the event camera 1306 is also used to determine the time interval between two changes in the pixel value of the pixel unit. It should be noted that the values of the counter values N corresponding to the different pixel units may be identical, different or partially different. In one embodiment, a resulting counter value, such as a minimum of all counter values N, may be selected from the values corresponding to the counter values N of a plurality of different pixel units, and the selected resulting counter value may be used to indicate a frame number in which all pixel values of all pixel units have not changed within a time point of a plurality of consecutive frames.
Further, for example, but not limited to, the frame rate may be equal to 30Hz, i.e., 30 frames per second, and if it is determined that the pixel values of the pixel units have not changed within a second, the counter value N may be sequentially incremented from zero by 1 to 30 in the second, at which time, if it is determined that the pixel values have changed at the next frame time, the counter value N will not be counted to, for example, 31, but the event camera 1306 will generate a trigger signal to wake up the digital processing circuit 1310 and transmit the currently counted counter value (i.e., 30) to the digital processing circuit 1310 before the counter value N is reset to zero. Furthermore, at the same point in time, the event camera 1306 also outputs information/data of the sensed/captured pixel values to the digital processing circuit 1310.
The information/data of the sensed/captured pixel values that are changed may be transferred from the analog image sensor circuit 1305 to the digital processing circuit 1310 using at least two different methods. In one embodiment, the event camera 1306 may send an actual value of the currently captured pixel value to the digital processing circuit 1310, and the digital processing circuit 1310 may directly use the actual value to replace a corresponding value of a pixel unit in a previous frame to generate a current frame. Furthermore, in other embodiments, the event camera 1306 sends a difference between an actual value of an actual captured pixel value and an actual value of a previously captured pixel value to the digital processing circuit 1310, and the digital processing circuit 1310 may add the difference to a corresponding value of a pixel unit in a previous frame to generate a current frame.
Referring to fig. 15, fig. 15 is a schematic diagram of two different operations of the digital processing circuit 1310 (or intelligent motion detector 1312) to update and generate data for a current frame. For a pixel cell, such as an active pixel cell, even if some of the circuitry within the smart motion detector 1312 is powered down, the data of the last previous frame may be stored in other portions of the circuitry within the smart motion detector 1312 as a static frame FS. The still image frame FS is generated by the digital processing circuit 1310 at an earlier point in time when the event camera 1306 detected a change in one or more pixel values, i.e., the still image frame FS is associated with one or more previous pixel values. As shown in a first scheme of fig. 15, when it is determined that the pixel value of the active pixel unit changes at a current frame time point, the event camera 1306 in the first scheme transmits the actual pixel value of the active pixel unit to the image buffer 1311 of the digital processing circuit 1310, so that the image data related to the change of the pixel value can be stored in the image buffer 1311. Then, after the intelligent motion detector 1312 completely enters the normal mode, the intelligent motion detector 1312 may generate the frame FC at the current frame based on the data of the static frame FS and the actual value of the pixel value. In practice, the intelligent motion detector 1312 is arranged to use the actual value to replace the previous value of the active pixel element in the static frame FS to generate a current frame FC.
Furthermore, as shown in a second scheme of fig. 15, for the active pixel unit, when it is determined that the pixel value of the active pixel unit changes at the current frame time point, the event camera 1306 in the second scheme transmits the difference between the previously captured pixel value and the currently captured pixel value corresponding to the active pixel unit to the image buffer 1311 of the digital processing circuit 1310. Then, after the intelligent motion detector 1312 completely enters the normal mode, the intelligent motion detector 1312 may generate a current frame FC based on the data of the static frame FS and the pixel difference. In practice, the intelligent motion detector 1312 adds the pixel difference to the previously captured pixel values of the same active pixel unit in the static frame FS to generate the current frame FC.
In one embodiment, when the current frame FC is generated, the intelligent motion detector 1312 (or digital processing circuit 1310) is used to determine whether a motion has occurred by generating one or more background frames based on the generated current frame FC and the counter value N and then comparing the current frame FS with the background frames. In practice, information/data of a previous background frame may be stored in a memory circuit of the smart motion detector 1312 so that the information stored in the memory circuit does not disappear even if the smart motion detector 1312 is powered off. If the smart movement detector 1312 has been maintained in the normal mode, the smart movement detector 1312 may perform a recursive moving average calculation operation based on a previous background frame and the current frame FC to generate a current background frame. By way of example (but not limitation), the data of the current background frame may be determined using the following equation:
Wherein FB is i Refer to the data, FB, of a current background frame generated at a current frame time point i i-1 Refers to the data of a previous background frame generated at a previous frame time point i-1, and FC refers to the data of a current frame (or referred to as a previous data frame). Similarly, the data of the next background frame generated based on the recursive moving average calculation operation can be determined by the following equation:
where FC' refers to the next data frame. Typically in a monitoring system, after a certain training time period has elapsed, the data of a curve corresponding to a plurality of background frames at a plurality of different frame time points will be converged to the data of a stable background frame.
However, if the smart movement detector 1312 has just left the power saving mode and there is no previous background frame FB at this time i-1 The smart motion detector 1312 does not perform the recursive moving average calculation operation, and the smart motion detector 1312 uses the counter value N to generate a weight value and then uses the weight value, the current data frame FC, and the last background frame FB stored in the memory circuit of the smart motion detector 1312 when the power of the smart motion detector 1312 is to be turned off i-N . At this time, the current background frame FB i May be determined by the following equation:
FB i ＝∝×FB i-N +(1-∝)×FC
wherein ≡is the weight value generated based on the counter value N. In a basic example (but not limited to), the weight value oc may be equal toOr->For example, if the value N is equal to 30, then ∈e.g. equal +.>The current background image frame FB i Can be determined by the following equation:
and
In addition, the current background image frame FB i It may also be determined by the following equation:
however, none of the above embodiments are limiting of the present application. In other words, the value of the weight value ≡ may be dynamically determined based on the number of frames in which the image/pixel value of one pixel unit is unchanged for several consecutive frames. Since performing a recursive moving average calculation at this time necessarily requires a recursive calculation to produce a plurality of background frames FB i-1 、…、FB i-(N-1) Can generate the current background image frame FB i So the counter value N is used to directly generate the current background frame FB i The calculation operation without performing the recursive moving average calculation can simplify the calculation.
For determining whether a motion has occurred in the current frame FC, the intelligent motion detector 1312 calculates the current background frame FB i And comparing the frame difference with a movement threshold THM to judge whether movement occurs. When the frame difference is higher than the motion threshold THM, the intelligent motion detector 1312 may determine that motion has occurred and then generate a warning signal to the backend system device 1315.
The movement threshold THM may be, for example, a fixed threshold value or may be a value that is dynamically adjusted. In one embodiment, the intelligent motion detector 1312 can use a curve fitting (curve fitting) operation to determine the current background frame FB based on a previous value of the motion threshold THM i The current frame FC and the counter value N dynamically adjust the motion threshold THM.
Next, for the background frame FB for generating the next sheet i+1 When using the slaveWhen the information/data of pixel values sent by the camera 1306 generates the next data frame FC', the intelligent motion detector 1312 may, for example, generate a current background frame FB based on i And the next data frame FC' to perform a recursive moving average calculation.
Fig. 16 is provided to make the operation of the present application described above more clear to the reader. Fig. 16 is a flowchart of the operation of the smart movement detector 1312 that has just entered the normal mode according to one embodiment of the present application. The step descriptions are described below:
step 1605: starting;
step 1610: generating a current data frame FC based on the previous data/static current frame FS and information or data (e.g., actual pixel values or pixel differences) of the changed image/pixel values of the pixels;
step 1615: generating a weight value oc based on the counter value N received from the event camera 1306 and buffered in the first image buffer 1311;
step 1620: generating a current background frame according to the weight value oc, the last background frame stored in the intelligent motion detector 1312 and the current data frame FC;
step 1625: calculating a frame difference between the current data frame FC and the current background frame;
step 1630: dynamically updating the value of the mobile threshold THM according to the previous or original excitation value of the mobile threshold THM, the last background frame and the current data frame FC;
Step 1635: judging whether the frame difference is equal to or higher than the movement threshold THM; if the frame difference is equal to or higher than the movement threshold THM, the flow proceeds to step 1645, otherwise, the flow proceeds to step 1640;
step 1640: the warning signal is not generated; and
step 1645: the warning signal is generated.
The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (14)
1. An analog image sensor circuit for coupling to a digital processing circuit for performing motion detection, the analog image sensor circuit comprising:
an event camera including a pixel unit;
the event camera is used for sensing pixel values of the pixel units to detect whether the pixel values are changed or not; when the pixel value changes, the event camera is used for triggering the digital processing circuit and transmitting the information of the pixel value to the digital processing circuit; when the pixel value changes, the event camera further transmits a counter value to the digital processing circuit; the counter value corresponds to a time interval between events of the digital processing circuit entering a power saving mode and events of the pixel value change; and when the pixel value is unchanged at a particular frame time point, the counter value is incremented by one and is not transmitted.
2. The analog image sensor circuit of claim 1, wherein said information of said pixel values and said counter values are stored in a first image buffer of said digital processing circuit before said digital processing circuit completely leaves said power saving mode and enters a normal mode, said digital processing circuit performing said motion detection based on said information of said pixel values and said counter values stored in said first image buffer after completely entering said normal mode.
3. The analog image sensor circuit of claim 1, wherein the event camera is to:
when the pixel difference between the reference pixel value of the pixel unit and the current pixel value of the pixel unit is higher than a pixel threshold value, judging that the pixel value is changed;
generating the information for the pixel values;
outputting the information of the counter value and the pixel value to the digital processing circuit;
updating the reference pixel value to the current pixel value; and
resetting the counter value to zero.
4. The analog image sensor circuit of claim 1, wherein the event camera is to:
When the pixel difference between the reference pixel value of the pixel unit and the current pixel value of the pixel unit is lower than a pixel threshold value, judging that the pixel value is not changed;
not outputting the information of the counter value and the pixel value to the digital processing circuit; and
the counter value is incremented by one.
5. The analog image sensor circuit of claim 1, wherein the event camera is to transmit a trigger signal to wake up the digital processing circuit in the power saving mode when the event camera determines that the pixel value is changed.
6. The analog image sensor circuit of claim 1, wherein the information of the pixel values is actual image capture values of the pixel values.
7. The analog image sensor circuit of claim 1, wherein the information of the pixel value is a difference between a current pixel value of the pixel cell and a previous pixel value of the pixel cell.
8. A method of simulating an image sensor circuit for external coupling to a digital processing circuit for performing motion detection, the method comprising:
Providing an event camera comprising pixel units;
sensing a current pixel value of the pixel unit using the event camera to detect whether the pixel value is changed;
triggering the digital processing circuit in a power saving mode using the event camera when the pixel value changes, and transmitting information of the pixel value to the digital processing circuit;
wherein the method further comprises:
transmitting a counter value to the digital processing circuit using the event camera when the pixel value changes; and
increasing the counter value by one when the pixel value does not change at a particular frame time point and not transmitting the counter value to the digital processing circuit;
wherein the counter value corresponds to a time interval between an event of the digital processing circuit entering a power saving mode and an event of the pixel value changing.
9. The method of claim 8, wherein the information of the pixel values and the counter values are stored to a first image buffer of the digital processing circuit before the digital processing circuit completely leaves the power saving mode and enters a normal mode, causing the digital processing circuit to perform the motion detection based on the information of the pixel values and the counter values stored in the first image buffer after the digital processing circuit completely enters the normal mode.
10. The method of claim 8, wherein the method further comprises:
when the pixel difference between the reference pixel value of the pixel unit and the current pixel value of the pixel unit is higher than a pixel threshold value, judging that the pixel value is changed;
generating the information for the pixel values;
outputting the information of the counter value and the pixel value to the digital processing circuit;
updating the reference pixel value to the current pixel value; and
resetting the counter value to zero.
11. The method of claim 8, wherein the method further comprises:
when the pixel difference between the reference pixel value of the pixel unit and the current pixel value of the pixel unit is lower than a pixel threshold value, judging that the pixel value is not changed;
not outputting the information of the counter value and the pixel value to the digital processing circuit; and
the counter value is incremented by one.
12. The method of claim 8, wherein the method further comprises:
when the pixel value changes, a trigger signal is transmitted from the event camera to wake up the digital processing circuit in the power saving mode.
13. The method of claim 8, wherein the information of the pixel values is actual image capture values of the pixel values.
14. The method of claim 8, wherein the information of the pixel value is a difference between a current pixel value of the pixel unit and a previous pixel value of the pixel unit.
Applications Claiming Priority (4)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US17/151,625 US11336870B2 (en) | 2017-12-26 | 2021-01-18 | Smart motion detection device and related determining method |
| US17/151,625 | 2021-01-18 | ||
| US17/376,127 US11622092B2 (en) | 2017-12-26 | 2021-07-14 | Image sensing scheme capable of saving more power as well as avoiding image lost and also simplifying complex image recursive calculation |
| US17/376,127 | 2021-07-14 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114827450A CN114827450A (en) | 2022-07-29 |
| CN114827450B true CN114827450B (en) | 2024-02-20 |
Family
ID=82526632
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111397742.9A Active CN114827450B (en) | 2021-01-18 | 2021-11-19 | Analog image sensor circuit, image sensor device and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114827450B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6388773B1 (en) * | 1999-02-24 | 2002-05-14 | Hewlett-Packard Company | Simultaneous multi-mode capture button behavior |
| CN102598074A (en) * | 2009-09-01 | 2012-07-18 | 德马荷工业相机股份有限公司 | Video camera system |
| CN109040583A (en) * | 2018-07-25 | 2018-12-18 | 深圳市共进电子股份有限公司 | Web camera energy-saving control method, device, web camera and storage medium |
| WO2019076954A1 (en) * | 2017-10-20 | 2019-04-25 | Defendec Oü | Intrusion detection methods and devices |
| WO2020195966A1 (en) * | 2019-03-27 | 2020-10-01 | ソニーセミコンダクタソリューションズ株式会社 | Imaging system, method of controlling imaging system, and object recognition system |
| CN112118380A (en) * | 2019-06-19 | 2020-12-22 | 北京小米移动软件有限公司 | Camera control method, device, equipment and storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2459701B (en) * | 2008-05-01 | 2010-03-31 | Pips Technology Ltd | A video camera system |
| US9743047B2 (en) * | 2013-05-27 | 2017-08-22 | Center For Integrated Smart Sensors Foundation | Network camera using hierarchical event detection and data determination |
-
2021
- 2021-11-19 CN CN202111397742.9A patent/CN114827450B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6388773B1 (en) * | 1999-02-24 | 2002-05-14 | Hewlett-Packard Company | Simultaneous multi-mode capture button behavior |
| CN102598074A (en) * | 2009-09-01 | 2012-07-18 | 德马荷工业相机股份有限公司 | Video camera system |
| WO2019076954A1 (en) * | 2017-10-20 | 2019-04-25 | Defendec Oü | Intrusion detection methods and devices |
| CN109040583A (en) * | 2018-07-25 | 2018-12-18 | 深圳市共进电子股份有限公司 | Web camera energy-saving control method, device, web camera and storage medium |
| WO2020195966A1 (en) * | 2019-03-27 | 2020-10-01 | ソニーセミコンダクタソリューションズ株式会社 | Imaging system, method of controlling imaging system, and object recognition system |
| CN112118380A (en) * | 2019-06-19 | 2020-12-22 | 北京小米移动软件有限公司 | Camera control method, device, equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 用于仿生视觉传感器的多方向运动目标识别系统设计;王含宇;《天津大学硕士论文》;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114827450A (en) | 2022-07-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11354993B2 (en) | Image surveillance and reporting technology | |
| US10424175B2 (en) | Motion detection system based on user feedback | |
| US20230209021A1 (en) | Image sensing scheme capable of saving more power as well as avoiding image lost and also simplifying complex image recursive calculation | |
| CN109963046B (en) | Motion detection device and related motion detection method | |
| US20090195382A1 (en) | Video sensor and alarm system and method with object and event classification | |
| WO2022011986A1 (en) | Wake-up method for surveillance camera, device, surveillance camera, and medium | |
| US11570358B1 (en) | Using remote sensors to resolve start up latency in battery-powered cameras and doorbell cameras | |
| US11405581B2 (en) | Motion detection methods and image sensor devices capable of generating ranking list of regions of interest and pre-recording monitoring images | |
| US11336870B2 (en) | Smart motion detection device and related determining method | |
| CN114827450B (en) | Analog image sensor circuit, image sensor device and method | |
| CN113963501A (en) | Shooting alarm method, shooting device and readable storage medium | |
| CN113923344B (en) | Motion detection method and image sensor device | |
| TWI716921B (en) | Detection device, detection system and detection method | |
| KR101699821B1 (en) | Signal Transfering Method of Wireless Motion Sensor for Reducing Battery Consume and Method thereof | |
| CN111225178A (en) | Video monitoring method and system based on object detection | |
| CN117689688A (en) | Mobile detection method, unit, intelligent monitoring equipment and storage medium | |
| US10832543B2 (en) | Activity sensor | |
| JP2003299074A (en) | Variation detecting apparatus and detecting method using camera apparatus | |
| JP2021010051A (en) | Imaging apparatus and method for controlling the same |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |