TW201801520A - Image device and method for detecting a chief - Google Patents

Abstract

A image method for detecting a chief, compressing comprising: a foreground object region and background object region generating step for detecting foreground object region(s) and background object region(s) from one or more image frames of a scene; a invasion region generating step for acquiring union(s) of the foreground object region(s) of the one or more image frames to generate invasion region(s); a disordered region generating step for confirm background modification region(s) from the invasion region(s) to generate disordered region(s); and a detection step for generating warning message(s) while area(s) of the disordered region(s) satisfying certain situations.

Description

Image thief detection device

The invention relates to the field of image analysis and recognition technology, in particular to the field of image analysis and recognition technology of thieves based on user behavior.

Image surveillance systems have been widely deployed in more important locations to monitor the presence of anomalies, such as unidentified users, intrusions of important items, or being taken away, due to image surveillance systems. Due to the widespread setting, manpower has been unable to take charge of the immediate monitoring work, requiring computer equipment to identify suspicious areas in advance, and then further confirming by human resources to effectively use limited human resources.

For example, a conventional image analysis system may use a previously established background reference frame, and then subtract the current frame from the background reference frame to obtain a difference frame. When the difference range of the difference frame is greater than a certain value, That is, the warning signal is issued, and the difference is displayed on the display device, so as to confirm the manpower of the guard. However, the biggest limitation of this method is that it can only be used in the monitoring of closed spaces such as treasury where no personnel are allowed to enter or exit. Moving objects such as cats, dogs, etc. can cause false positives in image analysis systems, rather than an effective intrusion detection system.

For example, another conventional image analysis system can use the face data that has been previously registered in the database, and when the system detects the non-registered face information, it sends a warning signal, but the limitation of this method is except for a camera. There are strict requirements on the installation position and resolution of the image sensor. It is also required that the system administrator log in to the system user in advance, when the system is installed. When people are free to pass or enter, such as when the company's door, or when the system user is equipped with a mask or bandage, it will cause system misjudgment, and it is not an effective intrusion detection system.

In order to solve the above-mentioned need to log in to the legitimate user in advance or the false positive rate is too high, this case proposes an image thief detection device, method and system, which often causes the display in the place to be changed greatly when the thief steals in a short time. The behavioral characteristics determine whether a thief is stealing, and if so, can issue a warning message to notify the guard, to protect, or to emit lights and sounds to scare off the thief, which is low-cost, simple and easy to use.

An image frame type thief theft analysis device comprises: a foreground object area and a background object area generating device for detecting a foreground object area and a background object area from one or more frames of a scene; an intrusion area a generating device, and the foreground object region is connected to the background object region generating device for acquiring a combination of the foreground object regions of the one or more frames in a time interval to generate an intrusion region;罝, and the foreground object area is connected to the background object area generating device and the intrusion area generating device for confirming an area where the background has changed from the intruding area to generate a messy area; a discriminating device, and the intruding area The generating device is connected to the messy area to generate a warning message, and when the area of the messy area meets certain conditions, a warning message is issued.

An image frame type thief stealing analysis method comprises: a foreground object area and a background object area generating step for detecting a foreground object area and a background object area from one or more frames of a scene; an intrusion area a generating step of obtaining a union of the foreground object regions of the one or more frames in a time interval to generate an intrusion region; a messy region generating step for confirming that the background has occurred from the intrusion region Changing area to produce a disordered area; a discriminating step of issuing a warning message when the area of the messy area satisfies certain conditions.

[Simple description]

Figure 1 system composition diagram

Figure 2 Relationship between foreground object area and intrusion area

Figure 3 First foreground object area, intrusion area, messy area relationship diagram

Figure 4: Appearance diagram of integrated image capturing device and frame analysis device

Figure 5 first thief stealing detection flow chart

Figure 6 Second foreground object area, intrusion area, messy area relationship diagram

Figure 7 second thief stealing detection flow chart

Figure 8 alert level diagram

[Common term definition]

Generally speaking, the terms "up, down", "front, back", "first, second", "left and right" only represent the relative direction and position between objects, and do not represent their position in real space. It should be apparent to those of ordinary skill in the art to which the invention pertains. The paragraphs in the specification refer to the paragraphs "upper and lower", which refer to the direction in which an object is subjected to gravity to produce a free fall; in addition, the term "before and after" "First, then" may also be synonymous with the term "early and late" in a particular case to describe the sequence or time relationship of the event, which should be easily distinguishable by those of ordinary skill in the art to which the invention pertains.

The term "connected" is equivalent to the term "coupled", which refers to the enumeration of one or more of the following elements, but not limited to: mechanical structure, spatial position, chronological order Relationships may also refer to the transmission or exchange of information between different components in the following (but not limited to) one or more of the following forms: electrical, magnetic, electromagnetic, optical, mechanical, thermodynamic, chemical In addition, the "connection" relationship is not only a "direct connection" relationship in which there is no other component between the two components, but also an "indirect connection" relationship between other components. On the other hand, the term "electrical connection" It emphasizes the connection and/or exchange of data between two components by means of electricity (especially voltage and current).

In the scope of the invention, the drawings, and the patent application, if there is an ordinal modification term such as "first" or "second", it is only for the purpose of identifying the component (or device, step), in some cases, by the above The components with the ordinal number modification are identical to each other, or there are some differences, but the difference does not cause the components to be interchangeable with each other, and also means that there is no connection, dependency, and processing order between the components. In another part of the case, the elements modified by the above ordered numbers are different from each other and cannot be interchanged with each other, and also represent a connection, dependency degree, and processing order relationship between the elements having a specific type. In the other direction, the inventors and the symbols in the figures are only for convenience of explanation. The ordering of numbers does not necessarily mean that the words corresponding to the numbers have a specific type of connection, dependency, and processing order. Those of ordinary skill in the art to which the invention pertains can easily ascertain the circumstances of the system based on the circumstances disclosed in the context of the disclosure, and do not need to be further described.

The term "frame" is equivalent to the terms "frame" and "frame". It is a two-dimensional data body used to record the data captured by the image capture device at a specific point in time. (video) data can be viewed as a series of frames with a chronological relationship.

[First Embodiment]

Referring to FIG. 1, FIG. 1 is a structural diagram of a system used in a preferred embodiment, which includes a The monitoring site 100, a frame analyzing device 200, a user output/input device 250, and one or more connecting elements 260a, 260b, wherein the monitored site 100 can represent the user's bedroom, but not limited thereto. In a practical example, the monitored site 100 does not have a compartment such as a wall, but is an outdoor venue, such as an outdoor parking lot. In another embodiment, the monitored site 100 can refer to various users. The location and scope to be monitored.

The user may install one or more image capturing devices 102a, 102b (hereinafter referred to as image capturing device 102) in the monitored site 100 for capturing one or more frame materials in the monitored site 100. In one embodiment, the image capturing device 102 refers to a camera, a camera, a mobile phone, and may have a PTZ function of panning, panning, and zooming. In another embodiment, the content captured by the image capturing device 102 is not limited to visible light or infrared light, but may be any physical or chemical characteristic in the location 100, as long as it can be displayed using a display device such as a screen. This feature can be distributed in the room, so it can also be a depth camera that uses optical or acoustic changes to estimate the depth of the object.

In an embodiment, the image capturing device 102 can periodically capture frame data at a certain sampling frequency according to a chronological relationship. The sampling frequency can be 0.5fps, 1fps, 2fps, 5fps, 15fps, 29.97fps, 30fps, 60fps. The sampling frequency is not limited to the above-mentioned sampling frequency; in another embodiment, the sampling frequency is not fixed and can be adjusted according to certain conditions, and the condition may be whether the image capturing device 102 is powered by a battery or an external power source. The condition may also be determined by information such as ambient temperature, ambient brightness, presence of an intruder, etc. provided by other types of sensors or remote devices of the monitored site 100.

When the plurality of image capturing devices 102 are installed in the monitored site 100, the image capturing device 102 may include the same field of view of the cover portion, so that the images captured by the different image capturing devices 102 are captured. Part of the same content can be included in the frame data. For example, the field of view of the image capturing device 102a includes the site spaces 104a and 104b. The field of view of the image capturing device 102b includes the site spaces 104c and 104b. That is, the common field of view of the image capturing devices 102a and 102b is the site space 104b. When no furniture or any object that hinders the light propagation is disposed in the space 104a, 104b, 104c, the frame captured by the image capturing device 102a is the area 106a, 106b of the wall 108b. The frame content captured by the capture device 102b is the regions 106c and 106b of the wall surface 108b. Therefore, the frame content captured by the image capture devices 102a and 102b includes the region 106b.

In another embodiment, the frame data captured by the different image capturing devices 102 does not include the same content but only has adjacent content, so that the frame captured by the image capturing devices 102a and 102b is captured. Only part of the edge information of the frame is shared, and the area 106b is approximately zero.

It is particularly emphasized here that in the present embodiment, the components in the monitored site 100 of FIG. 1 are drawn in a bird view manner from top to bottom. The image capturing device 102 in the monitored site 100 of FIG. It is fixed on a wall surface, and the fixing height is approximately equal to the height of the eye of a normal adult. However, in actual use, the image capturing device 102 can be fixed to a position that is convenient and hidden by any user, such as a ceiling. The image capturing device 102 may have an auxiliary light source such as an infrared light LED or a white light LED, or use a wide-angle lens, a super wide-angle lens, a fisheye lens or the like to increase the field of view of the image capturing device 102, or have an automatic The aperture is used to adjust the amount of light entering the image capturing device 102, and is not limited to the components illustrated in FIG. On the other hand, if there are multiple image capturing devices 102 in the monitored site 100, each image capturing device 102 may have its own mounting position, resolution, and field of view, but each image capturing device 102 captures The frame data needs to be spliced by an image stitching device to generate, for example, panoramic frame data, or to provide different viewing angles of the same background object. Frame information.

After the image capturing device 102 captures the frame, the connected frame 260a connected to the image capturing device 102 can be used to transmit the captured frame to the frame analyzing device 200 to determine the monitored site 100. Whether there is a thief who is stealing, in an exemplary case, the frame analysis device 200 can take unprocessed original data (such as the frame captured by the image capturing device 102) and transition data during processing ( For example, the foreground object area data, the final data after the processing is completed (such as whether a thief is found) is transmitted to the user output input device 250 via the connection device 260b connected to the frame analysis device 200, so that the user, the guard, and the security can understand In the case of the monitored site 100, the user, guard, and security can also use the user output input device 250 to transmit an operation command to the frame analysis device 200 or the image capture device 102.

The frame analysis device 200 may be a computer related information device, including but not limited to a desktop computer, a notebook computer, a palmtop computer, a mobile phone, a portable device, or a chip related product, such as a CPU. GPU, DSP, FPGA, custom chip (asic). The frame analysis device 200 includes a foreground object region and background object region generating device 202, an intrusion region generating device 204, a messy region generating device 206, and a discriminating device 208, and optionally includes an activity amount exclusion region generating device 2022 and a reference background image. The frame generating device 2024, the foreground object region pre-processing device 2042, the foreground object region post-processing device 2044, the weight parameter generating device 2062, the warning device 2082, the bus bar device 210, the output input devices 212a-212b, and the power supply device 214, but must be emphasized The above configuration naming manner is only for convenience of description of the embodiment, and the actual use is not limited thereto. Those having ordinary knowledge in the technical field of the invention can easily decompose one device into multiple devices, or By arranging a plurality of devices into one device, it is also possible to configure a plurality of devices having the same function, so that the devices can be operated in parallel or can be used as When the functions of one or more of the above devices are implemented in the manner of a pipeline or a thread, the above modifications are within the scope of the present specification.

The foreground object area and the background object area generating device 202 can receive one or more frames (hereinafter referred to as frames) captured by the image sensing device 102 via the output input device 212a, and run an object detection. An algorithm for separating the foreground and the foreground to detect the foreground object region and the background object region from one or more frames of the monitored site 100. The foreground object area generally refers to the area with motion in the frame, and the background object area refers to the area other than the foreground object area in the frame. When there is no foreground object area in the frame, the background object The zone is equivalent to the entire frame. The algorithm for object detection and front and back scene separation may use, for example, a difference frame of at least two adjacent frames in a time series, and an absolute error value of a specific pixel (pixel) on the difference frame. If the value is greater than a certain threshold, it means that the specific pixel is the foreground object area, and vice versa, it is regarded as the background object area, wherein the above threshold value can be 0 in one embodiment.

It is emphasized here that the algorithm for the front view separation mentioned in the previous paragraph is only a conventional technique with a small amount of calculation, and those having ordinary knowledge in the technical field of the invention can use any calculation with object detection or pre-background separation. The method for completing the foreground object area and the background object area generating device 202 is not limited by the techniques mentioned in the previous paragraph. For example, the algorithm may be edge detection technology, color detection method, and may be common in the field of video compression. The motion estimation technique can also be an optical flow method.

In a preferred embodiment, the foreground object area refers to an area of activity, which refers specifically to the amount of activity generated by the intruder's human body, appliances (such as carts, headlights), and does not include monitored The amount of activity generated by the object itself and the natural phenomenon in the site 100 is therefore a better In the embodiment, the activity amount exclusion area generating device 2022 can be selectively connected to the foreground object area and the background object area generating device 202, and the user can use the user output input device 250 to set the monitored place 100 or the frame. Excluded space or area, the excluded space or area may be, for example, a clock that operates twenty-four hours, a window that can see outdoor traffic, and a floor that is exposed to sunlight. In addition, the activity amount exclusion area generating device 2022 can also set the space, area, and object to be excluded in the monitored place 100 or the frame in an active or semi-active manner, for example, a template match can be used. The method is to exclude a self-propelled sweeper that can be started periodically.

In a preferred embodiment, the reference background frame generating device 2024 is selectively connectable to the foreground object region and the background object region generating device 202. The reference background frame generating device 2024 includes a dram, a flash, a hard disk, and the like. A memory storage device (not drawn) to store a reference background frame, which may be a frame provided by the foreground object area and background object area generating device 202 without any foreground object area, or may be a user The specific frame selected by the user output input device 250 from a plurality of frames without any foreground may also be the reference image input by the user via the user output input device 250, and the reference image may be It is different from other image capturing devices of the image capturing device 102, and is not limited to a case where the monitored site 100 is free of thieves in a frame format.

In a preferred embodiment, the number of frames or the number of maps stored in the reference background frame generating device 2024 may be greater than one pen, for example, a reference frame may be stored when different time points or illumination amounts are stored, when the camera has When the PTZ function is used, the reference background frame with different left and right rotation, up and down tilt and zoom characteristics can be stored.

What needs to be emphasized again is the foreground object area and background object area generating device 202. Other functions may be included, such as pre-processing operations such as de-noising, enhancement, or background correction operations such as Gaussian mixture model (GMM), not as mentioned in the specification. Function, operation or algorithm is limited.

The intrusion region generating device 204 can be coupled to the foreground object region and the background object region generating device 202 to obtain and/or receive information about the object region in front of the one or more frames. In a preferred case, The foreground object area information is taken from a plurality of frames in a time interval, and then a union operation is performed on the foreground object regions in different frames to generate an intrusion area, wherein the time interval is not timely The number of frames can be used, and the end of the time interval can be the time when the foreground object disappears or the time when the foreground object disappears slightly. The starting time of this time interval can be the foreground. The point at which the object appears or a slightly earlier point when the foreground object appears, and allows the foreground object to enter and exit the frame area multiple times.

Please refer to the schematic diagram of the front view set operation shown in FIG. 2a. The frame 300a includes an exemplary foreground object region 310a, and no foreground object is drawn. The foreground object region 310a is human cognitive and/or semantic. It is a simplified diagram of a human body, and only draws the more important edges and contours of the foreground object area (ie, the human body of the intruder) while drawing, ignoring changes in clothing, color, shadow, etc., the human body includes a head 312a, a neck The portion 314a, the trunk portion 316a, and the leg portion 318a, and because FIG. 2a assumes that the side of the human body faces the image capturing device 102, there is another arm portion 317a inside the trunk portion 316a, similarly, the frame 300b includes an exemplary foreground object region 310b including a head portion 312b, a neck portion 314b, a torso portion 316b, a leg portion 318b, and an arm portion 317b.

The time point of the frame 300a is slightly later than the time point of the frame 300b. In this case, the delay time is the sampling interval of one frame, that is, the foreground object area 310a and the foreground object area 310b do not actually appear in the same frame, but for convenience of explanation, the foreground object area 310a is especially 310b is drawn in the same frame 300. In addition, only the displacement motion occurs between the foreground object regions 310a-310b, and the foreground object region 310a and the foreground object region 310b actually have the same edge and contour, only because the foreground object region 310a ~310b will occupy part of the pixels in the frame at the same time (ie, the foreground object areas 310a-310b will partially overlap), so the foreground object area 310b only draws part of the edges and contours, which is supplemented here.

In an embodiment, the time interval of the intrusion area generating device 204 is set to 2, and as shown in FIG. 2b, a foreground object area 310a is first drawn on a blank, no background object area frame 300, and then The other foreground object area 310b is drawn on the same frame and covers the foreground object area which is already in the frame. Therefore, the right upper left lower oblique line area of FIG. 2b is the union of the foreground object areas 310a-310b after the joint operation. The area, and the set area is set as an intrusion area 320, indicating that the foreground object (ie, the intruder) has been active in the area, and further analysis is needed to determine whether the foreground object is a thief to be stolen, or normal. The user, the harmless third party, but in the embodiment, the frame analysis device 200 does not distinguish between a normal user or a harmless third party, so as to save the workload required to log in to the normal user in advance. However, it is particularly emphasized that the frame analysis device 200 can also work in conjunction with an image recognition device that uses a normal user profile (eg, a face) that is previously registered, for example, even if the frame analysis device 200 determines that the current intruder is stealing, It is technically feasible that the identity object is a legitimate user, does not issue a warning message, or allows a legitimate user to use a unique password to issue a warning message.

Referring to FIG. 2a, in some cases, the human body parts such as the head portion 312a, the neck portion 314a, the trunk portion 316a, and the leg portion 318a are connected to each other, but in some cases, the human body parts are partially connected. The segment is not always fully detectable and has a defective image. For example, the neck 314a in the frame 300a of FIG. 2a cannot be detected. In a preferred embodiment, the frame analysis device 200 can optionally include a foreground object area pre-processing device 2042 coupled to the intrusion area generating device 204 and operable to execute one or more algorithms for applying a pre-processed to each foreground object region of a single frame to fuse or split the foreground Interval. In this example, foreground object region pre-processing device 2042 can execute one or more algorithms for generating a virtual foreground object region that would otherwise be absent and connectable to head portion 312a and torso portion 316a, the location of the virtual foreground object region It may be exactly the same as the position of the neck 314a, and of course may be different and there is an error. In a preferred embodiment, the one or more algorithms may be a connected component or a watershed to combine smaller foreground object regions into a larger foreground object region and move Except for the invaders, they should belong to the area of the background object, but not limited to this.

Referring to FIG. 2b, in some cases, for example, when the frame capturing frequency of the image capturing device 102 is insufficient, the frame analyzing device 200 may generate even if the enabling foreground object region pre-processing device 2042 is selectively enabled. A hollow area 330, as shown in the upper left and lower right oblique lines of Fig. 2b, represents that the foreground object (i.e., the human body of the intruder) has been active in this area but is not considered to be a joint area. In a preferred embodiment, the foreground object region post-processing device 2044 can be selectively included and coupled to the intrusion region generating device 204 to adjust and fill the void region such that the post-processed intrusion region is comprised by region 320 and region 330. Composed of. The post-processing system may be determined according to whether the width of the intrusion area adjacent to the cavity area is greater than a certain value and/or the area of the hole area is less than a certain value, and if so, the video of the cavity area may be part of the intrusion area, but not To this end, for example, the post-processing can also be taken from a morphology, in particular from the dilation and erosion operations of the morphology.

Referring to FIG. 2c, in a preferred embodiment, the foreground object area is not represented by the human body 310a or the edges of the human body parts 312a-318a, and a simple graphic frame such as a rectangle or an ellipse may be used. To surround the human body 310a, and let the outer frame just surround the human body 310a. In a preferred embodiment, the frame may be slightly expanded outwardly to form an area that an intruder can touch by hand, equipment (such as a stick), or to form a range in which an item may be tossed. The extent of external expansion can be a percentage, or it can represent the length of a human arm or the extent to which an item can be thrown, but not limited thereto, and the expansion in each direction can be different.

Please refer to the frame 400 of FIG. 3a. Since the frame 400 has not been hacked by the thief at this time, it can be regarded as a background reference frame, and the illustrated monitored place 100 is a corner of the user's bedroom, and the table 420 is close to the wall. The faces 410a-410b and the floor 410c, the bed 430 abuts against the wall surface 410b and the floor 410c, and a picture 440 is attached to the wall surface 410b to cover the safe 442 installed in the wall surface 410b, and the space of the table 420 is accommodated. There are a plurality of drawers 422, 422a (hereinafter referred to as 422), and the above-mentioned grooves 422 can be opened and closed in a push-pull manner, and some items 424 can be placed on the table, and the bed 430 also has a receiving space. 432 and the rotating shaft (not drawn) is disposed on the right side, and the user can open the bed board clockwise to take the articles placed in the accommodating space 432.

The longer the burglary is, the higher the chance that the thief will be discovered by the user who just returned. In some cases, the thief will directly take away the item he is most likely to detect, such as item 424 on table 420, and will attempt to view the monitored location 100. All the items in the space, but due to the time factor, the thief will maintain the open space of the accommodating space as shown in Figure 3d, and even the items originally stored in the accommodating space will be randomly controlled and placed in the monitoring place. At the office.

Referring to FIG. 1, the messy area generating device 206 in the frame analyzing device 200 is connected to the invisible area generating device 204 and the foreground object area and the background object area generating device 202. In a preferred case, the background frame generating means 2024 can be connected to obtain and/or receive the intrusion area and the reference background frame respectively, and confirm that the background has changed from the intrusion area. The area to create a messy area that is caused by the invasion of the thief.

Referring to Figures 3a to 3d, assume that the thief 440 is stealing in the order of left first and then right again. Figure 3b shows a frame illustrating that a thief (foreground object) has stolen the table 420 and is preparing to steal. The bed 430, and the slot 422 have been opened, because in this embodiment, the rectangular frame is used to indicate the object of the thief, so the area 452 represents the intrusion area where the thief travels. In an embodiment, the intrusion area It does not include the area 450 where the thief is currently located, that is, the (more precise) intrusion area is only the area 452' of the light-colored wave, and the dark area 454 is the difference between the area 452' and the reference background frame, which represents A messy area in which the original items are messy due to theft of the thief. How to find the difference between the two frames is a conventional technique in the technical field of the invention, and therefore will not be described in detail herein. In addition, the area 452 is just The unions of the regions 452', 450 overlap, and are drawn separately for clarity of illustration, and are illustrated together.

Figure 3c shows a frame illustrating that a thief has also stolen the bed 430, the slot 422 and the bed 430 have been opened, and the area 452 represents the intruding area through which the thief travels, in one embodiment, The invading area does not include the foreground object area 450 where the thief is currently located, that is, the intruding area is only the area 452'. In addition, the area 454 is the difference between the area 452' and the reference background frame, that is, the messy area.

The discriminating device 208 can be connected to the intrusion region generating device 204 and the messy region generating device 206, and the discriminating device 208 can be configured according to the clutter region characterized by the messy region generating device 206. The characteristics of the intrusion area analyzed by 204 are analyzed together to determine whether the monitored site 100 has been stolen, and if so, a warning is generated. The warning message may cause the warning device 2082 built in or external to the frame analysis device 200 to emit a warning effect such as a light or a sound, and may also be linked with the security system. In addition, the alarm message may also be connected via the output input device 212b. The device 260b is transmitted and/or received by the user output input device 250, and is confirmed by the user, guard, and security near the user output input device 250 to eliminate false alarms.

The above-mentioned messy area characteristic and the intrusion area characteristic may be the size of the area, and the rule that the discriminating device 208 discriminates whether to issue the warning message may be selected or combined by the following conditions, but not limited thereto: the area of the messy area is larger than a certain value. The area of the intruding area is greater than a certain value, and the area of the messy area divided by the area of the intruding area is greater than a certain value.

In a preferred embodiment, the weight parameter generating means 2062 can be selectively coupled to the messy area generating means 206 to allow the user to set different weights for different background object areas, such as the weight of the area of the table 420 in FIG. 3a. 3, the weight of the area of the bed 430 is 2, the wall 410a~410b (excluding the drawing 440), the area of the floor 410c has a weight of 1 or 0, and even if the display has a movable part, the movable part can be Different weights are set in different positions, so that the discriminating device 208 can perform discriminating according to the weighted data. The weighting operation is a conventional technique in the technical field of the invention, and therefore will not be described in detail herein. The weight value can be based on user experience or experimental results. The decision is not limited by the numerical values listed in the above embodiments.

In a preferred embodiment, optical devices such as wide-angle lenses, super wide-angle lenses, and fisheye lenses used to increase the field of view of the image capturing device 102 may cause distortion of the frame image, and may be used on some image processing devices. First, use the calibration plate to obtain the deformation of the optical device, that is, the modulation transfer function (MTF) of the optical device, and then generate an inverse deformation function or matrix to correct the deformation. Therefore, the calculation in the weight parameter generating means 2062 can be performed. The method performs the weight calculation, and the above algorithm may be an algorithm for correcting the optical deformation, for example, the inverse matrix of the modulation conversion function, but not limited thereto.

In another preferred embodiment, the algorithm may automatically adjust the weight parameter according to the intrusion region characteristic of the intrusion region generating device 204. For example, the weight of the upper half of the intrusion region may be greater than the weight of the lower half, in a more advanced manner. In an embodiment, the area of the intruder area that the thief hand can cover may be greater than the area that the hand area cannot cover, and the information of the hand area may be generated by the foreground object area and the background object area generating device 202. provide.

Referring to FIG. 1, the frame analysis device 200 has two output input devices 212a-212b for respectively connecting with the connection devices 260a, 260b, and the output input device 212a is connected to the image capture device 102 through the connection device 260a and exchanges information. And letting the output input device 212b exchange information with other devices, in particular, through the connection device 260b to exchange information with the user output input device 250. However, the above distinction is only convenient for explanation, and is not limited thereto, for example, the invention Those of ordinary skill in the art can readily combine the output input devices 212a-212b into one output input device 212 (not shown).

In the frame analysis device 200, the devices 202 to 208 transmit information by using a dedicated connection pipe. However, the frame analysis device 200 may also have a bus bar device 210 shared between the devices. 202 to 208 can exchange information through the bus bar device 210. Of course, the frame analysis device 200 can also have a shared and exclusive information exchange connection pipe without conflict.

The frame analysis device 200 may include a power supply device 214, and the power supply device 214 may include one or more of an AC connection device, a DC connection device, a rechargeable battery, a non-rechargeable battery, a photoelectric conversion device, and a wireless charging device. Combination, for example when alternating current When the interruption is unable to supply power, it is powered by a rechargeable battery cell, and the rechargeable battery can be charged by an alternating current or photoelectric conversion device.

The frame analysis device 200 may be provided with a data storage device 216 for storing temporary and relayed data required for calculation by other devices, or for various users to view the data to be viewed by the user output input device 250 in the future. In addition, the data analysis device 218 may be provided in the frame analysis device 200, and the functions not belonging to the devices 202 to 208 may be performed by the data processing device 218, for example, the operation mode is determined, so that when the external power source is normally powered, the frame analysis device The 200 operates in the first mode of operation, and when the external power source is unable to supply power, the frame analysis device 200 operates in the second mode of operation to reduce the amount of power consumed.

Referring to FIG. 1, the user output input device 250 can display the unprocessed original data of the frame analysis device 200 (such as the frame captured by the image capturing device 102) and the transition data during processing. (such as foreground object area data), final data after processing (such as whether a thief is found), or an input device may be used to transmit an operation command to the frame analysis device 200 and/or the image capturing device 102 to set the frame analysis device. The parameters of 200 are either the left and right rotation, the up and down tilt and zoom characteristics of the PTZ camera.

The connection devices 260a-260b are used to connect the frame analysis device 200 with the image capturing device 102 and the user output input device 250. The connection may be performed by wire or wirelessly. The wired mode includes but is not limited to Ethernet. , power line network, etc., the wireless way includes but not limited to Bluetooth, wifi, mobile communication network, and the transmission mode includes one-to-one single-point mode, one-to-many multi-point mode, timely (real -time), non real time and other modes. In addition to the data transfer function, the connection devices 260a-260b also have a function of temporarily storing data and/or storing the transmitted data in an embodiment. In another In the embodiment, the connecting devices 260a-260b are not physically present, and the functions of the connecting devices 260a-260b can be transferred to the image capturing device 102, the frame analyzing device 200, and the user output input device 250, so that the image capturing is performed. The device 102, the frame analysis device 200, and the user output input device 250 can be directly connected.

FIG. 4 illustrates an appearance of a combination of the frame analysis device 200 and the image capture device 102, which is not significantly different from a general fire smoke alarm. The image capture device 102 can be installed in the LED working state indicator. And/or the test button 42, the frame analysis device 200 is mounted on the substrate inside the fire alarm, and can be connected and exchanged with the user output input device 250 by wireless transmission, except for the frame analysis device 200. It can also be connected to the loudspeaker 44 of the fire alarm to provide an alarm message without the need to additionally create a dedicated loudspeaker.

FIG. 5 is a first thief-scratch detection process 500 executed by the frame analysis device 200, and sequentially performs the following steps: foreground object area and background object area generation step S502, from one or more frames of a monitored place The foreground object area and the background object area are detected.

The intrusion region generation step S504 obtains a union of the foreground object regions of the one or more frames in a time interval to generate an intrusion region.

The messy area generates a step S506 from which an area where the background has changed is confirmed to generate a messy area.

In step S508, if the intrusion area and the messy area have specific conditions, a warning message is sent, and the specific condition may be selected and combined by the following conditions, but not limited to the limit: the area of the messy area is greater than a certain value, and the intrusion area is The area is larger than a certain value, and the area of the messy area divided by the area of the invaded area is greater than a certain value; one of the preferred cases is Ling The ratio of the area of the chaotic area divided by the area of the intruding area is greater than a certain value. The best case is that the area of the messy area is larger than a certain value, and the area of the intruded area is larger than a certain value, and the area of the messy area is divided by the invading area. The ratio of the area is greater than a certain value.

[Second embodiment]

The monitored site 100 illustrated in Figure 6a is the corner of the user's bedroom. The frame 600 is captured by a predetermined condition and the same image capturing device 102 of Figure 3, and most of the furnishings in the bedroom are illustrated in Figure 3a. The bedroom is the same, the difference is that the position of the painting 440 in the frame 400 has been replaced by the window 640 facing the road, so the same arrangement in Fig. 6a follows the component number of Fig. 3a without further explanation; Fig. 6a and Fig. Another difference of 3a is that a small volume and easy-to-carry valuables such as cash, gold, jewelry, etc. are placed in the groove 422a of the table 420, and the user uses the weight parameter generating means 2062 to set the groove 422a to the highest weight to play an early stage. Alerts and protects the effectiveness of important areas.

In an embodiment, the foreground object area and the background object area generating means 202 recognize the foreground by using only the motion between adjacent frames, and set a frame in which the foreground object area appears before the frame is the reference background frame. And stored in the reference background frame generating device 2024, since the activity amount exclusion region generating device 2022 is not enabled, the pedestrian passing by the road outside the bedroom will appear in the window 640 and be recognized as the foreground object. The area 650, as shown in Figures 6b to 6c, is infiltrated into the bedroom only by passing by the road, and the inside of the bedroom is not moved. Therefore, the intrusion area generated by the pedestrian is as shown in the area 652 of Fig. 6c. The area area is zero, so the discriminating device 208 does not treat the foreground object area 650 as a thief, but only as a user or a harmless third party.

Figure 6d illustrates another situation where two thieves intend to invade the bedroom. Stealing, and one person has successfully invaded into the bedroom, but because the two thieves are not far from each other, the foreground object area and the background object area generating device 202 may regard it as the same foreground, but because of the two thieves' invading limbs The motion or moving direction is not exactly the same. As shown in FIG. 6e, one of the left and the other to the left, the foreground object area pre-processing device 2042 can divide it into two foreground object areas 650-1, 650. -2, the intrusion area and the messy area caused by the two small invasions are represented by areas 652 and 654 respectively, because the area of the intruded area has reached a certain value and/or the area of the messy area has reached a certain value and/or the area of the messy area is divided. The ratio of the area of the invaded area has reached a certain value, and the discriminating device 208 regards the foreground object areas 650-1, 650-2 as thieves.

Fig. 6f illustrates a case similar to Fig. 6e, except that the activity amount exclusion area generating means 2022 is enabled to reduce the amount of calculation, so that the window has been set as the excluded intrusion area to produce a smaller range of intrusion areas.

Fig. 6g illustrates the case similar to Fig. 6f, except that the positions of the various furnishings in the room have been completely modeled in advance, and the 3d model is mapped to the background objects in the frame, and the thief opens the table 420. At the moment of the groove 422a, the image capturing device 102 cannot capture the frame in which the groove 422a is opened due to visual obscuration. In an embodiment, the user can install the second image capturing device 102b at different positions in the bedroom to confirm whether the groove 422a has been opened. If it has been opened, the groove of the frame 6h can be 422a is marked as a messy area, so even if the thief closes the previously opened slot 422a before leaving, the slot 422a area can still be marked with a messy area. In another embodiment, the opening and closing detecting device attached to the table or the groove 422a, or other detecting devices such as an infrared sensor in the room may be used to provide information on the change of the background object region, and It is necessary to add the image capturing device 102b.

7 is a second thief stealing detection process run by the frame analyzing device 200. 700, the basic process is similar to the first thief-theft detection process 500, the difference is that the part of the optional device in the image analysis device 200 can be enabled according to requirements, and the following steps are performed: refer to the background frame generation step S7022, and generate the mess The region generating device 206 analyzes the reference background frame used. In an embodiment, the reference background frame may also be provided to the foreground object region and background object region generating step S702 to detect the foreground object region.

The activity amount exclusion area generation step S7024 generates an area that is not analyzed in the determination step 708. In an embodiment, the steps S7026 to S706 may be selectively not performed in the activity amount exclusion area.

The weight parameter generating step S7026 generates different weighting parameters for different image regions, and the determining step S708 can perform a weighting operation.

The foreground object area and the background object area generating step S702 detect the foreground object area and the background object area from one or more frames of a monitored scene.

The foreground object area pre-processing step S7042 performs an operation of dividing and merging the foreground object area detected by the foreground object area and the background object area generating step S702 to form a semantically preceding object area.

The intrusion area generation step S704 acquires a union of the one or more frames of the foreground object area in a time interval to generate an intrusion area.

The foreground object area post-processing step S7044 performs an action of filling the holes for the void existing in the union of the foreground object areas.

The messy area generates step S706 from which an area where the background has changed is confirmed to generate a messy area.

In step S708, if the intrusion area and the messy area have specific conditions, a warning message is sent, and the specific condition may be selected and combined by the following conditions, but not limited to the limit: the area of the messy area is greater than a certain value, and the intrusion area is The ratio is larger than a certain value, and the ratio of the area of the messy area/area of the invaded area is greater than a certain value; one of the better cases is that the ratio of the area of the messy area divided by the area of the invaded area is greater than a certain value, and the best case is The area of the messy area is larger than a certain value, and the area of the invaded area is larger than a certain value, and the ratio of the area of the messy area divided by the area of the invaded area is greater than a certain value.

In the second thief-stolen detection process 700 illustrated in FIG. 7, the reference background frame generation step S7022, the activity amount exclusion region generation step S7024, and the weight parameter generation step S7026 can be purchased or first at the frame analysis device 200. When the user is installed and used, the user uses the user output input device 250 to set, but not limited thereto. For example, the reference background frame generating step S7022 can be combined with the foreground object region and the background object region generating step S702. The operation amount exclusion area generating step S7024 may be combined with the foreground object area and background object area generating step S702, but may be performed after or before step S702, and the weight parameter generating step S7024 and the discriminating step S708 may be combined. The operation may be performed before the step S708, and the steps of the reference background frame generating step S7022, the activity amount exclusion region generating step S7024, the weight parameter generating step S7026, and the like may be automatically determined by the algorithm without being input by the user.

[Third embodiment]

In order to save power, the number of frames captured and analyzed by the image capturing device 102 and the frame analyzing device 200 per second is variable, for example, 0.5 fps, 1 fps, 2 fps, 5 fps, 15 fps, 29.97 fps, 30 fps, 60 fps. The number of frames captured by the image capturing device 102 and the number of frames analyzed by the frame analyzing device 200 per second are equal and/or unequal. For example, the frame analyzing device 200 may omit The partial frame is processed, so that the frame analysis device 200 analyzes the number of frames per second lower than the number of frames captured by the image capturing device 102, so as to omit the control of the number of frames captured by the image capturing device 102 per second. Circuit or device.

In an embodiment, the number of frames analyzed per second by the frame analysis device 200 may vary according to the alert level, for example, the green level may be 1 fps, the yellow level may be 5 fps, the red level is 30 fps, and the frame analysis may be performed. The device 200 can receive messages from other frame analysis devices 200 or other intrusion detection devices, and automatically change the alert level. Taking FIG. 8 as an example, the 25 frame analysis devices 200 in the figure are respectively installed in different monitored places, and are originally set to the green level, but the frame analysis device 200-r in the center position is found to be stealing the thief. And raising the alert level to the red level, and sending the signal to raise the eight frame analysis devices 200-y adjacent to the frame analysis device 200-r to a yellow level, and at a more peripheral and frame analysis device The adjacent 16 frame analysis devices 200-g of 200-y maintain the green level.

In the embodiment of FIG. 8, although the alert level is set according to the distance or the adjacent or the like, it is not limited thereto. For example, whether the monitored site 100 where the device 200 is located may be connected according to the frame. The setting is also set, and the alert level is not limited to three levels, but may be four or five levels, and the alert level set by the individual frame analysis device 200 may be different under predetermined conditions, such as having a window, The monitored site 100, which is easy for outsiders to enter, can be set to a higher level, and the other monitored sites 100 are set to a lower level.

Since the user is less clear about the furnishings in the home, the user does not have a large range of messy areas when searching for the items. In contrast, the thief with time pressure is more likely to produce because of the unclear placement of the items. A wide range of messy areas, using this behavior difference, the frame analysis device 200 can be used normally without having to log in to a legitimate user in advance, achieving a simple Single, easy to use device.

The above description is only for the purpose of enabling those skilled in the art to implement the technology of the patent application, but the actual implementation is not limited thereto.

100‧‧‧Monitored places

102a, 102b‧‧‧ image capture device

104a, 104b, 104c‧‧‧ space

106a~106c‧‧‧Area

108a~108d‧‧‧ wall

200,200-r, 200-y, 200-g‧‧‧ frame analysis device

202‧‧‧ Prospect object area and background object area generating device

2022‧‧‧ Activity-excluding area generating device

2024‧‧‧Reference background frame generation device

204‧‧‧Invasion area generating device

2042‧‧‧ Prospect object area pre-processing device

2044‧‧‧Prospective object area post-processing device

206‧‧‧ messy area generating device

2062‧‧‧weight parameter generating device

208‧‧‧ discriminating device

2082‧‧‧ warning device

210‧‧‧ Busbar device

212a~212b‧‧‧Output input device

214‧‧‧Power supply unit

216‧‧‧ data storage device

218‧‧‧ data processing device

250‧‧‧User output input device

260a, 260b‧‧‧ connection device

300, 300a, 300b‧‧‧ frame

310a, 310b‧‧‧ foreground object area (human body)

312a~318a, 312b~318b‧‧‧ Prospect object area (human body parts)

320‧‧‧Joint area of foreground object area

330‧‧‧The void area of the joint area of the foreground object area

400‧‧‧ frame

410a~410c‧‧‧Wall, floor

420~442‧‧‧ Furniture

450‧‧‧ Prospect object area (thief)

452‧‧‧Invasion area (with foreground object area)

452’‧‧‧Invasion area (excluding foreground object areas)

454‧‧ ‧ messy area

40‧‧‧Integrated image capture device and frame analysis device

44‧‧‧Amplifier

42‧‧‧ indicator light and / or test button

500‧‧‧First thief stealing detection process

S502~S508‧‧‧Steps

600‧‧‧ frame

640‧‧‧ windows

650,650-1,650-2‧‧‧ foreground object area

652‧‧‧Invasion area (with foreground object area)

654‧‧ ‧ messy area

700‧‧‧Second thief stealing detection process

S702~S708, S7022~S7026, S7042~S7044‧‧‧ steps

S502~S508‧‧‧Steps

Claims (5)

A frame analysis device includes: a foreground object area and a background object area generating device for detecting a foreground object area and a background object area from one or more frames of a scene; an intrusion area generating device, and The foreground object area is coupled to the background object area generating device for acquiring a combination of the foreground object areas of the one or more frames in a time interval to generate an intrusion area; a messy area generating a decoration, and the a foreground object area is connected to the background object area generating device and the intrusion area generating device for confirming an area where the background has changed from the intruding area to generate a messy area; a discriminating device, and the intruding area generating device and the The messy area is connected and installed, and a warning message is issued when the area of the messy area meets certain conditions. A frame analysis device according to claim 1, further comprising: a weight parameter generating device, connected to the determining device, configured to set a plurality of weight parameters to different regions in the frame, the plurality of weight parameters may correspond to To a plurality of weight values, the plurality of weight values are not equal to each other. A frame analysis device according to claim 1, wherein the foreground object area refers to an area having an activity amount, which is generated by an intruder's human body and an appliance; the background object area is deducted from the scene. Other areas outside the foreground object area. A frame analysis device according to claim 1, further comprising an activity amount exclusion region generating device, and the foreground object region and the background object region are generated The device or the discriminating device is connected to exclude an activity amount generated by a specific space, region, and object in the scene to form an activity amount exclusion region. A frame analysis device according to claim 1, wherein one or more frames of the scene refer to frames captured by one or more image capturing devices in the same space; When the number of devices is greater than 1, the frame captured by the plurality of image capturing devices may have the characteristics of frame data that allows an image stitching device to operate normally or provide different viewing angles of the same background object.