FI112018B - Method and apparatus for detecting motion - Google Patents

Description

112018

Method and apparatus for detecting motion

Area

The invention relates to a method for detecting motion from consecutive images, and to a device for detecting motion from consecutive images.

5 Background

With the rise of crime, such as violence, theft or robbery, the use of different security camera systems has increased dramatically. A typical system is one in which surveillance cameras are positioned to capture various subjects. Sequential images captured by the cameras, typically a video image, can be sent to a central control room, where a person monitors images from different cameras on different monitors and attempts to identify a hazardous situation. Sequential images captured by the camera can also be recorded, for example, on tape, and then routinely or after a crime occurs, the person looks at the tape and seeks out the images of the perpetrator or the perpetrators of the crime.

The problem with such systems is the great need for manpower and, moreover, that human error can occur, especially in the dark hours of the night, that is, a criminal event may be undetected or detected too late to be prevented. In addition, from the dark: * 20 is difficult to distinguish between motion. One solution is to illuminate controlled objects v: but it adds to the cost in the form of lighting fixtures and consumed electricity.

; * One solution to the problems is to use motion detectors that are placed in the area being monitored. The problem is the limited range of the sensors, which means that they have to be located in a relatively controlled area. · ·. 25 which is an additional cost.

• a

BRIEF DESCRIPTION: It is an object of the invention to provide an improved method of recognizing: motion from consecutive images and an improved device for detecting motion from consecutive images; ta.

One aspect of the invention provides a method of generating difference data representing the magnitude of the luminance pixel of each of the preceding image and the corresponding current image luminance pixel; adding the generated '* · eroded data to a cumulative motion register, wherein in the motion register a data item corresponding to each of the 112018 luminance pixels of Figure 2; and detecting data elements whose value exceeds a predetermined threshold value.

In one aspect of the invention, there is provided an apparatus comprising: means for generating difference data representing a magnitude difference between a luminance pixel of each preceding image and a corresponding current 5 image luminance pixel; means for adding the generated eroded data to a cumulative motion register, wherein in the motion register a data item corresponding to each luminance pixel of the image; and the means for detecting data elements whose value exceeds a predetermined threshold value.

In one aspect of the invention, there is provided a device configured to: 10 generate difference data representing the magnitude of difference between the luminance pixel of each preceding image and the corresponding current image luminance pixel; adding the generated difference data to a cumulative motion register, wherein the motion register corresponds to a data item corresponding to each luminance pixel of the image; and detecting data elements whose value exceeds a predetermined threshold value.

Preferred embodiments of the invention are the subject of the dependent claims.

The invention is based on analyzing sequential images produced by a surveillance camera by analyzing differences between their luminance pixels, whereby motion can be automatically detected. The procedure is somewhat reminiscent of the motion estimation used in video encoding, but the motion register used is cumulative, whereby a person who moves for a short time and then stops is also detected. The motion recognition performed in this way is also much faster than the traditional motion estimation because what motion does not matter, since the invention is only interested in motion recognition.

The method and apparatus of the invention provide several advantages. The procedure is capable of implementing a security camera system that requires less manpower in the control room than before. In addition, the work of the person working in the control room is made easier because the system is automatic. · ·. 30 detects motion, and the system can automatically trigger an alarm.

The procedure can also be used to analyze the tapes depicted by the surveillance camera, and to try to identify points of interest from the tapes, i.e., the target data which is in motion. Embodiments of the procedure also have other advantages which; \ will be described in more detail below.

3, 112018

List of figures

The invention will now be described in greater detail in connection with the preferred embodiments, with reference to the accompanying drawings, in which Figure 1 is a block diagram showing a device for detecting motion from successive 5 figures; Fig. 2 is a flowchart illustrating a method of detecting motion from consecutive images; Figure 3 illustrates the processing of a luminance pixel in a motion register; Figures 4A, 5A, 6A and 7A show four images selected from the sequence of pe-10 consecutive images; Figures 4B, 5B, 6B and 7B show what the motion identified in Figures 4A, 5A, 6A and 7A looks like in black and white; Figures 4C, 5C, 6C and 7C show a zoomed view of the motion detected in Figures 4A, 5A, 6A and 7A; Figure 8 illustrates the effect of camera movement; Figure 9 illustrates the effect of using optical zoom of a camera; and FIG. 10 illustrates processing of a luminance pixel in a motion register in conjunction with the example of FIGS. 4A, 5A, 6A and 7A.

20 Description of Embodiments

Referring to Fig. 1, a device for detecting motion from consecutive images is described. By its basic principle, the device is simple, it is configured to generate eroded data by subtracting from the luminance pixel i of each previous image the corresponding luminance pixel of the current image. The device is also configured to add the generated erod data to the cumulative motion register. In the cumulative: ': in the motion register, each data luminance pixel corresponds to a data item. The device is configured to detect data items whose value exceeds en -; '·; for a given threshold.

. Let us now take a closer look at the device illustrated in Figure 1. The device may • communicate with one or more image sources, in our example, two camcorders 158, 160. Any known device providing sequential images 100A, 100B may be used as the image source. Because the invention analyzes changes in the luminance pixels of consecutive images, the information obtained from the image source must include the luminance information, either as discrete luminance pixels, or else the luminance pixels must be capable of extracting or converting the information from the image source. The cameras 158, 160 can be polled one by one until motion is detected somewhere. If motion is detected in the image of several cameras, the image to be transmitted may be selected, for example, based on the motion field, or the image from these cameras 5 may be transmitted alternately for a certain length of time.

It is possible to use multiple speeds for processing images. For example, if no motion is detected, 1 frame / second is used, whereas during pre-alarm and alarm 15 frames / second are used. When motion is detected, the reading speed is increased, i.e. the so-called. esihälytystilaan. If the motion is continuous, 10 will enter alarm mode, whereupon the system will start transmitting an image and then the zoom can also be turned on. The images of the pre-alarm mode can be saved and sent in case of an alarm mode. When the motion is complete, the system returns to the pre-alarm mode, where it stays for a predetermined time. If motion is still not detected, the alarm can be turned off and the reading speed dropped to a minimum.

15 Encoding of successive images, such as video, is used to reduce the amount of data so that it can be more effectively stored on a storage medium or transmitted over a communication connection. An example of a video encoding standard is MPEG-4 (Moving Pictures Expert Group). There are different image sizes used, for example, cif is 352 x 288 pixels and qcif is 20-176 x 144 pixels. Typically, a single image is divided into blocks containing information about luminance, color, and location. The block data is compressed block by block using the desired coding method. Compression is based on the removal of less relevant data. The compression methods are mainly divided into three categories: Spectral Redundancy Reduction, Spatial: Redundancy Reduction, and Temporal Redundancy Reduction. Typically, various combinations of these methods are used for compression.

For example, '··, 30 YUV color schemes are used to reduce spectral redundancy. The YUV model takes advantage of the fact that the human eye is more sensitive to variations in luminance, or luminance, than to changes in chrominance, or color. The YUV model has one luminance component (Y) and two chrominance components (U and V, or Cb and Cr). For example, the H.263 video encoding standard luminance block is 16 x 16 pixels and each is chrome '; 35 nance blocks covering the same area as the luminance block, 8x8 pixels.

The combination of one luminance block and two chrominance blocks is called 5112018 macroblock. Each pixel, in both the luminance and chrominance blocks, can have a value between 0 and 255, which means that eight bits are needed to represent one pixel. For example, a value of 0 for a luminance pixel is black and a value for 255 is white.

Thus, the successive images 100A, 100B entering the device may follow the YUV model, in which the luminance pixels are already provided as their own component, for example, a 352 x 288 luminance pixel in a cif image.

Thus, there may be many image sources, in which case the device must have an image source selector 102 for selecting the information stream to be handled by the device. The image-10 source selector 102 is not required if there is only one image source. Further, in our example, reference numeral 102 also represents a control circuit for an image source, such as a camera.

The image 104 of the selected image source is applied to the frame buffer 106, where the luminance and chrominance components of the image are stored.

15 The luminance component of the previous image contained in the frame buffer 106 is applied 108 to the second frame buffer 110.

Then, in block 116, eroded data representing the difference between the luminance pixel of each previous image and the corresponding current image luminance pixel is formed. This difference data generation can be made to Example 20 by subtracting the luminance pixel 112 of the current image 112 from each of the luminance pixels of the previous image 114, or by any other mathematical operation yielding the same result.

i ': The generated erod data 118 is then added to the cumulative motion cross point 126. The cumulative motion point 126 is a memory whose size is the height of the image: 25 times the width of the image. A possible reading range is [-255,255] if an image in YUV format is used. The memory may also be smaller, but then ... ... the incoming data will have to be preprocessed.

»I

In one embodiment, the device is configured to reduce the values of non-zero data items in the motion register to a zero by a predetermined amount of 30 each time the difference data is added. In the example of Fig. 1, this is illustrated by block 120 into which the eroded data 118 is imported, and; wherein 125 eroded data items imported from each motion register 126 are subtracted. · ·. a predetermined number is entered, and the difference thus obtained, plus the difference data 124, is then exported to the motion register 126 as the new data item value. A predetermined number I ·: '* 35 can be, for example, a number one or another number greater than one. The data can also be subtracted by adding the eroded data to the motion register 6112018 126, after which a predetermined number is subtracted from each non-zero data item. The device has a control block 136 which controls the operation of different parts of the device. Block 120 is provided with a predetermined number 122 from control block 136 which is subtracted from each data item. The control block 136 also controls the image source selector 102.

If the image source is changed, the content of the motion register 126 is always reset before the difference image data of the new image source is added to the motion register 126.

After the previous image and the current image have been processed in the manner shown above in FIG. 10, the control block 136 scans 128 the motion register 126 in a predetermined manner, for example, row by column. The control block 136 then identifies as motion the data elements of the motion register 126 whose value exceeds a predetermined threshold value. In one embodiment, the control block detects motion by blocks, for example, luminance macrol blocks. The blocks may also be partially overlapping. In one embodiment, motion in a block of an image is detected if at least a predetermined number of data items corresponding to that block in motion register 126 exceed a predetermined threshold.

Figure 3 illustrates the processing of a luminance pixel in a motion register 20 126. The example is based on the prototypes and experiments made by the applicant. The example character is based on a series of events described by a camcorder that describes. light table. A person arrives and leaves a black briefcase on the table.

In Figure 3, the horizontal axis has the numbers 1-150 of the consecutive images and the vertical axis has the value range of both the luminance pixel and the motion element data element. Check: Variations in the value of 25 tensile luminance pixels are plotted with 300. This | the variation of the value of the data element corresponding to the luminance pixel in the motion register 126 is represented by curve 302.

The selected luminance pixel in Figures 1-13 is part of the light table. A small variation in pixel value is noise. In Figures 14-18, the snow V 30 nance pixel is part of a black briefcase placed on a table. Similarly, in the pictures • · * :: 19-150 luminance pixels are part of a black briefcase left on the table.

,. ·. As can be seen by examining curve 302, in our example, there is an embodiment in which the values of non-zero data elements in the motion register 126 are reduced to zero by a predetermined amount of '· 35 at each incremental data increase. In our example, this predetermined amount is one. Decreasing the value of the data item results in that the portfolio in Figure 137 has already completely disappeared from the motion register 126, i.e., merged into the background.

Since data elements with a value above a predetermined threshold are detected in motion, it is interesting to consider what would be a suitable threshold in the example of Figure 3. A suitable threshold value that eliminates noise but no movement could be, for example, 10-15, whereby motion would no longer be detected by approximately the size of Figure 130.

In one embodiment, if motion is detected in the image, control block 136 controls 144 frame buffer 106 to export the current image zoom area 130 to zoom block 132. Control information 144 includes which zoom area in the image is transmitted as image 130 to zoom block 132. Exported from control block 136 control information 134 to zoom block 132, which control information indicates the relationship between the sizes of the incoming and outgoing images. This ratio is 1: 1 if the zoom is not used. Automatic 15-to-Zoom Motion is an optional feature that can be activated via the device interface (not shown in Figure 1). Thus, the zoom block 132 zooms in to the detected motion, for example, by interpolating the zoomed area to the size of the original image using known interpolation methods.

Motion detection detects the area where motion occurs. Limit this range so that the ratio of height to width is always the same as in the original image. Also, the zoom area should be slightly larger than the area where the movement is. The magnification must also have: '; maximum, depending on the image size used and the resolution of the camera. Example- *. that's why a 100x magnification of a qcif image no longer makes sense.

: 25 In one embodiment, the control block 136 and / or the zoom block * I * 132 stores the zoom position in memory to control the change in the zoom area. Zoom position information can be utilized by allowing only a predetermined change of the "" zoom position between two consecutive images. In addition to or instead of the zoom position, control block 136 and / or: V 30 zoom block 132 can store the zoom ratio in memory. ., can be utilized by allowing only a predetermined change in the zoom ratio between two consecutive images. · · · By limiting the zoom position change and / or the zoom ratio within the image, »» : '· Based on 35 sensed motion, but for example in steps of a certain magnitude Change I t 8 112018 speed control is an optimization task between the detected motion tracking speed and the image quality and information content.

Also, it is good to allow the motion area to move freely within the zoom range within certain limits. By the above methods, 5 smoothly moving images are obtained.

The zoomed or non-zoomed image 142 is then passed to an optional encoding block 150 whereby the image can be encoded if desired. The advantage of encoding a zoomed image is that the final result of the encoding is improved by removing unnecessary information from the image, thus making it easier to identify the burglary visible in, for example, a zoom image.

The encoding block 150 may be, for example, a known video encoder, for example an mpeg4 encoder. Figure 142 may also be exported to a storage medium included in the device or to a storage medium connected to the device, such as a computer hard drive. Figure 142 may also be exported to a viewing device, such as a monitor, whereby the image may need to be converted, depending on its presentation.

In one embodiment, the device further comprises a block 146 for generating a black and white image 148 of the data elements contained in the motion register 126. As described in Figure 1, the data elements may be exported to a block 146 from the control block 136 since the control block 136 reads 128 data block 146 could also read them directly from the motion register 126. The black-and-white image may be implemented, for example, so that the data elements exceeding a predetermined threshold in the black-and-white image are shown in white. . na and other data items in black, or a da- above a predetermined threshold. ·. : 25 items are shown in black and other data items are shown in white. This embodiment achieves the advantage that the detected motion is much more easily detected ... in the black and white image 148 of the motion than in the original image 100A, 100B. If we briefly return to the example of Fig. 3, it can be noted that if the same value is used as the threshold for black-and-white image generation as for motion detection, e.g., 10-15, the motion of the person is detected in black and white and the briefcase remains on the table until approximately 130. Of course, if desired, the threshold for the black and white image can be set to a smaller number so that motion is visible in the formed black and white image for a longer time.

In one embodiment, adjusting a predetermined threshold value used in motion detection adjusts the sensitivity of motion detection 112018 g. In general, this may be expressed, for example, by generating a threshold value for the current image in the control block 136 using the magnitude representing the average value of the luminance pixels of the current image and the threshold value of the previous image. For example, for image k, the average of the luminans-5 pixels i <nj <m ak = (zLpij) 1 nxm, (1); = oj = o where pij is the luminance pixel and n x m is the image size. It is not necessary to use all the luminance pixels of the image to calculate the average.

Then, for image k, the threshold value 10 tk = pak + q, and (2) tk = (rtk + stk-i) Kr + s), (3) where tk-i is the threshold value of the previous image. In formula 2, a linear function is used in which p and q are constants, but a non-linear function can also be used. According to formula 3, the threshold can also be weighted, r and s are constants, which can be changed by adjusting the threshold and thus the sensitivity of the device to large variations in luminance. The device interface may include a controller for adjusting the threshold steplessly to achieve the desired sensitivity. Weighing should be done to prevent the system from responding to rapid large changes in brightness, such as lightning or cloud-induced luminance changes. In formulas 2 and 3, it is not necessary to use •. but some other statistical measure of average value may be used. Noise can also be measured from random pixels:. and adjust the sensitivity accordingly.

Τ ': In one embodiment, the device is configured, e.g.

*> I

*! The block 136 is configured to move the luminance pixels of the previous image over the luminance pixels of the current image in the current image, move the corresponding data items in the motion register 126 to that same image area, and reset only the luminance pixels in the current image. t ·> i V data pixels corresponding to the pixels in the motion register 126 if the image area in the current image has moved relative to that of the previous image, i.e., the camera that formed the images has been moved 158,160.

!! In one embodiment, the device is configured, e.g. block 136 is assigned to modify the image area of the previous image to match the image area of the current image by interpolating it with the current image area! 35 pixels, and modify the area of the motion register 126 to reflect the area of the current image by interpolating the missing data elements corresponding to the area of the current image if the image is zoomed optically relative to the area of the previous image, i.e. the camera that produced the images 158,160 uses optical zoom.

In one embodiment, the device is configured, for example, control block 136 to determine 140 selection blocks 102, to direct the camera 160 that formed the images 138 to move in the direction of the detected motion. The camera 160 then has, for example, an electric motor for turning the camera 160 in the direction of movement. The control command 138 indicates the magnitude of the required movement 10, for example in degrees.

In one embodiment, the device is configured, for example, control block 136 to determine 140 selection blocks 102, to direct the camera 160 formed by the images 138 to optically zoom in on the detected motion. In this case, the camera 160 has an electrically-controlled optical zoom that can be used to zoom the image produced by the camera 160 15 into motion. Control command 138 indicates the required zoom ratio change.

In one embodiment, the device further comprises a block 154 for performing an alarm if motion is detected. In Figure 1, the alarm is denoted by reference numeral 156. The same reference numerals 154, 156 also illustrate an embodiment in which the device 20 comprises a block 154 for transmitting consecutive images formed in black and white. image or zoomed image using communication link 156. Pictures can be sent with or without alarm. The image you send may be an initial * * «Image from a full source, a black-and-white image, or> I t zoomed image. The camera that generated the images may also be moved or 25 optically zoomed from the image in the direction of the detected motion. Telecommunication connection • 1 * 1 »* * uses well-known solutions such as wireless radio, LAN,

The Internet, a fixed dedicated line, or some other way to transfer data between two points. The device may also be configured so that only an image with motion detected is transmitted using a communication link 156: * * ': 30. The sensitivity of the system can also be automatically selected

<· I

'. Decide whether to send an image from the camera or a black and white image from the image in the motion register. If the brightness of the area you are shooting falls below a certain level, you can send a black and white image. This is how the device works almost in the dark: '. ace, making it easier for the person looking at the image to detect motion.

The device blocks shown in Figure 1 can be implemented as one or more customized integrated circuits (Application-Specific Integrated 11 112018

Circuit, ASIC). Other implementations are also possible, such as a circuit built from separate logic components, or a processor with software. A mixed form of these different embodiments is also possible. The skilled artisan will consider, for example, the size and power requirements of the device, the processing power required, the manufacturing costs, and the production volumes. It should be noted that Figure 1 illustrates mainly functional assemblies, whereby parts of a practical hardware implementation may deviate from what is shown, since it is ultimately a degree of integration: how to implement in a given application a device implementing the desired functionality to detect motion from successive images.

The above device is suitable for a variety of purposes. It allows you to search for recorded footage and automatically search for areas of motion. The device can be located in front of a surveillance camera or in a central control room. A very cheap version can also be made. One cheap version is one in which the device also comprises a cheap camera and is plugged in. Further, the device comprises parts by which it can act as a subscriber terminal in a cellular network. Thus, the device can be set up to guard, for example, its owner's summer cottage, and the device sends an image and / or alarm to the owner's subscriber terminal as soon as it detects motion. The device thus acts as a burglar alarm and a monitoring device to check the situation in the summer cottage. Of course, other sensors, such as a fire alarm, can also be connected to the device. Skilled in the art :·. may also invent other applications for the described basic device.

. The following describes the execution of some of the functions of the device in a C program; :. with a pseudocode that follows the syntax of the Malay language.

'25 *

Header t * »#define mmin (a.b) ((a) <(b)? (A) :( b)) / * macro for the minimum of two values * /

I I

: '' ': 30 typedef struct {int ** lum; int ** cb; int ** cr; ·: · '· 35 Jdata; 12 112018 typedef struct {int mid [2]; / * midpoint 7 int hor; / * horizontal magnitude 7 5 int ver; / * vertical magnitude 7} mot; typedef struct {10 int lines; / * image height 7 int columns; / * image width 7 mot motion; / * range with motion at 7 mot zoom; / * area where the zoom is 7 data previous; / * previous image 7 15 data present; / * current image 7 int '' movement; / * business register 7 int 'shape; / * filtered motion register 7 int sensitivity; / * sensitivity 7} secur; / * Variable defined in the main function 7 20

Business Register Function • · · * »·

Adds a difference between images in the motion register, and reduces the register! ''. data per pixel by pixels by one.

• · 25. ···. int Add_motion (secur'frame) {inti.j; / 'Counters 7 \' int r = frame-> lines; / * image lines 7 '··.' 30 int c = frame-> columns; / * Image Columns 7 • · *

I t I

; · ·: For (i = 0; i <r; i ++) {'· for (j = 0; j <c; j ++)': ': 35 {frame-> movement [i] lj] + = 13 112018 ( (frame-> present) .lum [i] [j] - (frame-> previous) .lum [i] D]); if (frame-> movement [i] [j]> 0) frame-> movement [i] ö] - = 1; if (frame-> movement [i] ö] <0) 5 frame-> movement [i] [j] + = 1; }} return (1); } 10

Motion check function int Motion_check (secur * frame) {15 int i, j, k, I; Γ counters 7 int r = frame-> lines; / * image rows 7 int c = frame-> columns; Γ columns of the image 7 int founcNO; / * whether movement was found or not 7 int sum; / * number of motion pixels in block 7 20 int min_k, max_k, minj, maxj; / * found movement variables 7 int sens = frame-> sensitivity; / * sensitivity 7

I

for (i = 0; i <r; i ++) / * copy and filter motion register 7: ϊ {► * Ί 25 for (j = 0; j <c; j ++) {:: frame-> shape [i] [j ] = frame-> movement [i] [j]) / sens; }> 30 »t * ', min_k = c; / * initialize minimum and maximum 7 max_k = 0; *; · 'Min_l = r; maxj = 0; for (k = 0; k <c-7; k + = 4) / * find the motion and its region in 8x8 blocks 7 35 14 112018 {/ * the blocks are overlapping 7 for (l = 0; l <r-7; l + = 4) {sum = 0; 5 for (i = 0; i <64; i ++) {if (frame-> shape [l + i / 8] [k + i% 8]! = 0) sum ++; 10} if (sum> sens) {found = 1; 15 if (k <min_k) min_k = k; if (k> max_k) max_k = k + 3; if (l <min_l) 20 min_l = l; i '· ,. if (l> maxj) max_l = l + 3; • I · f I I · t> '·: 25}': .7> * »•» · if (found == 1) I * if motion, calculate its width, height, and center * / Γν {; 30 (frame-> motion) .mid [0] = (min_k + max_k) / 2; (frame-> motion) .mid [1] = (min_l + max_l) / 2; • · a ;;; (frame-> motion) .hor = max_k-min_k; a * '; · '(Frame-> motion) .ver = maxj-min_l; Γ · ..} *: · 35 else / * if not in motion then width, height and center point are in the picture * / {15 112018 (frame-> motion) .hor = c; (frame-> motion) .ver = r; (frame-> motion) .mid [0] + = Mmax (((c / 2) - (frame-> motion) .mid [0]) / 8.1); (frame-> motion) .mid [1] + = Mmax (((r / 2) - (frame-> motion) .mid [1]) / 8.1); 5} return (1); } 10 Zoom function int Zoom_control (secur 'frame) {

int r = frame -> lines; / * lines of the image V

15 int c = frame-> columns; Γ columns of the image 7 int left, right, up, down; Γ zoom edges 7 int i; / * counter 7 / * calculate new zoom height and width (relative to motion) 7 20 if ((frame-> motion) .hor <(c * (frame-> motion) .ver) / r) * »·; 'V {ί * if ((frame-> zoom) .ver> = (frame-> motion) .ver + r / 6) ν · ί ί 25 (frame-> zoom) .ver- =. · ". Mmin (abs (((frame-> zoom) .ver- (frame-> motion) .ver -r / 6) / 4), 4); .., (frame-> zoom) .ver = Mmax (mmin ((frame-> zoom) .ver, r), r / 2); · ,,. '(frame-> zoom) .hor = (c * (frame-> zoom) .ver) / r;' 30 (frame-> zoom) .hor = Mmax (mmin ((frame-> zoom) .hor, c), c / 2);:;: (frame-> zoom) .hor = ((frame-> zoom) .hor / 2) * 2;: '1': (frame-> zoom) .ver = ((frame-> zoom) .ver / 2) * 2;> '1 35 if ((frame-> zoom). ver <(frame-> motion) .ver + r / 10) {16 112018 (frame-> zoom) .ver + = mmin {abs (((frame-> zoom) .ver- (frame-> motion) .ver - r / 10) / 4), 8): (frame-> zoom) .ver = Mmax (mmin ((frame-> zoom) .ver, r), r / 2); 5 (frame-> zoom). hor = (c * (frame-> zoom) .ver) / r; (frame-> zoom) .hor = Mmax (mmin ((frame-> zoom) .hor, c), c / 2); (frame- > zoom) .hor = ((frame-> zoom) .hor / 2) * 2; (frame-> zoom) .ver = ((frame-> zoom) .ver / 2) * 2; 10}} / * end of if statement 7 else {15 if ((frame-> zoom) .hor> = (frame-> motion) .hor + c / 6) {(frame-> zoom) .hor - = mmin {abs ((( frame-> zoom) .hor- (frame-> motion) .hor - c / 6) / 4), 4); 20 (frame-> zoom) .hor = Mmax (mmin ((frame-> zoom) .hor , c), c / 2);: .. ( frame-> zoom) .ver = (r * (frame-> zoom) .hor) / c; (frame-> zoom) .ver = Mmax (mmin ((frame-> zoom) .ver, r), r / 2); .:. (frame-> zoom) .hor = ((frame-> zoom) .hor / 2) * 2; (frame-> zoom) .ver = ((frame-> zoom) .ver / 2) * 2; • «· 25} • · '· · ·' if ((frame-> zoom) .hor <(frame-> motion) .hor + c / 10) {: (frame-> zoom) .hor + = 30 mmin ( abs (((frame-> zoom) .hor- (frame-> motion) .hor- c / 10) / 4), 8); • * * (frame-> zoom) .hor = Mmax (mmin ((frame-> zoom) .hor, c) Ic / 2); (frame-> zoom) .ver = (r * (frame-> zoom) .hor) / c; i '·· (frame-> zoom) .ver = Mmax (mmin ((frame-> zoom) .ver, r), r / 2); ':': 35 (frame-> zoom) .hor = ((frame-> zoom) .hor / 2) * 2; (frame-> zoom) .ver = ((frame-> zoom) .ver / 2) * 2; 17 112018}} / * else End of statement 7 / * move the zoom center toward the center of the motion 7 for (i = 0; i <2; i ++) {if ((frame-> zoom) .mid [i] <{frame -> motion) .mid [i] -4) {10 (frame-> zoom) .mid [i] + = mmin (abs (((frame-> zoom) .mid [i] - (frame-> motion) .mid [i]) / 4)), 8); } if ((frame-> zoom) .mid [i]> (frame-> motion) .mid [i] +4) {15 (frame-> zoom) .mid [i] - = mmin {abs ((( frame-> zoom) .mid [i] - (frame-> motion) .mid [i])) / 4.8); }} 20 / * hold the zoom within the image 7 • '· .. (frame-> zoom) .mid [0] =.': ·. mmin (Mmax ((frame-> zoom) .mid [0], (frame-> zoom) .hor / 2)) c- (frame-> zoom) .hor / 2); (frame-> zoom) .mid [1] = 'ί 25 mmin (Mmax ((frame-> zoom) .mid [1], (frame-> zoom) .ver / 2)) r- (frame-> zoom ) .ver / 2); • ·

I I

left = (frame-> zoom) .mid [0] - (frame-> zoom) .hor / 2; • · *: right = (frame-> zoom) .mid [0] + (frame-> zoom) .hor / 2; 30 up = (frame-> zoom) .mid [1] - (frame-> zoom) .ver / 2; . i ·, down = (frame-> zoom) .mid [1] + (frame-> zoom) .ver / 2; »· *» T ·; Convert (frame, left, right, up, down); Γ Call the interpolation function 7 I »I i

* It I

35 return (1); } 18 112018

Next, with reference to the flow chart of Figure 2, a method for detecting motion from consecutive images will be described. The process begins at 200. 202 then selects the image source to use, for example one of several cameras. If there is only one camera, 202 need not be performed.

5 When the camera is selected, 204 reads the image from the camera. The image must be such that the luminance component can be read directly from it, or else the luminance component can be generated based on the information contained in the image. The luminance component is stored in 206.

10 The 208 then reads the next image from the camera. The system sensitivity adjustment performed as described above is performed at 210.

Next, we move to 212, generating the difference data representing the magnitude difference between the luminance pixel of each previous image and the corresponding current image luminance pixel 15, for example, by subtracting the current image luminance pixel corresponding to each luminance pixel of the previous image. In the same 212, the generated erod data is added to the cumulative motion register, where in the motion register each data luminance pixel corresponds to a data item.

Subsequently, 214 identifies data items with a value exceeding a predetermined threshold value. If desired, the tag: '· ,. This movement can also be used to turn on the alarm.

The alarm is checked at 216. If the alarm is on, then 230 will be used, otherwise 222 will be used.

Optional zoom to motion is performed on 230 previous *; 25 min as described. After a possible zoom, it moves to 232 if you export the zoomed image to the encoder. In one embodiment, the camera

• · I

'···' can be movable or optically zoomed, so the 234 checks for these features.

: V If these features are not available, we move on to 208, which reads the next 30 images from the camera. If these features are turned on again, the .X will go to 236, which will control the camera as desired, for example, as shown in the ">" lii above, to rotate the camera towards the detected motion or zoom; · 'Optically into motion. Also, as previously described, 240 modifies the values of the previous image and motion index to correctly detect motion, 1: ·: 35, and not detect motion caused by camera movement or optical zoom. Then, from 240 to 208, the next image from the camera is read.

222 tests whether the camera in use needs to be replaced. If it does not need to be replaced, it proceeds to 234, from which operation continues as described above. If, on the other hand, the camera is changed to another, it moves to 228, where the motion register is cleared, so that the values of the motion register of the previous camera do not interfere with the operation of the new camera.

Fig. 2 does not illustrate the termination of the method, since it can in principle be terminated at any point. The natural end point for 10 is when you no longer want to explore sequential images. A device of the type described above is suitable for carrying out the method, but other types of device may also be suitable for carrying out the method. Preferred embodiments of the method are embodiments according to the appended dependent claims. Their operation has already been described above in connection with the Act, therefore the description will not be repeated here.

A large example of image processing will now be described so that the operating principle by which the method and device recognize motion from sequential images operate is fully understood.

Figures 4A, 5A, 6A and 7A show four images selected from the sequence of Fri-20 consecutive images. Since there is no room here to represent all the pictures in the image sequence, four representative pictures have been selected.

t * ♦

The camera shoots a table in a corner of the room. As shown *, the 4A shown shows a radio recorder on the table.

Next, Fig. 5A shows the sequence later in the sequence; 25 a picture of a person walking to a table.

'; 6A, the person has taken the radio recorder off his desk in his lap.

In the picture of Figure 7A, the person and the radio recorder have left the picture,: ': with only a corner of the room and an empty table left.

Figure 10 illustrates the processing of a luminance pixel in a motion register in connection with the example of Figures 4A, 5A, 6A and 7A. Fig. 10 is formed in the same manner as Fig. 3, i.e., the horizontal axis has the numerical values from 1 to 30 of the consecutive figures I t, and the vertical axis has the value range of both the luminance pixel and the motion element data element. The variations in the value of the luminance pixel under consideration are represented by ·: ·· 35 curves 1000. The variation in the value of the data element corresponding to that luminance pixel in the motion register is described by curve 1002.

20 112018

The selected luminance pixel in Figures 1-13 is part of a radio recorder. A small variation in pixel value is noise. 14-15, the luminance pixel is part of a person's hand. In the figures 16-30, the luminance pixel is part of the table.

As observed by examining curve 1002, in our example again an embodiment is used in which the values of non-zero data elements of the motion register are reduced to zero by a predetermined amount each time the difference data is added. In our example, this predetermined amount is five.

Figures 4B, 5B, 6B and 7B illustrate what Figures 4A, 5A, 6A look like

and 7A identified motion in black and white. In this example, the motion is chosen to represent motion in black and immobility in white.

Since there is no motion in Figure 4A, there is nothing other than white in Figure 4B.

In Fig. 5A, motion is detected, whereby it is shown in Fig. 5B as a black person.

Figure 6B shows that a person has taken the radio recorder in his lap. As can also be seen in Figure 6B, the outlet of the radio recorder remains visible in motion to where it was. As shown in Fig. 7B, once the motion of the person 20 has been lost, the outlet of the radio recorder is still visible. As previously described, adjusting the motion detection threshold, the black-and-white image generation threshold, and the predetermined amount to be used to reduce the difference in data input can be used to adjust how long •: * motion is displayed in the image. If the entire black-and-white sequence is thought to be moving. **: 25, it shows a person's motion as a black character coming up to the table, taking the radio recorder in his lap, and exiting the corner of the room. from the stake area. Decreasing the value of a data item causes the newly detected motion to eventually disappear from the black and white image, i.e. when the motion: v. the value of the data elements contained in the register decreases enough, and the mus, ··. 30 ta black and white radio recorder.

Figures 4C, 5C, 6C and 7C illustrate what Figures 4A, 5A, 6A look like

\ i =: and 7A zoomed image to recognized motion.

As can be seen by comparing Figure 4A and 4C, Figure 4C does not; still used motion zoom because, according to Fig. 4B, there is no i: 35 motion detected yet.

21 112018

Figure 5C shows how the image has been begun to be zoomed from the image to a recognized motion, i.e., to the person walking.

As shown in Figure 6C, zooming is continued.

As shown in Figure 7C, the zoom is oriented toward the retrieved radio-5 recorder, as in Figure 7B, the retraction remained visible as motion.

Then, when motion is no longer recognized in the image, the image area can be restored, for example, by steplessly zooming back to the original image area.

Next, Figure 8 illustrates the effect of moving the camera 10. Using the previously described embodiment, the camera that formed the images is guided from the image in the direction of the detected motion. The previous image is framed by frame 800, and the current image is framed by frame 802. The oblique area 804 is the old image area that is omitted in the motion detection detection. Horizontally crossed area 806 is only the area of the 15 new image, whereby only data elements corresponding to the luminance pixels of the current image are reset. The motion vector 808 between the previous image 800 and the current image 802 determines the direction in which the camera has been moved. The common area 810 of the previous image 800 and the current image 802 is the area where the luminance pixels of the previous image 800 are moved to the luminance pixels of the current image 802 in the current image 80, and similar data elements in the motion register are moved to that same image area. In practice, the motion register data elements and the luminance pixels in the frame buffer 800 of the previous image are moved in the opposite direction determined by the motion vector 808.

Finally, Figure 9 illustrates the effect of the camera's optical zooming on the procedures described above. Using the previously described embodiment, the camera that formed the images is guided optically from the image in the direction of the detected motion. The picture above is framed by 900,: v. and the current image with frame 902. The crossed-out region 904 is the region that is visible. *. The area of the previous image 900 and the current image 902 is the region 906. The image area 900 of the previous image is '' matched with the image area of the current image 902 by omitting the area 904, and ...: interpolating to it. the missing image luminance pixels corresponding to the image area of the current image 902, and the motion register image area is modified to correspond to the current image 902 image area by interpolating the missing data elements corresponding to the image field of the current image 902.

"112018 22

Although the invention has been described above with reference to the example of the accompanying drawings, it is clear that the invention is not limited thereto, but that it can be modified in many ways within the scope of the inventive idea set forth in the appended claims.

Claims (53)

A method for identifying a movement from consecutive images, characterized in that: difference data describing the magnitude difference between the respective preceding luminance pixels and the corresponding current luminance pixels are formed (212); formed difference data is added (212) to a cumulative motion register, in which motion register the respective luminance pixel of the image is matched by a data element; and as movement (214) such data elements, whose value exceeds a predetermined threshold value, are identified. The method according to claim 1, characterized in that a movement is identified blockwise. Method according to claim 2, characterized in that a movement is identified in the bill's block, if at least a predetermined amount of the data elements corresponding to the block in question in the movement register exceeds a predetermined threshold value. Method according to claim 1, characterized in that the values of the movement register on data elements that deviate from noil are reduced to zero with an amount of predetermined size in connection with the addition of the respective data. · [! Method according to Claim 1, characterized in that a black and white image is formed from the data elements included in the movement register. 6. A method according to claim 5, characterized in that the data elements that exceed it in advance are presented in the black and white image. determined the threshold value as white and the other data elements as black, or • You smile at the data elements that exceed the predetermined threshold value. are presented as black and the other data elements as white. * · · • '. f Method according to claim 1, characterized in that the identified motion is zoomed in by interpolating the zoomed area: to the size of the original image. Method according to claim 7, characterized in that the zoomed image is encoded. ; Method according to Claim 7, characterized in that the zoom location is stored in the memory and only a change of predetermined size of the zoom point is allowed between two consecutive images. Method according to claim 7, characterized in that the zoom ratio is stored in memory and only a change of predetermined size of the zoom ratio is allowed between two consecutive images. 5 Method according to claim 1, characterized in that the sensitivity of the motion detection is regulated by controlling the threshold value. Method according to claim 11, characterized in that the threshold value for the current image is formed using a quantity which describes the average value of the luminance pixels of the current image and the threshold value of the previous image. Method according to claim 1, characterized in that if the image area in the current image has moved relative to the image area in the previous image, that is, the camera that took the images has been moved, then the luminance pixels are moved in front of the luminance pixels in the current image. image presenting the same image area, corresponding data elements in the motion register are moved opposite the same image area in question, and only the data elements corresponding to the luminance pixels in the current image are zeroed in the motion register. Method according to claim 1, characterized in that if the image area in the present image has been zoomed optically in relation to the preceding image area, that is, in the camera that took the images optical zoom is used, then the previous image area of the image is processed to correspond to the current image area of the image. by interpolating thereto by missing luminance pixels which: corresponds to the image area of the present image, and the image area of the motion register is processed to correspond to the image area of the present image by interpolating thereto by: 25 missing data elements corresponding to the image area of the present image. 15. A method according to claim 1, characterized in that the camera taking the images is controlled to move in the direction of the motion identified from the image. Method according to claim 1, characterized in that The camera that captured the images is controlled to optically zoom in on the motion identified from the image. Method according to claim 1, characterized in that alarms are given if a movement has been identified. Apparatus for identifying one movement from one another in succession. 35 images, characterized in that the device comprises: 112018 means (116) for forming difference data describing the magnitude difference between the respective previous luminance pixels and the corresponding current luminance pixels; means (118, 124) for adding formed difference data to a cumulative motion register (126), in which motion register (126) the respective luminance pixel of the image is matched by a data element; and means (136) for identifying such data elements as motion whose value exceeds a predetermined threshold value. 19. Device according to claim 18, characterized in that the means (136) for identifying a movement operate in a block-wise manner. Device according to claim 19, characterized in that in the block of the image a movement is identified if at least a predetermined amount of the data elements corresponding to the block in question in the movement register (126) exceeds a predetermined threshold value. 21. Device according to claim 18, characterized in that the device further comprises means (120) for reducing the values of the movement register on data elements which deviate from zero to zero with an amount of predetermined size in connection with the addition of the respective difference data. Device according to claim 18, characterized in that the device further comprises means (146) for forming a black and white image of the data elements included in the movement register (126). Device according to claim 22, characterized in that the data element exceeding the predetermined threshold is presented as white and the other data elements as black, or the data elements exceeding the predetermined threshold. predetermined threshold: ": is presented as black and the other data elements as white. Device according to claim 18, characterized in that the device further comprises means (132) for zooming in on the identified movement by interpolating the zoomed area to the original vehicle. · * ·. Its size. • · 25. Device according to claim 24, characterized in that. i: the arrangement further comprises means (150) to encode the zoomed image. Device according to claim 24, characterized in that: -. the arrangement further comprises means (132, 136) for storing the zoom point in memory, and allowing only a change of predetermined size of the zoom point between two consecutive images. 112018 Device according to claim 24, characterized in that the device further comprises means (132, 136) for storing the zoom ratio in memory, and only allow a predetermined size of the zoom ratio between two consecutive images. . 28. Device according to claim 18, characterized in that the sensitivity of the motion detection is regulated by controlling the threshold. Device according to claim 28, characterized in that the device further comprises means (136) for forming the current image threshold value using a quantity which describes the average value of the current image's luminance pixels and the predicted image threshold value. Device according to claim 18, characterized in that the device further comprises means (136) for moving the preceding luminance pixels opposite the luminance pixels in the present image presenting the same image area, moving the corresponding data elements in the movement register opposite the same image area in question, and zero only the data elements corresponding to the luminance pixels in the current image in the motion register, if the image area in the current image has moved relative to the image area in the preceding image, that is, the camera that captured the images has been moved. Device according to claim 18, characterized in that the device further comprises means (136) for processing the image area of the preceding image to correspond to the image area of the present image by interpolating it. "tili of missing luminance pixels corresponding to the image area of the current image, and '*' process the image range of the motion register to correspond to the image area of the current image: In the case of a previous image area, that is, in the camera that captured the images, optical zoom was used. Device according to claim 18, characterized in that the device further comprises means (102, 136) for controlling the camera which has taken. · *. The images to move in the direction of motion identified from the image. Device according to claim 18, characterized in that the device further comprises means (102, 136) for controlling the camera which has taken the pictures to zoom in on the optical identified from the image. # 34. Device according to claim 18, characterized in that the device further comprises means (136, 154) for giving alarm, if a movement has been identified. 112018 Device according to any of the preceding claims 16-34, characterized in that the device further comprises means (154) for transmitting consecutive images, a formed black and white image or a zoomed image using a telecommunication connection. Apparatus for identifying a movement from consecutive images, characterized in that the apparatus is configured to: form difference data describing the magnitude difference between the respective preceding luminance pixels and the corresponding current luminance pixels; Adding formed difference data to a cumulative motion register, in which motion register the respective luminance pixel of the image is matched by a data element; and as movement identify such data elements, the value of which exceeds a predetermined threshold value. 37. The device according to claim 36, characterized in that the device is additionally configured to identify a movement blockwise. Device according to claim 37, characterized in that a movement is identified in the image block if at least a predetermined amount of the data elements corresponding to the block in question in the movement register 20 exceeds a predetermined threshold value. 39. Device according to claim 36, characterized in that the arrangement is further configured to reduce the values of the movement register on data elements which deviate from zero to zero with an amount of pre-determined. I 'matched size in connection with the addition of the respective difference data. :. ' · * 25 40. Apparatus according to claim 36, characterized in that the *: ··: arrangement is further configured to form a black and white image of the data elements: * '*; which is included in the business register. 41. Device according to claim 36, characterized in that in the black and white image the data elements exceeding the predetermined threshold are presented as white and the other data elements as black, or the data elements exceeding the predetermined one. the threshold value •.:.: is presented as black and the other data elements as white. Device according to claim 36, characterized in that: -! in addition, the arrangement is configured to zoom in on the identified motion, by interpolating the zoomed area to the large play of the original image. 112018 43. Apparatus according to claim 42, characterized in that the device is further configured to encode the zoomed image. 44. Device according to claim 42, characterized in that the device is additionally configured to store the zoom point in memory, and allow only a change of predetermined size of the zoom point between two consecutive images. The device of claim 42, characterized in that the device is additionally configured to store the zoom ratio in memory, and allow only a predetermined size of the zoom ratio between two consecutive images. 46. Device according to claim 36, characterized in that the sensitivity of the motion detection is controlled by controlling the threshold. 47. Apparatus according to claim 46, characterized in that the device is further configured to form the threshold value for the current image 15 using a quantity which describes the average value of the current image's luminance pixels and the preceding image's threshold value. 48. Device according to claim 36, characterized in that the device is additionally configured to move prior luminance pixels opposite the luminance pixels in the current image presenting the same image area, moving corresponding data elements in the movement register opposite the same image area in question. only the data elements corresponding to the luminance pixels in the current image in the motion register, if the image area in the current image has moved in "relative to the image area in the preceding image, that is, the camera that has taken the · Y images has been moved. Yi 25 49. Device according to claim 36, characterized in that the "1 · 1 arrangement is further configured to process the preceding image image area: to correspond to the present image image area by interpolating thereto by missing luminance pixels corresponding to the current image image area, and to process the image area of the motion register to correspond to the image area of the current image through. ·· 1. Interpolation thereon of missing data elements corresponding to the current image • In the image area, if the image area in the current image has been zoomed optically in the ratio Y: to the previous image image area, that is, in the camera that captured the images using :: optical zoom. ; ·. Device according to claim 36, characterized in that the device is additionally configured to give alarms if a movement has been identified. 112018 51. Device according to claim 36, characterized in that the device is additionally configured to control the camera that has taken the images to move in the direction of motion identified from the image. 52. Apparatus according to claim 36, characterized in that the device is further configured to control the camera which has taken the images to zoom in on the optical identified from the image. Device according to any of the preceding claims 36-52, characterized in that the device is further configured to transmit consecutive images, a formed black and white image or a zoomed image using a telecommunication connection. «·» T • · I ·> · * · * · 1 · * I · I I · t