FI112017B - Automatic zooming method and device - Google Patents

Description

112017

Method and device for automatic zoom

Area

The invention relates to a method and a device for automatic zooming.

5 Background IPC classes G03B and H04N describe various solutions for automatically tracking and auto zooming a subject. U.S. Patent No. 4,719,485 describes a camera using a thermal SOR to detect living entities in the image. The detection is based on the difference in temperature of 10 objects from the ambient temperature. After the object is detected, its distance from the camera is measured. The camera is then rotated on a motorized stand according to the motion of the object. Motorized zoom is used to keep the object being tracked constant throughout the image, regardless of the object's distance from the camera. The problem with the solution is that it requires the use of a thermal sensor, a motorized pedestal, and distance measuring devices in a cost-effective manner.

Short description

The object of the invention is to provide an improved method for automatic zooming and an improved device for automatic zooming.

In one aspect of the invention, there is provided a method of automatic zooming, comprising: generating eroded data representing the difference between the luminance pixel of each preceding image and the corresponding image luminance pixel; adding the generated difference data to a cumulative motion:: register, where in the motion register each data luminance pixel of the image:: 25 element; detecting data elements whose value exceeds a predetermined threshold value; and zoom in to the recognized motion.

In one aspect of the invention, there is provided a device for automatic zooming, comprising: means for generating erosional data representing the difference between the luminance pixel of each preceding image and the corresponding image luminance pixel; means for adding the generated eroded data to a cumulative motion register, wherein in the motion register a data item corresponding to each luminance pixel of the image; means for detecting data items whose value exceeds a predetermined threshold value; and means for zooming into the identified motion.

2 112017

In one aspect of the invention, there is provided a device for automatic zooming configured to: generate difference data representing the difference between the luminance pixel of each preceding image and the corresponding current image luminance pixel; adding the generated difference data to the cumulative motion register 5, where in the motion register each data luminance pixel corresponds to a data item; identify data items whose value exceeds a predetermined threshold value; and zoom into the recognized motion.

Preferred embodiments of the invention are claimed in the dependent claims.

The invention is based on analyzing the differences between the luminance pixels of successive images, whereby motion can be detected automatically. The procedure is somewhat reminiscent of motion estimation used in video encoding, but the motion register used is cumulative, in which case a person who moves for a short period of time and then the py-15 is fired is also identified as motion.

The method and apparatus of the invention provide several advantages. The procedure can be used to zoom into identified motion without the use of special equipment parts, for example, without the heat sensors required in the aforementioned U.S. Patent. Also, the invention does not necessarily require optical zoom, since the zoom can be performed digitally. Previously, it is not really known to zoom into a recognized motion, but known solutions: Ys operate on the basis of focus, i.e. they zoom in on what is called a "motion". to nearest point:. . a. The procedure of the invention can also automatically zoo. Mata other than the center of the image area, while prior art solutions * ·· '25 zoom in on the center of the image area.

. Embodiments of the process also have other advantages, which are described below;

BRIEF DESCRIPTION OF THE DRAWINGS The invention will now be described in more detail with reference to the preferred embodiments, with reference to the accompanying drawings, in which: 'FIG. 1 is a block diagram showing a device for automatic zooming; FIG. 2 is a flow chart illustrating a method for automatic zooming; Fig. 3 illustrates the processing of a luminance pixel in a motion register; Figures 4A, 5A, 6A and 7A show four images selected from a sequence of sequential images; Figures 4B, 5B, 6B and 7B show what Figures 4A, 5A, 6A and 7A would look like. identified motion in black and white;

Figures 4C, 5C, 6C and 7C illustrate what Figures 4A, 5A, 6A look like

and 7A a zoomed image of the detected motion; Figure 8 illustrates the effect of moving the camera; Figure 9 illustrates the effect of using optical zoom of a camera; and FIG. 10 illustrates processing of a luminance pixel in a motion register in connection with the example of FIGS. 4A, 5A, 6A and 7A.

Description of Embodiments

Referring to Figure 1, a device for automatic zooming is described. Basically, the device is simple, it is configured to form-15 ground erod data by subtracting the current image luminance pixel from each of the previous image's luminance pixel. The device is also configured to add the generated difference data to the cumulative motion register. In the cumulative motion register, each data luminance pixel corresponds to a data item. The device is configured to detect data items whose value exceeds a predetermined threshold value of 20. The actual zooming is done with the device in; configured to zoom into a recognized motion.

. ·. Let us now take a closer look at the device illustrated in Figure 1. The device is: connected to an image source, in our example, a camcorder 158. The image source can be any known device which produces sequential images 100. Since the invention analyzes the snow; changes in the nance pixels, the information from the image source must contain the luminance information, either as discrete luminance pixels, or else the luminance pixels must be capable of being extracted or transformed from the information from the image source.

30 Sequential images, such as video, are coded '. t reduce the amount of data so that it can be stored more efficiently ;;; to a storage medium or transfer using a data connection. An example of a video coding standard is MPEG-4 (Moving Pictures Expert Group). There are different image sizes used, for example, the cif size is 352 x 288 pixels and the qcif size is 35,176 x 144 pixels. Typically, a single image is divided into blocks containing information about luminance, color, and location. The block data is compressed 4112017 blocks by the desired encoding method. Compression is based on the removal of less relevant data. Compression methods are mainly divided into three categories: Spectral Redundancy Reduction, Spatial 5 Redundancy Reduction, and Temporal Redundancy Reduction. Typically, various combinations of these methods are used for compression.

For example, a YUV color scheme is used to reduce spectral redundancy. The YUV model takes advantage of the fact that the human eye is 10 more sensitive to variations in luminance, or luminance, than to changes in chrominance, or color. The YUV model has one luminance component (Y) and two chrominance components (U and V, or Cb and Cr). For example, the H.263 video encoding standard luminance block is 16 x 16 pixels and both chrominance blocks, which cover the same area as the luminance block, are 8 x 8 pixels.

The combination of one luminance block and two chrominance blocks is called a macroblock. Each pixel, in both the luminance and chrominance blocks, can have a value between 0 and 255, which means that eight bits are needed to represent one pixel. For example, a value of 0 for a luminance pixel is black and a value for 255 is white.

Thus, the successive images 100 entering the device may follow the YUV model, in which the luminance pixels are already ready as their own component: Y: for example, in a cif image, 352 x 288 luminance pixels.

: The image source image 100 is exported to a frame buffer 106 where the recording: ·. *. luminance and chrominance components of the image.

25 Frame buffer 106 contained the luminance component of the previous image. · ''. is inserted 108 into a second frame buffer 110.

Then, in block 116, difference data is generated depicting the difference between the luminance pixel of each previous image and the corresponding luminance pixel of the current image. This difference data generation can be done, for example, by subtracting the luminance pixel of the current image 112 corresponding to each of the luminance pixels of the previous image 114, or otherwise giving the same result; mathematical operation.

The generated erod data 118 is then added to the cumulative motion register 126. The cumulative motion register 126 is a memory the size of an image in the frame: 35 keus times the image width. A possible reading range is [-255,255] if I t 5 112017 YUV format image is used. The memory may also be smaller, but then the incoming data will have to be preprocessed.

In one embodiment, the device is configured to reduce the values of non-zero data elements of the motion register to zero by a predetermined amount each time the difference data is added. In the example of Figure 1, this is illustrated by a block 120 into which the eroded data 118 is imported, subtracting a predetermined number from the 125 eroded data items imported from each motion register 126, and the resulting difference added by the eroded data 124 is then applied to the motion register 126 A predetermined number of 10 may be, for example, a number one or another number greater than one. Of course, the subtraction can also be done by adding the eroded data to the motion register 126, after which a predetermined number is subtracted from each non-zero data item. The device has a control block 136 which controls the operation of different parts of the device. Block 120 is provided with a predetermined number 15 122 from control block 136 which is subtracted from each data item.

After the previous image and the current image have been processed as described above, the control block 136 scans 128 the motion register 126 in a predetermined manner, for example, row by column. Then, the control block 136 identifies as motion the data elements of the motion register 126 whose value exceeds 20 predetermined thresholds. In one embodiment, the control block detects motion by blocks, for example, luminance macrol blocks. The blocks may also: Y be partially overlapping. In one embodiment, motion is detected in a block of an image if at least a predetermined number of data items corresponding to that block in motion register 126 exceeds a predetermined threshold.

Figure 3 illustrates the processing of a luminance pixel in a motion register 126. The example is based on the prototypes and experiments made by the applicant. The exemplar is based on a series of events filmed with a video camera depicting a light table. A person arrives and leaves a black briefcase on the table.

: 30 In Fig. 3, the horizontal axis has the numbers 1-150 of the consecutive images and the vertical axis has the value range of both the luminance pixel and the motion element data element. The variations in the value of the luminance pixel under consideration are plotted with a curve 300. The variation 1: 'of the data element corresponding to the' Y, luminance pixel in the motion register 126 'is represented by the curve 302.

v: 35 The luminance pixel selected for viewing in Figures 1-13 is part of the light table. A small variation in pixel value is noise. In Figures 14-18, the 6112017 Nance Pixel is part of a black briefcase that is placed on a table. Similarly, in Figures 19-150, the luminance pixel is part of a black portfolio that was left on the table.

As can be seen by examining curve 302, our example employs an embodiment in which values of non-zero data-5 elements in the motion register 126 are reduced by zero by a predetermined amount each time the difference data is added. In our example, this predetermined amount is one. Decreasing the value of a data item causes the portfolio in Figure 137 to have already completely disappeared from the motion register 126, i.e., merged into the background.

Since data elements whose value exceeds a predetermined threshold value are detected in the device, it is interesting to consider what would be a suitable threshold value in the example of Figure 3. A suitable threshold value that eliminates noise but no movement could be, for example, 10-15, whereby motion would no longer be detected by approximately the size of Figure 130.

When motion is detected in the image, control block 136 controls 144 frame buffer 106 to bring the zoom area 130 of the current image into zoom block 132. Control information 144 includes information on which the zoomed target area in the image is transmitted as image 130 to zoom block 132. control information 134 to zoom block 132, which control information indicates the relationship between the sizes of the incoming and outgoing images. This ratio is 1: 1 if the zoom is not used. Thus, the zoom block 132 zooms in to the identified: Y: motion, for example, by interpolating the zoomed area to the size of the original: image using known interpolation methods.

Motion detection detects the area where motion occurs. Limit-. · · ·. Adjust this area so that the ratio of height to width is always the same relative to the original image. In addition, the area to be zoomed in should be slightly larger than the area with motion. The magnification must also have a maximum, which depends on the image size used and the resolution of the camera. For example, a 100x magnification of a qcif image no longer makes sense.

• 30 In one embodiment, the control block 136 and / or the zoom block 132 stores the zoom position in memory so that the change in the zoomable area is reduced. Zoom position information can be utilized by allowing Aino -. * ··. I set a predetermined change in zoom position between two consecutive '·' images. In addition to or instead of the zoom position, control block 136 and / or: 35 zoom block 132 may store the zoom ratio in memory. Zoom ratio information can be utilized by allowing only a predetermined change of the zoom ratio between two consecutive images. By limiting the change in the zoom position and / or the zoom ratio, it is more comfortable to view the image because the image is not changed too quickly based on the detected motion, but at certain magnitudes, for example. Change-5 speed control is an optimization task between the detected motion tracking speed and the image quality and information content.

Also, it is good to allow the motion area to move freely within the zoom range within certain limits. By the above methods, a smoothly moving image is obtained.

The zoomed or non-zoomed image 142 is then passed to an optional encoding block 150 whereby the image can be encoded if desired. The encoding of the zoomed image has the advantage that the final result of the encoding is improved because the unnecessary information has been removed from the image.

The encoding block 150 may be, for example, a known video-15 encoder, for example an mpeg4 encoder. Figure 142 may also be exported to a storage medium included in the device or to a storage medium connected to the device, such as a computer hard drive. Fig. 142 may also be provided to a viewing device, such as a monitor; then, depending on the presentation, the image may need to be converted.

In one embodiment, adjusting the predetermined threshold value used in motion detection adjusts the sensitivity of motion detection: Y. In general, this may be expressed, for example, by generating a threshold value for the current image in the control block 136 using the magnitude representing the average value of the luminance pixels of the current image and, · · ·. 25 previous image thresholds. For example, for image k, the luminance ♦ · can be calculated. '; mean of the pixels * · * i <nj <m ak = (ΣΡι ^ / ηχηι, (1) '· 1 / = 0.7 = 0 where py is the luminance pixel and nxm is the size of the image. all luminance pixels in the image.

Then, for image k, the threshold value tk = pak + q, and (2) I- ·, h = (^ + ^ -,) / (^ + ^), (3): 'where tk-1 is the threshold value of the previous image. In Formula 2, a non-linear function is used where p and q are constants, but a nonlinear function can also be used. According to formula 3, the threshold can also be weighted, r and s being constants, which can be changed by adjusting the threshold and thus the sensitivity of the device to large variations in luminance. The device interface may include a controller for adjusting the threshold steplessly to achieve the desired sensitivity. Weighing should be done to prevent the system from responding to rapid large changes in brightness such as lightning or cloud light. In formulas 2 and 3, it is not necessary to use an average, but some other statistical measure of average value can be used. Noise can also be measured from random pixels and the sensitivity adjusted accordingly.

In one embodiment, the device is configured, e.g., control block 136, to move the luminance pixels of the previous image to the luminance pixels of the same image in the current image, move the corresponding data items in motion register 126 to that same image area, and reset the pixels of the snow-15 corresponding to the data elements in the motion register 126 if the image area in the current image has moved relative to that of the previous image, i.e., the camera that formed the images has been moved 158,160.

In one embodiment, the device is configured, e.g., control block 136, to modify an image area of the previous image to correspond to the 20 fields of the current image by interpolating the missing luminance pixels corresponding to the field of the current image, and interpolating the image area of the image with missing data elements corresponding to the image area of the current image, if the current image has been zoomed optically relative to the image area of the previous image, i.e., the 158,160 camera used to create the image · ·, 25 .

Y; In one embodiment, the device is configured, for example, by a control Y [block 136] to determine 140 selection blocks 102, to direct the camera 160 that formed the images 138 to move in the direction of the detected motion. In this case, the camera 160 has, for example, an electric motor for turning the camera 160 in the direction of movement of the V 30. Control command 138 indicates the magnitude of movement required: ...: for example, in degrees.

. . In one embodiment, the device is configured, for example, by a control. ··· Block 136 assigns 140 selection blocks 102 to direct the camera 160 that formed the images 138 to optically zoom in on the detected motion. In this case, the v: 35 camera 160 has an electrically controlled optical zoom that can be used to zoom the image produced by the camera 160 9112017. Control command 138 indicates the required zoom ratio change.

In one embodiment, the device comprises a block 154 for transmitting consecutive images and / or a zoomed image using a communication link 156. The image to be sent may be an image from the original source or a zoomed image. The camera that produced the images may also be moved or optically zoomed from the image in the direction of the detected motion. The telecommunication connection utilizes known solutions, for example wireless radio, LAN, Internet, fixed dedicated line, or some other way of transferring data between two points 10.

The device blocks shown in Figure 1 may be implemented as one or more application-specific integrated circuits (ASICs). Other implementations are also possible, such as a circuit built from separate logic components, or a processor with software. 15 A mixed form of these different embodiments is also possible. The skilled artisan will consider, for example, the size and power requirements of the device, the processing power required, the manufacturing cost, and the production volume. It should be noted that Figure 1 illustrates mainly functional assemblies, whereby parts of a practical hardware implementation may deviate from what is shown, since this is ultimately a degree of integration: how to achieve the desired 'functional' device for automatic zooming in the most efficient and reasonable cost.

A: The above device is suitable for a variety of purposes. You can also make a very cheap version of your device. One cheap version is one where the device,. The 25 also comprises a cheap camera and can be connected to mains power. Further, the device: comprises parts by which it can act as a subscriber terminal of a cellular network. This allows the * »'device to be guarded, for example, at its owner's summer cottage, and transmits the image to the owner's subscriber terminal as soon as it senses movement and zooms in. The device thus acts as a burglar alarm and a monitoring device which can '30 check the situation in the summer cottage. Naturally, other sensors, such as a fire alarm, can also be connected to the device. Other applications for the described basic device may also be found by one skilled in the art.

, The following describes the execution of some functions of the device using a pseudocode that • complies with the C programming language syntax.

t i I

10 112017

Header

#define mmin (a, b) ((a) <(b)? (a) :( b)) / * macro for two values of minimum V

5 typedef struct {int ** lum; int ** cb; int ** cr; 10} date; typedef struct {int mid [2]; / * midpoint * / 15 inthor; / * horizontal magnitude * / int ver; / * vertical magnitude * /} mot; typedef struct 20 {int Iines; / * image height 7 int columns; / * image width 7: mot motion; / * range with motion at 7 mot zoom; / * area with zoomion 7, *. 25 data previous; / * previous image * /,: data present; / * current image * / int '' movement; / * business register * / '' int 'shape; Γ filtered motion register * / int sensitivity; / 'Sensitivity * /! · '30 Jsecur; / * variable defined in main function 7 h »

Motion register function "Adds a difference between the images in the motion register and reduces the data in the register: 35 blades by one pixel.

11 112017 int Add_motion (secur 'frame) {int i, j; / * counters * / int r = frame-> lines; / * image rows 7 5 int c = frame-> columns; / * image columns 7 for (i = 0; i <r; i ++) {for (j = 0; j <c; j ++) 10 {frame-> movement [i] [j] + = {(frame-> present ) .lum [i] [j] - (frame-> previous) .lum [i] [j]); if (frame-> movement [i] [j]> 0) frame-> movement [i] [j] - = 1; 15 if (frame-> movement [i] [j] <0) frame-> movement [i] [j] + = 1; }} return (1); 20}

Motion check function i '·: int Motion_check (secur' frame). 25 {; int i, j, k, I; / * counters 7,] int r = frame-> lines; / * image rows 7 int c = frame-> columns; / * image columns * / intfound = 0; / * whether or not movement was found * / 30 int sum; / * number of motion pixels in a block * / int min_k, max_k, minj, maxj; / * found movement variables * / int sens = frame-> sensitivity; / * sensitivity * / for (i = 0; i <r; i ++) / * copier filter motion register * /: 35 {for (j = 0; j <c; j ++) {12 112017 frame-> shape [i] ö ] = frame-> movement [i] D]) / sens; }} 5 min_k = c; / * initialize minimum and maximum * / max_k = 0; minj = r; maxj = 0; 10 for (k = 0; k <c-7; k + = 4) / * find the motion and its region in 8x8 blocks 7 {/ * the blocks are overlapping 7 for (l = 0; Kr-7; l + = 4) { 15 sum = 0; for (i = 0; i <64; i ++) {if (frame-> shape [l + i / 8] [k + i% 8]! = 0) 20 sum ++; } »* *:. **; if (sum> sens) IV: {. ·· *. 25 found = 1; , · ''; if (k <min_k) min_k = k; if (k> max_k) max_k = k + 3; :. * 30 if (l <min_l) min_l = l; if (l> max_l), ·, max_l = l + 3; } * '35} 13 112017} if (found == 1) / * if motion, calculate its width, height, and center * / {5 (frame-> motion) .mid [0] = (min_k + max_k) / 2 ; (frame-> motion) .mid [1] = (min_l + max_l) / 2; (frame-> motion) .hor = max_k-min_k; (frame-> motion) .ver = max_l-min_l; } 10 else / * if not in motion then the width, height and center point are shown in Figure 7 {(frame-> motion) .hor = c; (frame-> motion) .ver = r; (frame-> motion) .mid [0] + = Mmax (((c / 2) - (frame-> motion) .mid [0]) / 8.1); 15 (frame-> motion) .mid [1] + = Mmax (((r / 2) - (frame-> motion) .mid [1]) / 8.1); } return (1); } 20

Zoom Function • ·; int Zoom_control (secur * frame) tl · '· · LV {25 int r = frame-> Iines; / * image rows 7 int c = frame -> columns; / * image columns 7, · '* ·; int left, right, up, down; / * zoom edges 7 int i; / * counter 7 * · · • · »• * \.! 30 / * calculate new zoom height and width (relative to motion) * /

• I

• I ·,.; Γ if ((frame-> motion) .hor <(c * (frame-> motion) .ver) / r):. 'V {t |. t if ((frame-> zoom) .ver> = (frame-> motion) .ver + r / 6) 35 {'* ·' (frame-> zoom) .ver - = 14 112017 mmin {abs ((( frame-> zoom) .ver- (frame-> motion) .ver -r / 6) / 4), 4); (frame-> zoom) .ver = Mmax (mmin ((frame-> zoom) .ver, r), r / 2); (frame-> zoom) .hor = (c * (frame-> zoom) .ver) / r; 5 (frame-> zoom) .hor = Mmax (mmin ((frame-> zoom) .hor, c), c / 2); (frame-> zoom) .hor = ((frame-> zoom) .hor / 2) * 2; (frame-> zoom) .ver = ((frame-> zoom) .ver / 2) * 2; } 10 if ((frame-> zoom) .ver <(frame-> motion) .ver + r / 10) {(frame-> zoom) .ver + = mmin {abs (((frame-> zoom) .ver - (frame-> motion) .ver - r / 10) / 4), 8): 15 (frame-> zoom) .ver = Mmax (mmin ((frame-> zoom) .ver, r), r / 2 ); (frame-> zoom) .hor = (c * (frame-> zoom) .ver) / r; (frame-> zoom) .hor = Mmax (mmin ((frame-> zoom) .hor, c), c / 2); (frame-> zoom) .hor = ((frame-> zoom) .hor / 2) * 2; (frame-> zoom) .ver = ((frame-> zoom) .ver / 2) * 2; 20}::} / * end of if statement 7 »· else I ': 25 {if ((frame-> zoom). Hor> = (frame-> motion). Hor + c / 6) {* * * * '(frame-> zoom) .hor - = mmin (abs (((frame-> zoom) .hor- (frame-> motion) .hor - c / 6) / 4), 4); i 30,; (frame-> zoom) .hor = Mmax (mmin ((frame-> zoom) .hor, c), c / 2); .:. (frame-> zoom) .ver = (r * (frame-> zoom) .hor) / c; / · ', (Frame-> zoom) .ver = Mmax (mmin ((frame-> zoom) .ver, r), r / 2); • »: (frame-> zoom) .hor = ((frame-> zoom) .hor / 2) * 2; · 35 (frame-> zoom) .ver = ((frame-> zoom) .ver / 2) * 2; } 15 112017 if ((frame-> zoom) .hor <(frame-> motion) .hor + c / 10) {(frame-> zoom) .hor + = 5 mmin {abs (((frame-> zoom) .hor- (frame-> motion) .hor - c / 10) / 4), 8): (frame-> zoom) .hor = Mmax (mmin ((frame-> zoom) .hor, c), c / 2); (frame-> zoom) .ver = (r * (frame-> zoom) .hor) / c; (frame-> zoom) .ver = Mmax (mmin ((frame-> zoom) .ver, r), r / 2); 10 (frame-> zoom) .hor = ((frame-> zoom) .hor / 2) * 2; (frame-> zoom) .ver = ((frame-> zoom) .ver / 2) * 2; }} Γ else end of statement * / 15 / * move center of zoom towards center of motion 7 for (i = 0; i <2; i ++) {if ((frame-> zoom) .mid [i] <(frame-> motion) .mid [i] -4) 20 {(frame-> zoom) .mid [i] i: + = mmin (abs (((frame-> zoom) .mid [i] - (frame-> motion) .mid [i]) / 4), 8); :;>} '·': If ((frame-> zoom) .mid [i]> (frame-> motion) .mid [i] +4). '···': 25 {(frame-> zoom) ) .mid [i] - = mmin (abs (((frame-> zoom) .mid [i] - (frame-> motion) .mid [i])) / 4.8); · .. · *}} i V 30 :: / * keep the zoom within the image 7.:. (frame-> zoom) .mid [0] =. * · ·, Mmin (Mmax ((frame-> zoom) .mid [0], (frame-> zoom) .hor / 2) Ic- (frame-> zoom) .hor / 2); : 35 (frame-> zoom) .mid [1] = • t 1β 112017 16 mmin (Mmax ((frame-> zoom) .mid [1], (frame-> zoom) .ver / 2), r- ( frame-> zoom) .ver / 2); left = (frame-> zootn) .mid [0] - (frame-> zoom) .hor / 2; 5 right = (frame-> zoom) .mid [0] + (frame-> zoom) .hor / 2; up = (frame-> zoom) .mid [1Hframe-> zoom) ver / 2; down = (frame-> zoom) .mid [1] + (frame-> zoom) .ver / 2;

Convert (frame, left, right, up, down); Γ Call interpolation function 7 10 return (1); }

Referring now to the flowchart of Figure 2, a method for automatic zooming will be described. The process begins at 200. 15 Then the 204 reads the image from the camera. The image must be such that the luminance component can be read directly from it, or else the luminance component can be generated based on the information contained in the image. The luminance component is stored in 206.

The 208 then reads the next image from the camera. The sensitivity adjustment of the system 20, which is performed as described above, is performed at 210.

Next, we move to 212, generating difference data representing the difference between the luminance pixel of each preceding image and the corresponding current image luminance pixel, for example, by subtracting the luminance pixel of the current image from each of the previous 25 pixels. In the same 212, the generated erod data is added to the cumulative motion register, where in the motion register each data luminance pixel corresponds to a data item.

Thereafter, 214 identifies data items whose value exceeds a predetermined threshold value. Motion recognition for the V 30 is dipped in 216. If motion is detected, then we move to 230, otherwise we move to * 222.

. :, Zoom in motion is performed in 230, · * · as described previously. a. After a possible zoom, we move to 232, where a zoo-matt image is exported to the encoder. In one embodiment, the camera may be movable or optically zoomable, so 234 checks for these features.

17 112017

If these features are not available, we move on to the 208, which reads the following image from the camera. If these features are enabled again, the 236 will be moved to control the camera as desired, for example, as described above, to rotate the camera toward a recognized motion or to zoom into 5 optical movements. Also, as described above, 240 adjusts the values of the previous image and motion register so that motion detection works correctly and motion is not detected due to camera movement or optical zoom. Then, from 240 to 208, the next image from the camera is read.

Fig. 2 does not illustrate the termination of the method, since it can in principle be terminated at any point. The natural end point is when you no longer want to explore sequential images. A device of the type described above is suitable for carrying out the method, but other types of device may also be suitable for carrying out the method. Preferred embodiments of Method 15 are embodiments according to the appended dependent patent claims. Their operation has already been described above in connection with the device, so the description will not be repeated here.

The following is an extensive example of image processing to fully understand the principle of auto zoom.

Figures 4A, 5A, 6A and 7A show four images selected from a sequence of consecutive images. Since there is no room here to represent all the pictures in the left sequence, four representative pictures have been selected.

The camera shoots a table in a corner of the room. As shown in Figure 4A, there is a radio recorder on the table.

; Next, Figure 5A shows a later image of a person walking to a table in the sequence.

6A, the person has taken the radio recorder off his desk in his lap.

,,, In the picture of Fig. 7A, the person and the radio recorder have left the picture, t; 30 with only a corner of the room and an empty table left.

FIG. 10 illustrates the processing of a luminance pixel in a motion register in connection with the example of FIGS. 4A, 5A, 6A and 7A. Figure 10 is formed. on the same principle as Figure 3, that is, the horizontal axis has the numbers 1 through 30 of the consecutive images and the vertical axis has the value range f> '35 of both the luminance pixel and the motion register data. The fluctuations in the value of the luminance pixel under consideration are represented by the curve 1000 of 18112017. The fluctuation in the value of the data element corresponding to that luminance pixel in the motion register is represented by the curve 1002.

The selected luminance pixel in Figures 1-13 is part of a radio recorder. A small variation in pixel value is noise. In Figures 14-15, the snow-5 nuance pixel is part of a person's hand. In the figures 16-30, the luminance pixel is part of the table.

As can be seen by examining curve 1002, in our example again an embodiment is used in which the values of non-zero data elements of the motion register are reduced to zero by a predetermined amount at each increment of the erod data. In our example, this predetermined amount is five.

Figures 4B, 5B, 6B and 7B show what the motion identified in Figures 4A, 5A, 6A and 7A looks like in black and white. In this example, the motion is chosen to represent motion in black and immobility in white.

Since there is no motion in Figure 4A, there is nothing other than white in Figure 4B.

In Fig. 5A, motion is detected, whereby it is shown in Fig. 5B as a black person.

Figure 6B shows that a person has taken the radio recorder into the lap-20. As can also be seen in Figure 6B, the outlet of the radio recorder remains visible in motion to the place where it was. As shown in Figure 7B, once the person's motion has been lost, the radio recorder outlet is still visible. As previously described, liikk the motion detection threshold and the difference data are added, · By adjusting the predetermined amount used for reduction, '25 can be used to adjust how long the motion is detected. If the entire monochrome •: quency is thought to be a moving image, it shows the motion of the person in black. · * As a character comes to the table, takes the radio recorder in his lap and exits the camera field of view. Decreasing the value of the data element causes the motion detected in the image to eventually disappear from the black and white image, i.e. when the value of the data elements in the motion register decreases sufficiently, the black radio recorder also disappears from the black and white image.

Figures 4C, 5C, 6C and 7C illustrate what Figures 4A, 5A, 6A look like. ·, And 7A zoomed image to the detected motion.

As can be seen by comparing Fig. 4A and 4C, Fig. 4C has not yet used motion zoom because, according to Fig. 4B, motion has not yet been detected.

19 112017

Figure 5C shows how the image has been begun to be zoomed from the image to a recognized motion, i.e., to the person walking.

As shown in Figure 6C, zooming is continued.

As shown in Figure 7C, the zoom is oriented toward the retrieved radio-5 recorder, as in Figure 7B, the retraction remained visible as motion.

Then, when motion is no longer recognized in the image, the image area can be restored, for example, by continuously zooming back to the original image area.

Next, Figure 8 illustrates the effect of moving the camera 10. Using the previously described embodiment, the camera that formed the images is guided from the image in the direction of the detected motion. The previous image is framed by a frame 800 and the current image is framed by a frame 802. The oblique area 804 is the old image area that is omitted in the motion detection detection. Horizontally crossed area 806 is only the area of the 15 new image, whereby only data elements corresponding to the luminance pixels of the current image are reset. The motion vector 808 between the previous image 800 and the current image 802 determines the direction in which the camera has been moved. The common area 810 of the previous image 800 and the current image 802 is the area where the luminance pixels of the previous image 800 are moved to the luminance pixels of the current image 802 in the current image 80, and similar data elements in the motion register are moved to that same image area. In practice, motion elements data elements and luminance pixels in the frame buffer 800 of the previous image are moved by a motion vector; . 808 prescribed distance in opposite direction.

. · ·, 25 Finally, Figure 9 illustrates the effect of using the optical zoom of a camera on the procedures described above. Using the previously described embodiment, the camera that formed the images is directed to optically zoom in on the newly detected motion. The previous image is framed by frame 900 and the current image is framed by frame 902. The crossed area 904 is the area that you see. The previous image 900 and the current image 902 share a region 906. The image area 900 of the previous image is modified to match the image area of the current image 902 by omitting area 904 and interpolating it the missing luminance pixels corresponding to the image area of the current image 902, and adjusting the image area of the motion register to match the image area of the current image 902 by interpolating to the image area of the current image 902; data items.

20 112017

Although the invention has been described above with reference to the example of the accompanying drawings, it is clear that the invention is not limited thereto, but that it can be modified in many ways within the scope of the inventive idea set forth in the appended claims.

t * * ·> 1 · ·

· I

Claims (44)

1. Automatic zooming method, characterized by the method: difference (212) difference data describing the size difference between each preceding luminance pixel and corresponding current luminance pixel is formed; added (212) formed difference data to a cumulative motion register, in which motion register each luminance pixel in the image is matched by a data element; Such data elements (214) are identified as motion whose value exceeds a predetermined threshold value; and zoomed (230) an identified motion. 2. A method according to claim 1, characterized in that the movement is identified blockwise. 15 3. A method according to claim 2, characterized in that the image block is identified as moving, if at least a predetermined amount of the data elements corresponding to the block in question in the movement register exceeds the predetermined threshold value. The method according to claim 1, characterized by decreasing the values of the data elements in the movement register that deviate from noil are reduced to zero. ·. with a predetermined amount in connection with the addition of each ... difference data. Method according to claim 1, characterized in that the identified motion is zoomed in by interpolating the zoomed area 25 to the size of the original image. * t The method according to claim 1, characterized by the '='. 'The zoomed image is encoded. Process according to Claim 1, characterized by the offset | Each zoom location is stored in the memory, and only one change of the pre-determined: “”: 30 size of the zoom location is allowed between two consecutive images. 8. A method according to claim 1, characterized by a reduced zoom. the ratio is stored in the memory, and only a change of the predetermined size of the zoom ratio is allowed between two consecutively:! : pictures. : ""; 35 9. A method according to claim 1, characterized in that by controlling the threshold, the sensitivity of the motion detection is controlled. 112017 10. A method according to claim 1, characterized by decreasing the threshold of the current image by using a quantity which describes the average of the luminance pixels of the present image and the threshold of the preceding image. 5 11. A method according to claim 1, characterized in that if the image field in the current image has moved relative to the previous image image field, i.e. the camera that formed the images has been moved, then the preceding luminance pixels are moved opposite the luminance pixels in the current image representing the same image. image fields, corresponding data elements in the motion register are moved opposite the same image field, and the data elements corresponding to luminance pixels in the current image are only zeroed in the motion register. Method according to claim 1, characterized in that the image field in the current image has been zoomed optically in relation to the preceding image field, that is, optical zoom has been used in the camera which formed the images, so that the previous image image field is processed to correspond to the current image. image fields by interpolating therein by omitted luminance pixels corresponding to the current image image field, and the motion register image field is processed to correspond to the current image image field by interpolating therein by omitted data elements corresponding to the current image image field. 20 13. A method according to claim 1, characterized in that the camera which formed the images is controlled to move in a direction of motion identified from the image. . 14. A method according to claim 1, characterized in that the camera that formed the images is controlled to optically zoom in on the image from which the image identifies movement. Apparatus for automatic zooming, characterized in that the apparatus comprises: means (116) for generating difference data describing the size difference between each preceding luminance pixel and corresponding * V 30 current image luminance pixels; means (118, 124) for adding formed difference data to a cumulative motion register (126), in which motion register (126) is each a luminance pixel in The image corresponds to a data element; In means (136) for identifying such data elements as motion, whose value exceeds a predetermined threshold value; and: "" "means (132) for zooming in on an identified movement. 112017 Device according to claim 15, characterized in that the means (136) for identifying the movement operate blockwise. Device according to claim 16, characterized in that in the image block a movement is identified, if at least a predetermined amount of the data elements corresponding to the block in question in the movement register (126) exceeds the predetermined threshold value. 18. Device according to claim 15, characterized in that the device further comprises means (120) for reducing the values of the data elements in the movement register which deviate from zero to zero with a predetermined amount in connection with the addition of was also a difference data. 19. Device according to claim 15, characterized in that the device further comprises means (132) for zooming in the identified movement by interpolating the zoomed area to the size of the original image. 20. Device according to claim 15, characterized in that the device further comprises means (150) for encoding the zoomed image. 21. Device according to claim 15, characterized in that the device further comprises means (132, 136) for storing a zooming point in the memory, and allowing only a predetermined size change of the zooming point between two successive images. . Device according to claim 15, characterized in that the device further comprises means (132, 136) for storing the zoom ratio in the memory, and allowing only a change of predetermined size I »I; *. ·. of the zoom ratio between two consecutive images. , '··. 25 Device according to claim 15, characterized in that V-V '. By controlling the threshold, the sensitivity of the motion detection is regulated. Device according to claim 15, characterized in that the device further comprises means (136) for forming the threshold value for the current image using a quantity describing the average value for the current: 30 luminance pixels are formed and the threshold value is formed. : 25th Device according to claim 15, characterized in that the device further comprises means (136) for moving the preceding lumi-, '· <*, nance pixels opposite the luminance pixels in the current image representing the same «· in the image field, moving the corresponding data elements in the motion register in front of the same image field, and zero the data elements corresponding to luminance pixels only: -, only in the current image in the motion register, if the image field in the current image has moved relative to the preceding image field, that is, The camera as the photo-dat images has been moved. 26. Device according to claim 15, characterized in that the device further comprises means (136) for processing the foreground image fields to correspond to the current image field by interpolating therein the omitted luminance pixels corresponding to the current image field, and processing the motion register image field to correspond to the current image image field by interpolating therein by omitted data elements corresponding to the current image image field, if the image field in the current image has been optically zoomed in relation to the preceding image field, that is, optical zooming has been used in the camera that formed the images. Device according to claim 15, characterized in that the device further comprises means (102, 136) for controlling the camera which formed the images to move in the direction of motion identified from the image. Device according to claim 15, characterized in that the device further comprises means (102, 136) for controlling the camera which formed the images to optically zoom in on the motion identified from the image. Device according to claim 15, characterized in that the device further comprises means (154) for transmitting a zoomed image using a telecommunication connection. 30. Automatic zoom device, characterized in that: V: the device is configured to:; ; ·. forming difference data describing the size difference between each preceding luminance pixel and corresponding current luminance pixel is formed; ! ''. adding formed difference data to a cumulative motion register, in which motion register each luminance pixel in the image corresponds to a data element; '· · ·' Identify such data elements as motion whose value exceeds a predetermined threshold value; and: zoom in on an identified movement. 31. Device according to claim 30, characterized in that the a n arrangement is additionally configured to identify the movement in blocks. | -32. Device according to claim 31, characterized in that a movement is identified in the image block if at least one predetermined amount of the data elements corresponding to the block in question in the movement register *** [: exceeds the predetermined threshold value. . 112017 33. Device according to claim 30, characterized in that the device is further configured to reduce the values of the data elements in the movement register which deviate from noil to noil by a predetermined amount in connection with the addition of each difference data. 34. Device according to claim 30, characterized in that the device is further configured to zoom in on the identified movement by interpolating the zoomed area to the size of the original image. The device according to claim 30, characterized in that the device is further configured to encode the zoomed image. 36. Device according to claim 30, characterized in that the device is further configured to store a zoom point in memory, and only allow a predetermined size of the zoom point between two consecutive images. 37. Device according to claim 30, characterized in that the device is further configured to store the zoom ratio in memory, and allow only a predetermined size of the zoom ratio between two consecutive images. 38. Apparatus according to claim 30, characterized in that the sensitivity of the motion detection is controlled by controlling the threshold. 39. Apparatus according to claim 30, characterized in that the device is further configured to form the threshold value for the current image using a quantity which describes the average current. *. luminance pixels are formed and the threshold value is formed. • ·. 25 40. Device according to claim 30, characterized in that the device is additionally configured to move prior luminance pixels · *% opposite the luminance pixels in the current image representing the same image field, move ** · “corresponding data elements in the movement register opposite the same image field, and zero the data elements corresponding to luminance pixels only in the current image in the motion register, if the image field in the current image has moved relative to the preceding image field, that is, the camera that formed the images. moved. The device according to claim 30, characterized in that the device is additionally configured to process the foreground image field til v: 35 to correspond to the current image field by interpolating therein by omitted: ... · luminance pixels corresponding to current image image field, and process the image field of the motion register to correspond to the current image image field by interpolating therein of omitted data elements corresponding to the current image image field, if the image field in the current image has been zoomed optically in relation to the previous image's image field, that is, optical zoom used in the camera that formed the images. 42. Apparatus according to claim 30, characterized in that the device is further configured to control the camera which formed the images to move in a direction of motion identified from the image. 43. Apparatus according to claim 30, characterized in that the device is further configured to control the camera which formed the images to optically zoom in on the motion identified from the image. 44. Device according to claim 30, characterized in that the device is further configured to transmit a zoomed image by using a telecommunication connection. »·» I *