KR102141646B1 - Method and apparatus for detecting moving object from image recorded by unfixed camera - Google Patents

Abstract

A method of detecting a moving object from an image photographed by a non-fixed camera, the method comprising: dividing each of the frames constituting the image into grid regions for applying a background model, and resolution of each of the grid regions By setting the attention mask on the grid areas to determine the step, compensating for the movement of the background model according to the movement of the non-fixed camera, and updating the model parameters for the background model for which the motion is compensated, A method is disclosed that includes selecting a moving object from each of the frames.

Description

Method and apparatus for detecting moving object from image recorded by unfixed camera}

The present disclosure relates to a method and apparatus for detecting a moving object from an image captured by a non-fixed camera. More particularly, the present disclosure relates to a method and apparatus for detecting a moving object from an image captured by a non-fixed camera, having improved detection efficiency as an attention mask is utilized.

Research is being conducted on a technique for detecting a moving object from an image by a camera. When a portion representing an object moving from a background that does not move in the image is distinguishable, various application methods can be derived therefrom. Accordingly, the importance of moving object detection is increasing in fields such as computer vision and image surveillance.

Even in an environment in which the camera photographing the moving object is not fixed, the moving object detection may be required. In the case of a non-fixed camera in which the camera itself moves and photographs the moving object, unlike the fixed camera, not only the target moving object but also the background are moved in the image, so the process of detecting the moving object from the background with movement It can be more complicated.

In order to detect a moving object from the image of a non-fixed camera, a process of grasping the motion of the background and compensating it may be involved. In the process of compensating for the movement of the background, the amount of computation required for detecting a moving object may be greatly increased, and in the case of a non-fixed camera, it may be difficult to detect the moving object in real time from an image taken. Accordingly, in the case of a non-fixed camera, a technique for reducing the computational amount of moving object detection may be required to ensure real-time property of moving object detection.

Registered Patent Publication No. 10-1438451

Various embodiments are to provide a method and apparatus for detecting a moving object from an image photographed by a non-fixed camera. The technical problem to be achieved by the present disclosure is not limited to the technical problems as described above, and other technical problems may be inferred from the following embodiments.

As a means for solving the above technical problem, a method of detecting a moving object from an image photographed by a non-fixed camera according to an aspect of the present disclosure includes a grid for applying a background model to each of the frames constituting the image dividing into (grid) regions; Setting an attention mask on the grid areas to determine the resolution at which each of the grid areas is processed; Compensating for the movement of the background model according to the movement of the non-fixed camera; And selecting the moving object from each of the frames by updating model variables related to the motion compensated background model.

An apparatus for detecting a moving object from an image photographed by a non-fixed camera according to another aspect of the present disclosure includes: a memory storing at least one program; And a processor that detects the moving object by executing the at least one program, wherein the processor divides each of the frames constituting the image into grid areas for applying a background model, and each of the grid areas. In order to determine the resolution to be processed, an attention mask is set in the grid areas, the motion of the background model is compensated for according to the motion of the non-fixed camera, and model parameters related to the motion compensated background model are updated. By doing so, the moving object can be selected from each of the frames.

According to the method and apparatus for detecting a moving object according to the present disclosure, even when an image targeted for detecting the moving object is photographed by a non-fixed camera, resolution may be differently set for each part of the image, which is required for detecting the moving object. The amount of computation can decrease. As the amount of computation decreases, since it may not be delayed to detect the moving object from the captured image, real-time property of moving object detection can be secured even in the case of a non-fixed camera.

1 is a view for explaining a process of detecting a moving object from an image photographed by a non-fixed camera according to some embodiments.
2 is a flowchart illustrating steps configuring a method of detecting a moving object from an image photographed by a non-fixed camera according to some embodiments.
FIG. 3 is a diagram for explaining a method of setting an attention mask according to some embodiments.
4 is a diagram for describing a method of determining a resolution in which each of the grid regions is processed according to some embodiments.
5 is a view for explaining the performance of detecting a moving object from an image photographed by a non-fixed camera according to some embodiments.
6 is a block diagram illustrating elements constituting an apparatus for detecting a moving object from an image photographed by a non-fixed camera according to some embodiments.

Hereinafter, exemplary embodiments will be described in detail with reference to the accompanying drawings. It should be understood that the following description is only for the purpose of embodying the embodiments and does not limit or limit the scope of the invention. From the detailed description and examples, what can be easily inferred by experts in the art is interpreted as belonging to the scope of rights.

As used herein, terms such as'consist of' or'comprises' should not be construed as including all the various components or various steps described in the specification, and some of the components or some steps It may not be included, or it should be construed that additional components or steps may be further included.

As used herein, terms including an ordinal number such as'first' or'second' may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from other components.

The terms used in the present specification have been selected as general terms that are widely used at present considering the functions in the present invention, but this may be changed according to the intention or precedent of a person skilled in the art or the appearance of new technologies. In addition, in certain cases, some terms are arbitrarily selected by the applicant, and in this case, their meanings will be described in detail in the description of the corresponding invention. Therefore, the term used in the present invention should be defined based on the meaning of the term and the contents of the present invention, rather than a simple term name.

The present embodiments relate to a method and apparatus for detecting a moving object from an image photographed by a non-fixed camera, and detailed descriptions of matters well known to those of ordinary skill in the art to which the following embodiments belong Omitted.

1 is a view for explaining a process of detecting a moving object from an image photographed by a non-fixed camera according to some embodiments.

Referring to FIG. 1, a process of detecting a moving object from an image photographed by a non-fixed camera includes an input video 110, a detection loop 120, and a foreground estimation 130. It can be configured as shown.

The image input 110 may refer to a step of receiving an image photographed by a non-fixed camera. The non-fixed camera may mean that not only the moving object to be photographed, but also the camera photographing the moving object has movement. For example, the non-fixed camera may be a black box of a vehicle or the like. Alternatively, the non-fixed camera may be utilized in a robot that simultaneously receives an image of the surrounding environment while moving.

The detection loop 120 may mean a step of processing an image received through the image input 110 process. The image of the non-fixed camera may be a video having a plurality of frames per second, and the process of processing the image of the detection loop 120 may be repeatedly performed for each of the plurality of frames. Each of the plurality of frames may be composed of a specific number of pixels, and the image processing process may be performed based on the values indicated by the pixels.

The detection loop 120 may be composed of a homography estimation 121, an attention mask estimation 122, a background model movement 123, and a background model update 124. However, this is only an example, and other general-purpose processes or steps may be further included on the detection loop 120.

The homography estimation 121 may mean a process of deriving a homography matrix indicating movement of a non-fixed camera. When a homography matrix is derived, it can be used to offset the motion of a non-fixed camera. When the movement of the non-fixed camera is canceled, the background movement may be canceled, and accordingly, only the movement of the actual moving object may be reflected.

The attention mask estimation 122 may refer to a process of setting the attention mask on the input image by the image input 110 process. When the attention mask is set, input images may be separated at different resolutions, and accordingly, a complicated portion may be processed at a high resolution and a simple portion may be processed at a low resolution, so that the computation amount of moving object detection may be greatly reduced. The moving object detection may be performed in real time on the input image by reducing the amount of computation through the attention mask.

The movement 123 of the background model may mean a process of correcting the movement of the background according to the movement of the non-fixed camera. As described above, when a homography matrix is derived, the movement of the camera may be derived, and considering that the background moves in a direction opposite to the movement direction of the camera, the movement of the background may be corrected through the homography matrix. have. When the motion of the background is corrected, only the moving object in the image has motion, so that moving object detection can be performed.

The background model update 124 may refer to a process of updating model parameters of a motion-corrected background model. The background model may mean modeling a moving object based on the background and the background in the input image. When the background model is set, a portion representing the moving object from the background model may be determined through calculation on model variables of the background model. In particular, the operation on the model variables can be performed to different degrees according to the resolution by the attention mask set in the attention mask estimation process 122, and accordingly, the efficiency of the operation can be increased, thereby reducing the overall computation amount. Can.

The detection loop 120 may be performed by repeating the above-described processes every frame. The model variables of the background model may be updated by the pixels constituting the frame every frame, and accordingly it may be determined which of the pixels represents a moving object.

The foreground estimation 130 may mean a process of estimating a foreground from model variables of a background model updated through the detection loop 120. The foreground is contrasted with the background, and may represent a moving object. That is, estimating the foreground from the background may mean a process of detecting a moving object from stationary objects.

Background model (background model) is to model the background to detect a moving object with movement against the background, it can have a variety of model variables. Model variables may be determined by pixels constituting each of the frames of the input image. For example, the value of model variables may be determined by the pixel intensity of each pixel. The pixel intensity may be an indicator of the brightness of the pixel.

The model variables of the background model may include mean, variance and age for the pixel intensity of the pixels. For example, the background model may be a single Gaussian model (SGM) having a mean and variance of a specific value, and the single Gaussian model may have an age indicating the degree of learning. A single Gaussian model can update the values of the mean and variance through learning about the pixels of each frame, and the value of the age can increase according to the learning.

The background model may be a dual-mode single-gaussian model (SGM). More specifically, the background model may be a dual-mode SGM including an parent background model and a canddate background model. When the background model is implemented as a dual-mode SGM by two SGMs, the background is modeled by two SGMs, so that the background can be prevented from being damaged by noise or foreground objects.

The background model can be applied in a grid unit. The grid may mean a unit in which frames constituting the input image are divided into a grid. For example, when the frames of the input image are composed of 320*240 pixels, 16*16 pixels may form one grid, and the input image may be replaced by 20*15 grids. A background model can be applied to each grid formed to a predetermined size.

At a specific frame or at a certain time t, the group of pixels constituting the i-th grid is called G _i ^(t) , and the total number of pixels in the group of pixels is called |G _i ^(t) | When the intensity is I _j ^(t) , the mean μ _i ^(t) , variance σ _i ^(t) and age α _i ^(t) of the SGM background model applied to the i-th grid are expressed as in Equation 1 below. Can be.

,

및

는 각각 t-1에서 배경의 움직임이 보상된 배경 모델의 모델 변수들을 의미할 수 있다. 배경의 움직임 보상에 대한 구체적인 내용은 후술할 도 2의 단계 230을 통해 설명될 수 있다. 한편, M_i ^(t) 및 V_i ^(t)는 아래의 수학식 2와 같이 표현될 수 있다.As the frame or time progresses from t-1 to t by Equation 1, learning of the background model may be performed, and the average, variance, and age may be updated accordingly.

,

And

May denote model variables of a background model in which background motion is compensated at t-1. Details of motion compensation in the background may be described through step 230 of FIG. 2 to be described later. Meanwhile, M _i ^(t) and V _i ^(t) may be expressed as Equation 2 below.

Equation 2 is obtained by calculating the average and variance of pixel intensity in grid units, and M _i ^(t) and V _i ^(t) may refer to model variables of a background model specialized for the grid unit. As in the case of Equation 2, since model variables are applied in grid units, not pixel units, errors in learning of the background model may be prevented by some singular pixels in the grid.

The input image may be modeled by the background model by the model variables of Equation 1 and Equation 2. Specifically, pixels constituting the frames of the input image are divided into grids, and a background model may be applied to each grid. In addition, the background model may be implemented as a dual mode SGM in which a model by model variables of Equation 1 and Equation 2 is provided in duplicate.

2 is a flowchart illustrating steps configuring a method of detecting a moving object from an image photographed by a non-fixed camera according to some embodiments.

Referring to FIG. 2, a method of detecting a moving object from an image photographed by a non-fixed camera may include steps 210 to 240. However, the present invention is not limited thereto, and other general steps other than the steps illustrated in FIG. 2 may be further included in a method of detecting a moving object from an image photographed by a non-fixed camera.

The method of FIG. 2 may be performed by an apparatus for detecting a moving object from an image captured by a non-fixed camera. The apparatus for performing the method of FIG. 2 may be the apparatus 600 of FIG. 6 to be described later.

In step 210, the apparatus 600 may divide each of the frames constituting the image captured by the non-fixed camera into grid regions for applying a background model.

In step 220, the apparatus 600 may set an attention mask on the grid regions to determine the resolution at which each of the grid regions is processed.

In the process of setting the attention mask, the apparatus 600 may merge or divide at least a portion of the grid regions based on variance of pixels included in each of the grid regions.

In the process of setting the attention mask, the apparatus 600, when at least a part of the grid areas are merged, averages the model variables applied to the merged grid area to the model variables of each of the grid areas before being merged. Can be set to

In the process of setting the attention mask, the apparatus 600 may set the model variables of each of the divided grid regions as model variables of the grid region before division, when at least a portion of the grid regions are divided.

In the process of setting the attention mask, the apparatus 600 may set the attention mask according to the row direction or column direction of grid regions arranged in a matrix shape.

More detailed information on the step of setting the attention mask may be described with reference to FIGS. 3 to 4 to be described later.

In step 230, the device 600 may compensate for the movement of the background model according to the movement of the non-fixed camera.

In the process of compensating for the motion, the apparatus 600 mixes the model based on the movement of the grid regions according to the homography matrix and the moving grid regions and the regions where the grid regions before the movement overlap. Can be done. Model mixing can be explained through Equation 3 below.

,

및

는 수학식 1의 모델 업데이트에 활용되는 변수들로서, 각각 t-1에서 배경의 움직임이 보상된 배경 모델의 평균, 분산 및 에이지를 의미할 수 있다. 수학식 3의 가중치 w_k는 아래의 수학식 4를 통해 설명될 수 있다.Referring to Equation 3,

,

And

Is a variable used in the model update of Equation 1, and may mean average, variance, and age of the background model in which the motion of the background is compensated at t-1, respectively. The weight w _k of Equation 3 may be described through Equation 4 below.

Referring to Equation 4, the weight weight w _k may be proportional to a region in which the grid region R _i moved according to the homography matrix and the grid region G _k ^(t) before the movement overlap, and the sum of all weights is 1 Can be That is, a region having many overlapping portions may be largely reflected in a model variable after motion compensation through a high weight, and a region having few overlapping portions may be vice versa. In this way, motion compensation by model mixing may be performed in such a way that overlapping parts are mixed.

In step 240, the device 600 may update the model parameters related to the motion compensated background model to select a moving object from each of the frames.

In the process of selecting a moving object, the apparatus 600 may select pixels representing the moving object based on pixel intensity of the pixels included in each of the grid areas and updated model variables. Specific criteria for selecting a moving object may be described through Equation 5 below.

Referring to Equation 5, for each grid, pixels whose pixel intensity deviates by a predetermined level or more from the average may be regarded as a moving object and may be regarded as a foreground pixel.

More detailed information on steps 210 to 240 of FIG. 2 and equations 1 to 5 may be provided through patent registration number 10-1438451.

FIG. 3 is a diagram for explaining a method of setting an attention mask according to some embodiments.

Referring to FIG. 3, the attention mask 312 for the input image 311, the attention mask 322 for the input image 321, and the attention mask 332 for the input image 331 are illustrated. have.

The attention mask may be set based on the variance of pixels included in each of the grid regions. In the process of setting the attention mask, at least some of the grid regions may be merged or divided. The resolution for processing the input image may be changed by merging or splitting grid regions. The complex portion of the input image may be processed at a high resolution through division of the grid region, and the simple portion may be processed at a low resolution through merging of the grid regions.

Whether to process at high resolution or at low resolution may be determined based on the standard deviation or dispersion of pixels. For example, the standard deviation or variance may be calculated for each of the grids constituting the frame of the input image, and the higher the standard deviation or variance, the higher the resolution and the lower the resolution.

The attention mask may be set along a row direction or a column direction of grid regions arranged in a matrix shape. It can be confirmed that all of the attention mask 312 to the attention mask 332 of FIG. 3 are set along the row direction. Specifically, by calculating the standard deviation or variance for each of the matrix-shaped grids, and adding the standard deviation or variance in the row direction, a column vector having one column can be derived, and the column vector The value of can represent the standard deviation or variance summed in the row direction. Therefore, the attention mask can be set along the row direction based on the value of the column vector.

Referring to the attention mask 312 for the input image 311, the upper portion of the input image 311 is a relatively simple part, and the standard deviation or variance of the grids is small, and accordingly, it is displayed in a dark color indicating low resolution. , The lower end of the input image 311 is a rather complicated part, and the standard deviation or variance of the grids is large, and accordingly, it is displayed in a bright color indicating high resolution. It can be confirmed that the attention mask 322 for the input image 321 and the attention mask 332 for the input image 331 are also set in the same manner along the row direction.

In FIG. 3, the attention mask is illustrated as being set along the row direction, but is not limited thereto, and the attention mask may be set in the column direction according to a corresponding principle, or may be set according to other suitable methods. .

The attention mask is set, and the resolution at which the input image is processed is not collectively set, but may be set to different resolutions according to complexity. Accordingly, a portion in which high resolution processing is unnecessary may be set to a low resolution and processed more quickly, so that the overall computational amount of moving object detection can be significantly reduced.

4 is a diagram for describing a method of determining a resolution in which each of the grid regions is processed according to some embodiments.

Referring to FIG. 4, an example is shown in which a grid area is divided according to the attention mask setting to increase resolution, or grid areas are merged to decrease resolution.

When the standard deviation or variance of the grid area 411 is relatively large, the grid area 411 may be divided into grid areas 412 or grid areas 413, and vice versa, grid areas 412 ) Or the grid regions 413 may be merged into the grid region 411.

When the grid area 411 is divided into four grid areas 412, four independent background models may be applied to the four grid areas 412, respectively. In this case, values of model variables of the background model applied to the grid region 411 before division may be pasted to model variables of the four background models. This method can be applied to the case where the grid area 411 is divided into 16 grid areas 413. After division, the background model applied to the separated grid regions operates independently, so that the values of model variables of each region can be changed.

When the four grid regions 412 are merged into the grid region 411, the average of the model variables of the background models applied to each of the four grid regions 412 is applied to the grid region 411 after merging. Can. For example, as the model variables of the background models applied to the four grid regions 412, σ ₁ ^(t) , σ ₂ ^(t) , σ ₃ ^(t) , and σ ₄ ^(t) are 10 respectively. , 10, 10, 30, the variance may be 15 as a model variable of the background model applied to the grid region 411 after merging. The same method may be applied when the 16 grid regions 413 are merged into the grid region 411.

On the other hand, when the grid region 421 is divided into grid regions 422 or grid regions 423 or merged on the contrary, when pixels of some grid regions are not defined (NaN), undefined The grid regions may be merged and divided with the grid regions excluded.

5 is a view for explaining the performance of detecting a moving object from an image photographed by a non-fixed camera according to some embodiments.

Referring to FIG. 5, a detection result (b) according to the prior art for the input image (a) and a detection result (c) according to the present disclosure are illustrated.

In the performance of detecting a moving object from the input image (a), referring to (b) and (c) of FIG. 5, the performance of the method according to the present disclosure may have the same level of performance as that of the prior art. On the other hand, the detection result (c) according to the present disclosure has a speed improvement of 6 fps (frame per second) compared to the detection result (b) according to the prior art, and a speed improvement of about 2 times compared to the prior art can be confirmed.

A method of detecting a moving object from an image photographed by a non-fixed camera according to the present disclosure may set a attention mask to vary the resolution for each grid area, and set the resolution of a simple part to a low level, which is necessary for detecting the moving object. The amount of computation to be performed can be greatly reduced. The number of frames per second that can be processed in real time may increase as the amount of computation decreases, and accordingly, real-time property of moving object detection may be secured.

6 is a block diagram illustrating elements constituting an apparatus for detecting a moving object from an image photographed by a non-fixed camera according to some embodiments.

Referring to FIG. 6, an apparatus 600 for detecting a moving object from an image photographed by a non-fixed camera may include a memory 610 and a processor 620. However, the present invention is not limited thereto, and other general-purpose elements may be further included in the device 600 in addition to the elements illustrated in FIG. 6.

The steps constituting the method of detecting the moving object from the image photographed by the non-fixed camera described with reference to FIGS. 1 to 5 may be processed in time series in the apparatus 600 of FIG. 6. Accordingly, even if omitted for the device 600 of FIG. 6, the content described above with respect to the method of FIGS. 1 to 5 may also be applied to the device 600 of FIG. 6.

The memory 610 may store data related to model variables of an input image or a background model as data processed by the device 600 and data to be processed. The memory 610 may store at least one program. At least one program stored in the memory 610 may be executed by the processor 620.

The memory 610 includes dynamic random access memory (DRAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), CD-ROM, and Blu-ray ) Or other optical disk storage, hard disk drive (HDD), solid state drive (SSD), or flash memory, but is not limited thereto.

The processor 620 may be implemented as an array of multiple logic gates, or may be implemented as a combination of a general-purpose microprocessor and a memory in which programs that can be executed in the microprocessor are stored. Also, the processor 620 may be composed of a plurality of processing elements.

The processor 620 may perform steps configuring a method of detecting a moving object from an image photographed by a non-fixed camera by executing at least one program stored in the memory 610.

The processor 620 may divide each of the frames constituting the image captured by the non-fixed camera into grid regions for applying a background model.

The processor 620 may set an attention mask on the grid regions to determine the resolution at which each of the grid regions is processed.

The processor 620 may compensate for the movement of the background model according to the movement of the non-fixed camera.

The processor 620 may select a moving object from each of the frames by updating model variables related to a motion compensated background model.

Meanwhile, a method of detecting a moving object from an image photographed by a non-fixed camera according to the present disclosure may be recorded in a computer-readable recording medium in which one or more programs including instructions for executing the method are recorded.

Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs, DVDs, and floptical disks. Hardware devices specifically configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like may be included. Examples of program instructions may include high-level language code that can be executed by a computer using an interpreter, etc., as well as machine language codes made by a compiler.

Although the embodiments have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements of those skilled in the art using the basic concept of the present invention defined in the following claims are also provided. Belongs to

600: a device for detecting a moving object from an image captured by a non-fixed camera
610: memory
620: processor

Claims (11)

A method for detecting a moving object from an image photographed by a non-fixed camera,
Dividing each of the frames constituting the image into grid regions for applying a background model;
Setting an attention mask on the grid areas to determine the resolution at which each of the grid areas is processed;
Compensating for the movement of the background model according to the movement of the non-fixed camera; And
And selecting the moving object from each of the frames by updating model variables related to the motion compensated background model,
The step of setting the attention mask,
And merging or dividing at least a portion of the grid regions based on variance of pixels included in each of the grid regions. delete According to claim 1,
The step of setting the attention mask,
And when the at least a part is merged, setting the model variables applied to the merged grid area as an average for the model variables of each of the grid areas before being merged. According to claim 1,
The step of setting the attention mask,
And if the at least a portion is divided, setting the model variables of each of the divided grid regions to the model variables of the grid region before division. According to claim 1,
The step of setting the attention mask,
And setting the attention mask along a row direction or a column direction of the grid regions arranged in a matrix shape. According to claim 1,
Compensating the movement,
And performing model mixing based on movement of the grid regions according to a homography matrix and regions in which the moved grid regions and the grid regions before the movement overlap. Way. According to claim 1,
The step of selecting the moving object,
And selecting pixels representing the moving object based on pixel intensity of the pixels included in each of the grid regions and the updated model variables. According to claim 1,
The model variables include a mean, variance and age for the pixel intensity of pixels included in each of the grid regions. According to claim 1,
The background model is implemented as a dual-mode dual-mode single Gaussian model (SGM) model comprising an aparent background model and a canddate background model. A computer-readable recording medium in which a program for implementing a method for detecting a moving object from an image photographed by a non-fixed camera is recorded,
The above method,
Dividing each of the frames constituting the image into grid regions for applying a background model;
Setting an attention mask on the grid areas to determine the resolution at which each of the grid areas is processed;
Compensating for the movement of the background model according to the movement of the non-fixed camera; And
And selecting the moving object from each of the frames by updating model variables related to the motion compensated background model,
The step of setting the attention mask,
And merging or dividing at least a portion of the grid areas based on a variance of pixels included in each of the grid areas. In the apparatus for detecting a moving object from the image taken by the non-fixed camera,
A memory storing at least one program; And
And a processor for detecting the moving object by executing the at least one program,
The processor,
Each of the frames constituting the image is divided into grid regions for applying a background model,
Attention masks are set on the grid regions to determine the resolution at which each of the grid regions is processed,
Compensate the movement of the background model according to the movement of the non-fixed camera,
The moving object is selected from each of the frames by updating model parameters related to the motion compensated background model,
The processor
And setting the attention mask by merging or dividing at least a portion of the grid regions based on a variance of pixels included in each of the grid regions.

Images