JP6266946B2 - Surveillance camera and image monitoring device - Google Patents

Description

  The present invention relates to a monitoring camera that captures a monitoring area, and an image monitoring apparatus that recognizes the state of the monitoring area using an image captured by the monitoring camera.

  In recent years, the number of surveillance cameras installed has increased due to the increased awareness of crime prevention, and research on recognizing objects such as intruders from images taken by surveillance cameras has been actively conducted.

  In the image recognition, there is a problem that the recognition accuracy is lowered when a region similar in color to the object is in contact with the object to be recognized on the image. For example, the recognition accuracy decreases for an image in which a person wearing a white shirt stands in front of a white wall and an image in which a plurality of persons wearing white shirts are superimposed. The cause of this is a decrease in edge strength at the contour of the object.

  On the other hand, the resolution of surveillance cameras is increasing. If a high-resolution monitoring image is used, a reduction in edge strength in the contour of the object is suppressed, so that an improvement in recognition accuracy can be expected for an object that is in contact with a region having a similar color. However, since a high-resolution monitoring image has a large amount of data, if it is used with a high resolution, the processing amount and communication amount increase, resulting in an increase in cost of the apparatus.

  A monitoring system that reduces the amount of data on the monitoring camera side has been proposed from the viewpoint of reducing communication volume and storage capacity. For example, the monitoring system described in Patent Document 1 detects a motion area or a person area as a feature area from a low-resolution image read out by thinning out a light-receiving element of a monitoring camera that has captured the monitoring area, and reads the feature area from the light-receiving element. Are transmitted together with the low-resolution image of the monitoring area.

JP2007-243660

  However, in the prior art, since the entire motion area or person area is transmitted as a high resolution image from the monitoring camera side, the amount of data is not sufficiently reduced, and there is room for reducing the cost of the image monitoring apparatus. For example, high-resolution image recognition is required for high-accuracy image recognition in areas where similar-colored areas are in contact with each other, but high-precision image recognition is possible even in low-resolution areas. Therefore, there is room for reducing the amount of data accordingly.

  In the prior art, since both the high resolution image and the low resolution image are transmitted for the feature region, the amount of data is not sufficiently reduced, and there is room for reducing the cost of the image monitoring apparatus.

  In addition, in the prior art, since a feature region is detected by analyzing a low-resolution image, there is a high possibility that a part or all of an object having a color similar to the background may be missed and recognition accuracy may be reduced. It was.

  Further, in the prior art, when a feature region is detected, both a high resolution image and a low resolution image are transmitted, and when a feature region is not detected, only a low resolution image is transmitted. As described above, when the number of transmission frames varies depending on the presence / absence of a feature region, the design of the image recognition apparatus that receives and processes these frames becomes complicated, which causes a high cost of the image monitoring apparatus.

  The present invention has been made in view of the above problems, and provides a monitoring camera that can output an image of a resolution useful for recognition processing of an object with a small amount of data regardless of the object existing in the monitoring area and the color of the background. It is an object of the present invention to provide an image monitoring apparatus using the monitoring camera.

  In order to solve the problem, the present invention provides a photographing unit that photographs a monitoring region and generates a first-resolution photographed image, and changes in luminance between pixels constituting each block by dividing the photographed image into a plurality of blocks An edge level estimator that calculates the amount and compares the amount of change in brightness with one or more predetermined thresholds and divides the image into a plurality of stages so that the higher the strength of the edge included in each block, A resolution conversion unit that generates a recognition image by replacing at least a block having the lowest level with a second resolution image lower than the first resolution, and an image output unit that outputs the recognition image to the outside. Provided is a surveillance camera characterized by comprising the above.

Also, in a preferred aspect of the present invention, the edge level estimation unit compares the luminance change amount with two or more threshold values having different values, and divides the intensity of the edge included in each block into three or more stages, The conversion unit further generates a recognition image by replacing the block having the highest level in the captured image with an image having a third resolution lower than the first resolution.
The third resolution is preferably the same resolution as the second resolution or higher than the second resolution.

  In a preferred aspect of the present invention, the edge level estimation unit compares the luminance change amount with three or more thresholds having different values, and divides the intensity of the edge included in each block into four or more stages, The conversion unit further generates a recognition image by replacing a block that is an intermediate stage between the lowest stage and the highest stage in the captured image with a fourth resolution image that is set lower as the stage is higher.

  Further, the present invention is an image monitoring apparatus comprising the monitoring camera, an image communication network that is connected to the image output unit and transmits an image for recognition, and an image recognition apparatus. A communication unit connected to the image communication network for receiving a recognition image, a feature amount extraction unit for extracting an edge feature amount from the recognition image, and an edge feature amount extracted by the feature amount extraction unit An image monitoring apparatus comprising: a recognition unit for recognizing a state.

  According to the monitoring camera or the image monitoring apparatus of the present invention, it is possible to achieve both high-accuracy image recognition of an object and reduction in apparatus cost regardless of the object existing in the monitoring area and the color of the background.

It is a figure which shows the structural example of the image monitoring apparatus 1 to which this invention is applied. It is a figure which shows an example of the relationship between an edge level, a luminance variation, and resolution. It is an example of the flowchart which shows the operation | movement of the surveillance camera 2-1. It is a schematic diagram which shows an example of the resolution conversion process. It is a schematic diagram which shows an example of a picked-up image. It is a schematic diagram which shows an example of the high resolution area | region and low resolution area | region in the image for recognition. 6 is an example of a flowchart showing an operation of the image recognition device 4. It is a figure which shows another example about the relationship between an edge level, luminance variation | change_quantity, and resolution. It is a figure which shows another example about the relationship between an edge level, luminance variation | change_quantity, and resolution.

<Configuration of image monitoring device>
With reference to FIG. 1, the structure of the image monitoring apparatus 1 which is an example of suitable embodiment of this invention is demonstrated.

  The image monitoring apparatus 1 includes a plurality of monitoring cameras 2-1, 2-2,..., 2 -N connected to an image recognition apparatus 4 via an image communication network 3.

  Each of the monitoring cameras 2-1, 2-2,..., 2-N is installed in a monitoring area where detection of a suspicious person is requested, and the monitoring area is imaged at a preset imaging cycle. Output to the image communication network 3. The images output by the surveillance cameras 2-1, 2-2,. The image is hereinafter referred to as a recognition image.

  The image communication network 3 is a network that transmits the recognition images output from the respective monitoring cameras 2-1, 2-2,..., 2 -N to the image recognition device 4. The image communication network 3 includes, for example, a LAN (Local Area Network) in which each of the monitoring cameras 2-1, 2-2,..., 2-N and the image recognition device 4 are connected by an Ethernet cable (Ethernet is a registered trademark). can do. Alternatively, the image communication network 3 may be an analog network in which each monitoring camera 2-1, 2-2,..., 2 -N and the image recognition device 4 are connected by a coaxial cable.

  When the image recognition device 4 receives recognition images from the respective monitoring cameras 2-1, 2-2,..., 2-N, the image recognition apparatus 4 performs image processing on the images and recognizes persons existing in the monitoring area. Further, the image recognition device 4 determines whether or not the person is a suspicious person, and outputs a predetermined abnormality signal to the monitoring center when it is determined that there is a suspicious person.

[Configuration of surveillance camera]
Hereinafter, the configuration of the monitoring camera 2-1 will be described.

  The surveillance camera 2-1 includes a high-resolution imaging unit 20, a captured image storage unit 21, an edge level estimation unit 22, a resolution conversion unit 23, a recognition image storage unit 24, and an image output unit 25.

  The high-resolution image capturing unit 20 captures the monitoring area at a preset image capturing period, and generates a first resolution captured image. For this purpose, the high resolution photographing unit 20 forms an image of a monitoring area on the photoelectric conversion element array such as a CCD sensor or a C-MOS that outputs an electrical signal corresponding to the amount of received light. An imaging optical system and an A / D conversion circuit that converts an analog electric signal output from the photoelectric conversion element array into a digital photographed image are included.

  The captured image generated by the high-resolution imaging unit 20 has a higher resolution and a larger amount of data than the recognition image. The first resolution is, for example, horizontal 8000 × vertical 4500 pixels. In this example, it is assumed that the captured image is a grayscale image.

  The high-resolution image capturing unit 20 is connected to the captured image storage unit 21 and sequentially stores the generated captured images in the captured image storage unit 21.

  The captured image storage unit 21 is a memory device that stores captured images and the like, and includes a RAM (Random Access Memory) or the like. The captured image storage unit 21 is connected to the high-resolution imaging unit 20 and the edge level estimation unit 22, and the captured image written by the high-resolution imaging unit 20 is sequentially read out by the edge level estimation unit 22. The photographed image storage unit 21 has a storage capacity for two photographed images so that the current frame can be written and the previous frame can be read simultaneously.

  The edge level estimator 22 is an arithmetic unit that divides a captured image into a plurality of blocks and divides the intensity of the edge included in each block into a plurality of stages, and is a DSP (Digital Signal Processor) or MCU (Micro Control Unit) or the like. Composed. This stage (edge level) uses, for example, two threshold values, the highest stage level in which a strong edge is included in the block, the intermediate stage level in which a weak edge is included in the block, and the strong edge in the block is also weak. It is possible to have three levels at the lowest level without edges.

  The edge level estimation unit 22 is connected to the captured image storage unit 21 and the resolution conversion unit 23, reads the captured image from the captured image storage unit 21, estimates the edge level, and reads the captured image (luminance value of each pixel) and The edge level of each block is output to the resolution converter 23.

  The size of each block can be, for example, 12 × 12 pixels, and the photographed image is divided into 666 horizontal blocks × 375 vertical blocks.

  The edge level estimation unit 22 calculates a luminance change amount between pixels constituting each block, compares the luminance change amount with one or more predetermined threshold values, and sets the intensity of an edge included in each block in a plurality of stages. Divide. Specifically, the edge level estimation unit 22 calculates the average value E1 of the edge intensity of the captured image in each block as the luminance change amount of the block, and if E1 is less than the lower limit value T1L, the block is in the lowest stage. If the edge level E1 is equal to or higher than the lower limit value T1L and lower than the upper limit value T1H, the block is determined to be an intermediate edge level, and if E1 is equal to or higher than the upper limit value T1H, the block is determined to be the highest edge level.

  FIG. 2 shows the relationship between the edge level and the luminance change amount described above. The relationship between the edge level and the block estimation state will be described below with reference to FIG.

  The lower limit value T1L can be set in advance as a threshold value for determining the presence / absence of an edge based on a previous experimental result or the like. That is, it is estimated that the block with the lowest edge level is a flat region without even weak edges. Therefore, even if the resolution of this area is reduced, the accuracy of image recognition is not easily lowered, and the data amount of the recognition image can be reduced by reducing the resolution of this area.

  The upper limit value T1H (> T1L) can be set in advance as a threshold value for discriminating between weak edges and strong edges based on previous experimental results. That is, the block having the highest edge level is estimated to be a region where a strong edge exists. Therefore, since the edge of the recognition target object can be extracted even if the resolution of this area is reduced, the accuracy of image recognition is not easily lowered, and the data amount of the recognition image can be reduced by reducing the resolution of this area.

  On the other hand, a block with an intermediate edge level is estimated to be a region where weak edges exist, and if the resolution of this region is reduced, the edge of the recognition target object cannot be extracted and the accuracy of image recognition may be reduced. . Therefore, the accuracy of image recognition can be expected to be improved by setting this area to a high resolution.

  The resolution conversion unit 23 is an arithmetic device that generates a recognition image by replacing a part of a captured image with a low-resolution image, and includes a DSP, an MCU, or the like.

  The resolution conversion unit 23 is connected to the edge level estimation unit 22 and the recognition image storage unit 24. When a captured image and a weak edge region are input from the edge level estimation unit 22, the resolution conversion unit 23 generates a recognition image, and The image is stored in the recognition image storage unit 24.

  With reference to FIG. 2 again, an example of recognition image generation processing will be described.

  The resolution converter 23 replaces at least the block having the lowest level 1 in the captured image with an image having the second resolution R2 lower than the first resolution R1 that is the resolution of the captured image. For example, the resolution conversion unit 23 generates a recognition image by replacing the level 1 block with one pixel in which the luminance value is set to the representative luminance value of the block. The representative luminance value can be an average luminance value in the block in the captured image. Alternatively, the representative luminance value may be the most frequent luminance value in the block in the captured image, or may be the luminance value of any one pixel in the block in the block in the captured image.

  In addition, the resolution conversion unit 23 replaces at least the block having the highest level 3 in the captured image with the image of the third resolution R3 lower than the first resolution R1. For example, the resolution conversion unit 23 generates a recognition image by replacing the level 3 block with one pixel in which the luminance value is set to the representative luminance value of the block. Since the luminance change in the block is larger at level 3 than at level 1, it is preferable that R3 ≧ R2 from the viewpoint of improving image recognition accuracy. In this example, R3 = R2, assuming that R2 has sufficient resolution to analyze the strong edge of the recognition target object.

On the other hand, the resolution (fourth resolution R4) of the block whose edge level is the intermediate level 2 remains the first resolution R1, and the resolution conversion unit 23 recognizes the luminance value of 12 × 12 pixels in the captured image as it is. The luminance value of the corresponding pixel in the image for use is set.
The fourth resolution R4 may be set lower than the first resolution R1 in a range higher than the third resolution R3.

  To summarize the above-described resolution relationships, R1 ≧ R4> R3 ≧ R2.

  The recognition image storage unit 24 is a memory device that stores a recognition image and the like, and includes a RAM or the like. The recognition image storage unit 24 is connected to the resolution conversion unit 23 and the image output unit 25, and the recognition image written by the resolution conversion unit 23 is sequentially read out by the image output unit 25. Note that the recognition image storage unit 24 has a storage capacity for two recognition images so that writing of the current frame and reading of the previous frame can be performed simultaneously.

  The image output unit 25 is a communication interface circuit that outputs a recognition image from the camera device 2-1 to the outside. The image output unit 25 is connected to the recognition image storage unit 24 and the image communication network 3, reads the recognition image from the recognition image storage unit 24, and matches the read recognition image to the communication protocol of the image communication network 3. It is converted into a signal addressed to the image recognition apparatus and sent to the image communication network 3.

  Note that the edge level estimation unit 22 and the resolution conversion unit 23 may be realized by a single arithmetic device. Further, the captured image storage unit 21 and the recognition image storage unit 24 can be realized by a single memory device.

  The configuration of the monitoring cameras 2-2,..., 2-N is the same as that of the monitoring camera 2-1, and thus the description thereof is omitted.

[Configuration of image recognition device]
Hereinafter, the configuration of the image recognition device 4 will be described.

  The image recognition device 4 includes an image input unit 40, a feature extraction unit 41, a suspicious person recognition unit (recognition unit) 42, and an abnormality output unit 43.

  The image input unit 40 is a communication circuit that receives a recognition image. When the image input unit 40 is connected to the image communication network 3 and the feature extraction unit 41 and receives recognition images from the monitoring cameras 2-1, 2-2,. The recognition image is output to the feature extraction unit 41. The data amount of the authentication image is much smaller than the captured image, and the communication load of the image input unit 40 is reduced.

  The feature extraction unit 41 is a calculation device that extracts a predetermined feature amount useful for image recognition of a suspicious person from an image for recognition, and includes a CPU (Central Processing Unit), a DSP, an MCU, or the like. The feature extraction unit 41 is connected to the image input unit 40 and the suspicious person recognition unit 42. When a recognition image is input from the image input unit 40, the feature extraction unit 41 extracts a predetermined feature amount and outputs it to the suspicious person recognition unit 42.

  The feature quantity extracted by the feature extraction unit 41 is edge strength, edge direction, HOG (Histograms of Oriented Gradients) feature quantity, edge feature quantity such as Haar-like feature quantity, luminance histogram, and the like.

  As described above, the recognition image includes a high resolution area and a low resolution area. The feature extraction unit 41 divides the recognition image for each resolution, and extracts feature amounts for each of the divided images. As described with reference to FIG. 4, the recognition image has a data configuration in which data indicating the number of pixels of each block and luminance value data corresponding to the number of pixels are arranged in the block order. Determines the resolution of each block from the number of pixels, and arranges the luminance values indicated by the number of pixels on the coordinates corresponding to each block to generate a divided image for each resolution.

  In this way, the feature extraction unit 41 extracts the feature quantity of the recognition target object existing in the monitoring area for each resolution, so that not only the strong edge feature quantity but also the areas with similar colors are in contact with each other and become weak. The feature quantity of the recognition target object including the edge feature quantity can be reliably extracted. Further, since the extraction processing from the high-resolution area with a large processing load is only necessary for ensuring the image recognition accuracy, an increase in the processing load of the feature extraction unit 41 is appropriately suppressed.

  The suspicious person recognizing unit 42 is an arithmetic device that recognizes the presence of a suspicious person using the feature amount extracted by the feature extracting unit 41, and includes a CPU, DSP, MCU, or the like. The suspicious person recognizing unit 42 is connected to the feature extracting unit 41 and the abnormality output unit 43. When a feature amount is input from the feature extracting unit 41, the suspicious person recognizing unit 42 determines whether there is a suspicious person and determines that there is a suspicious person. If this happens, a predetermined signal indicating that is output to the abnormality output unit 43. Note that the feature extraction unit 41 and the suspicious person recognition unit 42 may be realized by one arithmetic device.

  The suspicious person recognizing unit 42 detects a person by, for example, pattern matching processing that compares the edge strength and edge direction extracted from the recognition image with a preset shape pattern of the person, and performs a front-rear luminance histogram of the detected person. It is possible to analyze the movement of the person by the tracking process that compares in the frames to be performed, and to determine the person whose movement is abnormal as a suspicious person. Alternatively, the suspicious person recognition unit 42 detects a person by inputting the HOG feature amount extracted from the recognition image to a classifier that has previously learned the person's HOG feature amount, and performs a motion analysis of the person by tracking processing. You can also go to determine the presence of a suspicious person.

  In this manner, the suspicious person recognition unit (recognition unit) 42 recognizes an object existing in the monitoring region using the feature amount extracted for each resolution, and thus the colors are similar between the objects and between the object and the background. It is possible to recognize the object to be recognized with high accuracy by reducing the influence of the decrease in edge strength.

  The abnormality output unit 43 is a communication circuit that outputs an abnormality signal to the outside. The abnormality output unit 43 is connected to the suspicious person recognizing unit 42 and the center apparatus. When a predetermined signal indicating the presence of the suspicious person is input from the suspicious person recognizing unit 42, the abnormal output unit 43 transmits a predetermined abnormal signal to the center apparatus.

<Operation of image monitoring device>
Hereinafter, the operation of the image monitoring apparatus 1 will be described.

[Operation of surveillance camera]
First, the operation of the monitoring camera 2-1 will be described with reference to the flowchart of FIG. 3 and the schematic diagram of FIG.

  In the monitoring camera 2-1, the operations in steps S <b> 100 to S <b> 108 described below in FIG. 3 are repeatedly performed in the imaging cycle of the high resolution imaging unit 20.

  In step S100, the high-resolution imaging unit 20 of the monitoring camera 2-1 images the monitoring area, and stores the captured image of the first resolution R1 in the captured image storage unit 21 of the monitoring camera 2-1.

  When a new captured image is captured, the edge level estimation unit 22 of the monitoring camera 2-1 sets the first block among the above-mentioned 666 × 375 blocks as a target block in step S 101, and Start loop processing. Thereafter, the loop processing of steps S102 to S107 is performed on all blocks by sequentially changing the block of interest in a predetermined order.

  In step S102, the edge level estimation unit 22 calculates an average value E1 of the edge strength of the captured image in the block of interest.

  For this purpose, the edge level estimation unit 22 reads out 12 × 12 (= 144) pixels corresponding to the block of interest and the surrounding pixels in the captured image stored in the captured image storage unit 21, and determines each pixel in the block of interest. The edge strength is calculated and the average value thereof is obtained.

  The edge strength can be calculated using various known filters such as a Sobel filter, a Laplacian filter, and a Canny filter. In addition, the filter size for calculating the edge strength can be appropriately set according to conditions such as the camera installation height, the distance to the recognition target object, and the resolution of the captured image.

  In step S103, the edge level estimation unit 22 compares the calculated average value E1 of the edge strength with a predetermined lower limit value T1L and an upper limit value T1H to determine the edge level of the block of interest.

  That is, the edge level estimation unit 22 determines that the block is at level 2 including a weak edge if the average value E1 of the edge strength of the block of interest is greater than or equal to the lower limit value T1L and less than the upper limit value T1H (in S103). YES), otherwise, it is determined that the block is level 1 having no edge or level 3 including a strong edge (NO in S103).

  The edge level estimation unit 22 outputs the coordinates of the target block and the luminance values of the captured image constituting the target block together with the edge level of the target block to the resolution conversion unit 23 of the monitoring camera 2-1.

  If the target block is determined to be level 2, the process proceeds to step S104. In step S104, the resolution conversion unit 23 sets each luminance value of the captured image in the block corresponding to the target block in the recognition image, thereby keeping the resolution R4 of the block in the recognition image as the resolution R1. Maintain high resolution. For example, in FIG. 4, when the edge level of the block 50 of the captured image is determined to be level 2, the corresponding block 51 of the recognition image remains 144 pixels. The resolution conversion unit 23 causes the recognition image storage unit 24 to store a numerical sequence 52 in which 144 luminance values from the upper left pixel to the lower right pixel are arranged in order following the number of pixels in the block 51.

  On the other hand, if the target block is determined to be level 1 or 3, the process proceeds to step S105. In step S105, the resolution conversion unit 23 sets the representative luminance value of the captured image in the corresponding block in the recognition image, and reduces the resolution of the block in the recognition image to resolutions R2 and R3 (R2 = R3). Image. For example, in FIG. 4, when the edge level of the block 53 of the photographed image is determined to be level 1 or 3, the corresponding block 54 of the recognition image is one pixel. The resolution conversion unit 23 causes the recognition image storage unit 24 to store a numerical sequence 55 in which the representative luminance values of the block 53 are arranged following the number of pixels of the block 54.

  When the resolution conversion of the block of interest is performed, the resolution conversion unit 23 confirms whether or not the processing of all the blocks has been completed in S106. If the processing of all the blocks has not been completed, the resolution conversion unit 23 notifies the edge level estimation unit 22 (NO in S106), and the edge level estimation unit 22 that has received this notifies the next block in S107. The target block is set, and the process returns to step S102 to perform the process for the next block.

  On the other hand, when the processing of all the blocks is completed, the resolution conversion unit 23 notifies the image output unit 25 (YES in S106), and the image output unit 25 that has received this notifies the image storage unit for recognition in step S108. The image for recognition is read from 24, and the read image for recognition is output to the image recognition device 4 via the image communication network 3.

  The image output unit 25 that has output the recognition image notifies the edge level estimator 22 to that effect, and the edge level estimator 22 that has received the notification returns the process to step S100 and performs the process on the captured image of the next frame. . Note that the output processing of the image for recognition by the image output unit 25 may be performed in parallel with the processing of the edge level estimation unit 22 and the resolution conversion unit 23, and in this case, the edge level at the time when the image output unit 25 starts reading. What is necessary is just to notify the estimation part 22 of an output.

  With reference to FIG.5 and FIG.6, a picked-up image and the image for recognition are illustrated.

  A photographed image 60 in FIG. 5 is a photograph of a monitoring area where a person wearing a white shirt 62 and black pants stands in front of a white wall 61.

  FIG. 6 shows the high resolution areas 71, 72, 73 and the low resolution area 74 of the recognition image 70 output from the monitoring camera 2-1 for the captured image 60 of FIG.

  In the recognition image 70, the vicinity of the boundary between the wall 61 and the shirt 62, the vicinity of the structure line in the shirt 62, and the vicinity of the structure line in the trousers are determined as the intermediate level edge level 2 and become the high resolution regions 71, 72, 73. . On the other hand, an area where the color is greatly different and a strong edge is generated, such as the boundary between the wall 61 and the window, the boundary between the wall 61 and the pants, and the boundary between the wall 61 and the skin, is determined to be the highest edge level 3, and the area without the edge is the lowest. The edge level is determined as 1 and these areas become the low resolution area 74.

  In the high resolution areas 71, 72, and 73, color mixing is unlikely to occur at the boundary pixels of the object (for example, the boundary pixels between the wall 61 and the shirt 62 are less likely to be an intermediate color between the white of the wall 61 and the white of the shirt 62). In the image recognition device 4, even an outline of an object that is in contact with a region similar in color can be recognized, and erroneous recognition can be prevented.

  On the other hand, since the edge of the low resolution region 74 is originally strong (thick), the outline of the object can be recognized even if color mixing occurs in the boundary pixel of the object, or the image is recognized by reducing the resolution because there is no edge originally. The communication amount in the image monitoring device 1 and the processing amount in the image recognition device 4 can be reduced without reducing the recognition accuracy in the device 4.

  Since the operations of the monitoring cameras 2-2,..., 2-N are the same as those of the monitoring camera 2-1, description thereof is omitted.

[Operation of Image Recognition Device]
Next, the operation of the image recognition apparatus 4 will be described with reference to the flowchart of FIG.

  In the image recognition apparatus 4, the operations in steps S200 to S207 in FIG. 7 described below are executed every time a recognition image is received.

  In step S <b> 200, the image input unit 40 of the image recognition apparatus 4 receives the recognition image via the image communication network 3. When an image for recognition is received from any one of the monitoring cameras 2-1, 2-2,..., 2-N, the image input unit 40 outputs the image to the feature extraction unit 41 of the image recognition device 4, and the processing is step. The process proceeds to S201.

  In step S201, the feature extraction unit 41 divides the input recognition image by resolution to generate a divided image. For example, when the recognition image shown in FIG. 6 is input, it is effective only for the divided image having the resolution R4 (= R1) having the luminance value effective only in the high resolution areas 71, 72, and 73 and only in the low resolution area 74. A divided image having a resolution R2 (= R3) having a large luminance value is generated.

  In step S <b> 202, the feature extraction unit 41 extracts an edge feature amount from each divided image, and outputs the extracted edge feature amount to the suspicious person recognition unit 42 of the image recognition device 4.

  In step S203, the suspicious person recognition unit 42 performs person detection processing using the extracted edge feature amount. That is, the suspicious person recognizing unit 42 first generates a composite edge image by unifying edge feature amounts extracted from divided images having different resolutions to any one resolution. For example, if the recognition image shown in FIG. 6 is a processing target, it can be unified to the lower resolution R2. In this case, the suspicious person recognizing unit 42 obtains an average value of the edge feature amount for each block in the high resolution regions 71, 72, and 73, and sets the average value as the value of the corresponding pixel in the divided image of the resolution R2. A synthesized edge image may be generated. Next, the suspicious person recognizing unit 42 compares the edge feature amount of the synthesized edge image with the human shape pattern, or inputs the edge feature amount into the discriminator that has learned the human feature, thereby obtaining the human feature. It is determined whether the edge feature value to be represented is included.

  When the human edge feature amount is not detected (NO in step S204), the suspicious person recognition unit 42 returns the process to step S200, and the image recognition apparatus 4 waits until a new recognition image is received. On the other hand, if a human edge feature amount is detected (YES in step S204), the process proceeds to step S205.

  In step S205, the suspicious person recognizing unit 42 analyzes the detected movement of the person. That is, the suspicious person recognizing unit 42 causes the feature extracting unit 41 to extract a luminance histogram in a region where a person is detected, and compares the luminance histogram with a luminance histogram of a person previously detected from a recognition image having the same transmission source. To track the position of the person. Further, for tracking, the suspicious person recognition unit 42 stores the detected luminance histogram of the person in the storage unit (not shown) together with the information indicating the transmission source and the position history of the person.

  In step S206, the suspicious person recognizing unit 42 determines whether or not there is a suspicious person from the position history of the person obtained by analyzing the movement. For example, the suspicious person recognizing unit 42 determines that the person is a suspicious person when any of the position histories of the person represents stay for a long time, high speed movement, or the like. When it is determined that there is no suspicious person (NO in step S206), the suspicious person recognizing unit 42 returns the process to step S200, and the image recognition apparatus 4 waits until a new recognition image is received. On the other hand, if it is determined that there is a suspicious person (YES in step S206), the process proceeds to step S207.

  In step S207, the fact that the suspicious person recognition unit 42 exists is output to the abnormality output unit 43 of the image recognition device 4, and the abnormality output unit 43 that has received this generates an abnormality signal and sends the abnormality signal to the center device. Send.

  When the above process is completed, the suspicious person recognition unit 42 returns the process to step S200, and the image recognition apparatus 4 stands by until a new recognition image is received.

<Modification of luminance change amount>
In the above embodiment, the edge level estimation unit 22 calculates the average value E1 of the edge strength of each block as the luminance change amount, and compares these with the lower limit value T1L and the upper limit value T1H to estimate the edge level. In another embodiment, the edge level estimator 22 may use the following indicators as the luminance change amount.

(1) Luminance variance value The edge level estimation unit 22 calculates the luminance variance value E2 of the captured image in each block as the luminance change amount. The edge level estimator 22 sets the block having the luminance variance value E2 less than the lower limit value T2L to the lowest level 1, the block having the luminance variance value E2 greater than or equal to the lower limit value T2L and less than the upper limit value T2H to the intermediate level 2. A block whose value E2 is equal to or greater than the upper limit value T2H is determined to be the highest level 3.
Since the luminance dispersion value can be calculated by accumulating the difference between the luminance values of adjacent pixels, the processing load is small and suitable for sequential processing. Therefore, if the luminance change amount is used, the processing of the edge level estimation unit 22 can be sequentially performed, and the memory amount of the captured image storage unit 20 can be reduced.

(2) High-frequency component amount by FFT The edge level estimation unit 22 calculates the high-frequency component amount as a luminance change amount by performing FFT (Fast Fourier Transform) analysis on the luminance value of the captured image in each block. In addition, when using FFT, it is suitable to make the number of pixels of a block into the power of 2. For example, a block size can be made into 16x16 pixels. That is, the edge level estimation unit 22 performs FFT analysis on the data in which the luminance values of 256 pixels of the captured image in each block are arranged, and calculates the high-frequency maximum intensity value E3 (high frequency component amount). The edge level estimator 22 sets the block having the high frequency component amount E3 less than the lower limit value T3L to the lowest level 1, the block having the high frequency component amount E3 equal to or higher than the lower limit value T3L and less than the upper limit value T3H to the intermediate level 2. A block in which the component amount E3 is equal to or higher than the upper limit value T2H is determined as the highest level 3.

(3) High-frequency component amount by wavelet transformation The edge level estimation unit 22 calculates the high-frequency component amount as a luminance change amount by wavelet transforming the luminance value of the captured image in each block. The edge level estimation unit 22 calculates an average value E4 (high frequency component amount) of a high frequency component image obtained by performing discrete wavelet transform on the captured image of each block. Then, the edge level estimation unit 22 sets the block having the high frequency component amount E4 less than the lower limit value T4L to the lowest level 1, the block having the high frequency component amount E4 equal to or higher than the lower limit value T4L and less than the upper limit value T4H to the intermediate level 2. A block in which the component amount E4 is equal to or greater than the upper limit value T4H is determined to be the highest level 3.
Since wavelet transform is becoming more and more LSI, using wavelet transform can reduce apparatus cost. Further, since the low frequency component image obtained together with the high frequency component image in the process of the discrete wavelet transform can be used in the resolution reduction processing in the resolution conversion unit 23, the edge level estimation unit 22 and the resolution conversion unit 23 are partially shared. By doing so, the apparatus cost can be reduced.

(4) Difference value between low frequency image and captured image (high frequency component amount)
The edge level estimation unit 22 uses the difference value between the low-frequency image generated from the captured image and the captured image as the amount of change in luminance. That is, the edge level estimation unit 22 generates a low-frequency image by applying a median filter having a filter size of 3 × 3 centered on each pixel of the captured image, and the generated low-frequency image and the corresponding luminance value of the captured image. The absolute value of the difference is obtained, and the average value E5 is calculated for each block. Then, the edge level estimation unit 22 sets the block having the difference average value E5 less than the lower limit value T5L to the lowest level 1, the block having the difference average value E5 equal to or higher than the lower limit value T5L and less than the upper limit value T5H to the intermediate level 2. A block whose difference average value E5 is equal to or higher than the upper limit value T5H is determined to be the highest level 3. A smoothing filter may be used instead of the median filter.
Since the low-frequency image obtained in the course of this process can be used in the resolution reduction process in the resolution conversion unit 23, the device cost can be reduced by sharing a part of the edge level estimation unit 22 and the resolution conversion unit 23. .

  In addition to the above examples, it is possible to calculate the luminance change amount by applying various methods for analyzing the luminance distribution of the captured image or the high-frequency component amount of the captured image.

<Modification of edge level>
In the embodiment described above, the edge level estimation unit 22 estimates the edge level by dividing the luminance change amount into three stages using the upper limit value and the lower limit value. In another embodiment, the edge level estimation unit 22 can estimate the edge level by dividing the luminance change amount into two stages using only the lower limit value. FIG. 8 shows an example of the relationship between the edge level, the luminance change amount, and the resolution in this case. That is, the edge level estimation unit 22 compares the average value E1 of the edge strength of each block with the lower limit value T1L, determines that a block where E1 ≧ T1L is level 2 (a region including at least a weak edge), and E1 < The block which is T1L is determined as level 1. In response to this, the resolution conversion unit 23 generates a recognition image by replacing the resolution of the level 1 block in the captured image with an image having a resolution R2 lower than the resolution R4 of the level 2 block.

  Similarly, the edge level estimator 22 can compare the luminance variance value E2 of each block with the lower limit value T2L, and determine that a block where E2 ≧ T2L is level 2 and a block where E2 <T2L is level 1. . Alternatively, the edge level estimation unit 22 compares the high frequency component amount E3 obtained by performing FFT analysis of each block with the lower limit value T3L, and sets a block where E3 ≧ T3L is level 2 and a block where E3 <T3L is level 1. It can also be determined. Furthermore, the same applies to the case of using the high frequency component amount E4 obtained by wavelet transform of each block, the average value E5 obtained by averaging the difference values between the low frequency image and the captured image in each block, and other luminance change amounts. It is possible to estimate the edge level by dividing the luminance change amount into two stages using only the lower limit value.

  In this way, when the amount of change in luminance is divided into two stages, all areas where edges exist in the captured image become high-resolution areas, but a sufficient data amount reduction effect can be achieved in a monitoring area with a simple background.

  In another embodiment, the edge level estimator 22 can compare the luminance change amount with three threshold values and divide the edge level of each block into four stages. FIG. 9 shows an example of the relationship between the edge level, the luminance change amount, and the resolution in this case. That is, the edge level estimation unit 22 compares the average value E1 of the edge strengths of each block with the lower limit value T1L, the intermediate value T1M, and the upper limit value T1H (T1L <T1M <T1H). The block satisfying T1L ≦ E1 <T1M is determined as level 2.1, the block satisfying T1M ≦ E1 <T1H is determined as level 2.2, and the block satisfying E1 ≧ T1H is determined as level 3. . In response to this, the resolution conversion unit 23 replaces the level 1 block with the resolution R2 image and replaces the level 2.1 block with the resolution R4.1 image or maintains the resolution R1 in the captured image. The level 2.2 block is replaced with an image having an intermediate resolution R4.2 lower than the resolution R4.1, and the level 3 block is replaced with an image having a resolution R3 to generate a recognition image. That is, the resolution conversion unit 23 subdivides the level 2 block, which is an intermediate stage, and generates a recognition image by substituting the lower level image with the fourth resolution R4.1, R4.2 as the stage is higher. To do. The resolution conversion unit 23 subdivides a 12 × 12 pixel block into 4 blocks of 6 × 6 pixels and the like, and sets the representative luminance value of the captured image to 4 pixels or the like for each subdivided block. It can be converted to an intermediate resolution R5.2.

  Similarly, the edge level estimation unit 22 compares the luminance variance value E2 of each block with the lower limit value T2L, the intermediate value T2M, and the upper limit value T2H (T2L <T2M <T2H), and sets the block where E2 <T2L to level 1 , T2L ≦ E2 <T2M can be determined as level 2.1, T2M ≦ E2 <T2H can be determined as level 2.2, and E2 ≧ T2H can be determined as level 3. Alternatively, the edge level estimation unit 22 compares the high-frequency component amount E3 obtained by FFT analysis of each block with the lower limit value T3L, the intermediate value T3M, and the upper limit value T3H (T3L <T3M <T3H), and E3 <T3L. It is also possible to determine that a block is level 1, a block where T3L ≦ E3 <T3M is level 2.1, a block where T3M ≦ E3 <T3H is level 2.2, and a block where E3 ≧ T3H is level 3. Furthermore, the same applies to the case of using a high-frequency component amount E4 obtained by wavelet transform of each block, an average value E5 obtained by averaging the difference value between the low-frequency image and the captured image in each block, and other weak edge indices. The edge level can be divided into four stages by setting an intermediate value.

  The relationship between the intermediate value applied to the luminance change amount and the block size to be subdivided in order to realize the intermediate resolution can be appropriately set through a prior experiment.

  Furthermore, the number of steps can be increased, for example, by using four or more threshold values within a range in which block re-division is possible, and increasing the number of steps to five or more. If the intermediate resolution is set in this way, the data amount reduction effect can be further enhanced.

<Other variations>
In the above-described embodiment, an example in which the resolution of the captured image is 8000 × 4500 pixels in the horizontal direction is shown. However, the resolution of the captured image is not limited to this, for example, 7680 × 4320 pixels (8K UHDTV), 3840 × 2160 (4K). Other resolutions such as UHDTV) and 1920 × 1080 pixels (HD) may be used.

  In the above-described embodiment, an example in which the captured image is a grayscale image has been described. However, the captured image may be a color image. In this case, the edge level estimation unit 22 may calculate the luminance change amount after performing gray scale conversion on the color image of the block of interest in step S102.

  As described above, the recognition image generated by the surveillance camera has a high resolution in a region where colors similar to each other are in contact with each other. Therefore, it is difficult to extract the feature of the recognition target object in the image recognition apparatus using this. The image can be reduced, and high-accuracy image recognition can be performed in the image monitoring apparatus. Also, areas with weak edges are determined based on captured images with high resolution, and the areas in the recognition image are made high-resolution areas, so areas with similar colors touching and areas with weak edges are used for recognition. A high-resolution area can be reliably obtained in the image, and high-accuracy image recognition can be performed in the image monitoring apparatus.

  On the other hand, in the image for recognition, the area other than the above-mentioned area where the features of the recognition target object can be sufficiently extracted even if it is not high resolution has low resolution, so the amount of data that is the target of communication and image recognition can be reduced. An image monitoring apparatus can be provided at a low cost. Further, since the high resolution area in the recognition image is not limited to the entire recognition target object but limited to a part thereof, the data amount can be greatly reduced, and the image monitoring apparatus can be provided at low cost. Also, since one frame of recognition image is generated for one frame of captured image, the processing is not complicated in the image recognition apparatus that receives the recognition image, and the image monitoring apparatus can be provided at low cost.

In the above embodiment, an example in which the image monitoring apparatus recognizes a suspicious person has been shown. However, the present invention is not limited to human recognition, but can be applied to recognition of various objects such as vehicles, ships, disasters, and various events, and edges. This is useful for various applications for recognizing the state of the monitoring area using the feature amount.

DESCRIPTION OF SYMBOLS 1 ... Image monitoring apparatus 2-1, 2-2, ..., 2-N ... Surveillance camera 20 ... High-resolution imaging | photography part 21 ... Captured image memory | storage part 22 ... Edge level estimation part DESCRIPTION OF SYMBOLS 23 ... Resolution conversion part 24 ... Image storage part for recognition 25 ... Image output part 3 ... Image communication network 4 ... Image recognition apparatus 40 ... Image input part 41 ... Feature extraction Part 42 ... Suspicious person recognition part 43 ... Abnormal output part 50, 51, 53, 54 ... Block 52, 55 ... Block data 60 ... Photographed image 61, 62 ... Similar color Area 70 ... Recognition image 71, 72, 73 ... High resolution area 74 ... Low resolution area

Claims (3)

A photographing unit for photographing a monitoring area and generating a photographed image of the first resolution;
The captured image is divided into a plurality of blocks, a luminance change amount between pixels constituting each block is calculated, and the luminance change amount is compared with two or more predetermined thresholds having different values. An edge level estimator that divides the intensity of the edge included in 3 into three or more stages so that the higher the intensity is,
A block having at least the lowest stage in the captured image is replaced with an image having a second resolution lower than the first resolution , and a block having the highest stage in the captured image is replaced with the first resolution. A resolution conversion unit that generates a recognition image by replacing the image with a low third resolution ;
An image output unit for outputting the recognition image to the outside;
A surveillance camera characterized by comprising: The edge level estimation unit divides the intensity of the edge included in each block into four or more stages by comparing the luminance change amount with three or more thresholds having different values.
The resolution converting unit further replaces the block in which the stage is an intermediate stage between the lowest stage and the highest stage in the photographed image with an image of a fourth resolution that is set lower as the stage is higher. The surveillance camera according to claim 1 , which generates Monitoring camera according to claim 1 or 2, and video communication network for transmitting said recognition image being connected to the image output unit, an image monitoring apparatus comprising an image recognition apparatus, a
The image recognition device includes:
A communication unit connected to the image communication network and receiving the recognition image;
A feature quantity extraction unit for extracting edge feature quantities from the recognition image;
A recognition unit for recognizing the state of the monitoring region using the edge feature amount extracted by the feature amount extraction unit;
An image monitoring apparatus comprising: