JP4410755B2 - Image processing apparatus and monitoring system - Google Patents

Description

  The present invention relates to a monitoring system that monitors a vehicle with a monitoring camera and a recognition camera.

  Conventionally, images of running vehicles are captured with cameras installed in various locations, and the route of the vehicle with the number registered in advance is traced, and the model, color, and characteristics of the vehicle are recorded for theft. There are monitoring systems that detect cars, criminal vehicles, etc.

This monitoring system includes a combination of two types of cameras: (1) a monitoring camera (color camera or the like) that captures the entire vehicle and (2) a recognition camera that captures the license plate portion of the vehicle. .
Since the surveillance camera (1) is intended to capture the entire image of the vehicle (vehicle type, vehicle color, vehicle characteristics), the image is captured in color but the number of pixels (resolution) is relatively low. On the other hand, the recognition camera (2) is intended to capture the character information of the license plate portion of the vehicle with high accuracy, so that it is captured in monochrome, but the number of pixels (resolution) is relatively high. Note that the recognition camera (2) uses a near-infrared camera or the like so that character information can be accurately recognized even in a relatively dark place. In this way, in the monitoring system, by using the monitoring camera and the recognition camera together, it is possible to accurately recognize the vehicle number and capture the entire image of the vehicle (vehicle type, color, and vehicle characteristics). It is possible.

However, since a vehicle image captured by such a monitoring system includes many personal information elements, it is necessary to prevent such personal information elements from being used for purposes other than the original purpose. There is. In view of this, a technique for concealing the area of the license plate, which is an element of personal information included in the image, or making it available to only a limited user has been proposed (see Patent Document 1, etc.).
JP 2001-23072 A

However, since the image of the surveillance camera used in the above-described technique has a low number of pixels, it is not always possible to accurately identify and conceal the license plate area. Further, the above-described technique is merely a method for concealing the entire license plate imaged by the surveillance camera.
In other words, if a large number of users will be involved in the monitoring system in the future, the license plate, etc. will be monitored according to the user's authority. For this reason, it is necessary to finely divide an area that is not concealed (or an area where concealment can be restored) and control display. However, as described above, since the image of the monitoring camera has a low number of pixels, for example, display control cannot be performed with respect to each license plate code in the image of the monitoring camera.
Accordingly, an object of the present invention is to provide a monitoring system or the like that solves the above-described problems.

  In order to solve the above-described problems, the present invention obtains first image data that is image data of an entire image of a vehicle and second image data that is an image of a front portion of the vehicle, and performs image processing. An image processing apparatus for performing a data input unit that receives input of the first image data and the second image data, and an area specification that specifies an area of the license plate of the vehicle in the second image data A region position specifying unit that matches the world coordinates of the first image data and the second image data, and specifies in which region of the first image data the specified license plate region is located A storage unit that stores disclosure area information indicating a license plate area permitted to be disclosed for each level of the user of the monitoring system; and the specified first image data. The image data subjected to the image processing for concealing the license plate region in the image data is created, and the license plate region permitting the restoration of the concealment in the image data subjected to the image processing is referred to with reference to the disclosure region information. An image processing unit that creates restoration data shown for each level, and a transmission unit that transmits the image data subjected to the image processing and the restoration data are provided. Other configurations will be described in the embodiments described later.

  According to the present invention, the monitoring system can control display of a personal information area such as a license plate in the captured image according to the level of the user. Therefore, even if a large number of users will be involved in the monitoring system in the future, the personal information shown in the image can be appropriately protected.

The best mode for carrying out the present invention (hereinafter referred to as an embodiment) will be described in detail with reference to the drawings. FIG. 1 is a diagram showing the overall configuration of the monitoring system of the present embodiment.
As shown in FIG. 1, the monitoring system includes a recognition camera (second camera) 2 that captures an image of a front portion of a vehicle including a license plate, and a monitoring camera (first camera) that captures an image of the entire vehicle. Camera) 1, image processing device 3 that performs predetermined image processing (for example, mosaic processing) on image data output from each camera, and data from image processing devices 3 at each base, It includes a management device 4 that restores image processing according to the level, and a network 5 that connects each image processing device 3 and the management device 4.

As the monitoring camera 1, a color camera or the like is used so that the color and appearance characteristics of the vehicle can be easily understood. As the recognition camera 2, a near-infrared monochrome camera or the like is used so that the vehicle license plate can be easily imaged even in a relatively dark place.
The image processing device 3 is installed at each site where the vehicle is monitored. The image processing apparatus 3 may perform image processing of image data from a plurality of recognition cameras 2 and monitoring cameras 1. The management device 4 is realized by a computer having a communication function for performing communication via the network 5.

  Next, the hardware configuration of the image processing apparatus 3 will be described. FIG. 2 is a diagram illustrating a hardware configuration of the image processing apparatus of FIG.

  The image processing device 3 includes a processing device 50 that performs image processing such as mosaic processing, and a communication control device 60 that allows the processing device 50 to communicate with the management device 4 of FIG. .

The processing device 50 includes an A / D (Analog to Digital) converter 52 that converts analog signals output from the recognition camera 2 and the monitoring camera 1 into digital signals, an image memory 53 that stores digital signals, and image processing. An image processor 54 for performing the processing, a D / A (Digital to Analog) converter 55 for converting a digital signal for displaying a processing result by the image processor 54 into an analog signal, and the D / A converter 55 The monitor 56 etc. which output and display the image data output from are provided.
In addition to the image memory 53, the processing device 50 further includes a storage unit (not shown). The storage unit stores various data that are referred to when the image processor 54 performs image processing.
Further, the processing device 50 and the communication control device 60 are connected by an input / output interface (not shown). The communication control device 60 is assumed to be realized by a PC (Personal Computer) including a CPU (Central Processing Unit) 57, a RAM (Random Access Memory) 58 for storing a communication program, and a communication interface 59. The function of the communication control device 60 may be incorporated in a part of the processing device 50.

  The image processing in the processing device 50 is described as being performed by the image processing processor 54 using the image memory 53. However, a CPU (not shown) of the processing device 50 may execute a program stored in the RAM. Good.

  Next, functions of the image processing apparatus 3 and the management apparatus 4 will be described with reference to FIGS. 1 and 2 and FIG. FIG. 3 is a block diagram showing functional development of the image processing apparatus and management apparatus of FIG. In the following description, the image processing performed by the image processing apparatus 3 is described as mosaic processing, but the present invention is not limited to this.

<Image processing device>
The image processing device 3 receives input of a monitoring camera image (first image data) output from the monitoring camera 1 and a recognition camera image (second image data) output from the recognition camera 2. The input unit 31, the region specifying unit 32 for specifying the region of the license plate in the recognition camera image from the character recognition result of the number recognition unit 34, and the world coordinates of the monitoring camera image and the recognition camera image are made to coincide with each other. A region position specifying unit 33 for specifying which region of the surveillance camera image the license plate region in the surveillance camera image is located; a number recognition unit 34 for performing license plate number recognition of the recognition camera image; An image processing unit 35 that creates image data obtained by performing mosaic processing on image data, and image data that has undergone the image processing. And the transmission area 36 for transmitting the number recognition result by the number recognition section 34 via the network 5 and the disclosure area information indicating the image area to be disclosed to the user of the level for each level of the user of the monitoring system And a disclosure area information storage unit 37 for storing.

  The image processing unit 35 described above creates image data obtained by performing mosaic processing on the monitoring camera image, and also creates restoration data used for restoration of the mosaic processing. This restoration data includes information on the area of the license plate that permits mosaic restoration for each user level, information on the type of mosaic processing, and the like. For example, the image processing unit 35 allows the restoration of the mosaic in the license plate business code identification area (see reference numeral 11 in FIG. 14 to be described later) to the level 1 user as restoration data. Create information.

  The region specifying unit 32, the region position specifying unit 33, and the image processing unit 35 are realized by the image processing processor 54 shown in FIG. 2, and the data input unit 31 is realized by the A / D converter 52 shown in FIG. . Moreover, the transmission part 36 is implement | achieved by the communication control apparatus 60 of FIG.

  The management device 4 includes a reception unit 41 that receives transmission data from the image processing device 3, a restoration unit 42 that restores a mosaic of image data included in the transmission data, and an image that displays image data that has been restored from mosaic processing. A display unit 43 and a data storage unit 44 that stores transmission data in a storage unit (not shown) of the management device 4 are provided.

  The restoration unit 42 restores the mosaic-processed image data according to the level of each user based on the restoration data. For example, the restoration unit 42 receives an input of a user ID via an input unit (not shown) of the management device 4 and refers to information indicating a user level for each user ID. The level of the ID user is determined. Then, the mosaic process is restored according to this level. Here, the level of the user indicates the authority to view information at several stages. The monitoring system according to the present embodiment allows a user with a high level (low level value) to view more information, and views a user with a low level (high level value). Control is done to reduce the amount of information that can be generated.

  The management device 4 is realized by a computer including a CPU, a storage unit, an input / output interface, a communication interface, and the like. The receiving unit 41 described above is realized by the communication interface of the management device 4. The storage unit of the management device 4 stores a mosaic restoration program for restoring a mosaic based on the restoration data and an image display program for displaying an image on an image display device such as a liquid crystal monitor. The restoration unit 42, the image display unit 43, and the data storage unit 44 described above are realized by the CPU of the management device 4 executing the program of the storage unit.

<Explanation of camera>
Here, the monitoring camera 1 and the recognition camera 2 according to the present embodiment will be described with reference to FIGS. FIG. 4 is a diagram illustrating the arrangement of the monitoring camera and the recognition camera in FIG. As shown in FIG. 4, the visual field of the monitoring camera 1 is an area indicated by reference numeral 7 so that the entire vehicle 100 can be imaged. The field of view of the recognition camera 2 is an area indicated by reference numeral 6 (an area near the front portion of the vehicle) so that the license plate number can be recognized.
FIG. 5 is a diagram illustrating an image captured by the recognition camera of FIG. 4, and FIG. 6 is a diagram illustrating in detail a license plate in the image of FIG. 5.
Generally, the height and width in the visual field of the recognition camera 2 are about 2 to 3 m, and when the recognition camera 2 images the vehicle 100 of FIG. 4, an image 8 of FIG. 5 is obtained. That is, the image captured by the recognition camera 2 is written with the codes (land support code 9, vehicle type code 10, business code 11, serial number 12 and the like in FIG. 6) included in the license plate indicated by reference numeral 80 in FIG. The image has a resolution that can be recognized or visually confirmed by a human.

The installation of the recognition camera 2 at this time will be described with reference to FIG. FIG. 7 is a diagram illustrating the installation position of the recognition camera in FIG.
As shown in FIG. 7, when the camera installation height H, the camera depression angle θ, and the recognition camera 2 recognizes the license plate, the position of the license plate is obtained, so that the approximate position L1 from the camera can be found from the detected position.

On the other hand, the field of view of the monitoring camera 1 in FIG. For example, when the monitoring camera 1 in FIG. 4 images the vehicle 100, an image 13 in FIG. 8 is obtained. FIG. 8 is a diagram illustrating an image captured by the monitoring camera of FIG. FIG. 9 is a diagram illustrating the field of view of the monitoring camera of FIG. FIG. 10 is a diagram for explaining the position of the license plate in the image of the monitoring camera shown in FIG.
As shown in FIG. 9, the visual field (solid line area) of the monitoring camera 1 is larger than the visual field (dashed line area) of the recognition camera 2. As shown in FIG. 10, the position (height) of the license plate of the vehicle 100 in the image 13 captured by the monitoring camera 1 is the position indicated by the symbol L2.

<Description of operation>
Next, the operation of the monitoring system will be described using FIGS. 11 and 17 with reference to FIGS. FIG. 11 is a flowchart showing the operation of the image processing apparatus of FIG. FIG. 17 is a flowchart for explaining the operation of the management apparatus of FIG.

  The data input unit 31 of the image processing apparatus 3 in FIG. 3 receives input of images (monitoring camera image and recognition camera image) output from the monitoring camera 1 and the recognition camera 2 (S150).

Next, the number recognition part 34 performs a vehicle detection process (S160), and determines whether there is a license plate in the recognition camera image output from the recognition camera 2 (S170). If there is no license plate in the recognition camera image (No in S170), the process returns to S150 to wait for image data to be input again.
On the other hand, if the recognition camera image includes a license plate (Yes in S170), character recognition of each code of the license plate is performed (S180). For the license plate number recognition processing, the technique described in Japanese Patent Application Laid-Open No. 6-215293 “Car Number Recognition Device” or the like can be used. Next, the area specifying unit 32 specifies a license plate area in the recognition camera image from the character recognition result in S180 (S190).

  Next, the area position specifying unit 33 matches the world coordinates of the monitoring camera image and the recognition camera image, and performs coordinate conversion of the license plate area specified by the area specifying unit 32 in S190 (S200). That is, the region position specifying unit 33 specifies in which region of the monitoring camera image the license plate region in the recognition camera image is located. For example, as shown in FIG. 12, the license plate area is specified as an area indicated by reference numeral 240 on the monitoring camera image 24.

  For this coordinate transformation, a technique such as perspective transformation is used. Perspective conversion is described as camera calibration in “Computer Vision”: New Technology Communications, pp. 77-53. The details of the coordinate conversion in this embodiment will be described later with reference to the drawings.

  Next, the image processing unit 35 performs a mosaic process on the license plate area in the monitoring camera image (S220). The mosaic processing method at this time is not limited to the above-described mosaic processing as long as the character information can be concealed so that it cannot be visually identified and can be restored on the management apparatus 4 side.

Further, when the image processing unit 35 performs the mosaic process in S220, data for restoration is also created. The restoration data is information indicating a region where mosaic restoration is permitted in the image data and the type (method) of the mosaic. The area where the restoration of the mosaic is permitted is set for each user level (user authority) based on the disclosure area information by the administrator of the monitoring system or the like. This disclosure area information will be described later with reference to the drawings.
In addition, in the restoration data, instead of the area where the restoration of the mosaic is permitted, information regarding the area where the restoration of the mosaic is not allowed, that is, the area which is kept hidden may be described.

The image processing unit 35 performs mosaic processing on the entire area of the license plate image, and then refers to the above-described disclosure area information to create data for restoring the image. The mosaic-processed image data, restoration data, character recognition result, and the like (hereinafter collectively referred to as transmission data) are transmitted to the management apparatus 4 via the transmission unit 36 (S230). Thereafter, the process returns to S150, and the image processing apparatus 3 waits for the next image data to be input.
In this way, the image processing apparatus 3 performs mosaic processing of each image data.

Here, the coordinate conversion in S200 of FIG. 11 will be described with reference to FIG. FIG. 13A illustrates the world coordinate system, and FIG. 13B illustrates the camera coordinate system. Gx and Gy in FIG. 13A indicate coordinates in the world coordinate system, and Cx and Cy in FIG. 13B indicate coordinates in the camera coordinate system. The camera coordinates are relative coordinates within the field of view of the camera, and the world coordinates are absolute coordinates with the camera installation position as the origin.
This coordinate conversion consists of three steps: (1) camera parameter calculation, (2) camera coordinate → world coordinate conversion, and (3) world coordinate → camera coordinate conversion. In the following description, it is assumed that the installation positions of the monitoring camera 1 and the recognition camera 2 are substantially the same.

  First, from the reference points A to D in FIG. 13A installed on the road surface, the world coordinates (Gx, Gy) with the camera (the monitoring camera 1 and the recognition camera 2) as the origin and the camera image shown in FIG. The camera parameters (transformation matrix parameters for perspective transformation) of the monitoring camera 1 and the recognition camera 2 are calculated from the correspondence with the coordinates (Cx, Cy) of the coordinates a to d of b).

  Next, the camera coordinates of the recognition camera 2 are converted into world coordinates using the camera parameters of the recognition camera 2 obtained by the equation (1). That is, the world coordinates are calculated by substituting the camera parameters of the recognition camera 2 and the camera coordinates of the recognition camera 2 into the following equation (2). For example, the coordinates of the license plate area in the recognition camera image are converted into world coordinates.

  Subsequently, the world coordinates in the recognition camera 2 are converted into the camera coordinates of the monitoring camera 1 using the camera parameters of the monitoring camera 1 obtained by the expression (1). That is, the camera parameters of the monitoring camera 1 and the world coordinates are substituted into the following equation (3) to calculate the camera coordinates of the monitoring camera 1. That is, the world coordinates of the license plate area are converted into coordinates in the monitoring camera image.

  The coordinate conversion described above may be calculated from the height, installation position, depression angle, side view angle, field of view width, etc. of the monitoring camera 1 and the recognition camera 2.

  The area position specifying unit 33 specifies each code area in the license plate area of the monitoring camera image after performing coordinate conversion by the above-described calculation. The identification of each code area uses information on each code area identified when the number recognition unit 34 performs character recognition. Each code area is specified for each license plate. FIG. 14 is a diagram showing a specific example of each code area in the present embodiment. As shown in FIG. 14, the area position specifying unit 33 includes a land code area (reference numeral 9), a vehicle type code area (reference numeral 10), a business code area (reference numeral 11), and a serial number area (reference numeral 12). Is identified. After this, the serial number area (reference numeral 12) may be specified separately for each character.

  Next, the disclosure area information will be described with reference to FIG. FIG. 15 is a diagram for explaining the disclosure area information in the present embodiment. As shown in FIG. 15, the level set in the disclosure area information is two or more stages. That is, two levels such as level 0 and other levels may be set, or two or more levels such as level 0 to level 4 may be set. In the example in the figure, five levels from level 0 (high) to level 4 (low) are set, and the mosaic area (area to be concealed) is increased as the level becomes lower. In other words, as the level increases, the disclosed areas (areas not concealed) are increased.

For example, in FIG. 15, level 0 indicates that all the code areas of the license plate can be viewed.
Level 1 indicates that the serial number area (reference numeral 12) is concealed and the other areas (land support code “Adachi”, vehicle type code “500”, business code “O”) are disclosed.
Level 2 indicates that the serial number area (reference numeral 12) and the vehicle type code area (reference numeral 10) are concealed, but the other areas (land support code "Adachi", business code "O") are disclosed.
Level 3 covers the serial number area (reference numeral 12), the vehicle type code area (reference numeral 10), and the business code area (reference numeral 11), but discloses the other areas (land support code "Adachi"). Indicates.
Level 4 indicates that all areas (reference numerals 9 to 12) are concealed.
The disclosed area information can be input via an input unit (not shown) of the image processing apparatus 3.
An example of smoothing indicated by reference numeral 310 in FIG. 15 will be described later.

Next, transmission data transmitted from the transmission unit 36 will be described with reference to FIG. FIG. 16 is a diagram illustrating transmission data transmitted by the transmission unit of FIG. As shown in FIG. 16, the transmission data includes image data 161 subjected to mosaic processing, restoration data 162, character recognition result 163, and the like. The restoration data 162 includes a mosaic technique 165 that is information relating to a mosaic technique and a mosaic area 166 that is information relating to an area where mosaic restoration is permitted. Information regarding the area where the restoration of the mosaic is permitted is described by position coordinates or the like in the image data.
Further, the mosaic-processed image data 161 and the restoration data 162 may be transmitted separately to maintain security.

Next, the operation of the management apparatus 4 that has received the transmission data from the image processing apparatus 3 will be described using FIG. 17 with reference to FIGS.
The receiving unit 41 of the management device 4 receives the transmission data from the image processing device 3 via the network 5 (S420), and if there is an input requesting data storage via an input unit (not shown) (Yes in S430), the data The storage unit 44 stores the transmission data received by the management device 4 in a storage unit (not shown) (S440). On the other hand, if there is no input for requesting data storage (No in S430), the process proceeds to S450. In S450, it is determined whether or not there is an input requesting display of image data included in the transmission data. If there is an input requesting display of image data (Yes in S450), the process proceeds to S460, and the restoration unit 42 determines the level (S460). For example, the restoration unit 42 determines the level of the user who requests display of this image data, using the user ID or the like input via the input unit as a key. On the other hand, if there is no input requesting display of image data (No in S450), the process returns to S420. Then, the restoration unit 42 performs a mosaic restoration process on the image data according to the level of the user (S470). The mosaic restoration process at this time is performed with reference to the information of the mosaic technique 165 and the mosaic area 166 included in the restoration data.

For example, when the restoration unit 42 determines that the level of the user is “1”, the license plate area of the image data 161 is restored to an image as indicated by reference numeral 27 in FIG. In other words, an image having a serial number of reference numeral 12 is concealed and an image in which a mosaic is restored is created for the other regions. Then, the image display unit 43 performs display processing of the image data restored by the restoration unit 42 (S480). For example, the restored image data is displayed on a display device (not shown) connected to the management device 4. Then, the process returns to S420.
The management device 4 repeats the above processing every time transmission data is received from the image processing device 3.

  Thus, the monitoring system can perform display control according to the level of the user with respect to the number plate of the imaged vehicle.

<Second Embodiment>
Next, a second embodiment of the present invention will be described. The second embodiment is characterized in that the image processing device 3 synthesizes a part of the recognition camera image with the monitoring camera image and transmits it to the management device 4. That is, the image processing apparatus 3 transmits an image of the entire vehicle that is the monitoring camera image and an enlarged image of the area of the license plate portion in the recognition camera image. Therefore, when the user wants to visually confirm the license plate in the management device 4, even if the license plate area is too small in the image of the entire vehicle and character information cannot be confirmed, the number can be obtained by looking at the enlarged image portion. It becomes easy to confirm the character information of the plate.

  FIG. 18 is a flowchart illustrating the operation of the image processing apparatus according to the second embodiment. The processing from S150 to S190 in FIG. 18 is the same as the processing from S150 to S190 in FIG. 11, and thus the description thereof will be omitted. The processing from S191 onward, which is a feature of the present embodiment, will be described.

  The image processing unit 35 in FIG. 3 reads the license plate image in the recognition camera image specified in S190 (S191), and synthesizes this image with the monitoring camera image (S192). FIG. 19 is a diagram illustrating a composite image created by the image processing unit according to the second embodiment. As shown in FIG. 19, the image processing unit 35 synthesizes the image of the license plate area in the recognition camera image with the position of the area 39 of the monitoring camera image. The position of the region 39 may be changed according to the traveling direction of the vehicle and the road conditions. Further, it is preferable to appropriately enlarge the number plate in the region 39 so that the number plate is larger than the number plate in the region 40 so that the user can easily see it on the screen.

  Next, the area position specifying unit 33 performs coordinate conversion between the monitoring camera image and the recognition camera image in the same manner as S200 in FIG. 11 (S200). Then, the image processing unit 35 performs mosaic processing of the image synthesized in S192 (S320). That is, the image processing unit 35 performs mosaic processing on both the license plate region (region 40 in FIG. 19) and the combined license plate region (region 39) in the overall image of the vehicle.

  At this time, the position of the area 39 in the monitoring camera image and the area of each character information (land support code, vehicle type code, business code, serial number, etc.) in the area 39 are determined by the area position specifying unit 33 of the monitoring camera image. It is specified based on the coordinates of the synthesis area start point 190 and the relative coordinates of each character information obtained during character recognition of the recognition camera image. In addition, the image processing unit 35 creates data for restoring mosaic processing as in the first embodiment. The restoration data here includes, in addition to restoration data relating to the license plate area in the area 40, restoration data relating to the license plate area in the area 39.

Then, similarly to S230 in FIG. 11, the image processing unit 35 transmits the transmission data including the mosaic-processed image data, the restoration data, and the recognition result to the management apparatus 4 (S330 in FIG. 18).
Thereafter, the management device 4 receives the transmission data, and performs a mosaic restoration process and an image display process for the license plate areas in the areas 40 and 39 in FIG. 19 according to the same procedure as in the first embodiment.

  As described above, the image processing apparatus 3 synthesizes and transmits the enlarged image of the license plate captured in the recognition camera image to the monitoring camera image, so that even if the monitoring camera image is unclear, the user visually observes the license plate. It becomes easy to confirm the information of each code.

  Note that the image processing apparatus 3 may synthesize a driver image extracted from the monitoring camera image instead of the above-described license plate. FIG. 20 is a diagram illustrating a composite image in a modification of the second embodiment. For example, the image processing unit 35 extracts a region (region 200) in which the driver is captured from the monitoring camera image, expands the extracted image, and places the side of the monitoring camera image (for example, the region 61 in FIG. 20). Etc.). Then, the image processing unit 35 performs a mosaic process on the region (region 200 and region 61) where the driver is shown and the license plate region (region 40).

  As described above, the image processing device 3 performs the mosaic process on the license plate region of the monitoring camera image and the region where the driver is reflected, and transmits it to the management device 4, so that the privacy of the driver is protected and the privacy of the management device 4 is protected. Users will be able to see the characteristics of the driver. Therefore, the user can easily conduct criminal investigation using this system. Note that the area where the driver is shown may be an area of the windshield of the vehicle.

  Further, for the area where the driver is shown in the monitoring image data, after the area position specifying unit 33 specifies the license plate area, the upper area of the license plate area is subjected to horizontal / vertical differentiation, etc. It is assumed that the amount is extracted and specified from the density information or the like in the feature amount.

  Further, the image processing unit 35 may combine the image of the front grille of the vehicle extracted from the recognition camera image with the monitoring camera image. FIG. 21 is a diagram illustrating a composite image in a modification of the second embodiment. For example, the image processing unit 35 extracts the area of the front grille portion from the recognition camera image, enlarges the extracted image, and synthesizes it on the side of the monitoring camera image (for example, the area 62 in FIG. 21). . Then, the image processing unit 35 performs a mosaic process on the front grille portion (region 63 and region 62) of the vehicle in the composite image.

  As described above, the image processing device 3 performs mosaic processing on the front grill portion of the monitoring camera image and transmits it to the management device 4, so that the user of the management device 4 can protect the driver's privacy while the user of the management device 4 Can be confirmed. In other words, since the user can check the characteristics of the front grille portion of the vehicle (for example, scratches or dents on the front grille portion), it is easy to conduct criminal investigations.

Note that the front grill portion in the recognition camera image can be extracted by specifying the vehicle type and the rough vehicle size from the license plate recognition result by the number recognition unit 34. Alternatively, as in the above-described specification of the area where the driver is shown, the vehicle width is specified using horizontal / vertical differentiation, or the shape of the front part is registered in advance for each vehicle type, and pattern matching of this shape is performed. It can also be extracted by specifying.
As described above, the image processing device 3 may perform mosaic processing on image data of a portion that is a feature of the vehicle other than the license plate of the vehicle, and may transmit the image data to the management device 4.

<Other embodiments>
The present invention is not limited to the embodiments described above, and can be widely modified. For example, the disclosure area information described above is information indicating an area to be disclosed for each level. For each area, what level the area is (that is, up to which level users are allowed to restore the mosaic) ) May be used. In this way, the administrator of the monitoring system can easily adjust the level for each area.

  Further, the mosaic processing performed by the image processing unit 35 may use a smoothing technique as indicated by reference numeral 310 in FIG. 15, for example. Furthermore, the mosaic processing technique used by the image processing unit 35 may be different for each region. As the mosaic processing method, encryption using an encryption key, bit rotation of density values, or the like may be used. In this case, the image processing unit 35 includes an encryption key, a bit rotate key, and the like in the restoration data.

  Furthermore, the image processing apparatus 3 may be configured to select a predetermined method from a plurality of mosaic processing methods. That is, a plurality of mosaic processing programs are stored in the storage unit of the image processing apparatus 3, and in S220 of FIG. 11 and S320 of FIG. 18, which mosaic processing is performed before the image processing unit 35 performs the mosaic processing. It is possible to select whether to use the method. For example, a selection input regarding which mosaic processing method to use is received from the administrator of the monitoring system via the input unit of the image processing apparatus 3, and the image processing unit 35 executes a predetermined mosaic processing program based on the selection input. To do.

  Further, the mosaic processing executed by the image processing unit 35 may use a technique such as smoothing of irreversible transformation and partial pixel filling. In this case, the image processing unit 35 encrypts and adds the current image (image data before the mosaic process) in the area subjected to the mosaic process to the restoration data. Also by such a method, the image processing apparatus 3 can perform the mosaic process on the image and can restore the mosaic process on the management apparatus 4 side.

  The image processing apparatus 3 according to the present embodiment may be realized by an image processing program that executes the processing as described above. Further, the program can be provided by being stored in a computer-readable storage medium (CD-ROM or the like). It is also possible to provide the program through a communication network such as the Internet.

It is a figure which shows the whole structure of the monitoring system of this Embodiment. It is the figure which showed the hardware constitutions of the image processing apparatus of FIG. FIG. 2 is a block diagram showing functions of the image processing apparatus and management apparatus in FIG. It is a figure explaining arrangement | positioning of the monitoring camera of FIG. 1, and the camera for recognition. It is the figure which illustrated the image imaged with the camera for recognition of FIG. It is the figure which showed the license plate in the image of FIG. 5 in detail. It is a figure explaining the installation position of the camera for recognition of FIG. It is the figure which illustrated the image imaged with the monitoring camera of FIG. It is a figure explaining the visual field of the surveillance camera of FIG. It is a figure explaining the position of the license plate in the image of the monitoring camera of FIG. 3 is a flowchart illustrating an operation of the image processing apparatus in FIG. 1. It is the figure which illustrated the image imaged with the monitoring camera of FIG. (A) is a figure explaining a world coordinate system, (b) is a figure explaining a camera coordinate system. It is the figure which showed the specific example of each code area | region in this Embodiment. It is a figure explaining the disclosure area information in this embodiment. It is the figure which illustrated the transmission data which the transmission part of FIG. 3 transmits. It is the flowchart which showed operation | movement of the management apparatus of FIG. 6 is a flowchart illustrating an operation of the image processing apparatus according to the second embodiment. It is the figure which illustrated the synthesized image which the image processing part of 2nd Embodiment produced. It is the figure which illustrated the synthesized image in the modification of 2nd Embodiment. It is the figure which illustrated the synthesized image in the modification of 2nd Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Monitoring camera 2 Recognition camera 3 Image processing apparatus 4 Management apparatus 5 Network 31 Data input part 32 Area identification part 32 Number recognition part 33 Area position identification part 34 Number recognition part 35 Image processing part 36 Transmission part 37 Disclosure area information storage Unit 41 Receiving unit 42 Restoring unit 43 Image display unit 44 Data storage unit

Claims (7)

An image processing device that obtains first image data that is image data of a whole image of a vehicle and second image data that is an image of a front portion of the vehicle and performs image processing,
A data input unit for receiving input of the first image data and the second image data;
An area specifying unit for specifying an area of the license plate of the vehicle in the second image data;
An area position specifying unit that matches the world coordinates of the first image data and the second image data, and specifies in which area of the first image data the area of the specified license plate is located;
For each user level, a storage unit that stores disclosure area information indicating a license plate area for which disclosure is permitted;
Image data that has been subjected to image processing for concealing the area of the license plate in the identified first image data is created, and the concealment is restored in the image data that has been subjected to the image processing with reference to the disclosure area information An image processing unit for creating restoration data indicating the license plate area for each level,
A transmission unit that transmits the image data subjected to the image processing and the restoration data;
An image processing apparatus comprising: An image processing apparatus according to claim 1;
A receiving unit for receiving image data and restoration data transmitted from the image processing device;
Based on the restoration data, a restoration unit that restores concealment of the image data according to the level of the user,
And an image display unit for displaying the image data with the concealment restored,
A management device comprising:
A monitoring system comprising:   The image processing unit creates image data by pasting an image of a license plate area in the specified second image data to the first image data, and the license plate area in the created image data is created. Create image data that has been subjected to concealment image processing, and refer to the disclosure area information to create restoration data that indicates the area of the license plate that permits restoration of the concealment in the image data that has undergone image processing The monitoring system according to claim 2, wherein: The image processing to conceal is mosaicing, encryption, smoothing or filling processing of a portion to be concealed in each image data,
The image processing unit selects any one of the image processes based on an instruction input from an input unit, and further includes the type of the selected image processing in the restoration data. The monitoring system described in.   The first image data is image data output from a color camera, and the second image data is image data output from a monochrome camera that supports near infrared rays. Monitoring system.   The monitoring system according to claim 3, wherein an image of a front grill area of the vehicle is used instead of an image of a license plate area in the second image data. The disclosure area information is information indicating an image area of a vehicle permitted to be disclosed for each level of the user of the monitoring system,
The area position specifying unit further specifies the area of the vehicle windshield in the first image data based on the area of the license plate in the specified first image data,
The image processing unit creates image data obtained by copying and pasting the image of the identified windshield area to the first image data, and the license plate area and the windshield area in the created image data. Create image data that has undergone image processing to conceal the area, and reference the disclosure area information to create restoration data that indicates the image area that is allowed to be restored in the image data that has undergone image processing The monitoring system according to claim 2, wherein:

Images