JP7187154B2 - Image processing device, image processing method and program - Google Patents

Description

The present invention relates to an image processing apparatus, an image processing method and a program, and more particularly to a technique of hiding a specific object in an image to protect privacy.

In recent years, surveillance cameras have been installed all over town. Here, the importance of protecting the privacy of individuals captured by surveillance cameras is increasing. For example, Patent Literature 1 discloses a method of comparing a background image and an image to be processed, and performing concealment processing on a specific area of the image to be processed according to the comparison result to protect privacy. In particular, the human body and moving objects displayed in the image are detected, and the accuracy of concealment processing is improved.

JP 2016-115214 A

However, in Patent Literature 1, the background of areas other than the human body and the moving object in the image is updated at a uniform time, and it was not possible to process the object or area that should be immediately backgrounded to preferably protect the privacy.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide an image processing apparatus capable of setting a suitable background rendering time for each of a photographed subject subject to privacy protection and a subject not subject to privacy protection.

The image processing apparatus of the present invention includes detection means for detecting an object from a captured image captured by an imaging unit, and setting a time threshold for a region corresponding to the object detected from the captured image based on the characteristics of the object. and setting means for capturing the image so that the object in the area is reflected in the background image when the time period during which the object in the area in the captured image is stationary reaches the time threshold. updating means for updating a background image based on the image, wherein the setting means sets a first value as the time threshold for the region corresponding to the object when the object is subject to privacy protection. and if the object is not subject to privacy protection, a second value smaller than the first value is set as the time threshold for the region corresponding to the object.

According to the present invention, it is possible to more preferably perform processing for protecting privacy by setting a backgrounding time corresponding to an object in an image.

(a) It is a block diagram which shows the functional structure of an image processing apparatus. (b) It is a schematic diagram which shows schematic structure of an image processing system. 2 is a block diagram showing the hardware configuration of the image processing device; FIG. 4A to 4G are schematic diagrams for explaining an outline of privacy protection processing; FIG. 4A to 4G are schematic diagrams for explaining an example of privacy protection processing; FIG. 4 is a flowchart showing an outline of privacy protection processing; FIG. 4 is a schematic diagram showing a GUI for setting a time threshold; FIG. 4 is a schematic diagram showing a GUI for making settings related to a background image;

Embodiments will be described in detail below with reference to the accompanying drawings. Note that the configurations shown in the following embodiments are merely examples, and the present invention is not limited to the illustrated configurations.

<Embodiment 1>
FIG. 1B is a schematic diagram showing the image processing system of this embodiment. At least one or more image output devices 120 are communicably connected to the image processing device 101 via the network 110 . The image output device 120 includes a network camera for surveillance, an NVR (Network Video Recorder), and the like. In the following description, a network camera (simply referred to as camera 120) will be used as an example. The network 110 transmits distributed video from the camera 120 to the image processing apparatus 101 and transmits control signals and setting parameters of the camera 120 from the image processing apparatus 101 to the camera 120 by communication according to the Internet protocol. The image processing apparatus 101 acquires moving image data from the image output apparatus 120 and performs privacy protection processing on the acquired moving image data. Image data (moving image data) subjected to privacy protection processing is output to other client devices connected to the network 110, displayed on a display locally connected to the image processing device, or recorded on the NVR 120. You can

FIG. 2 is a block diagram showing the hardware configuration of the image processing apparatus 101. As shown in FIG. The image processing apparatus 101 comprises a CPU 201 , a ROM 202 , a RAM 203 , an external memory 204 , a network interface 205 , an input device 206 , a display device 207 and a system bus 208 .

The CPU 201 comprehensively controls operations in the image processing apparatus 101 and controls each component (202 to 207) via a system bus 208. FIG. A ROM 202 is a non-volatile memory that stores control programs and the like necessary for the CPU 201 to execute processing. Note that the programs executed by the CPU may be stored in the external memory 204 or a removable storage medium.

A RAM 203 functions as a main memory and a work area for the CPU 201 . That is, the CPU 201 loads necessary programs from the ROM 202 to the RAM 203 and executes the loaded programs to realize various functional operations when executing the processes described later.

The external memory 204 stores, for example, various data and information necessary for the CPU 201 to perform processing using programs. Also, the external memory 204 stores various data, various information, and the like obtained by the CPU 201 performing processing using programs and the like, for example.

A network interface 205 is an interface for communicating with an external device. Network interface 205 is, for example, a LAN interface. In this embodiment, an example in which the image processing apparatus 101 is a client PC (personal computer) will be mainly described.

The input device 206 functions as an input device to the CPU 201 . The input device 206 is, for example, a keyboard, mouse and buttons.

A display device 207 is a display that displays the output of the CPU 201 . Video may be output from the network interface 205 to the outside, and may be unnecessary depending on the configuration.

Note that the image output device 120 also has the same configuration as in FIG. If the image output device 120 is a network camera, it does not have the display device 207, but has an image sensor, a lens, and a processor (not shown) for image processing. The lens may be a removable lens.

FIG. 1A shows a schematic configuration of the image processing apparatus 101. The image processing apparatus 101 includes an image acquisition unit 102, a moving object detection unit 103, an object detection unit 104, a parameter setting unit 105, a background update unit 106, and an image comparison unit 107. , an image synthesizing unit 108 and an output unit 109 .

The image acquisition unit 102 sequentially acquires images from the image output device 120 at predetermined intervals, and sends the acquired images to the moving object detection unit 103 , the object detection unit 104 , the background update unit 106 and the image comparison unit 107 .

The moving object detection unit 103 detects a moving object by comparing the image acquired by the image acquisition unit 102 with a background image (background model) (background subtraction method). That is, the moving object detection unit 103 detects a moving object region based on comparison between the image acquired by the image acquisition unit 102 and another image captured at a different time from the acquired image. Note that the background model is appropriately updated by the moving object detection unit 103 following changes in the image, and may be a feature (information obtained by extracting the luminance component of each pixel) instead of the image.

The moving object detection unit 103 sends the moving object information obtained by the moving object detection process to the background updating unit 106 . The moving body information of this embodiment includes information on the center coordinates (position information) and the circumscribed rectangle (shape information) of the moving body detected from the image. The moving object detection unit 103 executes moving image detection processing on each of the plurality of images acquired by the image acquisition unit 102 and sends moving object information obtained as a result to the background updating unit 106 .

The object detection unit 104 performs object detection processing for detecting a specific object from the image acquired by the image acquisition unit 102 . The object detection unit 104 of this embodiment performs object detection processing from the image acquired by the image acquisition unit 102 by a method described below.

The first is the use of pattern matching methods. An object is detected by comparing the image acquired by the image acquisition unit 102 with a predetermined pattern image (or feature). The object to be detected is any object that has a pattern image that can be detected, such as a human body, a chair, a desk, or the like. The human body may be the whole body or a part of the whole body such as the upper body, head, face, and hands. However, the object detection method is not limited to the pattern matching method, and may be detection using image color, brightness, density gradient, texture, or machine learning feature amount.

The second is distance detection. It is known that there are currently cameras that can detect the distance from the camera image (distance is obtained by triangulation using binoculars or two pixels of a CCD). The distance for each area is measured from the image acquired by the image acquisition unit 102 . Object detection is performed according to the distance as follows. Areas that are farther than a predetermined distance are not subject to privacy protection and should be used as background areas. A region that is closer than the previous frame by a predetermined distance or more may have a human body in it, and is a privacy protection target region.

A score is created for each region based on the feature amounts (features) of the image regions detected by the first method and the second method. A high score is set for areas where privacy protection is required, such as a human body area, and a low score is set for areas where privacy protection is not required, such as distant views, chairs, and desks. In addition, the score is calculated with more certainty added, and if it is determined that there is a high possibility that it is an area that requires more privacy protection, the higher the score, the higher the possibility that it is an area that does not require privacy protection. If it is judged as, it will be a lower score.

The score information calculated for each area by the method described above is sent to the background updating unit 106 .

The background update unit 106 generates a background image based on the image acquired by the image acquisition unit 102, the moving object information acquired from the moving object detection unit 103, and the image area score information acquired from the object detection unit 104. The background updating unit 106 then sends the background image to the image comparing unit 107 and the image synthesizing unit 108 . The details of the background image generation processing by the background updating unit 106 will be described later.

The image comparison unit 107 combines a predetermined image with the protection area indicated by the protection area information acquired from the image comparison unit 107 in the background image acquired from the background update unit 106 to generate a protection image. A protected image is an image whose privacy is protected compared to the input image before processing, and the details of the generation method thereof will be described later. The image synthesizing unit 108 sends the generated protected image to the output unit 109 .

The output unit 109 displays the protected image acquired from the image synthesizing unit 108 . Note that when the image output device 101 is mounted on a network camera, the output unit 109 can transmit a protected image to a monitor device connected via the network camera. Further, when the image processing apparatus 101 is mounted on a monitor apparatus, a protected image is generated by synthesizing a predetermined image with a background image acquired from a network camera, and the generated protected image is displayed. A parameter setting unit 105 sets parameters required for object detection by the object detection unit 104 (details will be described later with reference to FIG. 6).

Next, a method of generating a background image by the background updating unit 106 will be described. The background updating unit 106 manages the stable background time for each area (block) obtained by dividing the background image into a predetermined size. The stable background time is set for each image area based on the moving object information and the score information of the image area, and corresponds to backgrounding of the object in the image area.

In the following description, the background image generation processing by the background updating unit 106 will be described in two cases: (A) the first image, and (B) the images subsequent to the first image.

(A) Processing of Creating Initial Background Image (Background Image Created for the First Time After Starting the Apparatus or Changing Shooting Conditions) In this case, the background image has not yet been created. In this case, the background update unit 106 stores the image acquired from the image acquisition unit 102 as the background image. Note that the background updating unit 106 sets 0 as the stable background time for all blocks of the background image.

(B) Subsequent Background Image Creation Processing This case is a case in which the first image is set as the background image and the subsequent background images are created. In this case, the background update unit 106 updates (increments) the stable background time corresponding to the blocks other than the block to which the moving object region identified by the moving object information belongs. Then, the background updating unit 106 identifies a block for which the post-update stable background time is equal to or greater than the time threshold, and based on the pixel values of the area corresponding to the identified block in the image newly obtained by the image obtaining unit 102, Update the background image. Here, the block in which the stable background time is equal to or greater than the time threshold means an area in which the moving object undetected state continued for the time threshold. Then, the background updating unit 106 sets 0 to the stable background time corresponding to the block used for updating the background image.

The background updating unit 106 sets a time threshold for each image area based on the score information of the image area. The higher the score, the higher the need for privacy protection, and the lower the score, the lower the need for privacy protection. The background updating unit 106 sets a longer time as the time threshold for a higher score, making it more difficult to combine with the background image (harder to create a background). On the other hand, the lower the score is, the shorter time is set as the time threshold, and the background image is more likely to be combined with the background image (become the background) in a shorter time. The background update unit 106 sets a score of the degree of privacy protection according to the feature amount of the object in the image area, and sets a time threshold value suitable for the situation. The time threshold may also be referred to as backgrounding time.

Objects that require a high degree of privacy protection include the human body, objects on which confidential information is written, and the like. Chairs, clocks, and the like are examples of objects that move but have a low degree of privacy protection. For example, the background updating unit 106 sets the stable background time to infinity in the image area with the maximum score. For example, the stable background time is set to 0 in the image region with the lowest score. For example, the background updating unit 106 sets the maximum score in an image region in which a human body is detected. For example, image areas detected as chairs and clocks are set to the minimum score.

Next, the details of the processing in case (B) will be described with reference to FIGS. 3 and 4. FIG. 3A and 4A are images (acquired images) acquired from the image acquisition unit 102, and FIGS. This is the result (recognition result) of drawing on a white image of size. Also, FIGS. 3(c) and 4(c) show stable background times, FIGS. 3(d) and 4(d) show time thresholds, and FIGS. 3(e) and 4(e) show update It shows a later background image.

FIGS. 3A to 3G show the state at the timing when the background image acquisition process according to this embodiment is started. FIG. 3 corresponds to case (A) described above. Typically, case (A) is the timing when the image processing apparatus 101 starts generating the background image or the timing immediately after the existing background image is reset. As described above, the background update unit 106 of this embodiment stores the image acquired from the image acquisition unit 102 in case (A) as it is as a background image. It is the same as the image. Further, as shown in FIG. 3B, no moving body or object is detected here. It is desirable that the original background image does not include the foreground.

FIGS. 4(a) to 3(g) show the state after a certain period of time has elapsed since FIG. 3, and assume that the background image shown in FIG. 3(e) is set. A person 401 and a chair 402 are shown in the obtained image shown in FIG. FIG. 4B shows that a moving object 403 is detected by the moving object detection processing of the moving object detection unit 103, and objects 404 and 405 are detected by the object detection processing of the object detection unit 104. FIG.

FIG. 4(c) shows that the stable background time of the block to which the moving object region detected by the moving object detection process belongs is set (reset) to zero. Also, as shown in FIG. 4C, the stable background time of the block for the non-moving object area is incremented at regular time intervals, increasing from the initial value of 0 to 9. FIG.

FIG. 4D shows the time threshold set by the score calculated from the moving body information detected by the object detection process. The standard time threshold is set to 60 in this embodiment. Since the object 404 is determined to be a human body and the maximum score value is set, the time threshold for the area where the object 404 is located is set to ∞ (infinity). On the other hand, the object 405 was determined to be a chair, and a score lower than the standard was set, so the time threshold for the area where the object 405 is located is set to 9.

FIG. 4(e) shows the background image updated based on the comparison result between the stable background time and the time threshold shown in FIG. 4(c). The stable background conversion time of the area of the chair 402 becomes 9, and the partial image of the area of the chair is synthesized (backgrounded) as the background image because the time threshold of 9 for the area is reached. Since the other areas have not reached the time threshold, they remain the previous background image (Fig. 3(e)).

Next, a method for generating protection area information by the image comparison unit 107 will be described. Protected area information is information indicating the position and shape of an object to be protected in an image. In this embodiment, a protected area is an area whose privacy should be protected.

The image comparison unit 107 extracts a luminance component from the acquired image acquired from the image acquisition unit 102 to generate a brightness image, and similarly generates a brightness image from the background image acquired from the background update unit 106 . Then, the image comparison unit 107 generates a difference image including the absolute value of the difference in pixel units (pixel by pixel) between the luminance image based on the acquired image and the luminance image based on the background image. Then, the image comparison unit 107 determines a pixel having a difference value larger than a predetermined threshold among the pixels (pixels) of the difference image as a pixel to be protected. Then, the image comparison unit 107 connects adjacent pixels to be protected, specifies an area having a certain area (the number of pixels) or more as a protected area, and generates protected area information indicating the position and shape of the protected area.

FIGS. 3(f) and 4(f) show protected area information generated based on the background images of FIGS. 3(d) and 4(d) and the acquired images of FIGS. 3(a) and 4(a). shows the protected area corresponding to . FIG. 3(f) shows that there is no protection area, and FIG. 4(f) shows that the area corresponding to the person 401 as a protection area needs to be protected for privacy. The image synthesizing unit 108 generates a protected image obtained by synthesizing a predetermined image with the protected area indicated by the protected area information of the background image. Note that the predetermined image is a silhouette image generated (anonymized) to enhance anonymity by, for example, filling the protected area corresponding to the position and shape specified by the protected area information with a single-color image. That is, after generating (updating) the background image, the image synthesizing unit 108 sets the specific area (protection area) corresponding to the comparison between the acquired image to be processed and the background image as the protection image (concealment image). Perform processing (also called silhouetted).

FIGS. 3(g) and 4(g) show the image synthesizing unit based on the background images of FIGS. 3(e) and 4(e) and the protected area information of FIGS. 3(f) and 4(f). 108 is the generated protected image. As shown in FIG. 4(f), the chair, which should be turned into the background quickly, is turned into the background so that the actual object can be seen, and the whole body (or part) including the face of the human body, which should be protected for privacy, is hidden. are anonymized so that individuals cannot be identified.

Next, the flow of processing of the image processing apparatus 101 of this embodiment will be described. FIG. 5 is a flowchart for explaining the operation of the image processing apparatus 101 of this embodiment. The CPU 201 of the image processing apparatus 101 implements the processing of FIG. 5 by reading out and executing a program related to the processing of FIG. 5 from the ROM 202 and the RAM 203 . The image processing apparatus 101 of the present embodiment starts the process of FIG. 5 in response to the user's instruction to start the privacy protection process. However, the start timing of the processing in FIG. 5 is not limited to the above example. For example, with regard to images of exhibitions, if the timing at which an important person appears is known in advance, the processing may be started with the arrival of a predetermined time (VIP time) as a trigger. may start processing with the entry of business hours as a trigger.

First, the image acquisition unit 102 acquires an image transmitted from the image output device 120 via the network 110 (S101). When the image output device 120 is an imaging device, it acquires a captured image in real time, and when the image output device 120 is a recording device, it acquires a recorded image (recorded video). Other methods may be used to acquire the image, and the image may be obtained by H.264. If the image is compressed by a known encoding method such as H.264, JPEG, HEVC, or HEIF, the image acquisition unit 102 may decode the image.

The moving object detection unit 103 detects a moving object in the image acquired by the image acquisition unit 102 (acquired image) based on the background image held in the RAM 203 by the background subtraction method. The object detection processing is performed based on the feature amount of (S102). That is, the moving object detection unit 103 detects a moving object region (foreground region) based on the difference between images (acquired image and background image), and detects moving object information (center coordinates (position information) of the moving object and circumscribed rectangle (shape information)) to the background updating unit 106 . Further, the object detection unit 104 detects an object region based on the features in the image, and updates the object information obtained by detection (the center coordinates (position information) of the object, the circumscribed rectangle (shape information), and the score) to the background update unit. 106.

The background update unit 106 performs a background image update process (generation process) based on the moving object information and the object information (S103). More specifically, the background updating unit 106 sets a time threshold value corresponding to a block newly detected as an object region, and sets a stable background time value corresponding to a block not detected as a moving object region to Increment at regular intervals. As a result of incrementing, the background image is updated by replacing the pixel values at the position corresponding to this block using the partial image corresponding to the position of the block in the obtained image where the value of the stable background time is equal to or greater than the threshold. . In addition to the method of replacing 100% of the pixel values, the update may be performed by synthesizing the pixel values of the background image at a predetermined ratio (for example, 40% of the background image and 60% of the partial image of the acquired image). Then, the background update unit 106 saves the updated background image in the RAM 203 and transmits it to the image comparison unit 107 . In this embodiment, an example in which the stable background time is managed for each block is mainly described, but the size and shape of the block are arbitrary.

The image comparison unit 107 generates protection area information based on comparison between the acquired image acquired from the image acquisition unit 102 and the background image acquired from the background update unit 106 (S104). The image comparison unit 107 then transmits the generated protection area information to the image composition unit 108 .

The image synthesizing unit 108 generates a protected image based on the background image obtained from the background updating unit 106 and the protected area information obtained from the image comparing unit 107 (S105). More specifically, the image synthesizing unit 108 performs concealment processing on a partial image at a position corresponding to the protection area indicated by the protection area information in the acquired image (captured image, recorded image). Then, the image synthesizing unit 108 synthesizes the concealment-processed image (mask image, silhouette image) with the background image to generate a protected image. The image synthesizing unit 108 displays the generated protected image on a display via the output unit 109 or outputs it to another external device.

The output unit 109 displays, for example, a protection image (S106). That is, after the background image is generated (after the update), the output unit 109 protects the protected area (specific area) corresponding to the comparison between the acquired acquired image (processing target image) and the background image as a concealed image. display an image. For example, if the image processing apparatus 101 is a PC, a protected image will be displayed on the display.

When the process of step S106 is completed, it is determined whether to continue or end the process of FIG. 6 (S107), and if it is to be continued, the process returns to S101.

FIG. 6 is an example of a GUI (graphical user interface) that operates as the parameter setting unit 105 of this embodiment. come true. A window 600 is a GUI (receiving means) for receiving instructions from the user to set parameters for the object detection unit 104 to execute detection processing, and sets a time threshold (background time) for each object or each feature. can do. The conditions in columns 601 to 605 arranged next to the ID 606 are AND conditions (conditions that are satisfied when all the conditions in columns 601 to 605 are satisfied). A column 601 indicates the type of object to be set and the area identifier, and the user selects from a pull-down menu or freely inputs text. Column 602 is a time threshold value (threshold value for determining whether or not to create a background) corresponding to the object type and area identifier set in column 601 .

A column 603 can set whether or not shape recognition is performed, and a pattern image used for pattern matching for shape recognition. When the user clicks a blank column, the user can refer to a file held in the image processing apparatus 101 or outside, or can draw and set a rough pattern. A column 604 allows setting whether or not to perform color recognition, and pixel values (for example, RGB pixel values) used for color recognition. When the user clicks a blank column, a color palette (not shown) is displayed, or an eyedropper tool for specifying a color in the image (captured image, recorded image) acquired by the image acquisition unit 102 is activated. A column 605 allows setting whether or not to perform further distance measurement, and the distance.

The ID 606 is a unique value for each entry involved in the time threshold, and in this embodiment conditions with a higher ID take precedence over conditions with a lower ID. When the user presses a button 607, an image (captured image, recorded image) acquired by the image acquisition unit 102 is launched in a separate window. ) to set the position and size of the region for which the time threshold is set.

When the user presses a button 608, the parameter setting unit 105 sets the contents set in the window 600 as parameters in the object detection unit 104 and the background update unit 106 (conditions may be discarded by pressing a button 609). . Details of the parameters set by the parameter setting unit 105 include parameters for the object detection unit 104 to execute detection processing (detected shape patterns, colors, and distances), and parameters for the background update unit 106 to update the background. (time threshold and target block).

Here, a supplementary description of each setting in FIG. 6 will be provided. The condition with the ID 606 of "6" sets the time threshold for region B to zero. Since the background image always matches the comparison image in the image area with the time threshold set to "0 seconds", it is never a protected area (not silhouetted). For example, regarding a captured image captured by a surveillance camera installed in a station at an angle of view that includes the railroad, it is necessary to emphasize safety over privacy protection on the railroad, so this setting may be used.

Since the time threshold is set to "infinite (seconds)" for the condition where the ID 606 is "5", the area is always protected (silhouetted). In addition, the condition where the ID 606 is "4" sets a time threshold value of "600 seconds" because the degree of privacy protection for a subject at a distance of 2 m or less is high. Also, for the condition where the ID 606 is "3", a time threshold of "600 seconds" is set as an object with a high degree of privacy protection for the region detected by pattern matching based on the "upper body shape feature" because it is a "human body". is doing. The condition where the ID 606 is "2" is that the area detected based on the "shape feature of the top of the chair" with the color "black" is regarded as an object with a low degree of privacy protection because it is a "chair", and the time threshold is set to "20 seconds". ” is set. In addition to chairs, desks and conference room fixtures can also be applied. The condition with ID 606 of "1" sets a standard time threshold of "60 seconds" for areas that do not meet the conditions with ID 606 of "2 to 6".

As described above, according to this embodiment, the image processing apparatus 101 generates and updates a background image using a plurality of detection units (the moving object detection unit 103 and the object detection unit 104). At this time, since the background time threshold value can be set for each property of the object, it is possible to more easily protect privacy by setting whether or not the object is subject to privacy protection for each property of the object. Specifically, the subject of privacy protection is less likely to be included in the background image, and the subject that is not subject to protection is more likely to be included in the protected image.

It should be noted that, as in the condition of ID "6" in FIG. 6, the area for which the time threshold is set to "0" is set to be displayed in one of the following three patterns in the window 600 in FIG. You may make it possible. The first is a form of "displaying the background image as it is", in which case the captured image or the recorded image is displayed as it is. The second is a mode of "fixing the first image as it is", in which an image corresponding to FIG. 3(a) is displayed. The third is a mode of "making into a blurred image", in which an image is displayed in which blur processing, mosaic processing, or the like is performed on an area for which "0" is set as the time threshold.

In the above-described embodiment, the background image updated by the background updating unit 106 is used by the image synthesizing unit 108 to generate the protected image. You may do so.

FIG. 7 shows an example of a GUI (Graphical User Interface) for setting a background image.

An area 701 is a place for setting the moving object detection unit 103 and the background update unit 106 . Here, either "arbitrary image" or "automatic update" of the above background image can be selected. When an arbitrary image is selected, the user presses a setting button and sets the image by referring to a predetermined image file from the image processing apparatus 101 or an external device. A reduced image of the set image is also displayed in the area 701 .

On the other hand, when the background image is set to "automatic update", the user can set whether or not to use the object detection unit 104 by switching the check box of "human body detection option". In the example of FIG. 7, the human body detection is set to "OFF", so the pattern matching processing load is reduced accordingly. Further, when the background image is set to "automatic update", it is possible to select whether the image acquisition unit 102 acquires the image from the "captured image" or from the "recorded image". As shown in FIG. 1(b), when both the imaging device 120 and the recording device 120 exist on the network 110, the image source can be selected. When the recording device 120 is selected, it is possible to specify the target frame from the video data held by the recording device by the start time and length.

A region 702 is a place for setting a background image to be used by the image synthesizing unit 108 . Here, there are a "set" button for setting an arbitrary image file and a "set live image" button for directly setting a live image captured by the imaging device 120 as a background image. It is also possible to set the background image set in the area 701 to be used as it is.

An area setting 703 displays the window 600 shown in FIG. 6 by pressing a setting button in a setting area related to a background image to be automatically updated in the area 701 . A region 704 allows the user to select a camera that reflects the settings of the regions 701 to 703 . Even if there are a plurality of surveillance cameras on the network 110, similar settings can be easily made. “File output interval” is a setting indicating an interval at which the image composition unit 108 writes an image file (JPEG or HEIF) corresponding to the protected image to a predetermined location in the external memory 204 . The image synthesizing unit 108 can store the image file in parallel with the distribution of the protected image. "File number upper limit" is a setting related to the upper limit number of image files that can be held in the external memory 204, and can be set to an integer value between 1 and 60480. When the upper limit is reached, older files are overwritten.

Also, in the above-described embodiment, the description of updating (overwriting) the time threshold is omitted. . For example, when the image output unit 120 captures an image in Full HD at 60 fps, the image acquisition unit 102 acquires a captured image in Full HD at 10 fps, and sends the acquired captured image to the moving object detection unit 103 and the object detection unit 104. offer. Then, the background update unit 106 updates the time threshold for each frame of the captured image input at 10 fps when the process is started. Specifically, an incremented score (stable background time) is maintained for a region in which the moving object detection unit 103 and the object detection unit 104 have detected the same as in the previous frame. On the other hand, for a region in which the moving object detection unit 103 or the object detection unit 104 detects a difference from the previous frame, the incremented score is reset to 0, and a new time threshold is set based on the detected content. Note that the background updating unit 106 may be updated once every several frames to reduce the processing load. This trigger is specified by the user, or when the moving body detection unit 103 or the object detection unit 104 does not detect an object or a moving body larger than a certain area for a predetermined time.

(Other embodiments)
The present invention supplies software (programs) for realizing one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and one or more processors (CPUs) in the computer of the system or device. , MPU, etc.) reads and executes the program. It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

101 image processing device 102 image acquisition unit 103 moving object detection unit 104 object detection unit 105 parameter setting unit 106 background update unit 107 image comparison unit 108 image composition unit 109 output unit 120 image output device

Claims (10)

a detection means for detecting an object from a captured image captured by the imaging unit;
setting means for setting a time threshold for a region corresponding to the object detected from the captured image based on the feature of the object;
The background image is generated based on the captured image so that the object within the region is reflected in the background image when the time period during which the object within the region within the captured image is stationary reaches the time threshold. updating means for updating the image;
The setting means sets a first value as the time threshold for the area corresponding to the object if the object is subject to privacy protection, and if the object is not subject to privacy protection, sets the time threshold to the object. An image processing apparatus, wherein a second value smaller than the first value is set as the time threshold of the region. 2. The image processing apparatus according to claim 1, wherein said detection means detects said object from said picked-up image by pattern matching based on a specific shape feature or by a specific color. 3. The image processing apparatus according to claim 1, wherein the object of privacy protection is a human body. 4. The apparatus according to any one of claims 1 to 3, further comprising synthesizing means for synthesizing the background image updated by said updating means and an anonymous image obtained by anonymizing a subject of privacy protection in said captured image to generate a privacy protection image. The image processing device according to any one of items 1 and 2. 5. The setting means can set the predetermined area of the captured image so that it is always backgrounded so that the predetermined area of the captured image is not subject to privacy protection. The described image processing device. 5. The setting device according to claim 4, wherein the setting means can set the predetermined area of the captured image so that the predetermined area of the captured image is not always backgrounded, thereby setting the predetermined area of the captured image to be subject to privacy protection. image processing device. The setting means displays the background image updated by the updating means for the predetermined area or performs blurring processing when the predetermined area of the captured image is set so as not to be always backgrounded. 7. The image processing apparatus according to claim 6, wherein either display can be set. 8. The image processing apparatus according to any one of claims 1 to 7, wherein said detection means detects an object at a specific distance or more as an object with a low degree of privacy protection. a detection step of detecting an object from the captured image captured by the imaging unit;
a setting step of setting a time threshold for a region corresponding to the object detected from the captured image based on the characteristics of the object;
a background image based on the captured image such that the object within the area is reflected in the background image when the time period during which the object within the area within the captured image is stationary reaches the time threshold; and an updating step of updating the
In the setting step, if the object is subject to privacy protection, a first value is set as the time threshold for the region corresponding to the object; if the object is not subject to privacy protection, the An image processing method, wherein a second value smaller than the first value is set as the time threshold for the region. the computer,
a detection means for detecting an object from a captured image captured by the imaging unit;
setting means for setting a time threshold for a region corresponding to the object detected from the captured image based on the feature of the object;
a background image based on the captured image such that the object within the area is reflected in the background image when the time period during which the object within the area within the captured image is stationary reaches the time threshold; and updating means for updating the
The setting means sets a first value as the time threshold for the area corresponding to the object if the object is subject to privacy protection, and if the object is not subject to privacy protection, sets the time threshold to the object. A program for operating as an image processing apparatus, wherein a second value smaller than the first value is set as the time threshold of the area.

Images