JP6529526B2 - Background image generation device and object detection device - Google Patents

Description

  The present invention relates to a background image generating device for generating a background image of a space from a three-dimensional model of a background of a predetermined space, and an object for detecting an object in the space by comparing the generated background image with a photographed image. It relates to a detection device.

  There is known a technique for detecting an object such as a person or a suspicious object appearing in a monitoring space by subjecting a photographed image obtained by photographing the monitoring space to background subtraction processing or background correlation processing for the purpose of crime prevention and the like. In these processes, it is necessary to generate a background image free of people and suspicious objects at any time.

  However, it is difficult to generate a background image in a crowded space where many people come and go, such as at event venues, because the frequency of appearance of scenes where people and suspicious objects are not appearing is low.

  Therefore, conventionally, a three-dimensional model (background model) of the background object of the monitoring space is rendered to generate a background image. For example, in the object detection apparatus described in Patent Documents 1 and 2 below, first, the illumination condition of the background model and the semi-fixed object are varied and rendered in a plurality of ways, and the rendering image most similar to the photographed image is used as the background image. elect. Alternatively, the rendered image most similar to the captured image is corrected to be close to the captured image to generate a background image.

Unexamined-Japanese-Patent No. 2016-194778 JP, 2016-194779, A

  However, in the above-described prior art, the generated background image may have an excessive approximation with respect to the captured image due to the influence of an object other than the background appearing in the captured image, which may cause an error in the background image or an object This has been a factor in the decrease in detection accuracy.

  For example, in the case where many people wearing bright clothes are included in the captured image, the captured image may be brightened even if the light is not lit. In such a case, if the rendering image most similar to the captured image is selected as the background image, or if the correction is made to make the rendered image closer to the captured image without restriction, a background image having a higher luminance than the actual lighting condition is generated. As a result, the error increases, making it difficult to detect parts of human clothes or making it easier to detect parts other than people.

  In addition, in the prior art, the range of searching for the lighting conditions and the state of the semi-fixed object is wide, so the processing load for generating the background image is heavy.

  The present invention has been made in view of the above problems, and it is possible to generate a background image based on a three-dimensional model of a background with high accuracy and low load even if an object other than the background appears in the captured image. An object of the present invention is to provide an image generation device and an object detection device capable of detecting an object other than the background with high accuracy using the generated background image.

  (1) The background image generating apparatus according to the present invention is an apparatus for generating a background image at a timing corresponding to the photographing, which is an image of a background object existing in the space with respect to photographing of a predetermined space by a camera. Storage means for storing a background model which is a three-dimensional model of the background and which includes optical feature quantities or three-dimensional coordinate values of each of the background as model parameters, and a predetermined one included in the background Control signal acquisition means for acquiring a control signal for controlling the operation of the device; Model generation means for generating a background model in which the model parameter of the device is changed according to the operation state of the device generated by the acquired control signal Background image generation means for rendering the background model generated by the model generation means on a shooting surface of the camera to generate the background image; Provided.

  (2) The background image generating device described in the above (1) further includes a photographed image acquiring unit that acquires a photographed image obtained by photographing the space with the camera, and the model generation unit is the control signal acquiring unit. When the control signal is acquired, a plurality of the background models having different model parameters in a variation range of the model parameter assumed in advance with respect to the control signal are generated, and the background image generation unit generates the plurality of backgrounds Each of the models may be rendered on the imaging plane to generate a plurality of rendering images, and the background image may be generated using one of the rendering images that has the least degree of difference from the imaging image. .

  (3) In the background image generation device described in (2), the background image generation unit changes the pixel value of the rendered image with the smallest degree of difference within a predetermined limit, thereby capturing the rendered image. The background image may be generated by performing correction close to the image.

  (4) In the background image generation device described in the above (2) and (3), the change range of the model parameter with respect to the control signal is a target range corresponding to the control signal and a transition to the target range. The model generation unit is configured to continuously generate a new value when the change of the model parameter corresponding to the background image is equal to or less than a predetermined threshold and the model parameter is within the target range continuously for a predetermined period or more. The plurality of background models having different model parameters may be generated only for the target range within the change range until the control signal instructing transition to the target range is acquired.

  (5) In the background image generating device described in the above (1), the background image generating means generates a rendered image by rendering the background model generated by the model generating means on the photographing surface, and the rendered image The background image can be generated by performing correction that brings the rendering image closer to the captured image by changing the pixel value of the pixel within a predetermined limit.

  (6) In the object detection device according to the present invention, the camera generates the background image generated by the background image generation device described in the above (1) to (5) in response to the shooting timing of the camera. The object other than the background object is detected in the space in comparison with the photographed image obtained by photographing the space in

  According to the background image generation device of the present invention, even when an object other than the background is shown in the photographed image, it is possible to generate the background image with high accuracy and low load. Further, according to the object detection device of the present invention, it is possible to detect an object other than the background with high accuracy using the background image.

It is a block diagram showing a schematic structure of an object detection device concerning an embodiment of the present invention. FIG. 2 is a schematic block diagram showing an example of configuration of a device whose operation is controlled by a control signal. 1 is a schematic functional block diagram of an object detection device according to an embodiment of the present invention. It is a schematic diagram explaining model control information in case an apparatus is a lighting apparatus comprised with one light. It is a schematic diagram explaining model control information in case an apparatus is a lighting apparatus comprised with three lights. It is a schematic diagram explaining model control information in case an apparatus is an automatic door. It is a schematic diagram explaining the example of production | generation of the background model by a model production | generation means. It is a flowchart which shows the general | schematic operation | movement of the object detection apparatus based on embodiment of this invention. It is a flowchart which shows the general | schematic operation | movement of the object detection apparatus based on embodiment of this invention.

  Hereinafter, embodiments of the present invention (hereinafter, referred to as embodiments) will be described based on the drawings. As an embodiment of the present invention, an example of an object detection apparatus 1 for detecting an object (an object other than a background object) such as a person appearing in a predetermined space (monitoring space) from an image captured by a surveillance camera will be described. Background objects are components constituting a surveillance space, and are permanent objects such as building structures and facilities. On the other hand, objects other than background objects are objects such as people and vehicles that temporarily appear in the monitoring space.

  FIG. 1 is a block diagram showing a schematic configuration of the object detection device 1. The background object that constitutes the surveillance space includes the device 2. The device 2 in the present embodiment is operated based on the control signal among the background objects to change the operation state, and the device 2 causes a change in the image by the operation. The devices 2 present as background objects in the monitoring space are, for example, lighting devices and automatic doors. On the other hand, backgrounds other than the device 2 are, for example, a floor, a wall, a desk, and a manual door.

  The object detection device 1 includes, for example, a background image generation device that generates a background image adapted to a change in brightness due to a lighting device, and detects a region changed due to the appearance of an object by comparing the background image with a photographed image. Specifically, the object detection device 1 includes a device interface unit 3, a camera 4, a storage unit 5, an image processing unit 6, and a user interface unit 7. The background image is ideally an image obtained by capturing only the background, and in the present invention, an image obtained by capturing only the background is simulated by computer graphics.

  The device 2 is connected to the device interface unit 3 of the object detection device 1, and the image processing unit 6 of the object detection device 1 obtains a control signal for controlling the operation of the device 2 from the device 2 via the device interface unit 3. .

  FIG. 2 is a schematic block diagram showing a configuration example of the device 2. The device 2 a shown in FIG. 2A is a lighting device that controls the on / off operation of one lamp 21 by one switch 20. Moreover, the apparatus 2b shown in FIG.2 (b) is a lighting apparatus which controls collectively the lighting / light extinction operation | movement of three lights 21 (lights AC) by one switch 20. FIG. For example, the light 21 is a fluorescent lamp or an incandescent lamp. For example, when the switch 20 is turned on, the light 21 is energized and turned on when the switch 20 is turned on, and when the switch is turned off, the light 21 is stopped and turned off. By the way, lighting equipment such as an office or a factory is often configured in a form like the equipment 2b.

  When the switch 20 is a mechanical switch, the voltage / current applied to the lamp 21 corresponds to a control signal for controlling the on / off operation of the lamp 21. In this case, the devices 2a and 2b may output the electrical signal itself applied to the light 21 as a control signal to the object detection device 1, or separately generate a signal representing the presence or absence of the application from the electrical signal. It may be output to the object detection device 1. For example, when the switch 20 is in the ON state, the devices 2a and 2b have a predetermined high voltage (high level), and when the switch 20 is in the OFF state, the devices 2a and 2b have electric signals that become the predetermined low voltage (low level) Output to the detection device 1. In addition, when the switch 20 is configured by an element such as a transistor that electronically switches ON / OFF, an electric signal that switches ON / OFF of the switch 20 can be extracted as a control signal.

  The device 2 c shown in FIG. 2C is an automatic door of a type in which the door 23 opens and closes in accordance with the output of the human sensor 22. That is, the sensor output signal of the human sensor 22 is a control signal for controlling the opening and closing operation of the automatic door 2c. Specifically, for example, the human sensor 22 outputs a high level voltage while detecting a human, and outputs a low level voltage while not detecting a human. The voltage change from the low level to the high level of the sensor output signal is used as a control signal representing an operation instruction to open and close the door 23. Further, the low level voltage when the door 23 is in the closed state can be used as a control signal representing an operation instruction for maintaining the door 23 in the closed state.

  The image processing unit 6 can recognize from the change of the control signal of the device 2 an opportunity for the operation of the device 2 to shift, that is, an opportunity for the change of the environment (background) of the monitoring space. The change of the control signal can be detected by storing the control signal in the storage unit 5 each time the captured image is acquired and comparing the current control signal with the previous control signal. Alternatively, the change in the control signal can be detected by differentiating the waveform of the control signal from acquisition of the previous captured image to acquisition of the current captured image and performing threshold processing. Further, the image processing unit 6 can recognize the maintenance of the operation of the device 2 from the control signal of the device 2.

  The device interface unit 3 is a control signal acquisition unit that acquires a control signal for controlling the operation of a predetermined device included in the background, and more specifically, is connected to the device 2 and the image processing unit 6. It is a circuit which acquires a control signal and outputs it to the image processing unit 6.

  The camera 4 is a so-called surveillance camera, and is an imaging means for imaging the surveillance space at predetermined time intervals. Specifically, the camera 4 is connected to the image processing unit 6, photographs the monitoring space, generates a photographed image, and outputs the photographed image to the image processing unit 6. For example, the camera 4 is installed on the ceiling of each monitoring space set in the event hall in a fixed state in a field of view over the monitoring space, and images the monitoring space at an interval of 5 frames / second. In the present embodiment, the camera 4 is provided as a photographed image acquisition unit for acquiring a photographed image obtained by photographing a space in the background image generation device of the present invention, but the photographed image acquisition unit is an interface for acquiring a photographed image from the camera 4 It may have only a function.

  The storage unit 5 is a storage device such as a read only memory (ROM) or a random access memory (RAM). The storage unit 5 stores various programs and various data, and inputs and outputs such information from the image processing unit 6. For example, the storage unit 5 stores in advance a background model which is a three-dimensional model of a background object, and camera parameters of the camera 4.

  The image processing unit 6 is configured using an arithmetic device such as a central processing unit (CPU), a digital signal processor (DSP), and a micro control unit (MCU). The image processing unit 6 is connected to the storage unit 5 and functions as each unit described later by reading and executing a program from the storage unit 5. Further, the image processing unit 6 reads various data from the storage unit 5 and stores various data in the storage unit 5. The image processing unit 6 is also connected directly or indirectly to the device 2 and the camera 4 and acquires a control signal from the device 2 and a photographed image from the camera 4 to generate a background image, detect an object, etc. Do. In addition, the image processing unit 6 is also connected to the user interface unit 7, shows the detection result and the like to the user via the user interface unit 7, and receives an operation input from the user.

  The user interface unit 7 is a user interface device including a keyboard, a mouse, a display and the like, and is connected to the image processing unit 6 to display data inputted from the image processing unit 6 and to display data inputted by the user. Output to For example, the user interface unit 7 is used by a user such as a security guard, and is used to display an object detection result, to display a warning of a failure of the device 2, to input a recovery signal of the device 2, and the like.

  FIG. 3 is a schematic functional block diagram of the object detection device 1. The device interface unit 3 functions as a control signal acquisition unit 30, the camera 4 functions as a photographing unit 40, and the storage unit 5 functions as a background model storage unit 50 or the like. Further, the image processing unit 6 functions as a model generation unit 60, a background image generation unit 61, an object detection unit 62, and the like. The user interface unit 7 functions as an output unit 70 that outputs the detection result of the object detection unit 62.

  The background image generation device used in the object detection device 1 includes an apparatus 2, a control signal acquisition unit 30, an imaging unit 40, a background model storage unit 50, a model generation unit 60, and a background image generation unit 61.

  The background model storage means 50 is a three-dimensional model of a background present in the monitoring space, and stores a background model including optical feature quantities or three-dimensional coordinate values of each background as model parameters.

  The background model consists of the information necessary to generate computer graphics of the surveillance space composed of background objects. Specifically, the background model is the position, posture, three-dimensional coordinate value represented by a three-dimensional shape and the color, texture, and reflection characteristics of the surface of each background object in an XYZ coordinate system that simulates a monitoring space. The data is stored in the background model storage means 50 in association with the corresponding background object number including the data.

  The more accurate the background model, the better. For example, each part with different color, texture, and reflection characteristics such as a door plate and door knob for a door and a top plate and a leg for a desk is defined as a background object It is preferable to store the model. On the other hand, it is preferable to define each surface as a background since a large background such as a wall has a large difference in distance from the light source on each surface.

  The model parameters constituting the background model are predefined, and appropriately updated according to the real environment of the monitoring space. In particular, regarding the device 2 that causes a change in the image due to the operation, the model parameter is appropriately set to a specific value according to the operation state, and is used to express the change in the operation state of the device 2 in the background image.

  For the light source, the background model includes the position of the light source in the XYZ coordinate system that simulates the monitoring space, and the illumination characteristics represented by the light distribution of the light source, the color temperature, and the like. Lighting equipment is also a light source. The light source may include the sun. A plurality of illumination conditions for the monitoring space can be simulated by setting the on / off state of each light source for the light source in the background model and calculating the output of the light source whose illumination state is set as the illumination characteristic.

  That is, the background model storage unit 50 is configured to calculate three-dimensional coordinate values of each of a plurality of background objects that are components constituting a predetermined space that the camera 4 is to capture and an optical feature amount (color and texture of each background object). , Reflection characteristics, and illumination characteristics of a light source illuminating the space).

  Here, the background model includes information on the operation of each device 2 in addition to the model parameters, and the information is referred to as model control information of the device 2 here.

  FIG. 4 is a schematic diagram for explaining model control information in the case where the device 2 is a lighting device configured by one light, and shows items of the model control information and their value ranges. The lighting apparatus shown in FIG. 4 corresponds to the apparatus 2a of FIG. 2 (a).

  The illumination intensity corresponds to the brightness of the illumination device. The illumination intensity is a model parameter for the illumination device, and its value range is 0% where 0% does not emit light when the switch is OFF, and 0 to 100% where 100% is the brightest state when the switch is ON. Here, the illumination intensity corresponding to 100% of the lighting device may be determined by actual measurement, or may be determined based on the catalog specifications of the lighting device or the like.

  The illumination intensity estimation range (parameter range) is an illumination when the model generation unit 60 generates a background model in which the illumination intensity as the model parameter of the illumination device is changed according to the operation state of the illumination device generated by the control signal. It is a range in which the intensity is changed. The illumination intensity estimation range is basically set according to the operating state of the illumination device. For lighting devices, for example, the lighting state (switch is turned on and the illumination intensity reaches a target range and is stable), the light-off state (switch is turned off), and the lighting device is in transition to the lighting state. It is possible to define the state (the state in which the illumination intensity changes towards the target range after the switch is turned on). The illumination intensity estimation range is set, for example, as 90% or more and 100% or less (90 to 100%) as a target range corresponding to the lighting state. The reason why the target range has a width in this way is that the actual illumination intensity does not necessarily have a value of 100% when the switch is ON, and the actual illumination intensity is 100% due to aging deterioration of the lighting equipment, etc. In some cases. Further, 0% is set for the unlit state, and 0% to less than 90%, which is defined as a range in the middle of transition, and 0 to 100%, which is the target range, are set.

  In addition to the fact that the illumination intensity can be estimated at high speed, which will be described later, by restricting the change range of the illumination intensity which is a model parameter by the illumination intensity estimation range, the illumination erroneously out of the range corresponding to the assumed operating condition It is possible to avoid the risk that the intensity is estimated. Then, by avoiding the estimation error of the illumination intensity, it is possible to make the erroneous extraction of the object less likely to occur in the background subtraction process using the background image generated based on the estimated illumination intensity.

  The previous illumination intensity (previous parameter) is used, for example, to determine whether the illumination intensity has become constant or the like when the operating state is “during transition”. Specifically, in the process repeated for each captured image in the operation described later, the latest estimated value is stored as the previous illumination intensity, and is read as the previous estimated illumination intensity in the next processing cycle. It is used.

  The operation state is set according to the control signal to one of the above-described "lighting", "lighting off", and "transition". In addition, the estimated illumination intensity is added to the setting as necessary. When the switch is off, "off" is stored in the operating state. Immediately after the switch is turned on, "during transition" is stored in the operating state as a period until the estimated illumination intensity reaches the target range and becomes stable. When the estimated illumination intensity is stabilized in the target range, “lighting” is stored in the operating state as the lighting device is sufficiently lighted and the transition is completed.

  The stability counter is a counter used to determine the operation state from “transitioning” to “lighting”, counts the number of times the illumination intensity has been stably estimated, and the count value is equal to or more than a predetermined value. If it does, the operation state is determined to be "lighting". On the other hand, the instability counter is a counter for determining that the lamp is not stable for a certain period of time, that is, the lamp is flickering. Specifically, when the difference between the newly estimated illumination intensity and the previous illumination intensity during transition is less than or equal to a predetermined threshold, the value of the stability counter is incremented, and conversely, the difference is greater than the threshold. If so, the value of the instability counter is incremented.

  The defect detection flag is set to ON when a defect such as no illumination, dark illumination, or flickering occurs when the illumination switch is turned ON, and notifies the administrator of the appliance through the user interface unit 7 It is a flag for

  FIG. 5 is a schematic diagram for explaining model control information in the case where the device 2 is a lighting device configured by three lights A to C, and shows items of the model control information and their value ranges. The lighting apparatus shown in FIG. 5 corresponds to the apparatus 2b of FIG. 2 (b).

  With regard to the device 2b, for example, the operating state of the device 2b is turned on if all the lights (lights A to C) are turned on, and conversely the operating state of the device 2b is turned off if all the lights are off If any of the lamps is in transition, the operating state of the device 2b can be defined as being in transition. Further, regarding the device 2b, among the model control information shown in FIG. 4, the illumination intensity, illumination intensity estimation range, previous illumination intensity, stability counter, instability counter, and malfunction detection flag are as shown in FIG. It is held. As described above, since the parameters are held for each lamp, it is possible to estimate the optimum illumination intensity even when there is a variation in brightness among the individual lamps, and it is possible to improve the accuracy of background image generation. In addition, since the failure detection flag is held for each light, it becomes easy to immediately identify the repair / replacement location when the failure occurs.

  FIG. 6 is a schematic view for explaining model control information in the case where the device 2 is an automatic door, and shows items of the model control information and their value ranges. The automatic door shown in FIG. 6 corresponds to the device 2c of FIG. 2 (c).

  The opening width is the width of the portion where the door 23 is open. The opening width is a model parameter for the automatic door 2c, and its value range is 0 to 100% where 0% is the state in which the door 23 is completely closed and 100% is the completely open state.

  The opening width estimation range (parameter range) is a range in which the opening width is changed when the model generation unit 60 generates a background model in which the opening width of the automatic door 2c is changed. The opening width estimation range is basically set according to the operating state of the automatic door 2c. The operation state of the automatic door 2c is, for example, a closed state (a state in which the human sensor 22 does not detect a person and a door 23 is closed), and a state in which it is opened or closed (a human sensor 22 detects a person and the door 23 is detected). Can be determined in the order of opening and closing). The opening width estimation range is set to, for example, 0 to 1% for the closed state. Note that the open state corresponds to the transition completion state while the open / close state is the transition state. The opening width estimation range of the open / close state is set to a range obtained by combining the target range and the transition midway until reaching the target range. Specifically, for the open / close state, 0% to 100%, which is a combination of 1% to 100% or less, which is defined as a transition midway, and 0 to 1%, which is a target range, is set.

  The previous opening width (previous parameter) is used, for example, to determine whether or not the opening width has become constant when the operating state is “during opening and closing”. Specifically, similarly to the previous illumination intensity described above, the latest estimated value is stored in the previous opening width and used in the next processing cycle.

  The operating state is set based on the estimated opening width as to whether the open / close state of the automatic door is “closed” or “opened” described above.

  The stability counter and the instability counter are basically the same as in the case of the lighting apparatus described above. Specifically, the stability counter counts the number of times that the opening width is stably estimated, and when the count value becomes a predetermined value or more, the transition completion of the opening / closing operation is determined, while the instability counter has the opening width It is used to determine a defect that is not stable for a certain period of time or more.

  The defect detection flag may cause, for example, an event that the automatic door does not open even if the human sensor detects a person, or an event that the automatic door does not close even if the human sensor does not detect a person It is a flag to be turned ON when notified and to notify the administrator of the appliance or the like through the user interface unit 7.

  Here, for the device 2 in which the defect detection flag is set to ON, the parameter range (illumination intensity estimation range, opening width estimation range) may be set in a range different from the range according to the above-described operation state . For example, the illumination intensity of the degraded lighting device may saturate at a value lower than the target range for changes in control signal from OFF to ON. In the present embodiment, the defect detection flag is turned ON for the device 2 whose operation state is stable in the middle of the transition in this way, and the parameter range of the device is, for example, the illumination intensity or the opening width reached until the defect detection. The upper and lower limits of ± 1% are set as the center. The parameter range can be set to 0 to 100%, for example, for an unstable failure of the device 2 such as flickering in which the lighting device repeatedly turns on and off.

  The model generation unit 60 generates a background model in which model parameters of the device 2 are changed according to the operation state generated by the control signal in the device 2. As a result, a background model imitating a space when the device 2 is in the operating state generated by the control signal is generated. The model generation unit 60 outputs the generated background model to the background image generation unit 61.

  In the present embodiment, when the control signal acquisition unit 30 acquires a control signal, the model generation unit 60 generates a plurality of background models having different model parameters in a change range of model parameters assumed in advance for the control signal. Do.

  When the device 2 is a lighting device, the model generation unit 60 changes the lighting condition of the lighting device as the model parameter according to the control signal. Here, the model parameter is the illumination intensity. For example, the model generation unit 60 changes the illumination intensity in 1% steps to generate a background model. Specifically, when the operating state is in transition from off to on, the model generation unit 60 changes the illumination intensity in 1% steps in the range of 0 to 100% set in the illumination intensity estimation range. The plurality of background models generated in this case are referred to as in-transit models. When the operating state is the lighting state, the model generation unit 60 changes the illumination intensity in steps of 1% in the range of 90 to 100% set in the illumination intensity estimation range. The plurality of background models generated in this case are referred to as transition completion models. When the operating state is the unlit state, the model generation unit 60 generates a background model with the illumination intensity set to 0%. In the present embodiment, no definition is made during the transition from lighting to lighting off. That is, when a change from lighting to turning off is detected from the control signal, the state is turned off without going through the transition. That is, when the control signal changes from ON to OFF, the model generation unit 60 does not generate a model during transition, and generates only a transition completion model with an illumination intensity of 0%. However, for example, a transitioning model in which the illumination intensity is set in the range of 0 to 100% may be generated as the transition from on to off is defined similarly to the transition from off to on.

  If the device 2 is a movable device such as an automatic door, the model generation unit 60 changes the three-dimensional coordinate values, which are used as model parameters for the movable device, according to the control signal. For example, for the automatic door, the opening width is used as a model parameter as described above. For example, the model generation unit 60 changes the opening width in 1% steps to generate a background model. Specifically, when the operating state is open or closed, the model generation unit 60 changes the opening width in 1% increments in the range of 0 to 100% set in the opening width estimation range, and the in-transition model Generate Further, when the operating state is in the closed state, the model generation unit 60 generates a transition completion model with the opening widths of 0% and 1%.

  Here, when the operation of a plurality of devices is controlled by one control signal, the model generation unit 60 generates a background model for the combination of the operation states of the individual devices. For example, in a lighting apparatus in which the lighting / extinguishing operation of the two lights A and B is collectively controlled by one switch, the model generation unit 60 determines the illumination intensity of the lights A when the operating state of both lights is on. For example, 121 background models are output combining 11 model parameters set in 1% steps in the range of 90 to 100% and 11 model parameters set similarly for the illumination intensity of the lamp B.

  FIG. 7 is a schematic view for explaining an example of generation of a background model by the model generation means 60. As shown in FIG. FIGS. 7A to 7C show examples of model generation for the devices 2a to 2c in FIGS. 2A to 2C, respectively. FIGS. 7 (a) to 7 (c) each show a horizontal direction as a time axis, and show examples of changes in the operation state when time passes rightward.

  With regard to the device 2a of FIG. 2A, as shown in FIG. 7A, when the switch 20 of the lighting device is changed from OFF to ON, the control signal acquisition unit 30 changes the state of the lighting device from off to on The control signal to be transferred is acquired, and the model generation unit 60 changes the illumination intensity, which is a model parameter of the illumination device, in the range of 0-100% set in the illumination intensity estimation range in the transition state, and is in transition Generate multiple background models to be models.

  When the illumination intensity is increased and turned on, the model generation unit 60 changes the illumination intensity in the range of 90 to 100% set in the illumination intensity estimation range, and generates a plurality of background models serving as a transition completion model.

  When the switch of the lighting device changes from ON to OFF, the control signal acquisition unit 30 acquires a control signal to shift the state of the lighting device from lighting to lighting off, and the model generation unit 60 calculates the illumination intensity of the lighting device. Generate a transition completion model with 0%.

  As for the device 2b of FIG. 2 (b), as shown in FIG. 7 (b), the model generation means 60 changes the illumination intensity of each of the lamps A to C in the same manner as one lamp of FIG. 7 (a). Generate a background model.

  With regard to the device 2c of FIG. 2C, as shown in FIG. 7C, when the output of the human sensor at the automatic door changes from the undetected state of the person to the detected state, the control signal acquisition means 30 The control signal to shift the state of the automatic door to the open / close state (transition state) is acquired, and the model generation unit 60 sets the opening width which is the model parameter of the automatic door to the opening width estimation range for the transition state. Change in the range of 0 to 100%, and generate multiple background models that become models during migration.

  That is, when the control signal acquisition unit 30 acquires the control signal for causing the state of the device 2 to transition, the model generation unit 60 is in the middle of transition to the target range corresponding to the control signal related to the operation state of the device 2. A model parameter is set with a parameter range that matches the range, and the in-transition model is generated as a background model that simulates the monitoring space in the in-transition state.

  In addition, after the transition of the operation state of the device 2 is completed, the model generation unit 60 continues to obtain model parameters in the middle of the transition until the control signal instructing the transition to the new target range is acquired. The generation of the background model is omitted, the model parameters are set only in the target range, and the transition completion model is generated as a background model imitating the monitoring space in the transition completion state. By doing this, the model generation unit 60 can output the background model limited to the range corresponding to the control signal from the device 2.

  The background image generation unit 61 virtually renders the background model generated by the model generation unit 60 on the photographing surface of the camera 4 based on the camera parameters stored in advance in the storage unit 5 to generate a background image. The camera parameters are internal parameters and external parameters of the camera 4, and include focal length, resolution (the number of pixels in the width direction, the number of pixels in the height direction), lens distortion and the like in addition to the photographing position and orientation. The background image generation unit 61 outputs the generated background image to the object detection unit 62.

  Specifically, first, the background image generation unit 61 derives the photographing surface of the camera 4 in the XYZ coordinate system of the background model from the camera parameters. Next, the background image generation means 61 generates ray data according to the position and the illumination characteristic of the light source included in the background model. At that time, for the illumination device which is the device 2 among the light source, the presence or absence and output of the ray data are adjusted based on the set illumination intensity for each background model. Subsequently, the background image generating means 61 adjusts the change in color and output of the ray data due to the reflection on the background according to the color, texture and reflection characteristics of the background. The background image generation unit 61 sets ray data of direct light and reflected light reaching the imaging surface from the light source to each pixel value of the imaging surface to generate a background image.

  As described above, the model generation unit 60 can generate a plurality of background models for the monitoring space at a certain timing in response to the control signal of the device 2. When a plurality of background models are generated corresponding to a certain imaging timing of the camera 4, the background image generation unit 61 renders each of the plurality of background models based on the camera parameters to generate a plurality of rendered images. Then, the background image generation unit 61 generates a background image using one of the rendered images that has the least degree of difference from the photographed image at the photographing timing of the camera 4.

  For example, the background image generation unit 61 calculates, as the degree of difference, the difference between the representative feature amount of the pixel value of the captured image and the representative feature amount of the pixel value of the rendering image. The representative feature amount can be, for example, an average pixel value. Alternatively, the average pixel value and the variance of the pixel values may be used as the representative feature amount. Also, the pixel values may be colors or gray values.

  In addition, it is desirable to calculate the representative feature amount excluding an area (such as a person area) in which the area other than the background is captured in the captured image. Therefore, the background image generation unit 61 calculates a representative feature amount using, for example, pixels in an evaluation area predetermined at a height of 2 m or more as an area where a person can not exist.

  Alternatively, for example, the background image generation unit 61 calculates the variance of pixel values of the region of the background object for each background object in the rendering image, and calculates the representative feature amount using pixels in the region where the variance is less than a predetermined value. Do.

  Furthermore, the background image generation unit 61 changes the pixel value of the rendering image with the smallest degree of difference within the predetermined limit for the device 2, thereby performing correction to bring the rendering image closer to the photographed image to generate a background image. Do.

  The reason for making this correction will be described. The space taken by the camera 4 is generally a complex structure composed of a plurality of three-dimensional objects. Therefore, the pixel value change of the background image to the change of the lighting condition is influenced by the unevenness of each background, the direction and distance to the light source of each background, the material of each background, multiple reflections between backgrounds, etc. It is also affected by objects other than the background, such as the person appearing in.

  Therefore, the pixel value change of the background image is not uniform but complex non-linear change in the entire background image. By using a background image generated based on a background model holding the three-dimensional coordinate values of a plurality of background objects, the effects of unevenness, orientation, distance, material, and multiple reflections of each background object can be reproduced relatively accurately. It is difficult to reproduce with the accuracy required for object detection processing. In addition, it is also difficult to reproduce the effects other than the background.

  Therefore, the background image generation unit 61 performs primary approximation by selecting a rendering image of illumination conditions that matches the captured image, and obtains a highly accurate background image by performing correction that brings the selected rendering image closer to the captured image. .

  The background image generation unit 61 corrects the pixel value of each pixel in the rendering image selected by the difference degree minimum according to the pixel value of the pixel in the photographed image. Specifically, for each pixel, the background image generation unit 61 brings the pixel value of the rendering image at the pixel close to the pixel value of the captured image within the range of the preset correction limit. For example, the lower limit and the upper limit are determined for each pixel as the correction limit, and each pixel is corrected within a range of pixel values which is equal to or greater than the lower limit and equal to or smaller than the upper limit. For example, the difference between the pixel value of the photographed image at each pixel and the pixel value of the rendering image is calculated, and the difference is added to the pixel value of the rendering image at the pixel to obtain a corrected pixel value. In addition, when the correction pixel value is below the lower limit value, the correction pixel value is replaced by the lower limit value, and when the correction pixel value is above the upper limit value, the correction pixel value is replaced by the upper limit value.

  At this time, the correction can be performed with high accuracy by selectively correcting the pixels excluding the area (such as the human area) where the areas other than the background appear in the photographed image. For that purpose, the background image generation means 61 compares the rendered image selected by the difference degree minimum with the photographed image for each local, detects a difference area having a difference degree of not less than a predetermined threshold T1, and The correction is performed on the pixels of the portion other than the difference area.

  Also, the background image may include minor errors in the texture in the background model. Therefore, the background image generation unit 61 performs comparison in units of small areas in which a plurality of neighboring pixels are put together. The small area may be each block obtained by dividing the photographed image into a grid, but it is desirable to be a super pixel in order to detect the shape of a person or the like with higher accuracy. That is, the background image generation unit 61 divides the captured image into a plurality of small regions in which neighboring pixels having similar pixel values are grouped, and represents the representative feature amount of the captured image and the representative feature amount of the selected rendering image for each small region. And the small area whose degree of difference is equal to or greater than the threshold T1 is detected as a different area. The representative feature amount in this case can be, for example, the average pixel value of each small area and the variance of the pixel values. Alternatively, the average pixel value may be used as the representative feature amount. Also, the pixel values may be colors or gray values.

  The lower limit of the correction described above is the pixel value of the rendered image at an illumination intensity 1% weaker (0% if the illumination intensity is less than 0%) than the illumination intensity at the time of generating the selected rendering image. The pixel value of the rendering image can be set to an illumination intensity that is 1% stronger (100% when the illumination intensity exceeds 100%) than the illumination intensity when the selected rendering image is generated. Here, when the rendering image of the illumination intensity giving the lower limit and the upper limit of the correction pixel value is an illumination intensity not included in the background model output by the model generation unit 60, rendering of the illumination intensity is performed again to the model generation unit 60. Output an image.

  Alternatively, the lower limit and the upper limit may be determined by multiplying the pixel value of each pixel in the selected rendering image by a preset ratio, or subtracting and adding a predetermined value.

  For rendered images in which model parameter changes of a plurality of lighting devices are combined, the pixel values of the rendering images of which the lighting intensity of each lighting device is 1% weaker than the lighting intensity at the time of generating the selected rendering image The lower limit may be set, and the upper limit may be set to the pixel value of the rendering image having an illumination intensity that is 1% stronger than the illumination intensity when the selected rendering image is generated.

  As described above, the background image generated by the background image generation unit 61 is out of the range corresponding to the assumed operation state by limiting the change range of the model parameter with the parameter range when generating the background model. It is possible to avoid the risk of becoming a thing. That is, since the background model generated by the model generation means 60 is limited to the change of the range according to the control signal from the device 2, it is excessively close to the photographed image due to the influence of objects other than the background shown in the photographed image. It is possible to suppress a defect that a background image is generated.

  For example, the background model is limited to a lighting intensity of 0% even when the photographed image is bright because a plurality of persons dressed in white clothes simultaneously enter the monitoring space with the lighting equipment turned off. Be done. Therefore, no rendering image other than 0% illumination intensity is selected. In addition, no correction deviates from the illumination intensity of 0%. Therefore, it is possible to prevent an error in which a background image excessively brighter than the actual illumination condition is generated due to the influence of a person appearing in the photographed image.

  In addition, even if the photographed image is dark because a plurality of persons dressed in black are simultaneously entering the monitoring space while the lighting device is on, the background model has an illumination intensity of 90 to 100%. Limited to Therefore, a rendering image with an illumination intensity of less than 90% is not selected. In addition, correction that deviates from the range of the illumination intensity of 90 to 100% is not performed. Therefore, it is possible to prevent an error in which a background image excessively darker than the actual illumination condition is generated due to the influence of a person appearing in the photographed image.

  Also, by limiting the range of change of the background model, it is possible to reduce the load of rendering processing and the like.

  The object detection means 62 compares the background image generated by the background image generation means 61 from the background model corresponding to the photographing timing of the camera with the photographed image of the monitoring space photographed by the camera at the photographing timing. Objects in the surveillance space. An object other than a background object is an object such as a person who appears in the monitoring space, which is captured in a captured image but is not modeled in the background model and does not appear in the background image.

  As mentioned above, the background image generated from the background model may include minor errors in the texture in the background model. Therefore, the object detection means 62 performs comparison in small area units in which a plurality of neighboring pixels are put together. The small area may be each block obtained by dividing the photographed image into a lattice, but it is desirable to be a super pixel in order to detect the shape of a person or the like with higher accuracy.

  That is, the object detection means 62 divides the photographed image into a plurality of small areas in which neighboring pixels having similar pixel values are grouped, and the degree of difference between the representative feature quantity of the photographed image and the representative feature quantity of the background image for each small area. Is calculated, and it is determined that an object other than the background object exists in a small area where the degree of difference is equal to or greater than a predetermined threshold value T2.

  The representative feature amount can be, for example, the average pixel value of each small area and the variance of the pixel values. In this case, a weighted sum obtained by weighting and adding the difference between the average pixel values calculated from each of the photographed image and the background image and the difference between the pixel value dispersions can be used as the dissimilarity. . Alternatively, depending on the environment, only the average pixel value or only the edge density may be used as the representative feature amount. Also, the pixel values may be colors or gray values.

  Here, in the case where a person or the like who wears clothes having a color similar to that of the background object is present in front of the background object, a part of the detection target object may fail to be detected with only the representative feature value of the pixel value. Therefore, in addition to the detection of the small area by the threshold T2, the object detection means 62 is a small area which is at least the threshold T4 set lower than the threshold T2 in the small area where a significant edge difference equal to or greater than a predetermined threshold T3 is detected. Also, it is determined that there is an object other than the background object. As described above, by evaluating both the feature value of the pixel value and the feature value of the edge, a detection target object having a similar degree of complexity to the background object in front of the background object or a detection having a similar color to the background object Even if there is a target object, highly accurate detection is possible.

  Next, the operation of the object detection device 1 will be described. FIG. 8 and FIG. 9 are flowcharts showing the general operation of the object detection device 1.

  When acquiring the photographed image from the camera 4 (step S1), the image processing unit 6 acquires the control signal of the device 2 from the device interface unit 3 which is the control signal acquisition unit 30 (step S2). That is, the control signal of the device 2 is acquired basically in synchronization with the photographing timing by the camera 4.

  The image processing unit 6 operates as the model generation unit 60, and determines whether the control signal has changed since the previous acquisition (step S3). When it is determined that a change has occurred (in the case of YES at step S3), the model generation means 60 sets "during transition" as the operation state of the device 2 (step S4). Specifically, if the device 2 is a lighting device, the operating state in the model control information of the device of the background model storage unit 50 is “for the change of the control signal when the switch 20 is switched from OFF to ON. “Under transition” is set, and the illumination intensity estimation range is set to 0 to 100%. In addition, if the device 2 is an automatic door, the control state is "opened" when the output of the human sensor 22, which is a control signal, changes from a low level voltage when not detected to a high level when detected. Is set, and 0-100% is set to the opening width estimation range. Further, in conjunction with the setting of “transitioning” (or “opening and closing”) of the operation state in step S4, the model generation unit 60 resets the count value of the stability counter and the instability counter to 0, and the defect detection flag Set to OFF. As described above, in the present embodiment, the transition from ON to OFF of the lighting apparatus is not defined as being in transition, so in this case, the model generation unit 60 sets “off” to the operation state, and the illumination intensity Set the estimated range to 0%.

  On the other hand, when it is determined that the control signal has not changed (in the case of NO at step S3), the model generation means 60 skips step S4.

  When there are a plurality of devices 2 as background objects, the model generation unit 60 performs the processes of steps S2 to S4 for each device.

  After steps S2 to S4, the model generation unit 60 reads the background model from the background model storage unit 50 of the storage unit 5, confirms the state of the device 2 (S5, S6), and generates the background model according to the state (S7, S8, S9).

  Specifically, the model generation unit 60 reads, for example, the parameter range of the model control information included in the background model, the operation state, and the defect detection flag regarding the state of the device 2. Then, with regard to the device 2 whose operation state is in transition (or during opening and closing) and the defect detection flag is OFF (YES in step S5 and NO in step S6), the model generation unit 60 The model parameter of the device 2 is changed within the parameter range to generate a model during transition (step S7).

  For example, if the lighting apparatus is in transition and the defect has not yet been detected, the transition to the operating state is made, the illumination intensity estimation range is set to 0 to 100%, and the defect detection flag is set to OFF. ing. The model generation means 60 changes the illumination intensity sequentially to 0%, 1%,..., 100% according to the illumination intensity estimation range, and generates a transitioning model according to the illumination device.

  Further, for example, if the automatic door is in transition and the defect is not detected yet, the transition state is in transition to 0 to 100% in the estimated opening width range and OFF is set in the defect detection flag. It is done. The model generation unit 60 sequentially changes the opening width of the automatic door to 0%, 1%,..., 100% according to the opening width estimation range to generate a model during transition.

  With regard to the device 2 whose operation state is in transition and a failure is occurring (YES in step S5 and YES in step S6), the model generation unit 60 sets the model parameters of the device 2 within the parameter range. To generate a failure model (S8).

  For example, if the lighting apparatus is in a stable failure (deterioration) with a lighting intensity of 85%, the device 2 is "in transition" to the operating state, 84 to 86% for the lighting intensity estimation range, and for the failure detection flag ON is set. The model generation unit 60 sequentially changes the illumination intensity to 84%, 85%, and 86% in accordance with the illumination intensity estimation range, and generates a failure model according to the illumination intensity estimation range.

  Also, for example, if the device 2 is a lighting device in which an unstable defect of lighting intensity (repetition of light on / off) is occurring, the operation state is basically “during transition” and the lighting intensity estimation range is 0 to 100%, the defect detection flag is set to ON. The model generation unit 60 sequentially changes the illumination intensity to 0%, 1%,..., 100% according to the illumination intensity estimation range, and generates a failure model according to the illumination device.

  For example, if the device 2 is an automatic door in which a stable failure (stop) occurs with an opening width of 50%, the operating state is "during opening and closing", the open width estimation range is 49 to 51%, and a failure detection flag Is set to ON. The model generation unit 60 sequentially changes the opening width to 49%, 50%, and 51% according to the opening width estimation range for the automatic door, and generates a failure model.

  Further, for example, if the device 2 is an automatic door in which an unstable failure (open and close) of the opening width is occurring, the operating state is basically "during opening and closing" and the opening width estimation range is 0 to 0. 100%, the defect detection flag is set to ON. The model generation unit 60 sequentially changes the opening width to 0%, 1%,..., 100% according to the opening width estimation range for the automatic door, and generates a failure model.

  For the device 2 in which the state of the device 2 is not in transition (in the case of NO at step S5), the model generation unit 60 changes the model parameter of the device 2 within the parameter range to generate a transition completion model. (Step S9).

  For example, if the device 2 is a lighting device whose transition to the lighting state is completed with a lighting intensity of 99%, “lighting” is set in the operation state, and 90 to 100% is set in the lighting intensity estimation range. The model generation unit 60 sequentially changes the illumination intensity to 90%, 91%,..., 100% according to the illumination intensity estimation range, and generates a transition completion model according to the illumination device.

  Further, for example, when the device 2 is a lighting device that has completed transition to the light-off state, “off” is set in the operation state, and 0% is set in the illumination intensity estimation range. The model generation unit 60 generates a transition completion model with the illumination intensity set to 0% according to the illumination intensity estimation range for the illumination device.

  For example, if the device 2 is an automatic door that has completed transition to the closed state with an opening width of 1%, “closed” is set as the operating state, and 0-1% is set as the opening width estimation range. The model generation unit 60 sequentially changes the opening width to 0% and 1% according to the opening width estimation range for the automatic door, and generates a transition completion model.

  When there are a plurality of devices 2 in the background, the model generation unit 60 changes the model parameters of each device according to the operation state etc. as described above, and combines the model parameters of the plurality of devices 2 to generate a background model Do.

  As described above, the model generation unit 60 sets model parameters within a parameter range previously assumed for the control signal to generate a background model. The model parameters for one device 2 can be set in a plurality of ways in the parameter range, and a plurality of devices 2 can exist in the background, so basically, a plurality of operation states of the device 2 or a group of devices are different. A background model is generated. However, for example, when there is only a closed automatic door in the background, there may be a case where only one operation state is generated as the background model.

  When the background model is generated, the image processing unit 6 operates as the background image generation unit 61, and the image generation unit 40 basically performs each of a plurality of background models generated by the model generation unit 60 according to the operation state of the device 2. The rendering image is generated by rendering on the coordinate system corresponding to the imaging plane of the camera 4 (step S10).

  The background image generation unit 61 compares each rendering image with the captured image to calculate the degree of difference, and selects the rendering image with the smallest degree of difference (step S11). Although the selected rendering image can be used as a background image, in the present embodiment, the rendering image is subjected to correction processing to obtain a background image (steps S12 and S13).

  Specifically, in step S12, the background image generation unit 61 determines the correction limit of each pixel of the rendering image based on the model parameter corresponding to the background model that is the basis of the selected rendering image to be corrected. Do. Then, in step S13, the background image generation unit 61 performs correction in which the pixel value of each pixel of the selected rendering image approaches the pixel value of the corresponding pixel of the photographed image within the correction limit.

  When the background image is generated, the image processing unit 6 operates as the object detection unit 62, and detects the area of the object from the photographed image (step S14). Specifically, the object detection means 62 performs background subtraction processing between the photographed image and the background image obtained in the correction processing of step S13 to extract a change area, and for example, a range predetermined as the size of a person A change area having an inner size is detected as an object area.

  When an object such as a person or the like other than the background is detected in step S14, the image processing unit 6 fills in the object area with a single color, for example, to generate a privacy mask, and a mask image obtained by superposing the privacy mask on the photographed image It is output to the output means 70 to display the image. On the other hand, when the object is not detected, the image processing unit 6 displays the photographed image as it is.

  Next, the image processing unit 6 operates again as the model generation unit 60 to determine and update the state of the device 2 (steps S15 to S29). Hereinafter, this process will be described.

  The model generation means 60 confirms whether the operation state of the device 2 is "in transition" (step S15). If it is not in transition (in the case of NO at step S15), the process skips steps S16 to S26 and advances the process to step S27.

On the other hand, when the device 2 is in transition (in the case of YES in step S15), the model generation unit 60 stores the model parameters of the in-transition model (hereinafter, current parameters) corresponding to the selected rendering image in step S11. The difference with the model parameter of the in-transition model (hereinafter referred to as the previous parameter) corresponding to the previous selected rendering image stored in the unit 5 is calculated and compared with the threshold value T _D (step S16). The threshold value T _D is determined in advance in accordance with the fluctuation range of the image of the device 2 when the operation state is stable. For example, T _D is 1%.

When the absolute value of the difference is equal to or less than T _D (in the case of YES at step S16), the model generation unit 60 increases the count value of the stability counter of the device by 1 (step S17). The value is compared to a threshold T _S (step S18). The threshold value T _S is set in advance to a time length enough to discriminate the stability / unstable state of the device 2. For example, in the case of lighting equipment, 5 which is the number of frames equivalent to 1 second is set as T _S , and in the case of an automatic door, 50 which is the number of frames equivalent to 10 seconds is set as T _S Ru.

When the value of the stability counter is equal to or greater than T _S (in the case of YES in step S18), the model generation unit 60 determines whether the rendering image selected in step S11 corresponds to the background model in the transition completion state. It is confirmed whether or not it is (step S19). That is, it is checked whether the current parameter of the device 2 is within the target range. If the current parameter of the device 2 is in the target range (YES in step S19), the model generation unit 60 sets the operation state of the device to a state indicating the completion of the transition (step S20). Specifically, “lighting” and “lighting off” are set for lighting devices, and “closing” is set for automatic doors. In this case, the model generation unit 60 changes the parameter range of the device 2 to the target range. For example, when the device 2 is a lighting device whose transition to lighting has been completed, the lighting intensity estimation range is set to 90 to 100%.

  On the other hand, if the current parameter of the device 2 is not in the target range and is in the middle of the transition (in the case of NO at step S19), the decrease of the illumination intensity as in the above-described deterioration of the illumination device occurs. Since the steady state is reached before the operating state reaches the target range, the model generation unit 60 sets the failure detection flag of the relevant device to ON, assuming that the failure in which the operating state is stable has occurred ( Step S21). In this case, the model generation unit 60 changes the parameter range of the device 2 to a range having ± 1% as the upper limit / lower limit centering on the model parameter of the background model selected in step S11. For example, when the device 2 is a lighting device and the illumination intensity of the background model selected in step S11 is 85%, the illumination intensity estimation range is set to 84 to 86%.

  When the model generation unit 60 detects that a failure is occurring, the model generation unit 60 causes the user interface unit 7 as a warning unit to warn of that (step S22). For example, if the device 2 is a lighting device, a space model is rendered on a virtual shooting surface including the installation surface (ceiling etc.) of the lighting device in the field of view, and a problem occurs in the rendered image. At the same time as highlighting the image, the character string "Light is deteriorated" is superimposed on the image to display a warning. Also, if the device 2 is an automatic door, for example, an image of the automatic door having a defect is highlighted in the background image, and the character string "automatic door is stopped" is superimposed on the background image to display a warning Let

If the value of the stability counter is less than T _S (NO in step S18), steps S19 to S22 are skipped and the process proceeds to step S27.

Further, in step S16, when the magnitude of the difference between the current parameter of the device 2 and the previous parameter is larger than T _D (in the case of NO in step S16), the model generation unit 60 determines the instability counter of the device. and the value is increased by 1 (step S23), and compares the value of the instability counter the increased the threshold _{T U} (step S24). Threshold T _U is determined in advance to the time length of the extent to which the device 2 is required for the migration. For example, in the case of lighting equipment, 5 which is the number of frames equivalent to one second is set as T _U , and in the case of an automatic door, 50 which is the number of frames equivalent to 10 seconds is set as T _U Ru.

If the value of unstable counter device 2 was T _U or more (YES in step S24), and for example, unstable failure device 2 such as a flicker illumination device repeatedly turning on and off It can be said that it has occurred. Therefore, the model generation unit 60 sets the malfunction detection flag of the device to ON, assuming that the malfunction whose operation state is unstable occurs (step S25). In this case, the model generation unit 60 can set the parameter range of the device 2 to the same setting as the parameter range of the in-transition model, for example, 0 to 100%.

  When the model generation unit 60 detects that a failure is occurring, the model generation unit 60 causes the user interface unit 7 as a warning unit to warn of that (step S26). For example, if the device 2 is a lighting device, a space model is rendered on a virtual shooting surface including the installation surface (ceiling etc.) of the lighting device in the field of view, and a problem occurs in the rendered image. The image is highlighted and a warning "flicker on the light is occurring" is superimposed on the image to display a warning. Also, if the device 2 is an automatic door, for example, the image of the automatic door having a defect is highlighted in the background image and the character string "automatic door is repeatedly opening and closing" is superimposed on the background image Display a warning.

If the value of the instability counter device 2 is less than _{T U} (if at step S24 of NO), the process proceeds skips step S25, S26 to step S27.

  In step S27, the model generation unit 60 stores the current parameter in the storage unit 5 to be referred to as the previous parameter next time.

  In response to the defect warning in steps S22 and S26, the device administrator who is the user can take measures such as repair / replacement of the device 2 in which the defect is occurring. If the problem is resolved by the countermeasure, the user operates the user interface unit 7 to input a recovery signal. The model generation unit 60 confirms the input of the recovery signal (step S28), and sets the operation state of the device 2 to "during transition" when the recovery signal is input (in the case of YES at step S28). Step S29). On the other hand, when the recovery signal is not input (NO in step S28), step S29 is skipped.

  When there are a plurality of devices 2 in the background, the model generation unit 60 executes steps S15 to S29 for each device.

  After completing the model parameter recording process S27 and the recovery process response processes S28 and S29, the image processing unit 6 returns the process to step S1 and starts the process for the next captured image.

Note that the value of stability counter in the determination of step S18 that is greater than or equal to the threshold, to ensure that the size is equal to or less than the threshold value T _D state of the difference between the current parameter and the last parameter has continued more than a predetermined time period For this reason, when the magnitude of the difference between the current parameter and the previous parameter exceeds the threshold value T _D in step S16, the value of the stability counter may be reset to zero. That is, as a result, when the change of the model parameter is equal to or less than the threshold value T _D and the model parameter is within the target range continuously for a predetermined period or more, a control signal instructing transition to a new target range is acquired. The model generation means 60 generates a transition completion model which changes the operation state only for the target range.

[Modification]
(1) In the above embodiment, the background image generation unit 61 selects the rendering image with the lowest degree of difference from the captured image from the plurality of rendering images, and further selects the pixel value of the selected rendering image as the captured image. The example which produces | generates a background image by performing correction | amendment which closely approaches to a corresponding pixel value was shown. In the modification, the background image generation unit 61 may omit the correction, and may use the rendering image having the lowest degree of difference from the captured image among the plurality of rendering images as the background image. In such a modification, for example, priority is given to reduction of the processing load over improvement in accuracy of pixel values, such as detecting movement of the desk in the monitoring space by searching for an edge pattern of the desk on the background image. It is suitable for the intended use. Also in this modified example, it is possible to prevent excessive approximation of the photographed image, and even if an object other than the background appears in the photographed image, it is possible to generate the background image with low load and high accuracy based on the background model.

  (2) In the above embodiment and modification, an example is shown in which the model generation unit 60 generates a plurality of background models at a time when changing to a plurality of background models, but the model generation unit 60 generates a background image The background model may be changed in an exploratory manner in cooperation with the generation means 61. For example, in the case of setting the illumination intensity which is a model parameter in the range of 0 to 100% in 1% steps with respect to the lighting apparatus during transition, first, the model generation unit 60 outputs a background model in which the illumination intensity is set to 50%. The image generation means 61 calculates the difference in brightness between the rendered image and the photographed image. Next, when the difference is positive (the rendered image is brighter), the model generation unit 60 outputs a background model in which the illumination intensity is changed to 49%, and the difference is negative (the rendered image is In the case of dark), the illumination intensity is changed in the direction in which the difference approaches 0, such that the model generation means 60 outputs a background model in which the illumination intensity is changed to 51%. Then, the background image generation unit 61 generates a background image using a rendering image when the difference between positive and negative changes or when the lower or upper limit of the range is reached without changing the positive and negative of the difference. By doing this, the processing load can be further reduced.

  Reference Signs List 1 object detection device, 2 devices, 3 device interface units, 4 cameras, 5 storage units, 6 image processing units, 7 user interface units, 20 switches, 21 lights, 22 human sensors, 23 automatic doors, 30 control signal acquisition means , 40 shooting means, 50 background model storage means, 60 model generation means, 61 background image generation means, 62 object detection means.

Claims (5)

A background image generating apparatus that generates an image of a background object existing in a space and the timing corresponding to the imaging with respect to imaging of a predetermined space by a camera,
A photographed image acquisition unit that acquires a photographed image of the space photographed by the camera;
Storage means for storing a background model which is a three-dimensional model of the background, and which includes optical feature quantities or three-dimensional coordinate values of each background as model parameters;
Control signal acquisition means for acquiring a control signal for controlling the operation of a predetermined device included in the background object;
A plurality of model parameters of the device are changed according to the operation state of the device generated by the acquired control signal, and the model parameters are different in a change range of the model parameter assumed for the control signal in advance. Model generation means for generating a background model;
Each of the plurality of background models generated by the model generation unit is rendered on the photographing surface of the camera to generate a plurality of rendered images, and among the rendering images, the degree of difference with the photographed image is the smallest. and the background image generating means for generating the background image using,
A background image generation device characterized by comprising. The background image generation unit generates the background image by performing correction that brings the rendering image closer to the captured image by changing the pixel value of the rendering image with the smallest degree of difference within a predetermined limit.
Background image generation apparatus according to claim 1, wherein the. The change range of the model parameter with respect to the control signal includes a target range corresponding to the control signal and a range halfway through transition to the target range.
When the change of the model parameter corresponding to the background image is equal to or less than a predetermined threshold and the model parameter is within the target range, the model generation means continues to the new target range continuously for a predetermined period or more. Generating a plurality of the background models having different model parameters only for the target range within the change range, until the control signal instructing transition of
The background image generation device according to claim 1 or 2 , characterized in that The background image generation unit generates a rendering image obtained by rendering the background model generated by the model generation unit on the photographing surface, and changes the pixel value of the rendering image within a predetermined limit to generate the rendering image. Performing correction close to the photographed image to generate the background image;
The background image generation device according to claim 1, characterized in that The said background image generation apparatus as described in any one of the Claims 1-4 image | photographed the said space with the said imaging | photography timing the said background image which the background image produced | generated according to the imaging | photography timing of the said camera was said. An object detection apparatus for detecting an object other than the background object in the space, as compared with a photographed image.

Images