JP4403418B2 - Information processing system and method, recording medium, and program - Google Patents

Description

  The present invention relates to an information processing system and method, a recording medium, and a program, and more particularly, to an information processing system and method, a recording medium, and a program that can flexibly detect and notify an event and suppress power consumption. .

  Conventionally, as a home security system, a method of knowing sensor information by viewing a monitor TV (Television) that displays a monitoring image transmitted from an imaging device has been proposed (see, for example, Patent Document 1).

  In addition, a method has been proposed in which a monitoring device in which an infrared sensor and an image sensor are combined detects a human body that has entered a monitoring area by determining the presence or absence of a human body and the presence or absence of a plan (for example, Patent Document 2). reference).

In the inventions described in Patent Document 1 and Patent Document 2, a specific place or a specific abnormality (event) is mainly detected.
Japanese Patent Laid-Open No. 8-124078 JP 2000-339554 A

  However, in the inventions described in Patent Literature 1 and Patent Literature 2, adjustment at the time of installation is troublesome, and once installed, the sensor (imaging device or monitoring device) may be moved to another location. There was a problem that it was difficult.

  In addition, the wireless cameras used in the past have a problem that the battery has only a short time because it continuously transmits images and sounds by battery driving.

  The present invention has been made in view of such a situation, and is intended to flexibly detect and notify an event and suppress power consumption.

According to an information processing system of one aspect of the present invention, in an information processing system that performs monitoring processing based on sensor data, a sensor unit that acquires sensor data output by a sensor that detects a state of a region to be monitored, and the sensor unit and feature quantity calculating means for calculating a feature quantity obtained by a predetermined calculation with respect to the acquired sensor data, by comparing the characteristic amount calculated by the feature quantity calculating means and the determination parameter, the event Event determining means for determining whether or not an event has occurred, and presentation data creating means for creating presentation data including event data for notifying the event when the event determining means determines that the event has occurred And presenting based on the presentation data created by the presentation data creation means. And presentation means, an input acquisition means for acquiring a user input of the determination whether the future to present a more presented by said event to said presentation means, the judgment of the user acquired by the input obtaining unit Control means for controlling the change in the calculation of the feature amount or the update of the determination parameter so that the occurrence of the event is easily detected or the occurrence of the event is difficult to be detected based on the input. It is characterized by that.

The sensor includes a camera, and the event determination unit is configured to detect a person by comparing the feature amount with a determination parameter in a region unit obtained by dividing an image captured by the camera into a plurality of regions. When a person is detected in a predetermined area, it is determined that the event has occurred, and when the control means indicates that the user does not have to present the event in the future, Thereafter, even if a person is photographed in the predetermined area, the change in the calculation of the feature amount can be controlled so as not to determine that the event has occurred .

When the input of the user's judgment indicates that the event does not have to be presented in the future , the control unit can stop the calculation of the feature amount .

The sensor data is image data, and the feature amount extraction unit can calculate an inter-frame difference of the image as the feature amount.

The presenting data creating means further creates remote control presenting data composed of the event data for notifying the event to a remote controller that accepts an input of a user judgment as to whether or not to present the event in the future. Can be made.

It may further include a remote control presentation means for presenting the event based on the remote presentation data generated by the presentation data creating means to the remote controller.

There are a plurality of types of sensors, and the control means can further execute control for turning on or off the power of the sensors that do not need to be detected when the user determines that the event will not be presented in the future .

The plurality of sensors may include any one of a camera, a microphone, and an infrared sensor .

The control means can also execute control for turning on or off the power of the sensor according to the passage of time or the content of the sensor data .

When the event determination unit determines that the event has not occurred, the presentation data creation unit determines that the event has occurred by using the television broadcast signal as the presentation data and the event determination unit. In this case, the presentation data can be created by inserting the event data into a part of the television broadcast signal .

An information processing method according to one aspect of the present invention is an information processing method for an information processing system that performs monitoring processing based on sensor data, and a sensor step that acquires sensor data output by a sensor that detects a state of a region to be monitored ; determining a feature quantity calculation step, the feature amount calculated by the processing of the feature quantity calculation step of calculating a feature amount obtained by a predetermined calculation with respect to the sensor data acquired by the processing of the sensor step An event determination step for determining whether or not an event has occurred by comparing with a parameter, and event data for notifying the event when it is determined that the event has occurred by processing of the event determination step. A presentation data creation step for creating presentation data including the presentation data; A presentation step of controlling the presentation based on the presentation data generated by the process of forming step, an input for acquiring a user's input of the determination whether the future presents the events presented by the presentation control step Based on the acquisition step and the determination input from the user acquired by the process of the input acquisition step, the occurrence of the event is easily detected, or the occurrence of the event is difficult to be detected. And a control step for controlling the change of the calculation of the quantity or the update of the determination parameter .

A recording medium according to one aspect of the present invention is a program that executes monitoring processing based on sensor data, the sensor step for obtaining sensor data output by a sensor that detects the state of an area to be monitored, and the processing of the sensor step comparing the feature quantity calculation step of calculating a feature amount obtained by predetermined calculation with respect to the acquired sensor data, the feature amount calculated by the processing of the feature quantity calculation step and the decision parameter by The event determination step for determining whether or not an event has occurred, and when the event determination step determines that the event has occurred, presentation data including event data for notifying the event is created The presenting data creation step and the processing of the presenting data creation step A presentation step of controlling the presentation based on the presentation data was made, and the input obtaining step of obtaining user input for determined whether future presents the events presented by the presentation control step, the Changing the calculation of the feature amount so that the occurrence of the event is easily detected or the occurrence of the event is difficult to be detected based on the input of the judgment acquired from the user acquired by the processing of the input acquisition step Alternatively, a computer-readable program including a control step for controlling update of the determination parameter is recorded.

A program according to one aspect of the present invention is a program that executes monitoring processing based on sensor data, and includes a sensor step that acquires sensor data output by a sensor that detects a state of a region to be monitored, and processing of the sensor step. a feature value calculating step of calculating a feature amount obtained by predetermined calculation with respect to the acquired sensor data, by comparing said characteristic quantity determination parameters calculated by the processing of the feature quantity calculation step An event determination step for determining whether or not an event has occurred, and presentation data including event data for notifying the event when it is determined by the event determination step that the event has occurred The presentation data creation step and the processing of the presentation data creation step A presentation step of controlling the presentation based on the presentation data created, an input obtaining step of obtaining user input for determined whether future presents the events presented by the presentation control step, the Changing the calculation of the feature amount so that the occurrence of the event is easily detected or the occurrence of the event is difficult to be detected based on the input of the judgment acquired from the user acquired by the processing of the input acquisition step Alternatively , a control step for controlling the update of the determination parameter is executed by a computer.

In one aspect of the present invention, a feature amount obtained by a predetermined calculation is calculated with respect to sensor data output from a sensor that detects a state of an area to be monitored, and an event is generated by comparing with a determination parameter. It is determined whether or not. When it is determined that an event has occurred, presentation data including event data for notifying the event is created, and based on a user's determination input as to whether or not to present the presented event in the future The change in the calculation of the feature amount or the update of the determination parameter is controlled so that the occurrence of the event is easily detected or the occurrence of the event is difficult to be detected .

According to one aspect of the present invention, an event can be notified. Further, according to one aspect of the present invention, an event can be detected and notified flexibly based on an input of a user's judgment as to whether or not to present a presented event in the future . Furthermore, according to one aspect of the present invention, useless power consumption can be suppressed.

  FIG. 1 shows a configuration example of a monitoring system 10 to which the present invention is applied. In this configuration example, the multi-sensor camera 1 is provided on the left monitoring area side in the figure, and the processing box 2, the presentation unit 3, and the processing box 2 are remotely operated on the notification / presentation side on the right side in the figure. A remote controller 4 is provided. The multi-sensor camera 1 and the processing box 2 perform wireless communication.

  The multi-sensor camera 1 is provided with a plurality of sensors, and each sensor is driven by a battery. Each sensor is installed in an area (necessary place) where an event is to be monitored. When an event occurs, the processing box 2 constructs a presentation image and sound, notifies the presentation unit 3 and the remote controller 4 to present them. The presenting unit 3 is, for example, a general television receiver. In this case, the presentation unit 3 displays a general viewing signal (video based on a broadcast signal) when an event does not occur (normal), and when an event occurs, an event image is displayed on a part of the general viewing signal. The inserted picture-in-picture image is displayed. The presenting unit 3 is not limited to a television receiver, and may be a dedicated monitor. The displayed image may be an image of the entire screen, not a picture-in-picture image.

  The user can make a determination on the presented event and can input a command from the remote controller 4 based on the result of the determination. For example, when an event is desired to be notified in the future, this can be instructed by operating an OK button (an OK button 291 in FIG. 9 described later). Based on the judgment input from the user, the event detected by the processing box 2 changes with time, and whenever the user uses the monitoring system 10, only the event intended by the user is detected and notified. .

  Since a plurality of sensors and a signal processing unit (not shown) mounted on the multi-sensor camera 1 operate only when necessary, wasteful power consumption can be suppressed.

  FIG. 2 is a block diagram showing a functional configuration of the multi-sensor camera 1 of FIG.

  The multi-sensor camera 1 includes a camera 21, a microphone 22, an infrared sensor 23, other sensors 24, a signal processing unit 25, a transmission unit 26, a reception unit 27, and a battery 28.

  The signal processing unit 25 detects an event based on data obtained from a plurality of sensors such as the infrared sensor 23 and other sensors 24. When an event occurs, the signal processing unit 25 sends data necessary for event presentation to the transmission unit 26. To the processing box 2 (notifies the processing box 2 of the event). For example, when the sensor 24 is a photo sensor that detects brightness, it detects that the room has been turned on. When the sensor 24 is a microwave sensor, for example, the distance and movement of a moving object are detected. To detect. The battery 28 supplies necessary power to each part of the multi-sensor camera 1.

  The receiving unit 27 receives a user determination signal, a sensor control signal, or a signal processing control signal transmitted from the processing box 2 in response to the event notification, and supplies the signal to the signal processing unit 25. The signal processing unit 25 executes processing based on these signals. Since the multi-sensor camera 1 acquires the user judgment input, only the sensors necessary for event detection and presentation, the signal processing unit 25, the transmission unit 26, and the reception unit 27 can be operated. Power consumption can be reduced.

  In addition, the multi-sensor camera 1 is small and battery-driven, and further has a wireless communication function, so that a large space is not required and wiring processing for power supply is unnecessary. It can be installed in various places in the home. Furthermore, since a plurality of sensors are mounted, various events in the home can be detected.

  FIG. 3 is a block diagram showing a functional configuration of the processing box 2 of FIG.

  The reception unit 51 receives a signal transmitted by radio (radio wave) from the transmission unit 26 of the multi-sensor camera 1 and supplies the signal to the signal processing unit 52. The signal processing unit 52 receives a general viewing signal (television broadcast signal), and performs processing based on the signal supplied from the receiving unit 51 or construction of a presented image. When an event occurs, the signal processing unit 52 constructs presentation data to the presentation unit 3 and outputs the data to the presentation unit 3, constructs remote control presentation data to the remote controller 4, and transmits via the transmission unit 53. It transmits to the remote controller 4 by radio (infrared or radio wave).

  The receiving unit 54 receives a signal (user determination signal) based on a user's determination transmitted wirelessly (infrared rays or radio waves) from the remote controller 4 and supplies the signal to the signal processing unit 52. The signal processing unit 52 executes processing based on the user determination signal and transmits necessary data to the multi-sensor camera 1 via the transmission unit 55 by radio (radio wave).

  FIG. 4 is a block diagram showing a functional configuration of the remote controller 4 of FIG.

  The reception unit 81 receives a signal (for example, remote control presentation data) transmitted wirelessly (infrared rays or radio waves) from the transmission unit 53 of the processing box 2 and supplies the signal processing unit 82 with the signal. The signal processing unit 82 performs processing based on a signal such as decoding or encoding of the acquired signal.

  The remote control presentation unit 83 presents an image (including characters and symbols) based on the signal transmitted from the signal processing unit 82. The user input IF (interface) unit 84 receives an input based on a determination from the user with respect to the event, and supplies a signal based on the input to the signal processing unit 82.

  When an event occurs, the reception unit 81 receives remote control presentation data transmitted from the transmission unit 53 of the processing box 2, and the signal processing unit 82 causes the remote control presentation unit 83 to present an image based on the presentation data. . When determination is input to the user input IF unit 84 by the user, a user determination signal is supplied to the signal processing unit 82. The signal processing unit 82 transmits a user determination signal from the transmission unit 85 to the processing box 2 wirelessly (infrared rays or radio waves).

  For example, two buttons (for example, on the remote controller 4) can be input so that the user can input a determination such as “an event that I want to be notified in the future” or “an event that should not be notified in the future”. , An OK button 291 and an NG button 292 in FIG. 9 described later are provided. The multi-sensor camera 1 and the processing box 2 change the processing based on the input of judgment from the user.

  The principle configuration obtained by connecting the main part of the monitoring system 10 of FIG. 1 composed of the multi-sensor camera 1, the processing box 2, and the remote controller 4 of FIGS. 2 to 4 is shown in FIG. become. However, a wireless communication path is also shown as a wired communication path.

  The sensor 101 including the plurality of sensors 101a to 101n in the example of FIG. 5 corresponds to the multi-sensor camera 1 of FIG. 2, and includes a feature amount extraction unit 102, an event determination unit 103, a presentation data construction unit 104, and parameters. The control unit 108 configured by the control unit 121, the signal processing control unit 122, and the power supply control unit 123 corresponds to the processing box 2 in FIG. This can correspond to the controller 4. However, this correspondence is arbitrary, and for example, some or all of the feature amount extraction unit 102, the event determination unit 103, and the control unit 108 can be provided as a configuration on the multi-sensor camera 1 side. (This example is shown in FIG. 7 described later).

  Next, processing executed by the monitoring system 10 of FIG. 5 will be described with reference to the flowchart of FIG. This process is started when the user gives an instruction to start monitoring in the monitoring area.

  In step S1, the feature quantity extraction unit 102 acquires sensor data from each of the plurality of sensors 101a, 101b,..., 101n constituting the sensor 101 provided in the monitoring area.

  In step S2, the feature quantity extraction unit 102 calculates various feature quantities from the various sensor data acquired by the process in step S1. For example, the inter-frame difference of the image signal is taken, and the barycentric position of the inter-frame difference representing the position of the moving object is calculated. Alternatively, the amount of change in data of the infrared sensor is calculated.

  In step S <b> 3, the event determination unit 103 acquires various feature amounts calculated by the feature amount extraction unit 102.

  In step S <b> 4, the event determination unit 103 determines whether an event has occurred (whether it is an event) based on the acquired various feature amounts. At this time, the event determination unit 103 determines an event based on a determination parameter for determining an event held by itself. This parameter is controlled and updated by the parameter control unit 121. If it is determined in step S4 that the event has occurred, the process proceeds to step S5, and the event determination unit 103 presents an event occurrence signal indicating that the event has occurred and data necessary for event notification and presentation to construct the presentation data. Supplied to the unit 104.

  In step S <b> 6, the presentation data construction unit 104 acquires event generation signals, event notifications, and data necessary for presentation from the feature amount extraction unit 102, and uses the event presentation data as part of the general viewing signal (television broadcast signal). Create the inserted presentation data. In addition, the presentation data construction unit 104 creates presentation data (hereinafter referred to as remote control presentation data) for the remote controller 4 configured by event presentation data (not including a general viewing signal).

  In step S <b> 7, the presentation data construction unit 104 outputs the presentation data created by the process of step S <b> 6 to the presentation unit 3 and causes the presentation unit 3 to present it. Also, the presentation data construction unit 104 outputs the remote control presentation data to the remote control presentation unit 83 and causes the remote control presentation unit 83 to present it.

  Since the presentation data is configured by inserting event presentation data into a part of a general viewing signal, the presentation unit 3 displays picture-in-picture (for example, display as shown in FIG. 15 described later). Is presented. Further, since the remote control presentation data is composed of event presentation data, only the display indicating the event (for example, the image of the monitored place) is presented on the remote control presentation unit 83.

  The user looks at the event (image and sound) notified and presented to the remote control presenting unit 83, and determines (for example, the event that is currently presented is an event that the user wants to be notified in the future or will not let it be done in the future. To determine whether it is a good event). At this time, for example, an image that prompts the user to input a determination may be displayed on the remote control presentation unit 83. When the judgment from the user is input, in step S8, the user input IF unit 84 acquires the judgment input for the event from the user. The user input IF unit 84 supplies a user determination signal based on the acquired determination input from the user to the control unit 108 including the parameter control unit 121, the signal processing control unit 122, and the power supply control unit 123.

  In step S9, the parameter control unit 121 updates the determination parameter for determining the event held in the event determination unit 103 based on the user determination signal acquired by the process in step S8, and the user intends. Enable event detection. For example, when brightness that is equal to or higher than the reference value is detected, when the event detection signal is set to be generated, the reference value can be changed to a brighter reference value or a darker reference value.

  In step S10, the signal processing control unit 122 controls the feature amount extraction unit 102 based on the user determination signal (the user determination signal acquired by the process in step S8). Specifically, control is performed such that detection of unnecessary feature values is stopped, or light processing (for example, processing for detecting a small number of feature values) to heavy processing (a larger number of feature values) when necessary. And control to switch to a process that allows more accurate determination. For example, even if a person is photographed in the area 323 out of the area 322 (area that is desired to be detected as an event) and the area 323 (area that is not necessarily detected as an event) as shown in FIG. Can be prevented from being detected as a feature quantity.

  In step S11, the power supply control unit 123 performs control to turn on or off the power of the sensor 101 based on the user determination signal (the user determination signal acquired by the process in step S8). Specifically, among the sensors 101a to 101n, the feature amount extraction unit 102 and the event determination unit 103 perform control so that the power of sensors unnecessary is turned off. This can prevent wasted battery consumption.

  After the process of step S11, the process returns to step S1 and the same process is repeated.

  On the other hand, if the event determining unit 103 determines that the event is not an event in step S4, the process proceeds to step S12, and the presentation data constructing unit 104 directly uses the general viewing signal (television broadcast signal) as the presentation data. 3 is output. The presentation unit 3 presents a general viewing signal. Thereafter, the process returns to step S1, and the subsequent processes are repeated.

  With the processing in FIG. 6, the user can reliably detect an event intended by the user by performing a simple determination input, and controls each sensor and the processing unit to operate only when necessary. Therefore, the power consumption of the multi-sensor camera 1 can be reduced without consuming unnecessary power.

  A specific example of the monitoring system (home security system) will be described below with reference to FIGS.

  In this example, the sensors to be used are the camera 201 and the infrared sensor 202 (FIG. 7), and the user responds to the presented event with “OK (to be detected as an event in the future)” or “NG (from now on). May not be detected as an event). In this case, a configuration example of the multi-sensor camera 1 is shown in FIG. 7, a configuration example of the processing box 2 is shown in FIG. 8, and a configuration example of the remote controller 4 is shown in FIG. Furthermore, flowcharts for explaining processing in each device are shown in FIGS. 10 to 14, and images presented in the presentation unit 3 are shown in FIGS. 15 and 16. Further, images presented on the remote control presenting unit 283 are shown in FIGS. 17 and 18. In the figure, portions corresponding to those in FIG. 5 are given the same reference numerals, and the description thereof will be omitted because it will be repeated.

  In FIG. 7, sensor data acquired by the camera 201 and the infrared sensor 202 is supplied to the feature amount extraction unit 102. For example, the infrared sensor 202 is always turned on, and the camera 201 is normally turned off to reduce power consumption (controlled by the control unit 108 as described above). The feature amount extraction unit 102 extracts a feature amount from the sensor data acquired by the infrared sensor 202 and supplies the feature amount to the event determination unit 103.

  Notification data used for notifying that an event has occurred is supplied from the feature quantity extraction unit 102 to the encoding unit 203, and the encoding unit 203 encodes this and supplies it to the transmission unit 204. When an event occurs, the transmission unit 204 transmits the encoded notification data to the processing box 2. The user determination signal transmitted from the processing box 2 is received by the receiving unit 205, decoded by the decoding unit 206, and then supplied to the control unit 108. As described above, the control unit 108 executes processing based on the user determination signal.

  Specifically, as described with reference to FIG. 5, the control unit 108 controls the event determination unit 103 to update the event determination parameter, and the feature amount extraction unit 102 controls the feature of the image. The amount extraction process is controlled to start or stop, and the camera 201 or the infrared sensor 202 is controlled to be turned on or off.

  The encoding unit 203, the transmission unit 204, the reception unit 205, and the decoding unit 206 also operate the multi-sensor camera 1 without wasting power by causing the processing to be executed only when an event occurs. Can be made.

  When an event notification (notification data) is transmitted from the transmission unit 204 of the multi-sensor camera 1, the notification data is received by the reception unit 241 of the processing box 2 in FIG. 8 and decoded by the decoding unit 242. Thereafter, the data is supplied to the presentation data construction unit 104. As described above, the presentation data construction unit 104 creates presentation data in which an event signal is superimposed on a part of the general viewing signal, outputs the presentation data to the presentation unit 3, and encodes the event data itself as remote control presentation data. The data is encoded by 243 and transmitted to the transmission unit 244. When an event notification (data for notification) is not transmitted from the multi-sensor camera 1 (when no event has occurred), the presentation data construction unit 104 outputs the general viewing signal itself to the presentation unit 3 as presentation data. , To present.

  A user determination signal is transmitted from the remote controller 4 along with the transmission of the remote control presentation data. Therefore, the receiving unit 245 of the processing box 2 receives this and causes the decoding unit 246 to decode it. The decoded user determination signal is supplied to the presentation data construction unit 104 and the encoding unit 247. Based on the user determination signal, the presentation data construction unit 104 executes processing such as stopping processing for generating a picture-in-picture image, for example. The encoding unit 247 encodes the supplied user determination signal and causes the transmission unit 248 to transmit the encoded user determination signal. The user determination signal transmitted by the transmission unit 248 is received by the reception unit 205 of the multi-sensor camera 1.

  When remote control presentation data is transmitted from the transmission unit 244 of the processing box 2, the reception unit 281 of the remote controller 4 in FIG. 9 receives this and supplies it to the decoding unit 282. The decoding unit 282 decodes the remote control presentation data and causes the remote control presentation unit 83 to present an image based on the decoded remote control presentation data.

  On the other hand, the user can input a judgment (whether it is detected as an event (OK) or not (NG)) by operating the OK button 291 or the NG button 292.

  The user input IF 84 detects that any one of the OK button 291 or the NG button 292 is operated, and supplies it to the encoding unit 283 as a user determination signal. The encoding unit 283 encodes the user determination signal and causes the transmission unit 284 to transmit it. The transmission unit 284 transmits the user determination signal to the reception unit 245 of the processing box 2.

  Next, processing in the multi-sensor camera 1 in FIG. 7, the processing box 2 in FIG. 8, and the remote controller 4 in FIG. 9 will be described with reference to the flowcharts in FIGS.

  First, processing in the multi-sensor camera 1 of FIG. 7 will be described with reference to the flowcharts of FIGS. 10 and 11. This process is started when the user gives an instruction to start monitoring in the monitoring area.

  In step S <b> 51, the camera 201 and the infrared sensor 202 that are deemed necessary acquire image data and detection data as sensor data, and supply them to the feature amount extraction unit 102. In the present example, both the camera 201 and the infrared sensor 202 are operated (necessary). However, the present invention is not limited to this. For example, only the infrared sensor 202 or only the camera 201 operates. It is good as it is.

  In step S52, the feature amount extraction unit 102 calculates a necessary feature amount. Specifically, necessary feature amounts such as the inter-frame difference of the image data output from the camera 201, the barycentric position of the inter-frame difference, and the change amount of the sensor data output from the infrared sensor 202 are calculated. Note that necessary feature values are converted from time to time. This is because it is controlled by the control unit 108. The feature amount extraction unit 102 supplies the calculated various feature amounts to the event determination unit 103 and also supplies it to the encoding unit 203 as notification data.

  In step S53, the event determination unit 103 acquires the supplied various feature amounts (various feature amounts calculated and supplied by the processing in step S52).

  In step S54, the event determination unit 103 determines whether an event has occurred (whether or not it is an event) based on the acquired various feature amounts. At this time, the event determination unit 103 determines an event based on a determination parameter for determining an event held by itself. This parameter is controlled and updated by the control unit 108. If it is determined in step S54 that the event is an event, the process proceeds to step S55, and the event determination unit 103 transmits the event occurrence signal to the encoding unit 203, the transmission unit 204, the reception unit 205, and the decoding unit 206. The encoding unit 203, the transmission unit 204, the reception unit 205, and the decoding unit 206 are in an operating state when this event occurrence signal is supplied. Therefore, when the event occurrence signal is not supplied, it is in the non-operating state (off state), so that it is possible to suppress wasteful consumption of power.

  In step S <b> 56, the encoding unit 203 receives the notification data (event notification data) transmitted from the feature amount extraction unit 102 in the process of step S <b> 52, encodes it, and supplies it to the transmission unit 204.

  In step S <b> 57, the transmission unit 204 transmits the notification data encoded and supplied by the encoding unit 203 to the processing box 2.

  The processing box 2 receives the notification data (step S103 in FIG. 12 to be described later) and transmits a user determination signal corresponding thereto (step S112 in FIG. 12 to be described later).

  In step S58, the receiving unit 205 determines whether a user determination signal has been transmitted. When the user determination signal is transmitted, the process proceeds to step S59, and the reception unit 205 receives the user determination signal and supplies it to the decoding unit 206.

  In step S <b> 60, the decoding unit 206 decodes the user determination signal and supplies it to the control unit 108.

  In step S <b> 61, the control unit 108 executes control such as parameter change control of the event determination unit 103, signal processing control of the feature amount extraction unit 102, and sensor power control. Details have been described with reference to FIG. 5 and FIG.

  If it is determined in step S54 that the event is not an event, if it is determined in step S58 that a user determination signal has not been transmitted, or after the process in step S61, the process returns to step S51, and the subsequent processes are repeated. It is.

  Next, processing in the processing box 2 in FIG. 8 corresponding to the processing of the multi-sensor camera 1 in FIGS. 10 and 11 will be described with reference to FIGS. 12 and 13. This process is started when the user instructs the presenting unit 3 to start presenting an image corresponding to the general viewing signal (broadcast program signal) or starting in the monitoring area.

  In step S101, the reception unit 241 determines whether there is an event notification. Specifically, the reception unit 241 determines whether notification data has been transmitted from the transmission unit 204 of the multi-sensor camera 1. In step S101, when it is determined that the event notification is not made (when it is determined that the notification data is not transmitted), the process proceeds to step S102, and the presentation data construction unit 104 directly displays the general viewing signal as the presentation data. Is output to the presenting unit 3 and is presented to the presenting unit 3. Thereafter, the processing returns to step S101, and the subsequent processing is repeated.

  If it is determined in step S101 that there is an event notification, in step S103, the reception unit 241 receives the notification data and causes the decoding unit 242 to decode it. The decryption unit 242 supplies the decrypted notification data to the presentation data construction unit 104.

  In step S104, the presentation data construction unit 104 receives the supplied notification data and creates presentation data. Specifically, the presentation data construction unit 104 creates presentation data by creating an event image on a part of a general viewing image (creates image data to be a picture-in-picture). Thereafter, the presentation data construction unit 104 outputs the created presentation data to the presentation unit 3 for presentation.

  At this time, an image as shown in FIG. 15 or 16 is displayed on the presenting unit 3. In the case of the example of FIG. 15, an event presentation area 321 is presented on the presentation unit 3 in a part of the general viewing screen 320. In the event presentation area 321, the image of the person photographed by the camera 201 is displayed in the right area 323 of the left area 322 and the right area 323 in the drawing. In response to the event presentation, the user operates the OK (desired to be detected as an event in the future) button 291 or the NO (does not be detected as an event in the future) button 292 of the remote controller 4 to make a determination. Will be entered. This user determination signal is transmitted from the remote controller 4 (step S156 in FIG. 14 described later).

  Assuming that the left region 322 of the regions 322 and 323 is a region that the user wants to detect as an event, the user does not respond to the event presentation as shown in FIG. The button 292 is operated. When the NO button 292 is operated by the user, a user determination signal corresponding to this is transmitted from the remote controller 4. When receiving this, the processing box 2 transmits it to the multi-sensor camera 1 (step S112 described later). As described above, when receiving the user determination signal, the control unit 108 of the multi-sensor camera 1 controls the feature amount extraction unit 102 to exclude the area on the right side of the angle of view of the camera 201 from the feature amount extraction range. (Step S61 in FIG. 11). As a result, even if a person is photographed in the area on the right side of the angle of view thereafter, an event occurrence signal is not generated, and event notification is performed when a person is photographed in the area on the left side. Thereby, for example, a presentation as shown in FIG. 16 is performed.

  When the user operates the NO button 292, only the general viewing image is presented on the presentation unit 3. That is, the event presentation area 321 is not displayed. As a result, the user can view the event presentation area 321 only in a necessary place without being presented with a useless event.

  Returning to FIG. 12, in step S <b> 105, the presentation data construction unit 104 creates remote control presentation data and outputs it to the encoding unit 243. The remote control presentation data does not include a general viewing signal, and the remote control presentation data is composed of notification data itself. When receiving the remote control presentation data, the remote controller 4 displays the corresponding image as shown in FIG. 17 or 18 (step S153 in FIG. 14 described later).

  In step S106, the encoding unit 243 encodes the remote control presentation data supplied by the process of step S105, and supplies this to the transmission unit 244.

  In step S107, the transmission unit 244 transmits the encoded remote control presentation data supplied by the processing in step S106 to the remote controller 4. The remote controller 4 receives this (step S151 in FIG. 14 to be described later), and after presenting (step S153 in FIG. 14 to be described later), transmits a user determination signal that is a signal based on the determination input by the user. (Step S156 in FIG. 14 described later).

  Therefore, in step S108, the receiving unit 245 determines whether or not a user determination signal has been transmitted. If it is determined that the user determination signal has been transmitted, the reception unit 245 receives the user determination signal and supplies it to the decoding unit 246 in step S109.

  In step S110, the decoding unit 246 decodes the user determination signal supplied by the process of step S109, supplies this to the presentation data construction unit 104, and supplies it to the encoding unit 247.

  In step S <b> 111, the encoding unit 247 encodes the user determination signal supplied by the process of step S <b> 110 and supplies this to the transmission unit 248.

  In step S112, the transmission unit 248 transmits the supplied user determination signal encoded by the process of step S111 to the multi-sensor camera 1. As described above, the multi-sensor camera 1 executes processing based on the user determination signal (step S61 in FIG. 11).

  In step S113, the presentation data construction unit 104 acquires the user determination signal decoded by the decoding unit 246 by the process of step S110, and executes processing based on the user determination signal.

  At this time, as described above, the presentation data construction unit 104 includes a signal that stops construction of the presentation data (for example, a signal that an NG button 292 in FIG. 17 described later is operated) in the user determination signal. If it is included, processing such as stopping the construction of the presentation data including the event image is executed. At this time, the event presentation area 321 is not presented in the presentation unit 3.

  In addition, when the user determination signal includes a signal that continues the construction of the presentation data (for example, a signal that an OK button 291 in FIG. 17 described later is operated), A process of continuing construction of the presentation data including the image is executed. At this time, the event presentation area 321 as shown in FIG.

  After step S102, if it is determined in step S108 that a user determination signal has not been transmitted, or after step S113, the process returns to step S101, and the subsequent processes are repeated.

  Next, processing in the remote controller 4 in FIG. 9 corresponding to the processing in the processing box 2 in FIGS. 12 and 13 will be described with reference to FIG. This process is started when the process of step S107 in FIG. 12 is executed by the transmission unit 244 of the process box 2.

  In step S151, the reception unit 281 receives remote control presentation data transmitted from the transmission unit 244 of the processing box 2 (transmitted from the transmission unit 244 of the processing box 2 by the processing of step S107 in FIG. 12). This is supplied to the decoding unit 282.

  In step S152, the decoding unit 282 decodes the remote control presentation data received by the process of step S151, and supplies the decoded remote control presentation data to the remote control presentation unit 83.

  In step S153, the remote control presenting unit 83 presents an image based on the received remote control presentation data.

  At this time, an image as shown in FIG. 17 or 18 is displayed on the remote control presentation unit 83. This image is the same image as the image of the event presentation area 321 in FIGS. 15 and 16 described above. That is, the remote control presentation data includes only data corresponding to the event presentation area 321 among the presentation data output to the presentation unit 3.

  In the case of the example in FIG. 17, the remote control presentation unit 83 presents an event based on the remote control presentation data. Of the left region 293 and the right region 294 in the figure of the remote control presentation unit 83, if the left region 293 is a region that the user wants to detect as an event, as shown in FIG. 17, the right region When a person image is displayed on the 294, the user presents an OK (desired to be detected as an event in the future) button 291 and a NO (may not be detected as an event in the future) button 292 for the event presentation. Of these, the NO button 292 is operated. On the other hand, as shown in FIG. 18, when a person image is displayed in the left area 293, the user operates the OK button 291.

  Therefore, in step S154, the user input IF unit 84 determines whether the user's determination has been input (whether one of the OK button 291 or the NG button 292 has been operated). If it is determined that the user's determination is input, the process proceeds to step S155, and the user input IF unit 84 supplies the user determination signal, which is a signal based on the user's determination, to the encoding unit 283, and the encoding unit 283 Encodes this. Thereafter, the encoding unit 283 supplies the encoded user determination signal to the transmission unit 284.

  In step S156, the transmission unit 284 transmits the encoded user determination signal (the user determination signal encoded by the encoding unit 283 by the process of step S154) to the processing box 2. The multi-sensor camera 1 and the processing box 2 receive this (step S59 in FIG. 11 and step S109 in FIG. 12), and execute the processing based on the user determination signal as described above. For example, when the user determination signal of the NG button 292 is transmitted in response to the presentation as shown in FIG. 17, the multi-sensor camera 1 is the right region in the range (view angle) captured by the camera 201. Are excluded from the feature extraction processing. As a result, an image in which a person is displayed in the right area 294 as shown in FIG. 17 will no longer be transmitted as an event, and a person is displayed in the left area 293 as shown in FIG. The displayed image is presented as an event image.

  In the process of step S154, when it is determined that the user's judgment is not input or after the process of step S156, the process returns to step S151, and the subsequent processes are repeated.

  In addition, as an example of a change in processing of the multi-sensor camera 1, for example, during a predetermined period after the use of the monitoring system 10 is started (when the number of determination inputs by the user is less than the predetermined number), the camera 201 and the infrared ray The sensor 202 is always turned on, and after a predetermined period has elapsed (when the number of determination inputs by the user exceeds a predetermined number), the camera 201 is normally turned off, and the camera is not the first camera when the infrared sensor 202 reacts. It can also be controlled to turn on 201.

  In addition, regarding the processing of the event determination unit 103, for example, during a predetermined period after the use of the monitoring system 10 is started (when the number of determination inputs by the user is less than the predetermined number), an image of a person (intruder) is captured. An event is output wherever the position (area) is, and after a predetermined period has elapsed (when the number of determination inputs by the user exceeds a predetermined number), the user can operate the OK button 291 to operate the user. It is also possible to change the determination parameter of the event determination unit 103 so that an event is output only when a person (intruder) is imaged at the position (area) specified by.

  By the above processing, the sensor to be operated is switched based on the user's judgment, and further, the processing in the multi-sensor camera 1 is changed by repeating the user's judgment, so that only the event intended by the user is detected and notified. Since only necessary sensors, signal processing, and the like operate, it is possible to detect and notify an event flexibly, and to reduce wasteful power consumption. Since the multi-sensor camera 1 is driven by the battery 28, one or more of the feature amount extraction unit 102, the event determination unit 103, and the control unit 108 are accommodated in the processing box 2 in order to reduce power consumption as much as possible. It is preferable to do so.

  Moreover, since the power consumption of the multi-sensor camera 1 can be suppressed and the size can be further reduced, the multi-sensor camera 1 can be easily installed in various places.

  The above example is an example for realizing the monitoring system 10, and many other system configurations are conceivable. An example is shown below.

  For example, the sensor to be used is not limited to a camera, a microphone, and an infrared sensor, and other sensors can be mounted. Further, the event to be detected is not limited to the above-described example.

  Further, for example, communication between the multi-sensor camera 1 and the processing box 2 is not limited to wireless communication, and may be wired communication.

  Also, a plurality of multi-sensor cameras 1 can be provided instead of one.

  Furthermore, it is possible to provide a plurality of the presenting units 3 instead of one.

  In addition, the processing box 2 can be an integral type, not a separate housing from the presenting unit 3.

  Furthermore, the remote controller 4 may not be provided with the remote controller presenting unit 83 and only the presenting unit 3 may be presented.

  Further, instead of presenting an event to the remote controller 4, an event presenting section and an input IF section for inputting user judgment can be provided in the processing box 2.

  The series of processes described above can be executed by hardware or can be executed by software. When a series of processing is executed by software, a program constituting the software executes various functions by installing a computer incorporated in dedicated hardware or various programs. It can be installed from a recording medium such as a general-purpose personal computer. In this case, the processing described above is executed by a personal computer 500 as shown in FIG.

  In FIG. 19, a CPU (Central Processing Unit) 501 performs various processes according to a program stored in a ROM (Read Only Memory) 502 or a program loaded from a storage unit 508 to a RAM (Random Access Memory) 503. Execute. The RAM 503 also appropriately stores data necessary for the CPU 501 to execute various processes.

  The CPU 501, ROM 502, and RAM 503 are connected to each other via an internal bus 504. An input / output interface 505 is also connected to the internal bus 504.

  The input / output interface 505 includes an input unit 506 including a keyboard and a mouse, a display including a CRT (Cathode Ray Tube) and an LCD (Liquid Crystal Display), an output unit 507 including a speaker, a hard disk, and the like. A communication unit 509 including a storage unit 508, a modem, a terminal adapter, and the like is connected. A communication unit 509 performs communication processing via various networks including a telephone line and CATV.

  A drive 510 is also connected to the input / output interface 505 as necessary, and a removable medium 521 composed of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is appropriately mounted, and a computer program read therefrom is It is installed in the storage unit 508 as necessary.

  When a series of processing is executed by software, a program constituting the software executes various functions by installing a computer incorporated in dedicated hardware or various programs. For example, it is installed in a general-purpose personal computer from a network or a recording medium.

  As shown in FIG. 19, this recording medium is not only composed of a package medium composed of a removable medium 521 on which a program is recorded, which is distributed to provide a program to a user, separately from the computer. These are configured by a hard disk including a ROM 502 storing a program and a storage unit 508 provided to the user in a state of being pre-installed in the apparatus main body.

  In the present specification, the step of describing a computer program includes not only processing performed in time series according to the described order but also processing executed in parallel or individually even if not necessarily processed in time series. Is also included.

  Further, in this specification, the system represents the entire apparatus constituted by a plurality of apparatuses.

It is a figure which shows the structural example of the monitoring system to which this invention is applied. It is a block diagram which shows the functional structure of the multi sensor camera of FIG. It is a block diagram which shows the functional structure of the processing box of FIG. It is a block diagram which shows the functional structure of the remote controller of FIG. It is a block diagram which shows the fundamental structure of the monitoring system of FIG. It is a flowchart explaining the event detection process performed by the monitoring system of FIG. It is a block diagram which shows the structural example of the multi sensor camera of FIG. It is a block diagram which shows the structural example of the process box of FIG. It is a block diagram which shows the structural example of the remote controller of FIG. It is a flowchart explaining the process in the multi sensor camera of FIG. It is a flowchart explaining the process in the multi sensor camera of FIG. It is a flowchart explaining the process in the process box of FIG. It is a flowchart explaining the process in the process box of FIG. 10 is a flowchart for explaining processing in the remote controller of FIG. 9. It is a figure which shows the example of presentation in the process of step S104 of FIG. It is a figure which shows the example of presentation in the process of step S104 of FIG. It is a figure which shows the example of a presentation in the process of step S153 of FIG. It is a figure which shows the example of a presentation in the process of step S153 of FIG. And FIG. 16 is a block diagram illustrating a configuration example of a personal computer.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 Multi-sensor camera, 2 Processing box, 3 Presentation part, 4 Remote controller, 10 Monitoring system, 83 Presentation part, 84 User input IF part, 101 Sensor, 102 Feature quantity extraction part, 103 Event determination part, 104 Presentation data construction part , 108 control unit, 121 parameter control unit, 122 signal processing control unit, 123 power control unit, 291 OK button, 292 NG button

Claims (13)

In an information processing system that executes monitoring processing based on sensor data,
Sensor means for obtaining sensor data output by a sensor for detecting a state of an area to be monitored ;
Feature quantity calculating means for calculating a feature quantity obtained by a predetermined calculation for the sensor data acquired by the sensor means;
By comparing the characteristic amount calculated by the feature quantity calculating means and determination parameter and determining event determination means for determining whether or not an event has occurred,
When the event determination means determines that the event has occurred, presentation data creation means for creating presentation data including event data for notifying the event;
Presenting means for presenting based on the presenting data created by the presenting data creating means;
An input acquisition means for acquiring a user input of the determination whether the future to present a more presented by said event to said presentation means,
Based on the judgment input from the user acquired by the input acquisition means , the change in the calculation of the feature amount or the occurrence of the event is easily detected or the occurrence of the event is difficult to detect An information processing system comprising: control means for controlling update of the determination parameter . The sensor includes a camera,
The event determination means detects a person by comparing the feature amount with a determination parameter in a region unit obtained by dividing an image captured by the camera into a plurality of regions, and a person is detected in a predetermined region , Determine that the event has occurred,
The control means does not determine that the event has occurred even if a person is photographed in the predetermined area after that when the input of the user's judgment indicates that the event does not have to be presented in the future The information processing system according to claim 1 , wherein a change in calculation of the feature amount is controlled . 2. The information processing system according to claim 1, wherein the control unit stops the calculation of the feature amount when the input of the user's judgment indicates that the event does not have to be presented in the future. . The sensor data is image data,
The information processing system according to claim 1 wherein the feature quantity extraction means, the difference between frames of the image, and calculates as the feature quantity. The presenting data creating means further creates remote control presenting data composed of the event data for notifying a remote controller that receives an input of a user's judgment as to whether or not to present the event in the future. The information processing system according to claim 1. The information processing system according to claim 5, further comprising a remote control presentation means for presenting the event based on the remote presentation data generated by the presentation data creating means to the remote controller. There are multiple types of sensors,
2. The information processing according to claim 1, wherein the control unit further performs control to turn on or off the power of the sensor that does not need to be detected when the user determines that the event will not be presented in the future. system. The information processing system according to claim 7 , wherein the plurality of sensors include any one of a camera, a microphone, and an infrared sensor. The information processing system according to claim 7 , wherein the control unit also executes control to turn on or off the power of the sensor according to the passage of time or the content of the sensor data . When the event determining unit determines that the event has not occurred, the presenting data creating unit determines the television broadcasting signal as the presenting data and determines that the event has occurred by the event determining unit. 2. The information processing system according to claim 1, wherein the presentation data is created by inserting the event data into a part of a television broadcast signal . In an information processing method of an information processing system that executes monitoring processing based on sensor data,
A sensor step for acquiring sensor data output by a sensor for detecting a state of an area to be monitored ;
A feature amount calculating step for calculating a feature amount obtained by a predetermined calculation for the sensor data acquired by the sensor step processing;
An event determination step for determining whether an event has occurred by comparing the feature amount calculated by the processing of the feature amount calculation step with a determination parameter;
When it is determined that the event has occurred by the process of the event determination step, a presentation data creation step of creating presentation data including event data for notifying the event;
A presentation step for controlling presentation based on the presentation data created by the processing of the presentation data creation step;
An input acquisition step of acquiring an input of a user's judgment as to whether or not to present the event presented by the presentation control step in the future ;
Based on the determination input from the user acquired by the process of the input acquisition step, the feature amount calculation is performed so that the occurrence of the event is easily detected or the occurrence of the event is difficult to detect. A control step of controlling change or update of the determination parameter . A program for executing monitoring processing based on sensor data,
A sensor step for acquiring sensor data output by a sensor for detecting a state of an area to be monitored ;
A feature amount calculating step for calculating a feature amount obtained by a predetermined calculation for the sensor data acquired by the sensor step processing;
An event determination step for determining whether an event has occurred by comparing the feature amount calculated by the processing of the feature amount calculation step with a determination parameter;
When it is determined that the event has occurred by the process of the event determination step, a presentation data creation step of creating presentation data including event data for notifying the event;
A presentation step for controlling presentation based on the presentation data created by the processing of the presentation data creation step;
An input acquisition step of acquiring an input of a user's judgment as to whether or not to present the event presented by the presentation control step in the future ;
Based on the determination input from the user acquired by the process of the input acquisition step, the feature amount calculation is performed so that the occurrence of the event is easily detected or the occurrence of the event is difficult to detect. A recording medium on which is recorded a computer-readable program including a control step for controlling change or update of the determination parameter . A program for executing monitoring processing based on sensor data,
A sensor step for acquiring sensor data output by a sensor for detecting a state of an area to be monitored ;
A feature amount calculating step for calculating a feature amount obtained by a predetermined calculation for the sensor data acquired by the sensor step processing;
An event determination step for determining whether an event has occurred by comparing the feature amount calculated by the processing of the feature amount calculation step with a determination parameter;
When it is determined that the event has occurred by the process of the event determination step, a presentation data creation step of creating presentation data including event data for notifying the event;
A presentation step for controlling presentation based on the presentation data created by the processing of the presentation data creation step;
An input acquisition step of acquiring an input of a user's judgment as to whether or not to present the event presented by the presentation control step in the future ;
Based on the determination input from the user acquired by the process of the input acquisition step, the feature amount calculation is performed so that the occurrence of the event is easily detected or the occurrence of the event is difficult to detect. A program for causing a computer to execute a control step for controlling change or update of the determination parameter .

Images