CN107026971B

CN107026971B - Control device and control method

Info

Publication number: CN107026971B
Application number: CN201710029754.3A
Authority: CN
Inventors: 半田雅大
Original assignee: Canon Inc
Current assignee: Canon Inc
Priority date: 2016-01-29
Filing date: 2017-01-16
Publication date: 2021-06-08
Anticipated expiration: 2037-01-16
Also published as: JP6742739B2; US11043091B2; KR20170091026A; CN107026971A; JP2017135676A; KR102082690B1; US20170221329A1

Abstract

The invention provides a control device and a control method. The control device includes: an acquisition unit configured to acquire selection information for selecting an image pickup apparatus from among a plurality of image pickup apparatuses that take an image to be displayed and/or stored by a terminal apparatus, based on a detection result of a sensor; and a control unit configured to control in such a manner that: an amount of communication regarding a captured image from an image capturing apparatus that may not be selected from among the plurality of image capturing apparatuses based on the acquired selection information is made smaller than an amount of communication regarding a captured image from an image capturing apparatus that may be selected based on the acquired selection information.

Description

Control device and control method

Technical Field

The present invention relates to a technique for controlling an image pickup apparatus.

Background

A monitoring system including a plurality of image pickup apparatuses is widely used in a monitoring target facility. In such a system, images taken by a plurality of image pickup apparatuses are collected in a terminal apparatus that displays and records the images. The collected images are analyzed or visually inspected by a guard person, thereby enabling detection and recording of anomalies in the facility. Japanese patent laid-open publication No. 2011-217055 discusses the following configuration related to this technique. Specifically, in addition to the image pickup apparatus, a plurality of sensors are provided in the monitoring target facility. When the sensor detects an abnormality, an image pickup apparatus most suitable for checking a situation at the time of occurrence of the abnormality is selected from the image pickup apparatuses. Then, the captured image from the selected image capturing apparatus is stored.

However, the system that displays and/or stores a captured image from an image capturing apparatus selected from a plurality of image capturing apparatuses based on a detection result from a sensor has the following problem. For example, in the case where all image pickup apparatuses in the system uniformly transmit a captured image to a terminal apparatus that displays and/or stores the image, the captured image is transmitted by an unselected image pickup apparatus, increasing the load imposed on the image pickup apparatus, the terminal apparatus, and the network. On the other hand, in a case where all the unselected image capturing apparatuses stop transmitting the captured image, transmission of the captured image may be delayed. For example, when an image pickup apparatus is newly selected from among unselected image pickup apparatuses that stop transmitting a captured image, there may be a delay before the selected image pickup apparatus starts transmitting and the captured image can be displayed and/or stored.

Disclosure of Invention

A control device, comprising: an acquisition unit configured to acquire, based on a detection result of a sensor, selection information for specifying that one or more image pickup apparatuses are to be selected from among a plurality of image pickup apparatuses, wherein the selection information includes information that the detection result relates to the sensor specifying the one or more image pickup apparatuses; a first control unit configured to control, independently of a detection result, in a manner as follows: in the plurality of image pickup apparatuses, an amount of traffic relating to a captured image from an image pickup apparatus not included in the one or more image pickup apparatuses to be selected in the selection information is made smaller than an amount of traffic relating to a captured image from an image pickup apparatus included in the one or more image pickup apparatuses to be selected in the selection information; a communication unit configured to be able to communicate with a plurality of sensors; and a second control unit configured to control so that an amount of processing relating to detection processing performed by a sensor whose detection result does not relate to the designation of the one or more image pickup apparatuses is relatively reduced compared to an amount of processing relating to detection processing performed by a sensor whose detection result relates to the designation of the one or more image pickup apparatuses.

Further features of the invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings.

Drawings

Fig. 1 is a block diagram illustrating a configuration of an image capturing system according to an exemplary embodiment.

Fig. 2 is a block diagram illustrating a functional configuration of a control apparatus according to an exemplary embodiment.

Fig. 3 is a block diagram illustrating a hardware configuration of the control device.

Fig. 4 is a diagram illustrating an example of selection information according to an exemplary embodiment.

Fig. 5 is a table illustrating the contents of the selection information.

Fig. 6 is a sequence diagram illustrating processing executed in the image capturing system.

Fig. 7 is a flowchart illustrating processing related to acquisition of selection information by the control device.

Fig. 8 is a flowchart illustrating processing related to reception of the detection result by the control device.

Detailed Description

Exemplary embodiments of the present invention are described below with reference to the accompanying drawings. It should be noted that the present invention does not necessarily require all combinations of the exemplary embodiments described below.

[ System Structure ]

Fig. 1 is a block diagram illustrating the configuration of an image capturing system according to the present exemplary embodiment. The camera system includes a control apparatus 100, a terminal device 200, a camera device 300-1, a camera device 300-2. In the present exemplary embodiment, the n image pickup apparatuses including the image pickup apparatus 300-1 to the image pickup apparatus 300-n are collectively referred to as the image pickup apparatus 300 unless otherwise specified. The control apparatus 100, the terminal device 200, the image pickup device 300, and the pointing device 500 are connected to each other via the network 10. The control device 100 and the sensor group 400 are connected to each other via a network 20.

The above connection is not limited. For example, a network connecting the control apparatus 100, the terminal device 200, and the pointing device 500 to each other can be different from a network connecting the control apparatus 100 and the image pickup device 300 to each other. Alternatively, for example, the network 10 and the network 20 may be connected to each other to establish a single network. The control apparatus 100 and the terminal device 200 may be integrally formed. The terminal device 200 and the pointing device 500 can be integrally formed.

The network 10 and the network 20 are each the internet, a wired Local Area Network (LAN), a wireless LAN, a Wide Area Network (WAN), or the like. The communication protocols, sizes, and configurations of the network 10 and the network 20 are not limited. For example, the network 10 and the network 20 can each be an ad hoc (adhoc) network or an Ultra Wideband (UWB) network or the like. For example, the networks 10 and 20 may employ Ethernet

Bluetooth

Or purple bee

Etc. as the communication protocol. Network 10 and network 20 are inAspects such as communication protocols and configurations may differ from each other.

The control apparatus 100 according to the present exemplary embodiment is connected to each of the terminal device 200, the image pickup device 300, the sensor group 400, and the pointing device 500 via the network 10 or the network 20. The control device 100 receives captured images transmitted from the plurality of image capturing apparatuses 300. The control apparatus 100 acquires the selection information 600 based on an instruction from the pointing device 500. Then, based on the selection information 600 and the detection result received from the sensor group 400, the control apparatus 100 selects a specific image pickup apparatus 300 from the plurality of image pickup apparatuses 300.

As exemplarily illustrated in fig. 4, the selection information 600 includes information (conditional expressions 610-1, 610-2, and 610-3) that associates the detection result (condition 620-1) received from the sensor group 400 with information (action 630) for identifying the image capturing apparatus 300. The selection information 600 is described in detail below. The control device 100 transmits the captured image transmitted from the selected image capturing apparatus 300 among the received captured images to the terminal apparatus 200. Therefore, the control apparatus 100 can transmit the captured image from the image capturing device 300 that is most suitable according to the detection result from the sensor to the terminal device 200. The control apparatus 100 may perform image processing on an image to be transmitted to the terminal device 200.

In the case where the control apparatus 100 uniformly receives the captured images from all the image capturing devices 300 in the system and then selects the captured image to be transmitted to the terminal device 200, the load on the image capturing devices 300, the control apparatus 100, and the network 10 may increase. In the case where the control device 100 receives only a captured image to be transferred to the terminal apparatus 200, when the image capturing apparatus 300 that captures the captured image to be transferred is changed, the control device 100 needs to control the start/stop of transmission of the captured image from the image capturing apparatus 300. Such control may cause a delay before a captured image from the newly selected image capturing apparatus 300 is transmitted to the terminal apparatus 200, and thus the usability of the system is impaired. In view of the above, the control device 100 according to the present exemplary embodiment controls in such a manner that: the captured image is received from the image capturing apparatus 300 that may transmit the captured image to the terminal apparatus 200. Therefore, the amount of data of captured images transmitted from the other image capturing apparatuses 300 can be reduced.

Specific examples of the control apparatus 100 include a Personal Computer (PC), a micro server, a home gateway, a relay device such as a router, a server device including Video Management Software (VMS), and a mobile terminal such as a smart phone or a tablet computer. The control apparatus 100 is not limited to these specific examples, and may be other devices.

The terminal apparatus 200 is connected to the control device 100 and the image pickup apparatus 300 via the network 10. The terminal apparatus 200 receives the photographed image photographed by the image pickup apparatus 300 and transmitted from the control device 100, and displays and records the received image. The terminal apparatus 200 can directly receive the captured image transmitted from the image capturing apparatus 300 selected by the control device 100 without using the control device 100. Specific examples of the terminal device 200 include a display and a recording server.

The image pickup apparatus 300 is connected to the control device 100 and the terminal apparatus 200 via the network 10. The image pickup apparatus 300 conforms to standards such as the Open Network Video Interface Forum (ONVIF), universal plug and play (UPnP), or Digital Living Network Alliance (DLNA). The image capturing apparatus 300 is not limited to image capturing apparatuses compliant with these standards, and may be compliant with other standards or may employ other systems. The image pickup apparatus 300 picks up an image and transmits the picked-up image to the control device 100. In the exemplary configuration of the present exemplary embodiment, the captured image transmitted by the image capturing apparatus 300 is a moving image. However, the photographed image is not limited to a moving image, and may be a still image. The captured image may include sound. The image capturing apparatus 300 can transmit a captured image to the terminal apparatus 200.

The image pickup apparatus 300 is controlled by the control device 100. Examples of control performed by the control device 100 on the image capturing apparatus 300 include data control, image capturing control, and status control. The data control is related to the transmission function of the image pickup apparatus 300. Examples of the object of data control include start/stop of transmission and a frame rate, resolution, and bit rate of a captured image to be transmitted. The image capturing control is related to the image capturing function of the image capturing apparatus 300. Examples of the object of image capturing control include start/stop of image capturing, a frame rate of image capturing, and a resolution and a bit rate of an image to be captured. The state control is related to the state of the image pickup apparatus 300. Examples of the state control include start/stop of the image pickup apparatus and pan/tilt/zoom (PTZ) of the image pickup apparatus. The content of the control performed by the control device 100 on the image capturing apparatus 300 is not limited to the example described above, and other types of control may be performed.

The sensor group 400 includes a plurality of sensors, and is connected to the control device 100 via the network 20. The sensor group 400 includes a function of transmitting detection results from the respective sensors to the control device 100. For example, the detection result received by the control device 100 from the sensor group 400 relates to at least one of temperature, illuminance, vibration, infrared light, time of day, time period, and whether the door is opened/closed. However, the contents that the sensor group 400 can detect are not limited to the items described above, and the sensor group 400 may include sensors that detect other types of information. The plurality of sensors in the sensor group 400 may detect different contents, respectively, and may detect the same contents at different positions.

The pointing device 500 is connected to the control apparatus 100 via the network 10, and transmits an instruction for causing the control apparatus 100 to acquire the selection information 600. Examples of the pointing device 500 include a PC, a micro server, a server device including a VMS, and a mobile terminal such as a smart phone or a tablet computer. The pointing device 500 is not limited to these devices and may be of other types.

[ functional Structure of control device 100 ]

The functional configuration of the control device 100 will now be described with reference to fig. 2. The control apparatus 100 includes a selection information management unit 101 (hereinafter referred to as a management unit 101), a sensor control unit 102, an image pickup device control unit 103 (hereinafter referred to as an image pickup control unit 103), a sensor selection unit 104, an image pickup device selection unit 105 (hereinafter referred to as an image pickup selection unit 105), an image acquisition unit 106, and a transmission unit 107.

The management unit 101 acquires and manages the selection information 600 based on the instruction received from the pointing device 500. The selection information 600 according to the present exemplary embodiment is used by the control apparatus 100 to select, from among the plurality of image pickup apparatuses 300, the image pickup apparatus 300 that transmits a captured image to be displayed and/or stored by the terminal apparatus 200, based on the detection result from the sensor. Specific details of the selection information 600 are described below.

The sensor control unit 102 includes the following functions: sensor search, advertisement message monitoring, and operational checking for sensors are performed to identify a sensor group 400 that operates while connected to the network 20. The sensor control unit 102 manages the identified sensors, and receives detection results from the sensors. The sensor control unit 102 may receive the detection result by subscribing to the detection event from the sensor (in other words, the sensor control unit 102 can instruct the sensor to transmit the detection result according to the occurrence of the detection event), or may periodically request the sensor to transmit the detection result. The sensor control unit 102 controls the sensors based on the specifying result from the sensor selection unit 104 described below. Specifically, for example, the sensor control unit 102 controls start/stop of detection by the sensor, start/stop of transmission of a measurement sampling rate, and detection results. The sensor control unit 102 is not limited to the above control.

The imaging control unit 103 includes the following functions: the image pickup apparatus 300 is searched, the advertisement message is monitored, operation check and the like are performed on the image pickup apparatus 300 to identify the image pickup apparatus 300 that operates while being connected to the network 10. The image capture control unit 103 controls the identified image capture apparatus 300 based on the designation result obtained by the image capture selection unit 105 described below. The imaging control unit 103 also controls reception/transmission of an image by the image acquisition unit 106.

The sensor selection unit 104 has the following functions: based on the selection information 600 acquired by the management unit 101, the sensors in the sensor group 400, the amount of processing relating to the detection processing to be reduced, are specified. The image pickup selection unit 105 selects the image pickup apparatus 300 that picks up an image to be displayed and/or stored by the terminal apparatus 200, based on the detection result received by the sensor control unit 102 and the selection information 600 acquired by the management unit 101. The image-pickup selection unit 105 also performs processing different from the above-described selection. Specifically, the image capture selection unit 105 specifies, based on the selection result acquired by the management unit 101, the image capture apparatus 300 whose data amount of the transmitted captured image is to be reduced, from among the plurality of image capture apparatuses 300.

The image acquisition unit 106 receives a captured image transmitted from the image capturing apparatus 300 based on control performed by the image capturing control unit 103. Then, the image acquisition unit 106 transmits a captured image captured by the specified image capturing apparatus 300 from among the received captured images to the transmission unit 107. The transmission unit 107 transmits the image received from the image acquisition unit 106 to the terminal device 200. The image acquisition unit 106 may perform image processing on the image received from the image pickup apparatus 300 and then transmit the resultant image to the transmission unit 107.

[ hardware configuration of control device 100 ]

Fig. 3 is a block diagram illustrating a hardware configuration of the control device 100. The control apparatus 100 includes a Central Processing Unit (CPU)111, a Read Only Memory (ROM)112, a Random Access Memory (RAM)113, an auxiliary storage device 114, a display unit 115, an operation unit 116, a communication unit 117, and a bus 118.

The CPU 111 controls the control apparatus 100 by using computer programs and data stored in the ROM 112 and the RAM 113. The CPU 111 realizes the functions of the respective components of the control apparatus 100 shown in fig. 2. The ROM 112 stores therein programs and parameters that do not need to be changed. The RAM 113 temporarily stores programs and data supplied from the secondary storage device 114, data acquired from the outside of the control apparatus 100 via the communication unit 117, and the like. The auxiliary storage device 114 includes, for example, a hard disk drive device, and stores captured images and data such as detection results from a sensor.

The display unit 115 includes, for example, a Liquid Crystal Display (LCD), and displays a Graphical User Interface (GUI) or the like for a user to operate the control apparatus 100. The operation unit 116 includes, for example, a keyboard and a mouse, and provides various instructions to the CPU 111 based on operations by the user. The communication unit 117 communicates with external apparatuses such as the terminal apparatus 200, the image pickup apparatus 300, the sensor group 400, the pointing apparatus 500, and a relay apparatus (not illustrated). The communication unit 117 includes an antenna that enables the control apparatus 100 to wirelessly connect with an external device. On the other hand, to establish a wired connection between the control apparatus 100 and an external device, for example, a LAN cable is attached to the communication unit 117. The bus 118 connects the various components of the control device 100 so that information is sent between them.

At least one of the display unit 115 and/or the operation unit 116 provided in the control apparatus 100 in the present exemplary embodiment may be a separate device outside the control apparatus 100. In this case, the CPU 111 operates as a display control unit that controls the display unit 115 and/or as an operation control unit that controls the operation unit 116.

[ Contents of selection information ]

The contents of the selection information 600 managed by the management unit 101 will now be described. The management unit 101 may acquire the selection information 600 from the pointing device 500, or may generate the selection information 600 in response to an instruction from the pointing device 500.

Fig. 4 is a diagram illustrating an example of the selection information 600. Conditional expression 610-1, conditional expression 610-2, and conditional expression 610-3 are written into selection information 600. In the following description, conditional expressions 610-1, 610-2, and 610-3 are collectively referred to as conditional expressions 610 unless otherwise specified. Conditional statement 620, action 630, indicator 650, and priority 640 are also written into selection information 600. Conditional statement 620 includes condition 620-1 and operator 620-2. Fig. 5 illustrates a specific example of the condition 620-1, the behavior 630, and the priority 640 in the write selection information 600. The selection information 600 is not limited to the content in the example shown in fig. 5, and may include other items and content written in the items.

In FIG. 4, in selection information 600, a conditional statement 620 follows conditional expression 610-1 representing a branch. An action 630 representing processing performed in the case where conditional statement 620 is true follows conditional expression 610-2. An indicator 650 indicating the end of the selection information 600 follows the action 630. The priority 640 may be written in the selection information 600. The plurality of pieces of selection information 600 acquired by the control apparatus 100 may include a plurality of descriptions representing associations between detection results from the sensors and information for identifying the image capturing apparatus. In this case, the priority 640 represents the priority of each association. Therefore, the control device 100 performs processing based on the selection information 600 having the larger value of the priority 640 among the plurality of selection information 600 thus acquired.

In the selection information 600, a plurality of conditional expressions 610 may be written. In conditional statement 620, a plurality of conditions 620-1 and a plurality of operators 620-2 may be written. The management unit 101 can collectively acquire/manage a plurality of selection information 600 as a single content. Specific examples of the operator 620-2 include AND, OR, NOT, NAND, NOR, AND EXOR. Conditional expression 610 is not limited to a single branch and may be a plurality of branches, a case structure (SWITCH statement), or the like. In act 630, multiple types of processes can be written. The selection information 600 is not limited to the format and order shown in fig. 4, and may be written in other formats such as structured languages including extensible markup language (XML), scripting languages including Python, and program languages.

[ Process flow of System ]

Next, a process flow of the image capturing system is described with reference to fig. 6 to 8. Fig. 6 is a sequence diagram illustrating processing executed in the image capturing system. Fig. 7 and 8 are each a flowchart illustrating processing executed by the control device 100. The process shown in fig. 7 starts at the timing when the connection for communication is established between the control apparatus 100 and the pointing device 500. The process shown in fig. 8 starts at the timing when the connection for communication is established between the control device 100 and the sensor group 400. The timing at which the processing shown in fig. 7 and 8 starts is not limited to the timing described above. The processing shown in fig. 7 and 8 is implemented when the CPU 111 loads a program stored in the ROM 112 onto the RAM 113 and executes the program. Fig. 6 is described below together with fig. 7 and 8.

In step S100(S200), the management unit 101 receives an instruction from the pointing device 500, and acquires the selection information 600 based on the instruction. The selection information 600 according to the present exemplary embodiment is used by the control apparatus 100 to select, from the plurality of image pickup apparatuses 300, the image pickup apparatus 300 that transmits a captured image to be displayed and/or stored by the terminal apparatus 200, based on the detection result from the sensor group 400. When the previously acquired selection information 600 is currently being managed, the management unit 101 performs management based on the determination of whether to delete the previously acquired selection information 600 or to hold both the previously acquired selection information 600 and the newly acquired selection information 600. The management unit 101 may acquire the selection information 600 from the pointing device 500, or may generate the selection information 600 in response to an instruction from the pointing device 500.

In step S101(S201), the sensor control unit 102 searches for a sensor connected to the network 20. When a sensor connected to the network 20 has been searched for by the sensor control unit 102, the processing in this step may be skipped. In step S102(S202), the image pickup control unit 103 searches for the image pickup apparatus 300 connected to the network 10. When the image capture control unit 103 has searched for the image capture apparatus 300 connected to the network 10, the processing in this step may be skipped. The processing in step S101(S201) and the processing in step S102(S202) may be performed in reverse order, or may be performed simultaneously.

In step S103(S203), the image pickup selection unit 105 analyzes the selection information 600 acquired by the management unit 101 in step S200, and sets an evaluation value for each of the plurality of image pickup apparatuses 300. The evaluation value is set based on the selection information 600 acquired by the management unit 101. The evaluation value corresponds to the possibility that each of the image pickup apparatuses 300 is to be selected as the image pickup apparatus 300 that transmits a captured image to be displayed and/or stored by the terminal apparatus 200. For example, a low evaluation value is set for the image capturing apparatus 300 as follows: the image pickup apparatus 300 is not written as a control object in the selection information 600 acquired by the management unit 101, and a captured image is not transmitted from the image pickup apparatus 300 to the terminal apparatus 200 regardless of the value of the detection result from the sensor. For example, the evaluation value may be set based on a history indicating a situation in which each image pickup apparatus 300 has been selected or the number of times the image pickup apparatus 300 has been selected.

The image capture selection unit 105 specifies, based on the selection information 600 that has been acquired by the management unit 101, the image capture apparatus 300 that reduces the data amount of the captured image to be transmitted, from among the plurality of image capture apparatuses 300. For example, the image pickup apparatus 300 whose evaluation value is lower than a predetermined threshold value is specified. In this specifying method, an image pickup apparatus 300 that is not selected based on the selection information 600 acquired by the management unit 101 is specified as an image pickup apparatus that reduces the amount of data. Unless the management unit 101 newly acquires the selection information 600, a captured image is not transmitted from the specified image capturing apparatus 300 regardless of the detection result from the sensor. Therefore, such a reduction in the data amount of the captured image does not greatly impair the usability of the system. The method for specifying the image capturing apparatus 300 in which the data amount of a captured image is to be reduced is not limited to the example described above, and other methods of specifying the image capturing apparatus 300 based on other criteria may be employed. The system may include the image pickup apparatus 300 that does not reduce the amount of transmission data regardless of the content of the selection information 600.

The sensor selection unit 104 specifies a sensor, from among the plurality of sensors in the sensor group 400, for which the amount of processing relating to the detection processing is to be reduced, based on the selection information 600 acquired by the management unit 101. For example, a sensor that is not written in the condition 620 in the selection information 600 acquired by the management unit 101, that is, a sensor whose detection result does not involve the above-described designation by the imaging selection unit 105 based on the selection information 600 acquired by the management unit 101. Unless the management unit 101 newly acquires the selection information 600, the detection results obtained by these sensors are not used. Therefore, the reduction in the processing amount of the detection processing for obtaining such detection results does not greatly impair the usability of the system. The method of selecting the sensor by the sensor selection unit 104 is not limited thereto. For example, as in the case where the image capturing apparatus 300 is specified by the image capturing selection unit 105, the sensor selection unit 104 may set evaluation values for the respective sensors. Therefore, a sensor to reduce the amount of processing relating to the detection processing can be specified based on the evaluation value. The system may include sensors that do not reduce the amount of processing regardless of the content of the selection information 600.

In step S104(S204), the sensor control unit 102 controls the sensor based on the result of the designation made by the sensor selection unit 104 in step S103 (S203). More specifically, the sensor control unit 102 controls the sensor designated by the sensor selection unit 104 so that the sensor is turned off, stops detection, performs measurement at a lower sampling rate, or operates in another similar manner. Therefore, the amount of processing relating to the detection processing performed by the sensor thus controlled is reduced, thereby achieving lower power consumption. The sensor control unit 102 may control the sensors not designated by the sensor selection unit 104 so that the sensors are turned on, start detection, perform measurements at a higher sampling rate, or operate in other similar manners. The control by the sensor control unit 102 is not limited to the above-described example, and may be other types of control.

In step S105(S205), the image capturing control unit 103 controls the image capturing apparatus 300 based on the result of the designation made by the image capturing selection unit 105 in step S103 (S203). More specifically, the image capture control unit 103 controls the image capture apparatus 300 specified by the image capture selection unit 105 such that the image capture apparatus 300 is turned off, stops capturing an image, or stops transmitting a captured image. The control by the imaging control unit 103 is not limited to these. For example, control may be performed such that the resolution of a captured image to be transmitted is reduced, the resolution of an image to be captured is reduced, the frame rate of a captured image to be transmitted is reduced, or an image is captured at a lower frame rate. Therefore, the amount of data of the captured image transmitted from the thus-controlled image capturing apparatus 300 is reduced, so that the load on the network 10 and the power consumption of the image capturing apparatus 300 are reduced. The image capture control unit 103 can control the image capture apparatus 300 not designated by the image capture selection unit 105 so that the image capture apparatus 300 is turned on, starts transmitting a captured image, or captures an image with a higher frame rate/resolution. The processing in step S104(S204) and the processing in step S105(S205) may be performed in reverse order, or may be performed simultaneously.

In step S206, the management unit 101 determines whether to stop acquisition of the selection information 600. If the management unit 101 determines to stop the acquisition of the selection information 600 (yes in step S206), the processing in fig. 7 is terminated. If the management unit 101 determines not to stop the acquisition of the selection information 600 (no in step S206), the management unit 101 waits for a new instruction from the pointing device 500.

In step S106(S300), the sensor control unit 102 receives the detection result from the sensor group 400. To receive the detection results, the sensor control unit 102 may poll the individual sensors or subscribe to detection events for the individual sensors.

In step S301, the imaging selection unit 105 checks the detection result against the selection information 600 to check whether the conditional statement 620 in the selection information 600 is true. When the conditional statement 620 is true (yes in step S301), the processing proceeds to step S302. When none of the conditional statements 620 is true (no in step S301), the processing proceeds to step S304. In the case where there is a conditional statement 620 that has been previously determined to be true, the imaging selection unit 105 checks whether the condition is still true.

In step S107(S302), the image pickup selection unit 105 selects the image pickup apparatus 300 that transmits the captured image to be displayed and/or stored by the terminal apparatus 200, based on the detection result received by the sensor control unit 102 and the selection information 600 acquired by the management unit 101. More specifically, the image capturing selection unit 105 selects the image capturing apparatus 300 that writes the behavior 630 corresponding to the conditional expression 620 that is true in the selection information 600.

In step S303, the control device performs processing of writing a behavior 630 corresponding to the conditional statement 620 being true. In the sequence described as an example in fig. 6, in step S108, the image acquisition unit 106 receives captured images transmitted from the plurality of image capturing apparatuses 300. Then, in step S109, the transmission unit 107 transfers, to the terminal apparatus 200, the captured image transmitted by the image capturing apparatus 300 selected by the image capturing selection unit 105, among the captured images received by the image acquisition unit 106 in step S108. In a case where a plurality of behaviors 630 respectively corresponding to a plurality of conditional expressions 620 that are true cannot be executed simultaneously, the control apparatus 100 determines the behavior 630 to be prioritized according to the timing of setting the selection information 600, the priority 640, and the like, and executes the processing. In the case where the conditional statement 620 corresponding to the act 630 of writing the ongoing process is no longer true, the control device 100 terminates the ongoing process.

In step S110(S304), the image sensing selection unit 105 recalculates the evaluation value that has been set in step S103(S203) based on the detection result received in step S300. An example of how to calculate the evaluation value is described below. A description is given based on an exemplary case where the conditional statement 620 describing the condition of the image capturing apparatus 300 selected as the object includes 10 conditions 620-1 connected to each other with AND as an operator 620-2. When one of the 10 conditions 620-1 is true and the other conditions 620-1 are false, the image pickup selection unit 105 sets 10 as the evaluation value of the object image pickup apparatus 300. Each time the true condition 620-1 is increased by 1, the evaluation value is increased by 10. Alternatively, the evaluation value may be calculated based on a physical quantity represented by the detection result. In other words, the evaluation value of the image pickup apparatus 300 selected when the temperature exceeds 100 ℃ may be set based on the temperature indicated by the detection result. For example, when the temperature is 40 ℃, the evaluation value may be set to 40, and when the temperature is 60 ℃, the evaluation value may be set to 60. With the evaluation value thus calculated, the control device 100 can perform control while distinguishing an image pickup apparatus 300 that may be selected from an image pickup apparatus 300 that may not be selected among the image pickup apparatuses 300 that can be selected based on the selection information 600.

The sensor selection unit 104 calculates evaluation values of the respective sensors, similarly to the image pickup selection unit 105 that calculates evaluation values of the image pickup apparatus 300. For example, the evaluation value of a sensor that detects whether a certain door in a building is opened/closed may be set based on the detection result from a proximity sensor for detecting whether a door is opened/closed.

In step S111(S305), the image capture control unit 103 controls the image capture apparatus 300 based on the evaluation value calculated by the image capture selection unit 105. For example, the image capture control unit 103 controls the image capture apparatus 300 whose evaluation value is lower than a predetermined threshold value so that the data amount of a captured image transmitted from the image capture apparatus 300 is reduced as in the processing in step S105(S205) described above. In step S112(S306), the sensor control unit 102 controls the sensor group 400 according to the evaluation value calculated by the sensor selection unit 104. For example, the sensor control unit 102 controls a sensor whose evaluation value is lower than a predetermined threshold value so that the amount of processing relating to the detection processing performed by such a sensor is reduced as in the processing in step S104(S204) described above. By these types of control, the power consumption of the image pickup apparatus 300 and the sensor group 400 and the load of the network 10 can be reduced. The processing in step S111(S305) and the processing in step S112(S306) may be performed in reverse order, or simultaneously.

The image capture control unit 103 may control the image capture apparatuses 300 whose evaluation values are lower than a predetermined threshold value in unison or differently depending on differences in the evaluation values. For example, the imaging control unit 103 may control in the following manner: the image pickup apparatus 300 that stops transmitting the captured image is distinguished from the image pickup apparatus 300 that captures an image at a lower frame rate. The same applies to the control of the sensors by the sensor control unit 102.

In a case where the image pickup apparatus 300 serving as a control target is in the sleep mode, the image pickup control unit 103 may wait until the image pickup apparatus 300 recovers from the sleep mode to issue a notification for control to the image pickup apparatus 300. Alternatively, the image capture control unit 103 may issue a notification to start recovery of the image capture apparatus 300 from the sleep mode at an appropriate timing. The image capture control unit 103 can determine whether to wait for the timing at which the image capture apparatus 300 recovers from the sleep mode based on the evaluation value set for the image capture apparatus 300. For example, the image capture control unit 103 may wait for the recovery timing to issue a notification to the image capture apparatus 300 having a low evaluation value, and may issue a notification to the image capture apparatus 300 having a high evaluation value without waiting for the recovery timing. The same applies to the control of the sensors by the sensor control unit 102. The sensor control unit 102 may change the interval of polling of the sensors according to the evaluation value. For example, the sensor control unit 102 may set a short polling interval for a sensor having a high evaluation value, and may set a long polling interval for a sensor having a low evaluation value.

In step S307, the sensor control unit 102 determines whether to stop receiving the detection result from the sensor group 400. When the sensor control unit 102 determines to stop the reception (yes in step S307), the processing in fig. 8 is terminated. When the sensor control unit 102 determines not to stop the reception (no in step S307), the sensor control unit 102 waits until the detection result is newly received from the sensor group 400.

As described above, the control device 100 according to the present exemplary embodiment receives the detection result from the sensor group 400. The control apparatus 100 then acquires the selection information 600 to select, from the plurality of image capturing apparatuses 300, the image capturing apparatus 300 that captures the captured image to be displayed and/or stored by the terminal apparatus 200 based on the detection result. The control apparatus 100 specifies, based on the thus acquired selection information 600, the image capturing apparatus 300 of the plurality of image capturing apparatuses 300 whose data amount of captured images to be transmitted is to be reduced. Then, the control device 100 controls the image pickup apparatus 300 based on the designation result. Therefore, in an image pickup system that displays and/or stores a picked-up image from an image pickup apparatus 300 selected from a plurality of image pickup apparatuses 300 based on the detection result from the sensor group 400, the following effects can be obtained. Specifically, a lower load of the system can be achieved as compared with a case where all the image capturing apparatuses 300 uniformly transmit captured images. It is possible to reduce the delay in the transmission of the captured image, as compared with the case where all the unselected image capturing apparatuses 300 stop transmitting the captured image.

In the configuration described in the present exemplary embodiment, the control device 100 sets evaluation values for each of the image capturing apparatus 300 and the sensors. Therefore, the image pickup apparatus 300 that is to reduce the data amount of the captured image to be transmitted and the sensor that is to reduce the processing amount related to the detection processing are specified. The control apparatus 100 specifies the image capturing apparatus 300 and the sensor based on the selection information 600, and does not have to set the evaluation value. The control device 100 may control as follows: the amount of data transmitted from the image pickup apparatus 300 is reduced without performing control to reduce the amount of processing of the sensor. Also in this configuration, the effect of achieving a lower load of the system without impairing the usability of the system can be achieved.

In the present exemplary embodiment, a configuration is described in which the image capturing apparatus 300 that transmits a captured image with a reduced data amount is specified and controlled. However, the configuration is not limited to this, and the image capturing apparatus 300 that transmits a captured image with an increased data amount may be specified and controlled. For example, the control device 100 may control as follows: the amount of communication regarding the captured image of the image capturing apparatus that may not be selected based on the acquired selection information 600 is made smaller than the amount of communication regarding the captured image of the image capturing apparatus that may be selected based on the acquired selection information 600. For example, the control device 100 may control as follows: the amount of communication relating to a captured image of an image capturing apparatus whose evaluation value set based on the likelihood of being selected is lower than the threshold value is made smaller than the amount of communication relating to a captured image of an image capturing apparatus whose evaluation value is higher than the threshold value.

In the configuration described in the present exemplary embodiment, the control device 100 controls the traffic volume related to the transmission of the captured image by the image capturing apparatus 300. However, the configuration is not limited to this, and the control device 100 may control the traffic related to the transfer of the captured image to the terminal apparatus 200. For example, the plurality of image capturing apparatuses 300 transmit a plurality of captured images to the control device 100. Then, the control device 100 transmits a captured image, which is possibly selected based on the selection information 600, from among the plurality of captured images, to the terminal apparatus 200 with high image quality. The control device 100 may control as follows: a photographed image that may not be selected based on the selection information 600 from among the plurality of photographed images received thereby is transmitted to the terminal device 200 with a low image quality or is not transmitted to the terminal device 200.

According to the above-described exemplary embodiments, in a system that uses a captured image of an image capturing apparatus selected from a plurality of image capturing apparatuses, it is possible to reduce a load applied to the system without impairing the usability of the system.

Other embodiments

Or by reading and executing data recorded on a storage medium (also more completely referred to as a "non-transitory computer-readable storage medium")The computer of the system or apparatus that includes one or more circuits (e.g., Application Specific Integrated Circuits (ASICs)) for performing the functions of one or more of the above-described embodiments, and/or includes one or more circuits (e.g., ASICs)) for performing the functions of one or more of the above-described embodiments, and the embodiments of the present invention may be implemented using a method by which the computer of the system or apparatus, for example, reads and executes the computer-executable instructions from the storage medium to perform the functions of one or more of the above-described embodiments, and/or controls the one or more circuits to perform the functions of one or more of the above-described embodiments. The computer may include one or more processors (e.g., Central Processing Unit (CPU), Micro Processing Unit (MPU)) and may include a separate computer or a network of separate processors to read out and execute computer-executable instructions. The computer-executable instructions may be provided to the computer, for example, from a network or a storage medium. The storage medium may include, for example, a hard disk, Random Access Memory (RAM), Read Only Memory (ROM), memory of a distributed computing system, an optical disk such as a Compact Disk (CD), Digital Versatile Disk (DVD), or blu-ray disk (BD)^TM) A flash memory device, a memory card, and the like.

The embodiments of the present invention can also be realized by a method in which software (programs) that perform the functions of the above-described embodiments are supplied to a system or an apparatus through a network or various storage media, and a computer or a Central Processing Unit (CPU), a Micro Processing Unit (MPU) of the system or the apparatus reads out and executes the methods of the programs.

While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.

Claims

1. A control device, the control device comprising:

an acquisition unit configured to acquire, based on a detection result of a sensor, selection information for specifying that one or more image pickup apparatuses are to be selected from among a plurality of image pickup apparatuses, wherein the selection information includes information that the detection result relates to the sensor specifying the one or more image pickup apparatuses;

a first control unit configured to control, independently of a detection result, in a manner as follows: in the plurality of image pickup apparatuses, an amount of traffic relating to a captured image from an image pickup apparatus not included in the one or more image pickup apparatuses to be selected in the selection information is made smaller than an amount of traffic relating to a captured image from an image pickup apparatus included in the one or more image pickup apparatuses to be selected in the selection information;

a communication unit configured to be able to communicate with a plurality of sensors; and

a second control unit configured to control so that an amount of processing relating to detection processing performed by a sensor whose detection result does not relate to the designation of the one or more image pickup apparatuses is relatively reduced compared to an amount of processing relating to detection processing performed by a sensor whose detection result relates to the designation of the one or more image pickup apparatuses.

2. The control apparatus according to claim 1, further comprising a setting unit configured to set an evaluation value corresponding to a likelihood that an image capturing apparatus is selected based on the acquired selection information, for each of one or more image capturing apparatuses of the plurality of image capturing apparatuses,

wherein the first control unit controls in the following manner: the amount of communication relating to a captured image from an image capturing apparatus whose evaluation value is smaller than a threshold value is made smaller than the amount of communication relating to a captured image from an image capturing apparatus whose evaluation value is larger than the threshold value.

3. The control device according to claim 2, further comprising a receiving unit configured to receive a detection result from the sensor,

wherein the setting unit sets the evaluation value based on the physical quantity indicated by the received detection result and the acquired selection information.

4. The control device according to claim 1, further comprising:

a first receiving unit configured to receive a detection result from the sensor;

a second receiving unit configured to receive a plurality of captured images transmitted from the plurality of image capturing apparatuses;

a selection unit configured to select an image pickup apparatus based on the acquired selection information and the received detection result; and

a transmission unit configured to transmit the plurality of captured images transmitted from the selected image capturing apparatus to the terminal apparatus.

5. The control device according to claim 1, wherein the control of the traffic by the first control unit includes at least one of: the processing for stopping transmission of a captured image from the image capturing apparatus, the processing for reducing the resolution of the captured image transmitted from the image capturing apparatus, or the processing for reducing the frame rate of the captured image transmitted from the image capturing apparatus.

6. The control device according to claim 1, wherein the control of the traffic by the first control unit includes at least one of: a process for stopping image capturing by the image capturing apparatus, a process for reducing the resolution of an image captured by the image capturing apparatus, or a process for reducing the frame rate of image capturing by the image capturing apparatus.

7. The control device according to claim 1, further comprising:

a receiving unit configured to receive a plurality of captured images transmitted from a plurality of image capturing apparatuses; and

a transmitting unit configured to transmit at least one of the received plurality of captured images to the terminal device,

wherein the traffic volume related to the photographed image is a traffic volume related to transmission of the photographed image by the transmission unit, and

wherein the first control unit controls a traffic volume related to the transmission of the captured image by the transmission unit.

8. The control device according to claim 1, wherein the detection result relates to a detection result of at least one of temperature, illuminance, vibration, infrared ray, time of day, time period, or an open/close state of the door.

9. A control method of a control apparatus, the control method comprising:

an acquisition step of acquiring, based on a detection result of a sensor, selection information for specifying one or more image pickup apparatuses to be selected from among a plurality of image pickup apparatuses, wherein the selection information includes information that the detection result relates to the sensor specifying the one or more image pickup apparatuses;

a first control step of controlling, regardless of a detection result, the following: in the plurality of image pickup apparatuses, an amount of traffic relating to a captured image from an image pickup apparatus not included in the one or more image pickup apparatuses to be selected in the selection information is made smaller than an amount of traffic relating to a captured image from an image pickup apparatus included in the one or more image pickup apparatuses to be selected in the selection information;

a communication step of enabling communication with a plurality of sensors; and

a second control step of performing control so that an amount of processing relating to detection processing performed by a sensor whose detection result does not relate to the designation of the one or more image pickup apparatuses is relatively reduced compared to an amount of processing relating to detection processing performed by a sensor whose detection result relates to the designation of the one or more image pickup apparatuses.

10. The control method according to claim 9, the control method further comprising:

a setting step of setting, for each of one or more image pickup apparatuses of the plurality of image pickup apparatuses, an evaluation value corresponding to a possibility that the image pickup apparatus is selected based on the acquired selection information,

wherein in the first control step, control is performed in such a manner that: the amount of communication relating to a captured image from an image capturing apparatus whose evaluation value is smaller than a threshold value is made smaller than the amount of communication relating to a captured image from an image capturing apparatus whose evaluation value is larger than the threshold value.